Three Mile Island HRA using THERP: Part 1

US2018

HandbookofHumanReliabilityAnalysis.pdf

Home >Applied Sciences homework help >Three Mile Island HRA using THERP: Part 1

NUREG/CR- 1278 SAND80-0200 RX, AN Printed August 1983

Handbook of Human Reliability Analysis with Emphasis on Nuclear Power Plant Applications

Final Report A. D. Swain, H. E. Guttmann

Prepared by Sandia National Laboratories Albuquerque, New Mexico 87185 and Livermore, California 94550 for the United States Department of Energy under Contract DE-AC04-76DP00789

Wi ... "

P It II

.1 Prepa-red for U.,SQ.NUCLEAR

SF2900Q(8-81 )

REGULATORY COMMISSION",I IT,,I ; h " , , , . . . . , . . ; ,' • .

NOTICE This report was prepared as an account of work sponsored by an agency of the United States Government. Neither the United States Government nor any agency thereof, or any of their employ- ees, makes any warranty, exprcssed or implied, or assumes any legal liability or responsibility for any third party's use, or the results of such use, of any information, apparatus product or process disclosed in this report, or represents that its use by such third party would not infringe privately owned rights.

Available from 0 GPO Sales Program Division of Technical Information and Document Control U.S. Nuclear Regulatory Commission Washington, D.C. 20555

and

National Technical Information Service Springfield, Virginia 22161

September 1, 1985 A. D. Swain

Addendum #1 to NUREG/CR-1278, August 1983

Following are some corrections to the above document, listing the page (p) and paragraph (para) numbers. Understandable typos are excluded. Some updating of references is included.

p.para # Description of Corrections

3-14,5 Line 3: ... are doubled for step-by-step tasks and quintupled for dynamic tasks. However, this Line 6: ... of human reliability, e.g., an additional factor of 2 increase in the estimated HEP. This is an [For further explanation, see the change for p 17-7.]

3-35,5 Line 4: ... is generally correct, and is known as the Inverted-U Hypothesis or the Yerkes-Dobson Law (Welford, 1974; Fitts and Posner, 1967). This means

6-3,5 Line 7: Beare, Dorris et al, 1982, 1983). No ...

6-5,6 Line 6: (1984) under ...

7-15 Table 7-3, item (1)(a): .01 (EF=3, 5, or 10) [This addition agrees with Table 7-2 which indicates conditions when any one of these EFs is relevant.]

11-8 Table 11-2 title, line 2: ... displays (or annunciated displays no longer annunciating) for ...

11-16 Table 11-4, item (7): The ** should be followed by the same "cross" footnote symbol used for item (8).

11-51 Table 11-13, add to last footnote: The Pr[Fi] column

assumes that all of the ANNs (or completely dependent sets of ANNs) are equal in terms of the probability of being noticed. See page 11-52, paragraph 2, if this assumption does not hold.

12-8,4 Line 7: ... emergency operating procedures ...

12-12,3 Line 4 & 5: ... Wreathall (1981, p 104, and 1982),

12-13 Table 12-3: The 3 lines in the figure should meet at the 1.0 Pr[F] rather than at the .9 HEP. Also, the 1 (e.g., HEP of 1.0) is incorrectly placed; it should be in the same relative position above the .9 HEP as is the .1 HEP above the .09 HEP. The Pr[F] values for the lower bound and median HEPs at 10 minutes are correct. The Pr[F] values for the upper bound HEP at 20 minutes should be 1.0.

-2-

12-20 Table 12-4: The 3 lines in the figure should meet at the 1.0 Pr[F] rather than at the .9 HEP. Also, the 1 (e.g., HEP of 1.0) is incorrectly placed; it should be in the same relative position above the .9 HEP as is the .1 HEP above the .09 HEP. The Pr[F] values for the lower bound and median HEPs at 10 minutes are correct. The Pr[F] values for the upper bound HEP at 10 minutes should be 1.0.

13-6 Table 13-3: Items (2), (3), and (4) correctly pertain to the potential error, "Select wrong control on a panel from an array of similar-appearing controls." There should have been another item comparable to item (1) in Table 11-2 (p 11-8). In order not to change the item numbers in Table 13-3, insert the following item after item (1):

(1A) Select wrong control when it is Negligible dissimilar to adjacent controls

17-7 Footnote, line 3: ... multiplied by an additional factor of 2 because ... [The intent of this change is to enable an analyst to apply the factor of 2 for step-by-step tasks or 5 for dynamic tasks, as discussed in paragraph 6 for "heavy task loading," and then to further increase the resultant HEP by an additional factor of 2 for the extra stressors listed in the footnote.]

19-4 Table 19-1, item (6): Rising stem with or without a position indicator** [This change is based on the conservative assumption that the checker will just look at the rising stem even when there is a position indicator.]

20-3 Figure 20-1 (pl of 3): Insert a "Screening Required?" hexagon in the No line following the "Abnormal Event?" hexagon. The YES and NO lines from the new hexagon go to the "Rule-Based Actions?" hexagon. Insert a "Decide on Screening Rules" box in the YES line. [This change is necessary because many PRAs require some form of screening for pre-accident situations.]

20-4 Figure 20-1 (p2 of 3), "Other PSFs" box: Other PSFs (see text, especially pp 3-14, -17,- 19,- 72, and 17-7)

20-6,5 Item (2), line 7: ... Assume YES. (Note: Screening may also be required for non-abnormal tasks, as shown in the NO path following the ABNORMAL EVENT? haxagon.)

20-25 Table 20-9 title, line 2: ... displays (or annunciated displays no longer annunciating) ...

-3-

20-27 Table 20-11, item (7): The ** should be followed by the same "cross" footnote symbol used for item (8).

20-28 Table 20-12: Items (2), (3), and (4) correctly pertain to the potential error, "Select wrong control on a panel from an array of similar-appearing controls." There should have been another item comparable to item (1) in Table 20-9 (p 20-25). In order not to change the item numbers in Table 20-12, insert the following item after item (1):

(1A) Select wrong control when it is Negligible dissimilar to adjacent controls

20-37 Table 20-21, item (1)(a): .01 (EF=3, 5, or 10) [This addition agrees with Table 20-20 which indicates conditions when any one of these EFs is relevant.]

20-38 Item (6) in Table 20-22: Rising stem with or without a position indicator** [This change is based on the conservative assumption that the checker will just look at the rising stem even when there is a position indicator.]

20-39 Table 20-23, add to last footnote: The Pr[Fi] column

assumes that all of the ANNs (or completely dependent sets of ANNs) are equal in terms of the probability of being noticed. See page 11-52, paragraph 2, if this assumption does not hold.

20-44 See change for p 20-3.

20-45 See change for p 20-4.

22-5,1 Line 11: Beare, Dorris et al, 1982, 1983).

A-9 Step 5, the numerator in the first equality: WPr[FT]

[The 0 was left out.]'

H-2 7th reference, line 5: DC, January 1984.

H-2 9th reference, line 3: ... New York: Plenum Press, 1984

H-3 9th reference, line 4: ... and M. W. McCann, Review and Evaluation of the Zion Probabilistic Safety Study, Sandia National Laboratories, ... DC, January 1984.

H-4 6th reference, line 1: ... Kozinsky, D. S. Crowe,

H-5 2nd & 3rd references: Add "Inc." after "Human Performance Technologies,".

-4-

H-13 4th reference, line 3: ... and D. M. Kunsman,

H-15 13th reference, line 2: ... Society, 1984.

H-15 14th reference, line 3: ... Press, 1984.

H-16 4th reference, line 1: .... Comments on Draft NUREG/CR- 1278, ... line 2: ... NM, May 1985.

H-17 10th reference, line 2: Vols 1 and 2,

H-18 4th reference, line 2: Society and the Institute ...

H-21 8th reference, line 3: Nuclear Safety, ...

H-22 4th reference, lines 3 & 4: ... Laboratory, NUREG/CR- 3114, U.S. Nuclear Regulatory Commission, Washington DC, December 1982.

H-22 6th reference, line 5: DC, August 1983.

H-25 6th reference, line 2: ... Equipment, D&A-TR-60-36F,

H-26 6th reference, line 4: Press, 1984.

J-7 3rd definition, line 1: ... part of diaQnosis: ... line 2: ... alternative diagnoses, ... [remove underlining]

J-8 7th definition, line 2: ... identify those systems ...

J-8 14th definition, line 2: ... level in Tank A ...

J-16 10th definition, line 1 etc: ... of human error on all of the tasks in the complete-failure path.

J-16 lth definition, line 1 etc: ... of no human errors on all of the tasks in the complete success path.

Please notify A. D. Swain, Div. 7223, Sandia National Laboratories, PO Box 5800, Albuquerque, NM 87185, phone: (505) 844-3675 (FTS 844-3675, Autovon 244-3765), of any other major errors.

NUREG/CR-1278 may be purchased for $12.00 postpaid from: US Government Printing Office, PO Box 37082, Wash. DC 20013-7082, phone: (202) 275-2060. The companion document, NUREG/CR-2254, May 1983, may be purchased there for $5.50 postpaid.

NUREG/CR-1278 SAND80-0200 RX, AN

Handbook of Human Reliability Analysis with Emphasis on Nuclear Power Plant Applications

Final Report

Manuscript Completed: June 1983 Date Published: August 1983

Prepared by A.D. Swain, H.E. Guttmann

Sandia National Laboratories Albuquerque, NM 87185

Prepared for

Division of Facility Operations Office of Nuclear Regulatory Research U.S. Nuclear Regulatory Commission Washington, D.C. 20555 NRC FIN A 1188

ABSTRACT

The purpose of the revised Handbook is to present methods, models, and estimated human error probabilities to enable qualified analysts to make quantitative or qualitative assessments of occurrences of human errors that may affect the availability or operational reliability of engineered safety features and components in nuclear power plants. The Handbook provides most of the modeling and information necessary for the performance of human reliability analysis as a part of probabilistic risk assessment of nuclear power plants. Limitations of the Handbook and cautions to be observed in its use are explicitly stated.

FOREWORD

The revised Handbook is the product of research beginning in September

1976. An early draft version dated April 1980 was supplanted by the October 1980 draft with a distribution of about 2,000. Extensive use of

the October draft by practitioners in probabilistic risk assessment (PRA) and human reliability analysis (HRA) provided us with valuable information on which to base the present version of NUREG/CR-1278. So many users of

the draft Handbook have provided comments and suggestions for improvement

that we have devoted an appendix (E) to listing the major contributors.

During the "Handbook project," we have had guidance and support from sev-

eral program managers at the U.S. Nuclear Regulatory Commission (NRC), Office of Nuclear Regulatory Research. The original idea for the Handbook

came from Dr. William E. Vesely, then of NRC's probabilistic analysis staff (now with Battelle's Columbus Laboratories). Dr. Vesely took an active role in the development of the draft Handbook and wrote a draft of the

original chapter on unavailability. Other NRC program managers associated with the draft Handbook were Dr. Michael C. Cullingford and Charles B. Oh

(now with Technology for Energy Corporation). From October 1980 to the present, the NRC managers for the present issue of the Handbook have been

James P. Jenkins, James W. Pittman, and Dr. Thomas G. Ryan (the present

manager).

Special thanks are due several persons from Sandia National Laboratories

(SNL). Dr. Richard R. Prairie, Manager, Reliability Department, developed the methods in Appendix A for propagating uncertainty bounds in an HRA and for determining uncertainty bounds for dependent human activities. Dr. Louise M. Weston, of the human factors group in the Reliability Department,

wrote Chapter 8 about the use of expert judgment in HRA. This is based on

continuing research in this area she manages. Dr. Robert G. Easterling, of the statistical group in the Reliability Department, developed an alter-

native method for estimating the effects of dependence, which is presented here as Appendix B. While a member of SNL's human factors group, and later with Battelle's Columbus Laboratories, Barbara Jean Bell assisted in the rewriting of material from the draft Handbook and was instrumental in the

development of the search scheme and the organization of the data tables in the new Chapter 20. Dr. Dwight P. Miller managed a project to test how well people could use the draft Handbook. This project, conducted by Human Performance Technologies, Inc., provided valuable insights for preparation of the present issue of the Handbook. Or. Miller refined the Chapter 20 search scheme and also organized all of the several hundred comments on the draft Handbook in a usable format. This work is summarized in Appendix E. Finally, Bonnie P. Roudabush proofread the manual and checked all the numbers and figures, attempting to catch all of our many errors.

Tech Reps, Inc., Albuquerque, New Mexico, provided the word processing for the draft and revised Handbooks. Special thanks are due Bruce E. Howden, who provided editorial services, Wendy E. West, who was responsible for coordinating production, and Rosanne C. Rohac who was responsible for the artwork.

The intended applicability of the revised Handbook has benefitted from our

experience in teaching several HRA courses based on the draft Handbook and from our visits to several different types of reactors in Denmark, England, Finland, France, Japan, Norway, Scotland, Sweden, and the United States. We have studied human factors and human reliability aspects of one or more units in each of the following plants or simulators: four U.S. and five foreign boiling water reactors, six U.S. and four foreign pressurized water reactors, two foreign gas-cooled reactors, two foreign fast breeder reac- tors, three U.S. and one foreign research reactors, and six U.S. and five foreign nuclear reactor plant simulators. We wish to thank the personnel from the plants we have visited for taking the time to acquaint us with plant policies, operating practices, procedures, etc., that affect human reliability.

There are many differences between the draft and present versions of the

Handbook. While only a few changes have been made in the estimated proba- bilities and models in the draft Handbook, new estimates and new models have been developed. We have developed some interim models for the diag- nosis and decision-making processes involved in coping with an abnormal event in a nuclear power plant. We do not claim to provide more than a very rough model for this area of human behavior. Use of the model in PRAs will provide a starting point for the development of a more comprehensive model of the human diagnosis and decision-making process.

The most obvious difference between the draft and present versions of the

Handbook is the major reorganization of content--the chapters have been put into a more logical sequence and the chapters themselves have undergone major changes. Chapter 20, which includes nearly all of the estimated human error probabilities in the Handbook, has been completely reorganized. Of major help to the user is a search scheme to aid him in finding the estimates he needs to consider in a typical PRA. Appendix F presents a detailed description of all the significant differences between the draft and present versions.

We intend this Handbook to be a living document; we plan to update it as more data and better human performance models become available. For this reason, the document is not permanently bound but is punched for use in

ring binders.

We urge users to provide us with feedback on any problems they have in applying Handbook data, models, and methods to solving practical problems of HRA. We would also like to be informed of any error relative frequency data that become available for comparison with the estimated probabilities

of human error summarized in Chapter 20.

We have also prepared a companion Workbook to the Handbook. The Workbook presents a step-by-step procedure for conducting an HRA of operations in

nuclear power plants and provides several examples to guide the user of the Handbook. The Workbook is NUREG/CR-2254, first presented in draft form in December 1981 and now available in a revised version (Bell and Swain, 1983).

CONTENTS

Page

PART I. BASIC CONCEPTS I-I

CHAPTER 1. INTRODUCTION 1-1

Purpose of Handbook 1-1 Scope 1-2 Relationship of Handbook to WASH-1400 1-3 Limitations of the Handbook 1-3 Organization of the Handbook 1-6 Related Documents 1-8 How To Use This Handbook 1-9

CHAPTER 2. EXPLANATION OF SOME BASIC TERMS 2-1

Overview 2-1 Human Factors Engineering, Human Engineering, Human Factors, and Ergonomics 2-2

Man-Machine System and Interfaces 2-2 NPP Personnel Addressed in the Handbook 2-2 Displays, Manual Controls, and Written Materials 2-3 Human Reliability 2-3 Human Reliability Analysis 2-4 Probabilistic Risk Assessment 2-4 Human Performance Models 2-4 Performance Shaping Factors 2-5 Stressors and Stress (Physiological and Psychological) 2-5 Populational Stereotypes 2-6 Dependence, Independence, and Coupling 2-6 Human Error 2-7 Consequences of Human Errors and Recovery Factors 2-8 Types of NPP Operating Conditions Considered in HRA 2-9 Unavailability Because of Human Error 2-9 Types of NPP Tasks and Task Behaviors Addressed in HRA 2-10 Task Taxonomy 2-13 Task Analysis 2-13 Error-Likely Situations and People 2-14 Accident-Prone Situations and People 2-15 Categories of Incorrect Human Outputs Related to HRA 2-16 Human Error Probability 2-17 Basic, Conditional, and Joint Probabilities 2-18 Distribution of HEPs 2-18

CHAPTER 3. SOME PERFORMANCE SHAPING FACTORS AFFECTING HUMAN RELIABILITY 3-1

Overview 3-1 The Human As a System Component 3-1 The Role of Performance Shaping Factors in HRA 3-6 Classes of Performance Shaping Factors 3-10

External PSFs 3-11

vii

CONTENTS

Page

CHAPTER 3 (con't)

Internal PSFs 3-11 Stressors 3-11

Situational Characteristics 3-12 Architectural Features 3-12 Quality of the Working Environment 3-13 Work Hours and Work Breaks ,3-15

Shift Rotation and Night Work 3-18 Availability/Adequacy of Special Equipment/Tools

and Supplies 3-19

Manning Parameters 3-19 Organizational Structure and Actions by Others 3-22 Rewards, Recognition, and Benefits 3-22

Task and Equipment Characteristics 3-22 Perceptual Requirements 3-24 Motor Requirements 3-24

Control-Display Relationships 3-24 Anticipatory Requirements 3-24 Interpretation Requirements 3-25 Decision-Making 3-25 Complexity (Information Load) 3-25 Frequency and Repetitiveness 3-26 Task Criticality 3-26 Long- and Short-Term Memory Requirements 3-26 Calculational Requirements 3-27 Feedback 3-27 Dynamic Versus Step-by-Step Activities 3-27 Team Structure 3-28 Man-Machine Interface Factors 3-28

Job and Task Instructions 3-28 Stressors 3-33

Psychological Stressors 3-33 Physiological. Stressors 3-40 Summary of Human Reaction to Stress 3-42

Internal PSFs 3-42 Some Ergonomics Problems in Operating NPPs 3-48

Poor Design and Layout of Controls and Displays 3-48 Poor Labeling of Controls and Displays in the Control Room 3-52

Inadequate Indications of Plant Status 3-62 Presentation of Nonessential Information 3-65 Inadequate Labeling and Status Indications Outside the Control Room 3-66

Most Important Classes of PSFs Related to System Safety 3-68 Effects of Good Ergonomics Practices on Estimated HEPs 3-70

viii

CONTENTS

Page

PART II. METHODS FOR ANALYSIS AND QUANTIFICATION

OF HUMAN PERFORMANCE II-1

CHAPTER 4. MAN-MACHINE SYSTEMS ANALYSIS 4-1

Overview 4-I

Step 1 - Describe the System Goals and Functions 4-2 Step 2 - Describe the Situational Characteristics 4-4

Step 3 - Describe the Characteristics of the Personnel 4-4

Step 4 - Describe the Jobs and Tasks the Personnel Perform 4-5

Step 5 - Analyze the Jobs and Tasks To Identify Error-Likely Situations (ELSs) and Other Problems 4-9

Step 6 - Estimate the Likelihood of Each Potential Error 4-17 Step 7 - Estimate the Likelihood That Each Error Will Be Undetected (or Uncorrected) 4-1I

Step 8 - Estimate the Consequences of Each Undetected (or Uncorrected) Error 4-18

Step 9 - Suggest Changes to the System 4-19

Step 10 - Evaluate the Suggested Changes (Repeat

Steps 1 through 9) 4-22

CHAPTER 5. A TECHNIQUE FOR HUMAN RELIABILITY ANALYSIS 5-1

Overview 5-1

Background of THERP 5-1 Description of THERP 5-2

Model Versus Method 5-4 HRA Event Trees 5-5 Event Trees Versus Fault Trees 5-9 Nominal, Basic, Conditional, and Joint Probabilities 5-10 An Example of a Human Reliability Analysis 5-12

Use of HRA Outputs in System Reliability Studies 5-16

Design Trade-Off Studies 5-16 Probabilistic Risk Assessment 5-16

Sensitivity Analysis 5-18

General Form of an HRA for PRA 5-19

"Validity" of HRA Using THERP 5-22

CHAPTER 6. SOURCES OF HUMAN PERFORMANCE ESTIMATES 6-1

Overview 6-1

The Data Problem for HRA 6-1

Categories of Sources Used for Derived HEPs 6-2 Nuclear Power Plants 6-3 Dynamic Simulators of NPPs 6-5 Process Industries 6-6 Industrial and Military Data 6-6 Field Studies 6-7 Experiments Using Artificial Tasks 6-7

CONTENTS

Page

CHAPTER 6 (con't)

Use of Expert Opinion 6-8 Discretizing of Continuous Variables 6-9 Estimates of Performance Times for HRA 6-10

Time Taken To Begin a Task 6-10 Time Taken To Do a Task Correctly 6-11 Fixed Time Intervals for Correct Task Performance 6-12 Estimates of Performance Times in the Handbook 6-12

Some Data Related to Use of Displays 6-13 Data on Quantitative Reading of Analog Displays 6-14 Data on Check-Reading of Analog Displays 6-15 Data on Quantitative Reading of Digital Displays 6-15 Errors of Recording 6-16 Errors in the Use of CRTs 6-16

Estimates of HEPs Related to Valving Operations 6-17 Some Data Related to Oral Instructions and Written Procedures 6-17

Administrative Control 6-18 Some Data Related to Manual Controls 6-18 Models of Response to Abnormal Events 6-19 Use of the Nominal HEPs from Chapter 20 6-19 A Final Comment on Data Sources 6-20

CHAPTER 7. DISTRIBUTION OF HUMAN PERFORMANCE AND UNCERTAINTY BOUNDS 7-1

Overview 7-1 Data and Hypotheses about Human Variability 7-1

Intra- and Interindividual Variability 7-I The Normal Distribution 7-2 The Lognormal Distribution 7-5 Relative Unimportance of Assumed Distributions 7-6 Definition of Nominal HEPs 7-8

Uncertainty Bounds 7-9 Need for and Definitions of Measures of Uncertainty 7-9 Elements of Uncertainty in an HRA 7-9 Derivation of Uncertainty Bounds 7-11

Guidelines for Use of HEP Uncertainty Bounds 7-14 Bounding Analysis 7-14 Propagation of Uncertainty Bounds in an HRA 7-18

CHAPTER 8. USE OF EXPERT OPINION IN PROBABILISTIC RISK ASSESSMENT 8-1

Overview 8-1 Need for the Use of Formal Judgment Procedures 8-1 Potential Applications of Judgment Procedures 8-2

CONTENTS

CHAPTER 8 (con't)

General Requirements for the Use of Judgment Procedures General Descriptions of Recommended Judgment Procedures

Paired Comparisons Procedure Ranking/Rating Procedure Direct Numerical Estimation Procedure Indirect Numerical Estimation Procedure

Overall Evaluations of Judgment Procedures Quality of Judgments Difficulty of Data Collection Empirical Support

Recommendations in View of Practical Considerations and Types of Events To Be Judged

CHAPTER 9. UNAVAILABILITY

Overview Unavailability Equations Applications of the Unavailability Equations

Example No. 1 - Unavailability of a Diesel Generator Example No. 2 - Unavailability of a Primary Source

of Emergency Coolant

PART III. HUMAN PERFORMANCE MODELS AND ESTIMATED HUMAN ERROR PROBABILITIES

CHAPTER 10. DEPENDENCE

Overview Different Methods for Assessing Dependence Two Types of Dependence Characteristics of Dependence Conditional Probabilities of Error Based on Data Direct Estimation of Conditional Probabilities of Error The Positive Dependence Model

Levels of Dependence Complete-Failure Path and Complete-Success Path Dependence Dependence in Other Paths

Psychological Considerations for Levels of Dependence Qualitative Difference Between Zero and Nonzero Levels of Dependence

Zero Dependence Low Dependence Moderate Dependence High Dependence Complete Dependence

Functional Relationships among Tasks

Page

8-3 8-5 8-5 8-6 8-7 8-9 8-9 8-10 8-10 8-12

8-12

g-1

9-1

9-1 9-6 9-6

9-6

III-1

10-1

10-1 10-2 10-4 10-5 10-6 10-10 10-10 10-12 10-14 10-14

10-15 10-15 10-17 10-17 10-18 10-18 10-19

CONTENTS

Page

CHAPTER 10 (con't)

Awareness of One's Own Errors 10-20 General Guidelines in Assessing Level of Dependence 10-22

Relationship of Dependence Model to an Earlier Model 10-22 Equations for Zero and Complete Dependence 10-22 Different Methods for Determining Intermediate Levels of Dependence 10-25

Application of Dependence Equations 10-28 Application of the Dependence Model to a Parallel-Series

System 10-32 Example No. 1 10-36 Example No. 2 10-36 An Example of Sensitivity Analysis 10-37

Uncertainty Bounding in Assessing Dependence 10-38

CHAPTER 11. DISPLAYS 11-1

Overview 11-1

Some Basic Assumptions and Terms 11-1

PART 1. UNANNUNICATED DISPLAYS 11-4

Major PSFs for Unannunciated Displays 11-4 Selecting an Incorrect Unannunciated Display 11-5 Reading and Recording Quantitative Information

from Displays 11-7 Meters and Digital Readouts 11-10 Chart and Printing Recorders 11-10 Graphs 11-10 Indicator Lamps 11-10 "Funneling" of-Attention in Use of Displays 11-11 Recording Errors 11-11

Arithmetic Calculations 11-11 Calculations Without Use of a Calculator 11-12 Calculations Using a Calculator 11-12 Recovery Factors for Arithmetic Calculations 11-13

Some Recovery Factors for Errors in Quantitative Readings 11-13 Recovery Factors Based on Operator Expectancy 11-13 The Use of Human Redundancy in Quantitative Readings 11-14

Check-Reading Displays 11-14 Check-Reading Meters and Chart Recorders 11-15 Digital Readouts 11-15 Check-Reading Indicator Lights 11-17

Detection of Unannunciated Deviant Displays During Periodic Scanning 11-18

A General Display Scanning Model 11-19 One Unannunciated Deviant Display - Two Operators 11-22

Number of Operators 11-23

xii

CONTENTS

Page

CHAPTER 11 (con't)

Assignment of Panel Responsibilities 11-23 Level of Dependence Between Two Operators Assigned

to Same Reactor Panels 11-23 Percentage of Time Operators Are Available To Scan

the Panels 11-23 Multiple Unannunciated Deviant Displays - One Operator 11-24 Multiple Unannunciated Deviant Displays - Two Operators 11-25 Estimated Scanning HEPs 11-28

Status Lamps and Legend Lights 11-28 Digital Readouts 11-30

Analog Displays 11-30 Multiple Unannunciated Deviant Displays Present During Scanning 11-30

Detection of Unannunciated Deviant Analog Displays During Periodic Scanning 11-31

Scanning of Meters with Limit Marks 11-31 Unavailability Calculations 11-34 Scanning of Other Types of Analog Displays 11-34

PART 2. ANNUNCIATED DISPLAYS 11-37

Types of Annunciated Displays 11-37 Major PSFs for Detecting Annunciated Signals 11-39 Errors in Reading Annunciated Legend Lights 11-39 Scanning Errors for Unannunciated Conditions of Annunciated Displays 11.-42

Responses to Annunciating Indicators -- Power-Generating Mode 11-43

One Annunciating Indicator, One Operator 11-43 One Annunciating Indicator, Two Operators 11-48 Multiple Annunciating Indicators, One Operator 11-49 Multiple Annunciating Indicators, Two Operators 11-52

Responses to Annunciating Indicators - Maintenance or Calibration Operations 1.1-52

Responses to Annunciating Indicators - Transients, Startup, and Shutdown 11-53

Responses to Annunciating Indicators - Loss-of-Coolant Accidents 11-54

PART 3. RECENT DEVELOPMENTS IN DISPLAYS 11-55

Operational Support Systems 11-55 Handling of Alarms with Logic (HALO) 11-55 Safety Parameter Display System (SPDS) 11-56 Disturbance Analysis and Surveillance Systems 11-57

Comment on Recent Display Developments 11-57

xiii

CONTENTS

Page

CHAPTER 12. DIAGNOSIS OF ABNORMAL EVENTS 12-1

Overview 12-1 Abnormal Events Considered in HRA/PRA 12-1 Some Problems in Modeling Cognitive Behavior for PRA 12-2

Treatment of Cognitive Behavior for PRA 12-2 A Comment on Errors of Judgment and Other Cognition-Related Terms 12-4

Some Problems in Estimating Diagnostic Errors 12-9 Initial-Screening Models for Operator Diagnosis and

Postdiagnosis Performance 12-12 Initial-screening Model for Diagnosis of Abnormal Events 12-12 Initial-Screening Model for Postdiagnosis Performance 12-14

Nominal Model for Diagnosis of Abnormal Events 12-16 Concluding Comments 12-22

CHAPTER 13. MANUAL CONTROLS 13-1

Overview 13-1 Types of Manual Controls 13-1 Errors in Manual Control Activation 13-2 Performance Shaping Factors 13-2 Estimated Probabilities of Errors of Commission 13-5 Recovery Factors 13-9 Recent Improvements in Control Rooms 13-10

CHAPTER 14. LOCALLY OPERATED VALVES 14-1

Overview 14-1 General Assumptions 14-1 HEPs in Valve Manipulations 14-3

Selection Errors 14-3 Reversal Errors 14-4 Stuck Valves 14-6 Other Potential Errors in Valve Manipulation 14-8

CHAPTER 15. ORAL INSTRUCTIONS AND WRITTEN PROCEDURES 15-1

Overview 15-1 Oral Instructions 15-2

Types of Oral Instructions 15-2 Oral Instruction Model 15-3

Some Problems with Written Procedures 15-7 Preparation of Written Material 15-8

Qualifications of the Writers 15-8 Accuracy of Original Written Instructions 15-9 Updating of Written Procedures in a Plant 15-10

HEPs in Use of Written Procedures 15-12

xiv

CONTENTS

Page

CHAPTER 15 (con't)

Written Procedures Models 15-12 Adjustments of Nominal 1-EPs Using Written Procedures 15-13

CHAPTER 16. MANAGEMENT AND ADMINISTRATIVE CONTROL 16-I

Overview 16-1

Administrative Control 16-1 Types of Administrative Control 16-1 Examples of Poor Administrative Control 16-2 Purpose of Administrative Control 16-4

Plant Policy 16-4 Use of Available Written Procedures 16-7 Tagging Systems 16-9 Management Systems 16-14

CHAPTER 17. STRESS 17-1

Overview 17-1 The Four Levels of Stress 17-1 Very Low Task Load 17-3 Optimum Task Load 17-6 Heavy Task Load 17-6 Threat Stress 17-9 The Data Problem for Threat Stress 17-10

The Ronan Study 17-11 The Berkun Studies 17-12 Derivation of the Estimated HEPs under Extremely High Stress on NPP Tasks 17-13

The Stress PSF and the Large LOCA 17-14 The Doubling Rule 17-16 Concluding Comments 17-17

CHAPTER 18. STAFFING AND EXPERIENCE LEVELS 18-1

Overview 18-1 Staffing 18-1 Experience Level 18-3

Licensed Reactor Operators 18-3 Other Personnel 18-4

CR Personnel Interactions in an Abnormal Situation 18-4

CHAPTER 19. RECOVERY FACTORS 19-1

Overview 19-1 Human Redundancy 19-2

Some Psychological Considerations 19-2 Estimated HEPs for Checking 19-3

CONTENTS

Page

CHAPTER 19 (con't)

Cautions in Assessing Recovery Credit for a Checker 19-7 Annunciated Indications 19-8 Active Inspections 19-9 Passive Inspections 19-10

Periodic Scanning 19-10 Basic Walk-Around Inspection 19-11 Modifications of Basic Walk-Around for More Than One

Inspection per Shift 19-20 Effectiveness of the Basic Walk-Around Inspection 19-23

PRA Treatment of Recovery 19-24

PART IV. USE OF HANDBOOK TABLES OF HUMAN ERROR PROBABILITIES FOR HUMAN RELIABILITY ANALYSIS,

AND SOME APPLICATIONS IV-1

CHAPTER 20. TABLES OF ESTIMATED HUMAN ERROR PROBABILITIES 20-1

Overview 20-1 Performance Shaping Factors 20-1 Search Scheme for Use of Chapter 20 Tables 20-2 A Talk-Through of the Search Scheme 20-6 List of Chapter 20 Data Tables 20-10 The Data Tables 20-13

CHAPTER 21. EXAMPLES AND CASE STUDIES 21-1

Overview 21-1

Problem 1: Failure To Initiate Steam Generator Feed-and-Bleed Procedures 21-1

Background and Assumptions 21-2 The Analysis. 21-3 Application of the Chapter 20 Search Scheme 21-10

Problem 2: Reanalysis of a WASH-1400 HRA 21-14 Background 21-14 The Original Analysis 21-15 A Reanalysis 21-21

Problem 3: Case Study of a LOCA Procedure 21-24 Problem 4: Case Study of a Manual Switchover to AFWS 21-26 Problem 5: The Two Status Lamps at TMI 21-29

PART V. CONCLUDING COMMENTS AND APPENDICES V-i

CHAPTER 22. CONCLUDING COMMENTS 22-1

Overview 22-1 What the Handbook Is 22-1 Need for the Handbook 22-1

xvi

CONTENTS

Page

CHAPTER 22 (con't)

Limitations in Use of the Handbook 22-2

When To Use the Handbook 22-3 Needs for Further Research To Improve HEP Estimates 22-3 Prospects for the Future of HRA/PRA 22-6

APPENDIX A. METHODS FOR PROPAGATING UNCERTAINTY BOUNDS IN A HUMAN RELIABILITY ANALYSIS AND FOR DETERMINING UNCERTAINTY BOUNDS FOR DEPENDENT HUMAN ACTIVITIES A-i

Overview A-I Development of the UCBs Propagation Method for HRA A-2 Summary of Rules To Determine UCBs on Pr[F I A-7T A Method for Determining UCBs for Conditional HEPs

Based on the Dependence Model A-10 Review of Dependence Model A-10 Qualitative Aspects of UCBs for Dependence Levels A-1i The Method A-11 An Illustration of the Method A-12

An Example Application of the UCBs Propagation Method A-12 Comparison of the UCBs Propagation Method and Propagation Using a Monte Carlo Procedure A-21

APPENDIX B. AN ALTERNATIVE METHOD FOR ESTIMATING THE

EFFECTS OF DEPENDENCE B-i

Introduction B-i Probability Model B-2 Lognormal Model B-5

APPENDIX C. CALCULATIONS OF MEAN AND MEDIAN NUMBERS OF TRIALS TO DETECTION GIVEN IN TABLE 11-9 C-i

APPENDIX D. CALCULATIONS FOR BASIC WALK-AROUND INSPECTIONS AS A FUNCTION OF PERIOD BETWEEN SUCCESSIVE WALK-AROUNDS D-I

APPENDIX E. REVIEWS OF THE DRAFT HANDBOOK E-i

Overview E-1 Three Categories of Review E-i Handbook Exercises Project E-7 The UKAEA Review E-7 Informal Review E-8

APPENDIX F. A COMPARISON OF THE OCTOBER 1980 AND PRESENT VERSIONS OF THE HANDBOOK F-i

Overview F-I

xvii

CONTENTS

Page

APPENDIX F (con't)

General Comparison F-i Part I. Basic Concepts F-i

Chapter 1. Introduction F-i Chapter 2. Explanation of Some Basic Terms F-i Chapter 3. Some Performance Shaping Factors Affecting

Human Reliability F-i Part II. Methods for Analysis and Quantification of

Human Performances F-4 Chapter 4. Man-Machine Systems Analysis F-4 Chapter 5. A Technique for Human Reliability Analysis F-4 Chapter 6. Sources of Human Performance Estimates F-4 Chapter 7. Distribution of Human Performance and Uncertainty Bounds F-4

Chapter 8. Use of Expert Opinion in Probabilistic Risk Assessment F-5

Chapter 9. Unavailability F-5 Part III. Human Performance Models and Estimated

Human Error Probabilities F-5 Chapter 10. Dependence F-6 Chapter 11. Displays F-6 Chapter 12. Diagnosis of Abnormal Events F-6 Chapter 13. Manual Controls F-7 Chapter 14. Locally Operated Valves F-7 Chapter 15. Oral Instructions and Written Procedures F-7 Chapter 16. Management and Administrative Control F-7 Chapter 17. Stress F-7 Chapter 18. Staffing and Experience Levels F-7 Chapter 19. Recovery Factors F-8

Part IV. Use of Handbook Tables of Human Error Probabilities for Human Reliability Analysis, and Some Applications F-8

Chapter 20. Tables of Estimated Human Error Probabilities F-8 Chapter 21. Examples and Case Studies F-S

Part V. Concluding Comments and Appendices F-8 Chapter 22. Concluding Comments F-8 Appendix A. Methods for Propagating Uncertainty Bounds in

a Human Reliability Analysis and for Determining Uncertainty Bounds for Dependent Human Activities F-9

Appendix B. An Alternative Method for Estimating the Effects of Dependence F-9

Appendix C. Calculations of Mean and Median Numbers of Trials to Detection Shown in Table 11-9 F-9

Appendix D. Calculations for Basic Walk-Around Pr[F]s as a Function of Period Between Successive Walk-Arounds F-9

Appendix E. Reviews of the Draft Handbook F-9 Appendix F. A Comparison of the October 1980 and Present Versions of Handbook F-9

Appendix G. General Human Error Rates from WASH-1400 F-9

xviii

CONTENTS

APPENDIX F (con' t)

Page

Part V. References, Equations, Glossary, and Abbreviations F-10 References F-10 Equations F-10 Glossary F-10 Abbreviations F-10

APPENDIX G. GENERAL HUMAN ERROR RATES FROM WASH-1400 G-1

REFERENCES H-i

EQUATIONS I-I

GLOSSARY J-1

ABBREVIATIONS K-i

xix

LIST OF FIGURES

Figure Title Page

3-1 A simplified model of the human component in a man-machine system for the purpose of PRA 3-4

3-2 Relationship of external and internal PSFs 3-9 3-3 Steps from columnar style format for NPP

written procedures 3-31 3-4 Hypothetical relationship of psychological stress

and performance effectiveness 3-36 3-5 Vigilance effect for passive tasks with low

signal rate 3-39

3-6 Hypothetical effects of practice and no practice on maintenance of emergency skills 3-45

3-7 A 23-minute link analysis of reactor engineer

activities in a Swedish BWR 3-50 3-8 An 8-hour link analysis of reactor engineer

activities in a Swedish BWR 3-51 3-9 MOV switches on part of the ESF panel at the PWR

used in the WASH-1400 Study 3-53 3-10 Part of the lower ESF panel in a PWR 3-54

3-11 Location aids for lower ESF panel in a PWR 3-55 3-12 Location aids for upper panel in a PWR 3-56 3-13 Location aids for annunciator panel #1 3-57 3-14 Violation of reading stereotype in labeling of

monitor panels 3-59 3-15 Two adjacent displays with conflicting

representations of the real world 3-60

3-16 Reactor control switches for the four primary coolant loops 3-61

4-1 A job-task analysis format used for military system studies 4-7

4-2 A simplified model of the human component in a man-machine system for the purpose of

probabilistic risk assessment (Note: This is a repeat of Figure 3-1) 4-10

5-1 HRA event tree for series or parallel system 5-6 5-2 Simplified event trees for a large LOCA 5-11 5-3 HRA event tree for loss of steam generator feed 5-13 5-4 Four phases of human reliability analysis 5-20 5-5 Two options for incorporation of HRA into PRA 5-23 5-6 Outline of a THERP procedure for HRA, 5-24 7-1 Hypothesized lognormal probability density

function of HEPs for nuclear power plant personnel 7-7

7-2 HRA event tree for series or parallel system (This is a repeat of Figure 5-1) 7-16

9-1 Time line of equipment life to illustrate

unavailability 9-5 10-1 HRA event tree of hypothetical calibration tasks 10-7

LIST OF FIGURES

Figure Title Page

10-2 HRA event tree of hypothetical calibration tasks (small and large miscalibrations) 10-9

10-3 Continuum of positive dependence represented by five discrete points 10-11

10-4 HRA event tree showing complete-success path, and complete-failure path 10-13

10-5 Direction of operator action relationships

along the positive dependence continuum 10-21 10-6 A parallel-series system with potential failures

of equipment (E,, E2 , and E ) or of human tasks (A, B, or C) 10-35

11-1 Hypothetical curve representing detection effectiveness at initial audit and at hourly intervals thereafter, ending with last scan at beginning of the eighth hour 11-21

11-2 Initiation of action in response to annunciators in control room, given one operator, only one annunciating indicator, and power-generating mode 11-47

12-1 A simplified model of the human component in a man-machine system for the purpose of probabilistic risk assessment (Note: This is a repeat of Figure 3-1) 12-3

12-2 Cognition wheel 12-7 12-3 Initial-screening model of estimated HEPs and

UCBs for diagnosis within time T of one abnormal event by control room personnel 12-13

12-4 Nominal model of estimated HEPs and UCBs for diagnosis within time T of one abnormal event by control room personnel 12-20

15-1 Examples of narrative and columnar formats 15-17 15-2 An example of a less-than-average-quality written

procedure 15-19 17-1 Hypothetical relationship between performance and

stress (based on Figure III 6-1 from WASH-1400) with task stress and threat stress division 17-2

17-2 Estimated human performance after a large LOCA 17-15 19-1 Curve of retention as determined by the Recognition

Method 19-13 19-2 Percent recovery, r, of detection effectiveness as

a function of the number of days between basic walk-around inspections by the same person 19-19

20-1 Search scheme for use of Chapter 20 tables 20-3 20-2 Quick reference guide to Chapter 20 tables 20-14 21-1 Hypothetical plant procedures related to Problem 1 21-4 21-2 HRA event tree for Problem 1 21-5

xxi

LIST OF FIGURES

Figure Title Page W

21-3 MOV switches on part of the ESF panel at the PWR

used in the WASH-1400 study (Note: This is a repeat of Figure 3-9. 21-16

21-4 HRA event tree for Step 4.8.1 in LOCA procedure 21-19 21-5 Changes to HRA event tree in Figure 21-4 to

reflect changes in Handbook models and HEPs 21-22 A-i An HRA event tree with three complete-failure paths. A-3 A-2 Expanded HRA event tree for loss of steam generator

feed. (Note: This tree is an expansion of Figure 5-3.) A-15

A~iJ.

xx•

LIST OF TABLES

Table Title Page

3-1 Categories of incorrect human outputs related to human reliability analysis 3-7

3-2 Some PSFs in man-machine systems 3-9 3-3 Some task and equipment characteristics 3-23 3-4 Some psychological stressors 3-.34 3-5 Some physiological stressors 3-41 3-6 Some individual (organismic) factors 3-43 3-7 Most important PSF classes related to system safety 3-69 3-8 Estimated decreases in HEPs resulting from

application of good ergonomics practices to nuclear power plants 3-72

4-1 Steps in Man-Machine Systems Analysis (MMSA) 4-3 4-2 A checklist for evaluating error-likeliness

during observation or performance of tasks 4-13 4-3 Sample scales for profile of importance of

accident-prone situations 4-20 4-4 Consequence categories 4-21 4-5 System criteria for trade-off considerations 4-23 5-1 Symbology used in THERP method of HRA 5-7 7-1 Range ratio in production operations 7-3 7-2 General guidelines for estimating uncertainty

bounds for estimated HEPs 7-13 7-3 Approximate CHEPs and their UCBs for dependence

levels given FAILURE on the preceding task 7-15 8-1 Rank orderings of four judgment procedures on

three evaluation criteria 8-11 10-1 General guidelines in assessing level of dependence 10-23 10-2 Equations for conditional probabilities of

success and failure on Task "N," given success or failure on previous Task "N-i," for different levels of dependence 10-27

10-3 Conditional probabilities of success or failure for Task "N" for the five levels of dependence, given FAILURE on preceding Task "N-i" 10-31

10-4 Conditional probabilities of success or failure for Task "N" for the five levels of dependence, given SUCCESS on preceding Task "N-i" 10-33

11-1 Human factors engineering deficiencies of displays

in nuclear power plants 11-6 11-2 Estimated probabilities of errors in selecting

unannunciated displays for quantitative or qualitative readings 11-8

11-3 Estimated HEPs for errors of commission in reading and recording quantitative information from unannunciated displays 11-9

xxiii

LIST OF TABLES

Table Title Page

11-4 Estimated HEPs for errors of commission in check-reading displays 11-16

11-5 Estimated probabilities of detecting at least one of one to five unannunciated deviant displays, as a function of the BHEP for detection of a single deviant display during periodic scanning 11-26

11-6 Estimated probabilities of failing to detect at least one of one to five unannunciated deviant displays as a function of the BHEP for detection of a single deviant display during periodic scanning 11-27

11-7 Estimated probabilities of failure to detect one (of one) unannunciated deviant display at each scan, when scanned hourly 11-29

11-8 Estimated per scan and cumulative probabilities of detection for each hourly scan for one of one unannunciated deviant meter with limit marks 11-32

11-9 Mean and median numbers of trials to detection for unannunciated meters, with limit marks, that

become deviant prior to any given scan, T. 11-35 11-10 Estimated per scan probabilities of detection for

each hourly scan for one (or one completely dependent set) of one unannunciated deviant display 11-36

11-11 Human engineering deficiencies in annunciator warning systems 11-40

11-12 Estimated HEPs for annunciated legend lights 11-44 11-13 The Annunciator Response Model: estimated HEPs

for multiple annunciators alarming closely in time 11-51 12-1 Definitions of cognition-related terms and usage

in the Handbook 12-6 12-2 Initial-screening model of estimated HEPs and EFs

for diagnosis within time T by control room personnel of abnormal events annunciated closely in time 12-15

12-3 Initial-screening model of estimated HEPs and EFs for rule-based actions by control room personnel after diagnosis of an abnormal event 12-17

12-4 Nominal model of estimated HEPs and EFs for diagnosis within time T by control room personnel of abnormal events annunciated closely in time 12-19

12-5 Guidelines for adjusting nominal diagnosis HEPs from Table 12-4 12-23

13-1 PSFs related to controls 13-3 13-2 Deficiencies in layouts of manual controls

observed at NPPs 13-4 13-3 Estimated probabilities of errors of commission

in operating manual controls 13-6 14-1 Estimated HEPs for selection errors for locally

operated valves 14-5

xxiv

LIST OF TABLES

Table Title Page

14-2 Estimated HEPs in detecting stuck locally

operated valves 14-7 15-1 Estimated probabilities of errors in recalling

oral instruction items not written down 15-5 15-2 Estimated HEP per item (or perceptual unit) in

preparation of written material 15-11

15-3 Estimated probabilities of errors of omission per item of instruction when use of written procedures is specified 15-14

16-1 Estimated HEPs related to failure of administrative control 16-5

16-2 The four levels of tagging or locking systems 16-11 17-1 Modifications of estimated HEPs for the effects

of stress on skilled personnel 17-4 18-1 Modifications of estimated HEPs for the effects

of stress and experience levels 18-5 18-2 Number of reactor operators and advisors available

to cope with an abnormal event and their related

levels of dependence: assumptions for PRA 18-8 19-1 Estimated probabilities that a checker will fail

to detect errors made by others 19-4 19-2 Probabilities of successful and unsuccessful

recognition of the same deviant item during basic walk-around inspections performed on successive

calendar days by the same person 19-15 19-3 Estimated probability of detecting any particular

deviant condition within 30 days for various numbers of days between basic walk-around inspections 19-21

19-4 Estimated probabilities that the basic walk-around inspection will fail to detect a particular

deviant indication of equipment outside the control room within 30 days 19-22

20-1 Initial-screening model of estimated HEPs and EFs for

diagnosis within time T by control room personnel of abnormal events annunciated closed in time (from Table 12-2) 20-17

20-2 Initial-screening model of estimated HEPs and EFs for rule-based actions by control room personnel after diagnosis of an abnormal event (from Table 12-3) 20-18

20-3 Nominal model of estimated HEPs and EFs for diagnosis within time T by control room personnel of abnormal events annunciated closely in time (from Table 12-4) 20-19

20-4 Number of reactor operators and advisors available to cope with an abnormal event and their related levels of dependence: assumptions for PRA (from Table 18-2) 20-20

xxv

LIST OF TABLES

Table Title Page

20-5 Estimated HEP per item (or perceptual unit) in preparation of written material (from Table 15-2) 20-21

20-6 Estimated HEPs related to failure of administrative control (from Table 16-1) 20-22

20-7 Estimated probabilities of errors of omission per item of instruction when use of written procedures is specified (from Table 15-3) 20-23

20-8 Estimated probabilities of errors in recalling oral instruction items not written down (from Table 15-1) 20-24

20-9 Estimated probabilities of errors in selecting unannunciated displays for quantitative or qualitative readings (from Table 11-2) 20-25

20-10 Estimated HEPs for errors of commission in reading and recording quantitative information from unannunciated displays (from Table 11-3) 20-26

20-11 Estimated HEPs for errors of commission in check-reading displays (from Table 11-4) 20-27

20-12 Estimated probabilities of errors of commission in operating manual controls (from Table 13-3) 20-28

20-13 Estimated HEPs for selection errors for locally operated valves (from Table 14-1) 20-29

20-14 Estimated HEPs in detecting stuck locally operated valves (from Table 14-2) 20-30

20-15 The four levels of tagging or locking systems (from Table 16-2) 20-31

20-16 Modifications of estimated HEPs for the effects of stress and experience levels (from Table 18-1).. 20-32

20-17 Equations for conditional probabilities of success and failure on Task "N," given success or failure on previous Task "N-i," for different levels of dependence (from Table 10-2) 20-33

20-18 Conditional probabilities of success or failure for Task "N" for the five levels of dependence, given FAILURE on preceding Task "N-i" (from Table 10-3) 20-34

20-19 Conditional probabilities of success or failure for Task "N" for the five levels of dependence, given SUCCESS on preceding Task "N-i" (from Table 10-4) 20-35

20-20 General guidelines for estimating uncertainty bounds for estimated HEPs (from Table 7-2) 20-36

20-21 Approximate CHEPs and their UCBs for dependence levels given FAILURE on the preceding task (from Table 7-3) 20-37

20-22 Estimated probabilities that a checker will fail to detect errors made by others (from Table 19-1) 20-38

xxvi

LIST OF TABLES

Table Title Page

20-23 The Annunciator Response Model: estimated HEPs for multiple annunciators alarming closely in time (from Table 11-13) 20-39

20-24 Estimated HEPs for annunciated legend lights

(from Table 11-12) 20-40 20-25 Estimated probabilities of failure to detect one

(of one) unannunciated deviant display at each scan, when scanned hourly (from Table 11-7) 20-41

20-26 Estimated probabilities of failing to detect at least one of one to five unannunciated deviant displays as a function of the BHEP for detection of a single deviant display during periodic scanning (from Table 11-6) 20-42

20-27 Estimated probabilities that the basic walk-

around inspection will fail to detect a particular deviant indication of equipment outside the control room within 30 days (from Table 19-4) 20-43

21-1 Explanation of terms in Figure 21-2 21-6 21-2 Estimated probabilities of failure to initiate AFWS 21-28 A-I Examples of UCBs for dependence levels determined

by the method in this appendix A-13 A-2 Explanation of failure limbs in Figure A-2 A-16 A-3 Terms for calculating UCBs on PR[F T A-18• T D-1 Estimated values for use in calculating Pr[Sis

in Table 19-3 D-3 E-i Participants in "Handbook Exercises Project" E-2 E-2 Reviewers of the Draft Handbook in a study by the

UKAEA Safety and Reliability Directorate E-4 E-3 List of informal reviewers of draft Handbook E-5 F-i Correlation of chapters from draft NUREG/CR-1278

with those of the present version of this document F-2 F-2 Cross index of old and new Chapter 20 table numbers F-3 G-I General error rate estimates from Table III 6-i in

WASH-1400 G-2

xxvii

Part I. Basic Concepts

PART I. BASIC CONCEPTS

NOTE: This Handbook replaces a draft version (Swain and Guttmann, 1980) that received considerable tech- nical review. Appendix F summarizes of the dif- ferences between the draft and the present version.

This handbook of human reliability analysis (HRA) is oriented toward probabalistic risk assessments (PRAs) of nuclear power plants. Since both HRA and PRA are relatively new technologies, it is necessary to introduce some basic concepts, the subject matter of the first part of the Handbook. Part I consists of three chapters. Chapter 1 describes the purpose, scope, and organization of the Handbook. Chapter 2 defines many of the relatively new terms useful for HRA and PRA (a glossary defines all of the technical terms). Chapter 3 presents a general model of human performance for HRA/PRA, which serves as the background for the more specific models elsewhere in the Handbook. This chapter also discusses many of the factors that influence human performance in nuclear power plants and similar complex man-machine systems.

I-I

Chapter 1. Introduction Purpose of the Handbook

CHAPTER 1. INTRODUCTION

Purpose of Handbook

The primary purpose of the Handbook is to present methods, models, and estimated human error probabilities (HEPs)* to enable qualified analysts to make quantitative or qualitative assessments of occurrences of human errors in nuclear power plants (NPPs) that affect the availability or operational reliability of engineered safety features and components. The Handbook is intended to provide much of the modeling and information necessary for the performance of human reliability analysis (HRA) as a part of probabilistic risk assessment (PRA) of NPPs, as described in Chapter 4, "Human Reliabil- ity Analysis," of NUREG/CR-2300.

Although not a design guide, a second purpose of the Handbook is to enable the user to recognize error-likely equipment design, plant policies and practices, written procedures, and other human factors problems so that improvements can be considered. Many studies have indicated that in com- plex man-machine systems, human error has often been the overriding contri- butor to actual or potential system failures (e.g., Shapero et al, 1960; Meister, 1962; and Meister and Rabideau, 1965). Analyses of NPP operations indicate that NPPs are not exceptions to this general finding (WASH-1400; Rasmussen, 1976; and Rasmussen and Taylor, 1976). Finally, incidents such as those at Brown's Ferry and Three Mile Island (TMI) clearly show that humans have acted not only as accident initiators and accident propagators, but also as accident mitigators in NPPs.

The Handbook provides the methodology to identify and quantify the poten- tial for human error in NPP tasks. When used in association with design guides and related documents such as NUREG-0700, NUREG-0801, MIL-STD-1472C, Van Cott and Kinkade (1972), and Woodson (1981), the Handbook can provide valuable guidance in assessing the relative merits of different design features and operating methods.

Terms defined in the glossary are underlined the first time they appear in each chapter. All abbreviations are spelled out in the abbreviations section at the end of the Handbook and also the first time the abbreviation is used in each chapter.

1-1

Scope

It is our intent that this Handbook assist utilities in evaluating the role of operating personnel in currently operating power plants, help enable designers of future plants to avoid major human reliability problems, and provide a quantitative base for the assessment of human errors in NPP safety and productivity.

While it is our intention that the Handbook apply to both operating and future NPPs, we have emphasized HRA for the former. There are two major reasons for this emphasis. First, the human reliability problems in cur- rent operating plants are real and need to be addressed now to reduce further the already low risk to the public from these plants. Several PRAs of operating plants have been done, and most have used the draft Handbook for solving HRA problems. Second, some of the display and control concepts being considered for future plants are so new that insufficient information exists to develop quantitative indices of human performance. With the exception of these new concepts, the estimated HEPs and human performance models in the Handbook do apply to both operating and future plants. For this Handbook to be complete in its coverage of future plant designs, additions and revisions will be made when data and information become available.

The Handbook in its present form provides data, modeling, techniques, and a practical procedure that will enable qualified analysts to perform HRAs either for PRA purposes or to identify human factors engineering problems that are serious enough to warrant changes in equipment or procedures. The HRAs can be performed for operating plants or for plants in any stage of design or development.

Discussion in subsequent chapters will note that the above analyses are easier to accomplish in operating plants. This is because much of the information required for HRA is best obtained by interviews with plant personnel and demonstrations of selected operating procedures or of postu- lated abnormal events. However, the procedures in the Handbook can also be applied in estimating and quantifying human error problems in the design stages of future plants. Chapters 4 and 5 note some of the problems in this kind of application.

Although the Handbook is oriented toward engineered safety features, the models, procedures, and estimated HEPs are relevant to all aspects of NPP design and operation where there is an interaction of people with plant systems and equipment. Most of the material in this Handbook is also applicable to, and has been used in evaluating, human reliability in other large process plants, e.g., chemical plants, oil refineries, offshore oil production, and other power-generating plants.

1-2

Handbook and WASH-1400

Limitations of the Handbook

Relationship of Handbook to WASH-1400

Sandia National Laboratories (SNL) personnel were involved in the reliabil- ity analyses performed in WASH-1400. (The authors of this Handbook were the human reliability analysts for that study.) The human reliability assessments appear in various volumes of WASH-1400. Section 6.1, "Human Reliability Analysis," in Appendix III, Failure Data, describes in general terms how the estimates of HEPs for various system safety tasks were derived and incorporated into the system fault trees.

Since WASH-1400 presents only summaries of the human error analyses, it is sometimes difficult for readers to understand how the various HEPs were developed.* To utilize human reliability principles more fully in plant design and operations, more information is needed than that given in WASH-1400. Particularly, information is needed that can be applied to specific problems in NPPs. In this Handbook, we define the concepts involved, the data employed, and the calculations used in applying human error analyses to system evaluations of reliability and availability in NPPs. The methodology and estimated HEPs presented apply to NPPs in general, not only to light water reactors, the subject matter of WASH-1400.

Limitations of the Handbook

The state of the art in HRA is barely beyond its infancy. Until recently, many system analysts did not attempt to quantify the effects of human performance in system reliability or safety studies. Even today, some system reliability and system safety analyses omit human error analyses, or they make unrealistic assumptions about the probability of human error. Neither of these approaches is satisfactory since either can lead to erro- neous and possibly dangerous conclusions in PRA studies. Experience in military, space, and commercial man-machine systems indicates that the human has a major role in both accident propagation and mitigation. Des- pite limitations in the coverage and accuracy of human performance esti- mates, use of the models and estimated HEPs from this Handbook can lead to realistic risk assessments and reliability analysis in general.

Human performance is difficult to predict because of its variability. Each reactor operator in an NPP differs from all other reactor operators and will frequently show remarkable variability in his own behavior from day to day and from moment to moment. The human performs more different func- tions, in more different ways, under more different conditions than any other single element in a system. He has more interfaces, he accepts a greater variety of inputs, he provides a greater variety of outputs, and the possible relationships between his inputs and outputs are even more varied.

We include Table III 6-1, "General Human Error Rate Estimates," from WASH-1400, as Appendix G, Table G-1.

1-3

Limitations of the Handbook

Despite this variability, it is possible to predict, with varying degrees of uncertainty, the reliability of a human involved in a task for which he is adequately trained. The uncertainty will be smallest when predicting behavior in the performance of routine tasks such as test, maintenance, calibration, and normal control room operations and will be largest when predicting behavior in response to an abnormal event. While large degrees of uncertainty are not des ,irable, considerable imprecision in HRA is toler- able for many purposes of PRA. Further, there are techniques described in the Handbook that allow an analyst to assess the influence of variations in assumptions and estimates of human performance on system reliability and safety. While the user of the Handbook cannot expect his estimates of HEPs and other aspects of human performance to be precise, he should not be discouraged.

The reader must also understand that if he is inexperienced in analyzing human performance, his estimates could have greater uncertainties than those stated in the Handbook. Following are some mistakes we have observed in hRAs performed by persons not experienced in human performance analysis:

(1) To ignore the various types of interrelationships that exist among operators (including supervisors), between operators and equipment, and between operators and operating conditions, including the various formal and informal feedback paths that modify human behavior.

(2) To assume that everyone in the control room following the occurrence of an abnormal event will be involved in all tasks, even the most detailed of tasks.

(3) To assume that people will always do what they are told to do (either by oral instructions, by written procedures, or by plant policy).

(4) To overestimate the probability that a person checking the performance of another will detect errors made by the person being checked.

If any of the above mistakes are made, the analyst's estimates of HEPs are likely to be too optimistic, and he will ignore certain behaviors that could have serious effects on the system. if the user is aware of the difficulty of estimating failure probabilities of equipment but believes that human behavior is easier to understand and predict, he, too, will be subject to unjustified optimism.

The more the reader knows about human behavior in systems, especially in nuclear power systems, the more accurate his identification of human events and his estimation of HEPs are likely to be. There is no substitute for experience in any endeavor, especially in one with as large an element of subjectivity as HRA. Therefore, teams of experts that include qualified human factors personnel will be at advantage.

Along this line, some comments by reviewers of the draft Handbook are relevant. These comments are taken from Miller (1983). One reviewer, an authority in PRA, stated that the estimated HEPs in the Handbook should be

1-4

Limitations of the Handbook

used only by human factors specialists. This PRA expert (not a human factors specialist) doubted that persons outside the human factors field would be able to make the proper evaluation of the role of performance shaping factors. Another reviewer, a human factors specialist, was not so pessimistic. He stated, "The Handbook could certainly be used by persons who do not have a human factors background, particularly since it contains somewhat of a primer of human factors basics, and references to other human factors engineering handbooks. Given the complexity of the subject, how- ever, users are likely to vary considerably in their ability to perform the analyses properly. The consistency and reliability of the Handbook users would certainly be improved if they participated in some human factors training, preferably emphasizing the use of the Handbook itself (this would probably be beneficial to most users with previous human factors training as well)." Another reviewer, also a human factors specialist, stated, "In my view the major problem in human reliability analysis is not so much the quantification, but the modeling process against which the quantification is then applied. This requires practical industrial exprience as well as analytical ability. It also requires intellectual determination and patience. My feeling, therefore, is that uninformed use could be most dangerous." Another reviewer, a foreign human factors specialist said, "The Handbook is like an instrument, a 'vioiin.' It looks simple, but it requires skills and arts to use it." He also called out the need for training and exercises for using the Handbook. To this end, the U.S. Nuclear Regulatory Commission (NRC) has set up an HRA course as a part of a curriculum in PRA for its personnel. The HRA course includes instruction in the use of the Handbook and its companion document, NUREG/CR-2254. Similar courses are offered outside the NRC. The need for special training for those who are to perform the HRA portions of a PRA may be considered a limitation by some. However, HRA deals with analyzing and quantifying human performance, and the requirement for special training in this area is not unreasonable.

Another limitation of the Handbook is that models and estimated HEPs have not been developed for all NPP tasks. For example, we have studied refuel- ing, plant security, plant evacuation, emergency communications, and plant chemistry, but they have not been addressed directly in the Handbook. However, there are HEPs in the Handbook related to potential errors in performing these operations, e.g., errors in preparation and use of written procedures, to the carrying out of oral instructions, and to errors in arithmetic calculations. Further, the Handbook does not provide estimated HEPs related to the use of new display and control technology that is computer-based. Neither does the Handbook provide HEPs for corrective maintenance such as repairing a pump. There are three.reasons for not considering this type of maintenance:

(1) The HEPs for these actions are usually included in other parts of a PRA, and one would not want to count human errors twice.

(2) After repair, the equipment usually is tested, and most repair errors are detected.

1-5

Organization of the Handbook

(3) NRC-sponsored research by Oak Ridge National Laboratories and Applied Psychological Services (Siegel et al, 1983) is specifically addressed

to the pr oblem of estimating HEPs for maintenance tasks.

The Handbook addresses the kinds of tasks that have been studied in PRAs of NPPs to date. These are divided into tasks performed during normal oper- ating conditions and tasks to be performed after an abnormal event has occurred. For normal operating conditions, the tasks addressed by the Handbook are calibration tasks, tests performed after calibration, mainte- nance, or other operations, and the restoration of important safety compo- nents to their available states after carrying out tests, calibration, or maintenance. These tasks are all related to the availability of ESFs should an abnormal event occur. For abnormal conditions, the Handbook provides information to estimate HEPs for tasks to be performed to cope with abnormal events, many of which have not occurred in commercial plant operation. The tasks involve the recognition of patterns of signals in the control room so that the personnel will know what actions to take at vari- ous stages during an event, the carrying out of the required actions, and the avoidance of extraneous actions that could reduce control of the abnormal situation.

In general, our HEPs and models are based on studies and observations in the kinds of plants we have visited, which are listed in the Foreword. Some newer plants may incorporate human factors improvements that could make some of our estimates too pessimistic.

Despite the limitations, the user will be able to apply much of the material to tasks not specifically considered in this Handbook. This application is possible because there can be considerable similarity in the human factors aspects of different plants despite differences in equipment and other engineering aspects.

The scarcity of objective and quantitative data on human performance in NPPs is a serious limitation. most of the hEPs in this Handbook are what we call derived data. In some cases, they are extrapolations from per- formance measures, which may be only marginally related. In other cases the HEPs represent our best judgment based on our experience in complex systems and on our background in experimental and engineering psychology. The necessity to rely so heavily on judgment is a regrettable state of affairs, but a start needs to be made, and this Handbook is a first step toward what is really needed--a large data bank of human performance infor- mation directly related to NPP tasks.

A final point, which some may consider a limitation, is that the Handbook does not deal with malevolent behavior. This is a Handbook about human errors made by people who intend to do the correct thing but sometimes fail in this intent. Malevolent behavior is not due to error: it is deliberate behavior calculated to produce a harmful effect.

Organization of the Handbook

The Handbook is divided into six major parts plus prefatory material. The detailed table of contents serves as an index. Part I, "Basic Concepts,"

1-6

Organization of the Handbook

consists of this chapter and two others. Chapter 2 presents some basic definitions of terms, including a categorization of types of errors that one can expect in NPPs, fossil fuel power plants, or any man-machine sys- tem. Chapter 3 presents some philosophy and guidelines for viewing the human in a system context, including a discussion of factors that influence human performance in a system. Chapter 3 also lists principles of good human factors design, along with NPP examples of conformance with and deviation from these principles and their implications for quantification of human performance.

Part II, "Methods for Analysis and Quantification of Human Performance," consists of six chapters (4 through 9). Chapter 4 presents the analytical methods to identify the tasks and task elements to be used in HRA. Chapter 5 presents our method for HRA, including HRA event trees, and illustrates the mechanics of incorporating estimated HEPs into HRA event trees. The relationship of HRA to PRA is discussed. Chapter 6 discusses the sources of the HEPs and models in the Handbook. Chapter 7 presents the assumptions about size and shape of the distributions of HEPs in the Handbook and includes a discussion of uncertainty bounds and'sensitivity analysis. Chapter 8 explains the use of expert judgment in lieu of data on human performance and describes acceptable methods for using expert judgment. Finally, Chapter 9 briefly describes the use of HEPs to estimate the una- vailability of systems and components due to human error.

Part Ill, "Human Performance Models and Estimated Human Error Probabili- ties," consists of Chapters 10 through 19, which present models developed from available experimental literature, interviews with and obervations of NPP personnel in the U.S., Europe, Japan, and South Africa, and the experi- ence of the authors. The human performance modelsrepresent human activi- ties that are important for the detection of and response to abnormal conditions and for ensuring that important safety systems will be available should an abnormal event occur. In several of the models, performance is estimated as a function of time. The models are presented as mathematical statements, with uncertainty bounds when appropriate. These models involve considerable extrapolation from available data and experience and are regarded as hypotheses that must be subjected to rigorous testing in labo- ratory and plant settings.

Part IV, "Use of Handbook Tables of Human Error Probabilities for Human Reliability Analysis, and Some Applications," consists of two chapters. Chapter 20 consolidates the HEPs from the preceding chapters for convenient reference and provides a search scheme to help the analyst find the appro- priate HEPs in an HRA. Chapter 21 presents some case studies to illustrate task analysis, the qualitative application of HRA, and the application of the human performance models, the HEPs, and the human reliability technique to NPPs.

Part V, "Concluding Comments and Appendices," consists of Chapter 22 and the appendices. Chapter 22 presents concluding comments, includihg an assessment of current HRA techniques and apparent trends. There are seven appendices (A through G) to the Handbook, which include a list of the

1-7

Related Documents

Eight NRC/SNL documents are directly related to the Handbook and serve as supportive material. These are described below.

NUREG/CR-2254, "A Procedure for Conducting a Human Reliability Analysis for Nuclear Power Plants" (Bell and Swain, 1983), serves as a companion docu- ment to the Handbook. NUREG/CR-2254 shows how the methodology and data in the Handbook are used in performing an HRA as part of a PRA. An appendix provides several examples that can be used as a training aid for those who wish to perform a systematic HRA.

Two volumes of NUREG/CR-2744, "Human Reliability Data Bank for Nuclear Power Plant Operations," provide a discussion of the important data problem. Volume 1, "A Review of Existing Human Reliability Data Banks" (Topmiller et al, 1982), describes the current, inadequate state of human reliability data and includes reprints of three old banks of derived human performance data that are still useful. Volume 2, "A Data Bank Concept and System Description" (Comer et al, 1983), presents a task taxonomy that is designed especially for HRA purposes and proposes an approach to the set- ting up of a data bank to collect the kind of data useful for HRA.

Two reports deal with the use of psychological scaling, i.e., the use of expert judgment to derive estimates of HEPs. NUREG/CR-2255, "Expert Esti- mation of Human Error Probabilities in Nuclear Power Plant Operations: A Review of Probability Assessment and Scaling" (Stillwell et al, 1982), describes the various methods that have been employed for this purpose and their strong and weak points. NUREG/CR-2743, "Procedures for Using Expert Judgment to Estimate Human Error Probabilities in Nuclear Power Plant Operations" (Seaver and Stillwell, 1983), presents the best procedures to use for this purpose.

NUREG/CR-3309, "A Simulator-Based Study of Human Errors in Nuclear Power Plant Control Room Tasks" (Beare et al, 1983), presents the first empirical data on HEPs that were obtained specifically to provide comparisons with the Handbook estimates. The areas of agreement and disagreement with the Handbook estimates are described.

Finally, two SNL reports present information on reviews of the Handbook. SAND82-7056, "Peer Review Study of the Draft Handbook for Human Reliability Analysis With Emphasis on Nuclear Power Plant Applications, NUREG/CR-1278" (Brune et al, 1983), presents the results of a formal tryout of the draft Handbook by 29 selected experts in PRA and/or human factors. SAND82-1196,

1-8

How To Use This Handbook

"Peer Review Comments on NUREG/CR-1278" (Miller, 1983), summarizes the comments in the above study by Brune et al and also summarizes comments from over 60 reviewers of the draft Handbook. Both reports provided impor- tant guidance for the revisions to the draft Handbook that constitute the present document.

How To Use This Handbook

In writing this Handbook, it was not possible to follow a step-by-step sequence from the first to the last page. The subject matter is complex and the procedure highly iterative. We suggest that the user read through the entire volume, paying particular attention to the mechanics of HRA described in Part II and illustrated in examples throughout the Handbook. The user should work out some of these examples for himself because he must develop skill in HRA event tree diagramming, especially in its representa- tion of the conditional probabilities of events, to perform a satisfactory HRA. Users of the Handbook who plan to conduct HRAs will read the compan- ion document, NUREG/CR-2254, work the examples in that report, and attend a course in HRA, along with courses in elementary probability theory.

The examples of how human behavior and performance are estimated under various situations in NPPs constitute a "scenario-oriented" approach. We hope that the user can develop a "feel" for how humans behave, since pre- cise approaches do not as yet exist for modeling human behavior in all its complexities and with all its interactions. The Handbook presents basic principles, guidelines, a reasonable amount of modeling, a set of human performance data, and numerous examples to assist the user in performing an HRA. In this sense, this document is not like the usual handbook in which one can look up some set of data and apply it directly to a problem. Because of the diverse backgrounds of those interested in human reliability in NPPs, much of the Handbook is tutorial. Some of the human factors information will seem elementary to those with a background in human fac- tors technology, and some of the information on reliability will seem elementary to those with a background in that area. We have tried to integrate the information so that practitioners of both disciplines will have sufficient guidance to function as part of a team of human reliability analysts.

Once the user has developed a facility for HRA event tree diagramming and understands the limitations and rationale for the estimated HEPs in the Handbook, he should find that the summary tables and search scheme in Chapter 20 will be sufficient for solving most HRA problems for NPP operations.

1-9

Chapter 2. Explanation of Some Basic Terms

CHAPTER 2. EXPLANATION OF SOME BASIC TERMS

Overview

Although the glossary defines all the specialized terms used in the Handbook, this chapter elaborates on some that require additional dis- cussion and presents the point of view taken by the authors in assessing the reliability of human performance in nuclear power plant (NPP) opera- tions. The order of presentation of the terms in this chapter was chosen to facilitate their development:

Page

Human Factors Engineering, Human Engineering, Human Factors, and Ergonomics .................... ........................... 2-2

Man-Machine System and Interfaces ............ .................. .. 2-2 NPP Personnel Addressed in the Handbook ........ ............... .. 2-2 Displays, Manual Controls, and Written Materials ..... .......... 2-3 Human Reliability .................. .......................... .. 2-3 Human Reliability Analysis ............... ..................... 2-4 Human Performance Models ................. ...................... 2-4 Performance Shaping Factors .............. ..................... .. 2-5 Stressors and Stress (Physiological and Psychological) .. ....... .. 2-5 Populational Stereotypes ..................... ...................... 2-6 Dependence, Independence, and Coupling ......... ............... 2-6 Human Error ...................... ............................. 2-7 Consequences of Human Errors and Recovery Factors .... .......... .. 2-8 Types of NPP Operating Conditions Considered in HRA ... ......... .. 2-9 Unavailability Because of Human Error .......... ................ 2-9 Types of NPP Tasks and Task Behaviors Addressed in HRA .. ....... .. 2-10 Task Taxonomy ...................... ............................ 2-13 Task Analysis ...................... ............................ 2-13 Error-Likely Situations and People ........... ................. 2-14 Accident-Prone Situations and People ........... ................ 2-15 Categories of Incorrect Human Outputs Related to HRA ... ........ 2-16 Human Error Probability ................ ....................... .. 2-17 Basic, Conditional, and Joint Probabilities ...... ............. .. 2-18 Distribution of HEPs ................. ........................ 2-18

2-I

Human Factors Engineering, Human Engineering, Human Factors, and Ergonomics

All of the above terms describe a discipline concerned with designing machines, operations, and work environments so that they match human capa- cities and limitations (Chapanis, 1965, p 8). The first three terms are used most widely in the United States. The term "human engineering" is now falling into disuse by human factors practitioners since some persons have used it in the Procrustean sense of engineering humans, i.e., making people fit the environment--the antithesis of human factors engineering. The last term, ergonomics, is used most frequently in other countries but is now becoming popular in the United States as well.

People working in the human factors area are often called human factors specialists, engineering psychologists, or ergonomists. The latter term is more common outside the United States. In the Handbook, these terms are interchangeable.

Man-Machine System and Interfaces

The term man-machine system denotes a system in which people have a moni- toring and/or control function. The term man is used in the generic sense. The term man-machine interface refers to points of interaction between people and the system. Thus, a display, a control, written materials, or any other item a person observes or uses is a man-machine interface. Man-man interfaces refer specifically to person-to-person communication or other interaction; in the Handbook, the term man-machine interfaces includes man-man interfaces.

NPP Personnel Addressed in the Handbook

The Handbook emphasizes tasks performed by control roam personnel, both licensed reactor operators and auxiliary reactor operators, and the shift technical advisor. This emphasis is dictated by the recognition that they represent the most important "man" aspect of the man-machine interfaces in NPPs of interest to probabilistic risk assessment (PRA). These personnel have many opportunities to commit errors that can result in an abnormal event, and their performance after the occurrence of such an event can either reduce or increase the consequences of the event. The Handbook also addresses errors in tasks performed by technicians who calibrate and test safety-related systems and components. Less emphasis is placed on tasks performed by maintenance personnel and by management personnel (except for shift supervisors, who are also reactor operators). Chapter 18 includes a discussion of the interaction of these personnel and their influence on tasks important for PRA.

In some chapters in the Handbook, the term operator is used to designate anyone who performs tasks that must be evaluated by the user of the Hand- book. This generic use of the term "operator" is a conventional one in the human factors field.

2-2

Displays, Manual Controls, and Written Materials

A display is any instrument or device that presents information to any sense organ (visual, auditory, or other). In NPPs, displays are annun- ciated or unannunciated. Annunciated displays usually consist of panels of legend indicators (often called tiles) that have auditory signals asso- ciated with them. Unannunciated displays in NPPs include meters, digital readouts, chart recorders, graphs, indicator lights, computer printouts, and video presentations.

Manual controls are those components with which the human enters his inputs to a system. Types of controls in NPPs include switches (rotary, toggle, and other), pushbuttons, levers, knobs, cranks, connectors, and tools. Manual controls may be continuous (e.g., a synchronization control or a potentiometer) or discrete, e.g., a two-position switch for a motor- operated valve.

Three types of written materials are addressed in the Handbook: formal written procedures that may be used many times, ad hoc written procedures that are one-of-a-kind, informally prepared procedures for some special purpose, and written notes prepared in response to oral instructions. Since written materials direct the performance of people in a man-machine system, they are part of the man-machine interfaces in a system.

Human Reliability

Evans (1976) notes that the popular definitions of reliability and availa- bility are as follows:

Reliability is the probability of successful performance of a mission.*

Availability is the probability that the system or component is available for use when needed.

Meister (1966) defines human reliability as "the probability that a job or task will successfully be completed by personnel at any required stage in system operation within a required minimum time (if the time requirement exists)." We borrow from Evans and Meister to define human reliability as the probability that a person (1) correctly performs some system-required activity in a required time period (if time is a limiting factor) and (2) performs no extraneous activity that can degrade the system.**

In the Handbook we use the term "reliability" in the above sense of "accuracy" rather than the conventional use by psychologists who use the term to indicate the consistency (repeatibility) of some measure of human performance (English and English, 1958).

In other applications, other measures of human performance (e.g., inter- val or ordinal numbers) can be used to define human reliability, but in the Handbook, we use probabilities only.

2-3

Human Reliability Analysis

Probabilistic Risk Assessment

Human Reliability Analysis

Human reliability analysis (HRA) is a method by which human reliability is estimated. In carrying out an HRA, it is necessary to identify those human actions that can have an effect on system reliability or availability. The most common application of HRA is the evaluation of human acts required in a system context. The consideration of extraneous actions is also impor- tant. The person in a system may not only fail to do what he is supposed to do, or fail to do it correctly, but he may also do something extraneous that could degrade the system. The latter is the weak link in HRA. It is not possible to anticipate all undesirable extraneous human actions. The best anyone can do is to identify those actions having the greatest poten- tial for degrading system reliability and availability. The assignment of probability estimates to extraneous actions is difficult and uncertain. Often the best one can do is to estimate very broad ranges of probabilities of human errors that one believes include the true probability. Fortu- nately, the probabilities of extraneous actions are usually very low.

The method commonly used in solving practical human reliability problems is the one described in Chapter 5, known as THERP - Technique for Human Error Rate Prediction. Other HRA methods are described and compared in a series of reviews by Meister (1964, 1971, 1973, 1983a, b), Embrey (1976), Pew et al (1977), and Swain (1964b). The HRA methodology in the Handbook is the THERP method. The estimates of probabilities of human errors and the

modeling of human performance in the Handbook are amenable to any HRA technique.

Probabilistic Risk Assessment

In the Handbook, HRA is described in the context of a probabilirtic risk

assessment (PRA). As defined in the Zion Probabilistic Safety Study (1981, Vol. 1), a PRA is "a rigorous and systematic identification of the levels of damage that could conceivably result from NPP operation and a quantita- tive assessment of the likelihood of such occurrences." In HRA, we assess the effects of human errors in the assessments of risk in a PRA.

Human Performance Models

A model of a system is an abstraction that reproduces symbolically (simu- lates) the way in which the system functions operationally (Chapanis, 1961). In the Handbook, the term human performance model denotes a sche- matic representation or abstraction of human behavior in a system context. Our goal in modeling human performance for PRA is to develop descriptive models to predict (within wide limits) how well people will perform what they are supposed to do in normal and abnormal situations in NPPs. For this limited purpose of modeling, it is sufficient to represent only a few of the limitless combinations of human and system characteristics. Esti- mated probabilities of human error are assigned to certain of these combi- nations in the human performance models in Part III of the Handbook.

0 2-4

Performance Shaping Factors

Stressors and Stress

Performance Shaping Factors

In modeling human performance for PRA, it is necessary to consider those factors that have the most effect on performance. Many factors affect

human performance in such a complex man-machine system as an NPP. Some of these performance shaping factors (PSFs) are external to the person and some are internal. The external PSFs include the entire work environment, especially the equipment design and the written procedures or oral instruc- tions. The internal PSFs represent the individual characteristics of the person--his skills, motivations, and the expectations that influence his performance. Psychological and physiological stresses result from a work environment in which the demands placed on the operator by the system do not conform to his capabilities and limitations.

To perform an HRA, an analyst must identify those PSFs that are most rele-

vant and influential in the jobs studied. Chapter 3 discusses several of the PSFs that influence the reliability of NPP personnel. Chapter 4 pre- sents a method to analyze required tasks to identify those PSFs relevant to HRA.

Stressors and Stress (Physiological and Psychological)

One of the most influential PSFs is stress. Montaigne, a French essayist of the late 1500s, noted, "Men under stress are fools, and fool them- selves." This quotation reflects a commonly held view that stress is undesirable. In the Handbook, we consider stress to be the human response to a stressor, and note that the relationship between human performance and stress is curvilinear--too little stress and too much stress both lead to less-than-optimum performance. Some in-between level of stress is n-s-

sary to provide sufficient arousal to perform reliably.

The stressors are classified as psychological stressors and physiological

stressors. The distinction is often arbitrary. Psychological stressors include task speed, distractions, monotonous work, threats from super- visors, and emergency situations. Physiological stressors include fatigue, discomfort, constriction of movement, and high temperature. Chapters 3 and

17 present discussions of stress and stressors, including the facilitative and disruptive aspects of stress.

For modeling of human performance, we represent the stress variable with four levels: very low (insufficient arousal to keep alert), optimum (the facilitative level), and two levels of high stress: moderately high (mod- erately disruptive), and extremely high (very disruptive). The first three levels are designated as task stress, which results from a very low task load, an optimum task load, or a heavy task load. The highest level repre-

sents threat stress and implies emotional reactions to the task situation.

2-5

Populational Stereotypes

Dependence, Independence, and Coupling

Populational Stereotypes

Another strong PSF is the expectancy that certain groups of people have for

certain modes of control activation, or modes of display presentation, outcomes, or meanings. Such expectancies are called populational stereo- types. For example, we strongly expect that when we turn a valve counter- clockwise, water will begin to flow, and that flow will increase as the valve is turned further counterclockwise. On the other hand, when we rotate a volume control on a radio counterclockwise, we expect the volume to decrease. A populational stereotype that differs between cultures is related to the common two-position light switch. In the U.S., we expect the "UP" position of a light switch to turn the lights on. In Europe, the opposite populational stereotype holds. Cultural differences also exist for direction-of-reading stereotypes.

Any design that violates a strong populational stereotype means that the user must learn to inhibit his expectancies. Even with extensive training, it is difficult to change a populational stereotype completely. Under high stress levels, we tend to revert to our populational stereotypes despite training to the contrary.

Dependence, Independence, and Coupling

Dependence is another important PSF. Dependence between two tasks refers

to the situation in which the probability of failure on one task is influ- enced by whether a success or failure occurred on the other task. The dependence may exist between two tasks performed by one person, or between the tasks performed by different persons. In WASH-1400, the term coupling was used and is still used by some PRA specialists. In the Handbook, we use the term dependence.

Complete dependence between two tasks means that if failure occurs on one,

failure will occur on the other with certainty. Similarly, if success occurs on one task, success will occur on the other. The dependence may be negative, but for the usual situations, it will be positive.

Zero dependence (or independence) between two tasks means that the proba- bility of failure or success on one task is the same regardless of failure or success on the other. In HRA, zero dependence is often assumed for situations in which the analysis is not materially affected by an assump- tion that may not be valid.

Between complete dependence and zero dependence there is a continuum of

possible levels of dependence. We have developed a dependence model in which the infinite possible levels of dependence are represented by the following five levels: zero dependence (ZD), low dependence (LD), moderate dependence (MD), high dependence (HD), and complete dependence (CD). This model and other methods of assessing dependence are discussed in detail in

Chapter 10.

2-6

Human Error

We define human error as any member of a set of human actions that exceeds some limit of acceptability (Rigby, 1970). Thus, an error is merely an

out-of-tolerance action, where the limits of tolerable performance are defined by the system. In an objective assessment of human error, there is no connotation of blame or fault. If an error is made, the underlying causes of the error are identified so that the probability of recurrence of that error is reduced or eliminated. We regard errors as the natural outgrowth of some unfavorable combination of people and the work situation (Swain, 1969c, 1980a, b). Either the person making an error does not have sufficient skill or motivation for consistently acceptable performance, and/or aspects of his work situation are not in accord with what he can do reliably. Insufficient skill in performance may arise from some combina- tion of inadequate training, poor provisions for practicing infrequently performed tasks, poor motivation, or any number of characteristics that directly affect the capability of the performer. The signi Eicant aspects of the work situation include the design of the equipment, the written procedures, and the general environment, as mentioned in the definition of performance shaping factors.

By convention, the definition of human error excludes malevolent behavior. Such behavior is not due to error; it is deliberate behavior calculated to produce a harmful effect. PRAs of NPP operations characteristically do not include malevolent acts and their possible consequences. However, it is possible to estimate probabilities of human error in the defense against, and recovery from, malevolent acts. Many of the estimates of probabilities in the Handbook could be used in such a study, but it does not specifically address this topic.

Human errors include intentional errors and unintentional errors. The former occur when the operator intends to perform some act that is incor- rect but believes it to be correct or to represent a superior method of performance. In everyday language, he has good intentions, but the effect on the system of his performance may be undesirable. An erroneous belief in the correctness of a method of operation often results from misinter- pretation of written materials or failure to understand an oral instruc- tion. The oprator's belief that his way is better than the prescribed way can result in a deliberate violation of standard operating rules. Examples of such errors include (1) not using written procedures in the intended manner, (2) deliberately loosening some equipment tolerances (setpoints) to avoid shutting down a reactor when it may not be absolutely necessary, and (3) venting low-radioactive containment pressure to the atmosphere because the operator is not willing to wait for the automatic safety features to respond to the increasing containment pressure.

Most errors are unintentional--the error just happens; it was not intended. Another term used for this type of error is an action slip (Norman, 1981a). Examples of unintentional errors or slips include (1) inadvertent tripping of the reactor because an operator sat on the edge of the control panel, (2) activating an incorrect control because the intended control is located

2-7

Consequences of Human Errors and

Recovery Factors

nearby and the labels for the two controls are similar in appearance, and (3) inadvertently skipping a step in a written procedure. (The latter type of unintentional error is the most frequent and is important in HRA.)

Consequences of Human Errors and Recovery Factors

In HRA, we are concerned with potential errors and potential consequences* of these errors to system reliability or availability. We want to estimate the probabilities of various kinds of errors that could adversely affect the system. A large number of potential errors may never have occurred, yet in HRA, we must treat the possibility of each error that has potential system consequences that should be avoided. Often we must consider errors whose probabilities of occurrence are very low--lO 4 or lower on each occasion where such an error is possible. Despite low probabilities of occurrence, errors can be very important if their occurrence could lead to highly unfavorable system consequences.

Knowledge of the consequences of error can act as.aPSF for the performer.

This knowledge can lower or raise his probability of error, depending on the circumstances. For example, an operator may be reluctant to trip the reactor because of fear that the trip may prove to be unnecessary and that he will be blamed for the associated economic loss. Thus, the operator's knowledge of the consequences of a reactor trip could be an important PSF. The analyst may have to take this into account when estimating the time it will take for the operator to trip the reactor.

In NPP operations, few human errors cause damage or lower the availability

of individual systems because the potentially adverse effect of the errors is prevented or compensated for by other components or systems or by other human actions. We call these preventive or compensatory factors recovery factors. The error that did not result in some undesirable consequence to the system is a recovered error, or a no-cost error. If an error is not recovered, and i~t results in some undesirable consequence, it is called an unrecovered error. Many of the recovery factors in an NPP result from someone checking someone else's performance. We use the term human redun- dancy to designate this type of recovery factor, and several of the models in Part III address this factor.

In PRA, one estimates the probabilities for potential errors, potential

recovery factors, and potential consequences separately. In making these estimates, one considers any dependence among them. For example, in most cases, the probabilities of errors for a checker of someone else's work

Our use of the word "consequences" is restricted to the direct conse- quences of a human action, e.g., a valve left in the wrong position makes some safety function unavailable. In PRA, the word is often used in an indirect sense, i.e., the health and financial effects resulting from a release of radioactive material.

2-8

Types of NPP Operating Conditions

Considered in HRA

will be much higher than the probabilities of errors for the original performer. This is because the checker usually does not expect to find many errors when he is evaluating someone else's performance--a special case of dependence.

As mentioned, knowledge of the potential consequences of an error can affect one's behavior. A mistake made by some analysts is to assume that an operator will be more careful when he knows that the consequences of an error are serious and that the analyst should assess lower probabilities of error in such cases. Records of human performance in many complex systems do not provide support for such an assumption.

Types of NPP Operating Conditions Considered in HRA

For HRA, it is convenient to divide NPP operating conditions into normal

and abnormal. Normal operating conditions include startup, planned shut- down, power level control, and refueling. Abnormal operating conditions result from events that disrupt the normal conditions in a plant. In a PRA, only certain specified abnormal events are generally considered. These events are called initiating events, i.e., events that require the plant to trip. Initiating events are classified as external events (such

as a fire, flood, earthquake) or internal events. The latter are divided into loss-of-(primary-)coolant accidents (LOCAs) and transients. A tran- sient is a condition other than a LOCA that causes a requirement for reac- tor shutdown. For PRA purposes, a loss of secondary coolant is classified

as a transient, not a LOCA. There are some internal events that are not usually considered in PRAs, such as a leak in the spent fuel pool.

PRAs include consequence analysis (the analysis of health and financial

effects resulting from a release of radioactive material that resulted from some initiating event or events) and emergency response, e.g., the evacu- ation of personnel who could be affected by the release of radioactive material. To date, HRAs performed as parts of PRAs have been applied only to internal initiating events (LOCAs and transients), and the Handbook specifically addresses only the response to internal initiating events. However, some of the models and estimated probabilities of human errors

could be applied to tasks in other areas of abnormal operating conditions.

Unavailability Because of Human Error

Availability is defined as the probability that a system is available for use when needed. Its converse, unavailability, is 1 minus availability. In NPPs, any errors of operation, maintenance, or calibration can result in the unavailability of some safety-related system or component for some period of time. This unavailability continues until someone discovers that the system or component is not operative, or until its condition causes other changes to the plant that lead to the discovery. In addition, other system events can cause some component to be unavailable, and this unavail- ability may be displayed on some meter or result in some other visible

2-9

Types of NPP Tasks and Task Behaviors Addressed in HRA

change in the plant. Plant personnel then have the opportunity to note 0 this change and take steps to restore the component to its normal operating condition.

The role of human performance in the unavailability of safety-related systems and components is discussed in Chapter 9, "Unavailability."

Types of NPP Tasks and Task Behaviors Addressed in HRA

A task is a level of job behavior that describes the performance of a meaningful job function. It is any unit of behavior that contributes to the accomplishment of some system goal or function. Task behavior refers to the human activities involved in carrying out a task. The distinction between a task and task behavior is often blurred.

Tasks are often divided into subtasks and/or into steps. Different ana- lysts may designate the same set of task behaviors as a task, subtask, or step; the distinction is arbitrary.

We think of a step as consisting of the following:

(1) Some type of information to the task performer from his work situation (e.g., he reads a display or a written procedure, or he receives an oral instruction)

(2) Some degree of processing of the information

(3) Some type of response (e.g., activating a switch or deciding that no action is required)

There are many ways to classify NPP tasks and task behaviors. The classi- fications employed in the Handbook are those we have found to be useful for HRA.

NPP tasks may be performed under normal or abnormal operating conditions. Under normal conditions, several tasks are of special interest because of their potential effect on the availability of engineering safety features (5SFs). These tasks are the following:

(1) Routine control room tasks, e.g., scanning of panels to note if any display indications are not normal

(2) Preventive and corrective maintenance tasks, e.g., replacing a defec- tive part in a pump in a safety system

(3) Calibration tasks, e.g., ensuring that setpoints for bistable ampli- fiers for detection of unacceptable conditions in the reactor are within tolerance

2-10

Types of NPP Tasks and Task Behaviors Addressed in HRA

(4) Postmaintenance or postcalibration tests, e.g., a test to see that a repaired pump works properly after maintenance

(5) Change and restoration tasks, in which the normal states of valves, circuit breakers, pumps, or other components are changed to permit maintenance, calibration, or tests and then are restored to their normal states after completion of the work

(6) Recovery tasks - those involving the use of recovery factors to detect deviant conditions, e.g., checking someone's work (human redundancy), noticing annunciations of out-of-tolerance conditions, active inspections (in which a person is directed to inspect specific items of equipment, usually via written procedures), and passive inspections (in which the search for deviant conditions is more casual, as in the basic walk-around inspection)

Calibration and restoration tasks are of special interest in an HRA because of the possibility that human errors may cause several redundant safety systems or components to be inoperable or unavailable. In this case, the human error becomes a common-cause event (also called common-cause failure or common-mode failure) because a single event (a human error) has the potential to fail more than one safety function or device.

The task behaviors involved in carrying out tasks under normal operating

conditions usually involve little, if any, diagnosis or decision-making. What is to be done is usually prescribed either-by plant policy or in written procedures and schedules for performing certain routine tasks. A useful breakdown of tasks behaviors is to designate them as skill-based, rule-based, or knowledge-based (Rasmussen, 1981). (This classification can also be applied to tasks carried out under abnormal operating conditions.) Skill-based behavior consists of the performance of more or less subconscious routines governed by stored patterns of behavior, e.g., use of a hand tool by one experienced with the tool. Rule-based behavior requires a more conscious effort in following stored (or written) rules, e.g., calibrating an instrument or using a checklist to restore locally operated valves to their normal operating status after maintenance. Knowledge-based behavior pertains to cases in which the task situation is, to some extent, unfamiliar--where considerably more cognition is involved in deciding what to do. As in most attempts at task behavior classification, there is some overlapping between terms, and the distinction between skill-.and rule- based behavior and between rule- and knowledge-based behavior may not be well-defined. Most of the models and estimates of human error probabi- lities in the Handbook pertain to rule-based behavior.

In NPP operations, the term skill-of-the-craft is sometimes used to denote a level of expertise in skill- and rule-based activities such that the task performer works without the aid of written materials. In HRA, it is impor- tant to determine when skill-of-the-craft is being relied on -by task per- formers, since errors of omission are a significant factor in HRA.

2-11

Types of NPP Tasks and Task

Behaviors Addressed in HRA

Under abnormal operating conditions, the three basic classes of tasks are those involved in recognizing that there is an abnormal situation, in diagnosing the problem and deciding what to do about it, and in carrying out the actions required to mitigate the effects of the event and to con- trol the abnormal situation.

The task behaviors in coping with unusual conditions ranging from deviant displays to abnormal events and that are of primary interest in HRA fall into several overlapping categories:

(1) Perception - noting that some abnormal condition exists, e.g., noti- cing that some alarms are sounding and blinking

(2) Discrimination - distinguishing the characteristics of a signal (or set of signals), e.g., noting the level of coolant in a tank

(3) Interpretation - assigning a meaning to the signal that was discri- minated, e.g., realizing that a low level of coolant in a tank means that there is a leak somewhere

(4) Diagnosis - determining the most likely cause(s) of the abnormal event

(5) Decision-Making - choosing between alternative diagnoses and deciding which actions to carry out

(6) Action - carrying out the activities indicated by the diagnosis, operating rules, or written procedures

(These terms are explained in more detail in Table 12-1.)

In our general model of the human component in a man-machine system (Figure 3-i1), the perception and discrimination tasks are internal inputs to the human processor, interpretation and decision-making tasks are cognitive activities, and the actions carried out are the response. This classifica- tion, which also applies to. routine tasks, is discussed fully in Chapter 3.

Some other terms for types of tasks are useful in HRA. We distinguish between dynamic and step-by-step tasks as one measure of different levels of task complexity. A dynamic task is one that requires a higher degree of man-machine interaction than is required by routine, procedurally guided tasks. Dynamic tasks may require decision-making, keeping track of several functions, controlling several functions, or any combination of these. These requirements are the distinction between dynamic tasks, such as may be involved in coping with an abnormal event, and step-by-step tasks, such as restoring valves after maintenance.

Related to the above two terms are some terms associated with manual con- trol tasks and used in the human factors community. Manual control tasks may be either' continuous or discontinuous (i.e., discrete). Continuous tasks are a subset of dynamic tasks in which a person performs some sort of

2-12

Task Taxonomy Task Analysis

tracking activity, i.e., he watches a frequently changing display and makes control adjustments to maintain the display indication within limits. An everyday example is that of steering an automobile, in which the display is the view through the windshield. Rod control is a continuous task in some NPPs in which an operator manipulates the controls in response to one or more displays on the control board. Most manual control tasks in an NPP are discrete in that each task element is a discrete step. For example, in a calibration procedure, many discrete switching actions are carried out.

In the Handbook, we generally use the terms dynamic and step-by-step tasks in preference to continuous and discrete tasks.

Task Taxonomy

There have been several attempts to develop a 'task taxonomy (or task clas- sification). Some of the taxonomies (Chambers, 1969; Fleishman et al, 1968, 1970) were useful for psychological research but not very useful for HRA because they referred primarily to human variables. Other taxonomies brought in equipment variables and are more useful for HRA (Swain, 1956, 1971; Munger et al, 1962; Payne and Altman, 1962; Smith and Payne, 1962; Berliner et al, 1964; Irwin et al, 1964a, b; Blanchard et al, 1966; Meister, 1967; Rigby, 1967). More recent attempts to develop a taxonomy have been specifically directed at HRA as a part of a PRA of NPP operations (Rasmussen, 1981; Rasmussen et al, 1981; Topmiller et al, 1982, 1983; Coiner et al, 1983; and the draft and present versions of the Handbook).

The recent efforts to develop a task taxonomy useful for HRA categorize data on human behavior in terms of combinations of equipment and task variables. Chapter 20 in the Handbook represents our approach in this area. Figure 20-1, a search scheme for using the Chapter 20 data tables, directs the user to specific tables, based on type of operation (abnormal or normal), type of task (diagnosis or rule-based behavior), type of man- machine interface (written materials, displays, valves), and type of error (omissions or commissions). This search scheme and data tables constitute a brief and very specialized task taxonomy, designed specifically for use by HRA specialistis participating in a PRA. Obviously, the search scheme and tables can be used in other applications, but they are less general than the elaborate taxonomies intended for collection and display of data (e.g., the Coiner reference cited).

Task Analysis

Chapter 4 presents an analytical procedure called task analysis. This technique is used to identify the relevant human elements in tasks and to identify the potential for human error. The task analysis provides the necessary raw material for the type of HRA described in the Handbook. The level of detail in a task analysis for HRA is determined by the needs of the PRA.

Task analysis is an analytical process for determining the specific beha- viors required of the human components in a man-machine system. It

2-13

Error-Likely Situations and People

involves determining the detailed performance required of people and equip- ment, and the effects on them of environmental conditions, malfunctions, and other unexpected events. Within each task, behavioral steps are ana- lyzed for the perceptions, decisions, memory storage, and motor outputs required, as well as for expected errors.

The data from a task analysis can be used to establish equipment design

criteria, operating sequences, written procedures, and requirements for selection and training of personnel. Task analysis is the most commonly used tool of the human factors specialist for both qualitative and quanti- tative applications.

Task analysis may be employed during any stage in the development or use of

a system. Different methods of task analysis may be used for different stages.. For example, link analysis is used to depict the pathways that are generated by people walking about and communicating with each other within a system. This technique is employed in later phases of development when mockups are available or during the use phase. Talk-throughs or walk- throughs are used when there are personnel available who can demonstrate how they would operate in a system.

Error-Likely Situations and People

In performing a task analysis to identify the potential for error, the

analyst identifies work situations that are error-likely in the sense that the ergonomics are so poor that errors are likely to occur; hence, error- likely situations (ELSs). ELSs involve demands on humans that are not compatible with their capabilities and limitations. For example, any design that violates a populational stereotype must be considered error- likely. A design that forces an operator to keep several display indica- tions in mind, and to figure out the meaning of the combination of indi- cations, is more error-likely than a design that displays directly the information the operator needs, without forcing him to go through an inter- pretation process. In general, designs that are not in accord with accepted ergonomics principles are error-likely. Accepted ergonomics principles are described in human factors reference works such as MIL-STD-1472C (1981), NUREG-0700 (1981), Van Cott and Kinkade (1972), or Woodson (1981).

Sometimes a person is characterized as error-likely. In an NPP, a truly

error-likely person would soon be recognized and would be retrained, reas- signed, or discharged. Although chronic error-likeliness in people quali- fied, trained, and experienced in a job is not common, we are all error- likely from time to time. Anyone emotionally upset is usually more likely to make errors. If one is fatigued from unusually long hours or work or has not had enough sleep, certain types of errors are relatively likely. Error-likeliness in people who have had adequate training in a job is

usually temporary.

2-14

Accident-Prone Situations

and People

Accident-Prone Situations and People

Accident-proneness is a special case of error-likeliness. The accident- prone situation (APS) is one that fosters human errors likely to result in

injury to people or damage to equipment and facilities. An accident-prone person is one who has statistically "more than his share" of accidents when compared with others having the same degree of exposure to opportunities for the same types of accidents.

A recently publicized example of an APS is that of an airliner that made a forced landing because some O-rings had been omitted from the oil seal stems. Investigation revealed that the stems were usually issued to the mechanics with the O-rings in place on the stems but that occasionally they were issued without the O-rings, in which case the mechanics were required to obtain the rings at a different location and position them on the stems

before threading the stems into the engines. Because the O-rings usually were on the stems, the mechanics had a high expectancy for the stems to be complete. On several occasions, the mechanics had failed to notice that the stems did not have the O-rings in place. Since the forced landing, it has been made company policy that all stems be issued with the 0-rings in place, thus relieving the mechanics of the requirement to check the stems. This policy change should materially reduce the probability of future unsafe incidents of this type, provided that there is a reliable method to ensure that the O-rings are indeed positioned on the stems before the mechanics use them.

Accident-proneness in individuals is not a very useful concept. At times,

it has been used to justify negligent safety practices by blaming the person who was involved in an accident and classifying him as accident- prone. Although there are groups who seem to be accident-prone (for exam- ple, the class of male drivers in the United States under 25 years of age), studies of work situations show that chronically accident-prone people in industry are rare. Carefully controlled studies show that accident-prone- ness in people is usually due to temporary conditions such as illness or emotional disturbances.

In the early 1900s, the concept of the accident-prone individual in indus-

try arose in part because faulty statistical analyses were used that did not incorporate concepts of statistical significance. Subsequent analyses of these early data showed that certain individuals were stigmatized as accident-prone when the number of accidents they experienced was not signi- ficantly greater than the number expected due to chance alone (Mintz and Blum, 1961). Even when chance can be ruled out, it may be found that people seeming to have "more than their expected share" of accidents are those who have the greatest exposure to the risk of accidents.

Taking all of the above factors into consideration, most modern industrial

safety specialists conclude that it is more cost-effective to look for APSs than to look for accident-prone people. The emphasis in the Handbook is on techniques for identifying APSs and ELSs and for estimating their potential effect on the availability of ESFs.

2-15

Categories of Incorrect Human

Outputs Related to HRA

Categories of Incorrect Human Outputs Related to HRA

As illustrated earlier, there are several ways of classifying human error. In HRA, we are interested primarily in those human errors that constitute incorrect inputs to the system. A person can make an error if he does something incorrectly, fails to do something he should, or fails to do something in time. It is convenient to categorize incorrect human outputs into errors of omission and errors of commission, with finer breakdowns as follows:

Errors of Omission

Omits entire task Omits a step in a task

Errors of Commission

Selection error:

Selects wrong control Mispositions control (includes reversal errors, improperly made

connections, etc.) Issues wrong command or information (via voice or writing)

Error of sequence

Time error:

Too early Too late

Qualitative error:

Too little Too much

Any of these incorrect human outputs may be the result of other human errors: an error in interpretation of a pattern of signals, a misreading of a display, a misprint in an emergency operating procedure, etc. In an HRA, the incorrect human outputs and human errors leading to these incor- rect outputs must be analyzed.

From a systems point of view, any of the above incorrect human outputs and other human errors is considered to be an error only when it reduces or has the potential for reducing system reliability, system safety, or the like- lihood that some other system success criterion will be met. Obviously, a person in a system performs many extraneous acts, e.g., smoking a ciga- rette, scratching his nose, and the like. In a system context, these behaviors are not considered errors unless they have potential for degra- ding the system in some manner. Sometimes an incorrect human output can

2-16

Human Error Probability

result in an undedirable system consequence (an unrecovered error), but generally, because of recovery factors in a well-designed system, no seri- ous loss to the system will occur (the error was a recovered error).

Human Error Probability

In the Handbook, the basic index of human performance is the human error probability (HEP). The HEP is the probability that when a given task is performed, an error will occur. There are many ways to estimate the HEP; some are statistical and some are judgmental. We use the term "human error probability" to represent any estimate. The HEPs listed in the Handbook are nominal HEPs, that is, plant-specific and situation-specific PSFs have not been taken into account.

Ideally, HEPs would be determined from actuarial data consisting of error relative frequencies in which the known number of errors of a given type are divided by the number of opportunities for that error to occur. Unfor- tunately, very little of this type of data exists for NPP tasks. Data on the number of errors of a given type have been recorded in the Nuclear Regulatory Commission's (NRC's) Licensee Event Reporting (LER) system, a measure of the numerator of the error relative frequency. However, the number of unrecovered errors that occur is grossly underestimated, and there is no record in the LER system of the number of opportunities for each type of error, the denominator of the error relative frequency (Speaker et al, 1982). The serious problem of the scarcity of actuarial data for HRA is discussed in Chapter 6.

In our earlier reports, including Section 6.1 of WASH-1400, we used the term human error rate (HER) interchangeably with human error probability. Since the term "rate" is often used in the sense of frequency per unit of time, it is not used in the Handbook, to avoid confusion.

Because most of our estimates of HEPs are based on some combination of data

and judgment, we call them derived HEPs to differentiate them from HEPs that would be calculated from error relative frequencies for NPP tasks. In developing our derived HEPs for NPP tasks, we often use data and informa- tion from other types of tasks that are similar to NPP tasks. Similarity is judged in terms of the correspondence of behavioral variables. Two physically dissimilar items of equipment might be similar in terms of the human behaviors involved in their operation, calibration, or maintenance. Therefore, an observed HEP for one of these items of equipment can be used as the estimate of the HEP for the same task on other items of equipment.

The probabilities most often used in HRA can be classified as demand proba- bilities; that is, the probabilities that given human actions will be performed correctly when required. If time limitations are imposed on the performance of a task, one probability of interest is the probability that the task will be completed correctly within the allotted time. If required, the HEP per hour can be obtained. For most calculations, the interest is in the probability of at least one error (for a given task) per hour. In availability estimates, the HEP per hour is estimated even though

2-17

Basic, Conditional, and

Joint Probabilities

the task may be performed with a frequency of much less than once per hour. Some sample calculations are presented in Chapter 9.

The reliability of a task (the probability of its successful performance)

is generally expressed as 1 minus HEP. Thus, when we speak of the relia- bility of performance of a human task, we are speaking of the probability of successful performance per demand. When we speak of the error probabi- lity, we mean the probability of unsuccessful performance per demand, or task unreliability, which is 1 minus task reliability. The terms, human error probability, human failure probability, or task failure probability are often used interchangeably with human unreliability. (Similarly, human

success probability, task success probability, and human reliability are used interchangeably.)

Basic, Conditional, and Joint Probabilities

Three types of probability are important in performing an HRA. These are

the basic human error probability (BHEP), the conditional human error probability (CHEP), and the joint human error probability (JHEP).

The BHEP is the probability of a human error on a task that is considered

as an isolated entity, unaffected by any other task. If the task is the first in a series of tasks, there is no ambiguity in this definition. It the task is not the first task and its outcome may be dependent upon the outcome of other tasks, the BHEP would be that probability conjectured to exist if no other tasks were involved.

The CHEP is the probability of human error on a specific task, given fail- ure, or success, on some other task. Two tasks are independent if the CHEP is the same regardless of whether success or failure occurred on the other task; otherwise, they are dependent.

The JHEP is the probability of human error on all tasks that must be per-

formed correctly to achieve some end result. This is the probability of most interest in reliability work and is determined by using both BHEPs and CHEPs.

Distributions of HEPs

The nominal HEPs listed in the Handbook represent our best estimates of the

HEPs for the tasks or activities described. For reasons discussed in Chapter 7, we assume that if sufficient actuarial data could be obtained on

a large number of tasks performers, the HEPs for each task would be distri- buted lognormally, or approximately so. Our nominal HEPs are designated as the medians of the lognormal distributions, that is, 50% of the true HEPs should be above and 50% below our nominal HEPs.

For HRA/PRA, we express the amount of variation in estimated HEPs in the

form of uncertainty bounds (UCBs) or error factors (EFs). The estimated variation reflects the uncertainty in our estimates. This uncertainty

2-18

Distributions of HEPs

arises from three main sources. One source is associated with variability due to people and conditions. A second is the uncertainty in our assess- ment of the HEPs. A third source is modeling uncertainty--how well one can model human performance in an HRA application.

We express the overall uncertainty as a lower and an upper UCB, where the lower UCB represents the 5th percentile of HEPs on a hypothesized lognormal distribution of HEPs for a task, and the upper UCB represents the 95th percentile. Thus, the expression .003 (.001 to .01) means that our nominal HEP is .003 and that we believe it unlikely that the true HEP would be lower than .001 in more than 5% of the cases, nor would it be higher than .01 in more than 5% of the cases. Most of the estimated UCBs in the Handbook are symmetrical about the median HEP, and those that are can be represented by a convenient shorthand term, the EF. In the above example, the expression may be restated as .003 (EF = 3). The lower UCB is calcu- lated by dividing the nominal HEP by the EF. In the example, the actual upper UCB of .003 x 3 = .009 is rounded upward to .01 to preserve a range ratio of 10 between the upper and lower UCBs.

The spread between the lower and upper UCBs in the Handbook varies accord- ing to task conditions. In general, the spread increases with very small HEPs (<.001) and large HEPs (>.01). EFs range from 3 to 10. To express greater uncertainty, the analyst may assume the UCBs to encompass a smaller percentage of cases than the range defined by our nominal percentages of 5% and 95%, e.g., he may assume the UCBs to encompass the range of percentages from 10% and 90%, or any other values that seem to be reasonable for the situation.

The UCBs in the Handbook are based on judgment and should not be confused with statistical confidence limits, which are based on observed frequencies of occurrence. The assignment of UCBs to our nominal HEPs is to permit certain types of sampling and propagation techniques characteristic of PRAs, beginning with WASH-1400 (see Chapter 12 in NUREG/CR-2300).

2-19

Ch. 3. Some Performance Shaping Factors Affecting Human Reliability

CHAPTER 3. SOME PERFORMANCE SHAPING FACTORS AFFECTING HUMAN RELIABILITY

Overview

This chapter describes several factors that affect human reliability in nuclear power plant (NPP) operations. Any factor that influences human performance is designated as a performance shaping factor (PSF). This chapter outlines some qualitative aspects of human performance measurement and provides a background for the quantitative models presented in Part III. We emphasize those PSFs that our experience in human reliability analysis (HRA) indicates have the largest influence on human performance in NPP operations and that can be at least approximately estimated. There- fore, not all of the PSFs listed in this chapter are discussed fully or represented in the quantitative models in Part III. For purposes of prob- abilistic risk assessment (PRA) of NPP operations, this limitation is not crucial.

In this chapter, we first discuss the human as a system component and in- troduce a simplified model to depict the human's role in a system in suffi- cient detail for the qualitative aspects of a PRA. Next, we describe vari- ous classes and categories of PSFs that affect human performance in NPPs. A large section of the chapter is devoted, to descriptions of some devia- tions in online NPPs from the best possible human factors engineering. These are provided so that the reader can understand how certain PSFs affect human reliability in these plants. This emphasis is in keeping with the Nuclear Regulatory Commission's (NRC's) interest in applications of HRA to existing plants. We understand fully that future NPPs will incorporate many human factors improvements. For the time being, however, the primary interest in PRA is to assess plants that are now, or will be shortly, pro- ducing power. We conclude the chapter with a general evaluation of those classes of PSFs that appear to have the most influence in a PRA and with estimates of the ranges of human reliability increases occurring if certain human factors improvements are made. The reader will note that many sig- nificant improvements could be made in human reliability through changes to equipment, written procedures, practice, training, and administrative con- trol, which should not involve major expenditures.

The Human As a System Component*

In many complex systems, human activities are required for maintenance, calibration, testing, and other normal operations as well as for coping with unusual events that place a system at risk. In general, each human, individually or as a member of a team, must perceive information (from displays, environmental conditions, or written or oral instructions) about

Some of this material was presented at the Eighth Congress of the Inter- national Ergonomics Association (Swain, 1982b).

3-1

The Human As a System Component

the state of that part of the system for which he has some responsibility. He must then process that information to determine what action, if any, he should take and then take it. While there is general agreement with this gross description of the role of the human in a complex system, the details of human interaction with a system are less easy to understand or describe. This problem is complicated by the fact that, for many important system functions, especially those called upon in response to abnormal events,, the interaction of two or more people is required.

These difficulties are somewhat lessened if the object is to model the role of the human only for the purpose of PRA. In this case, some of the de- tailed human response mechanisms can be represented by more general state- ments, and those statements can be presented in the form of estimates of human error probabilities (HEPs) as functions of the time available to respond to system events.

An important part of any PRA is the manner in which the influence of the human component is handled. Various methods exist for HRA, all of which assume that the human can be treated analytically, much as are other com- ponents in a complex system. This assumption has led some to the con- clusion that the human functions in the same manner as any other system component. Considering the greater variability and interdependence in human performance, such a conclusion is invalid. Still, human failures can be studied objectively and quantified within limitations, as can any other component failure.

If there were considerable quantitative data describing the performance of tasks by NPP personnel, we could develop data-based models of all activi- ties relevant to PRA. Since so little data are available at the NPP task level, we have had to develop different types of human performance models for PRA purposes. Our general approach is to divide human behavior in a system into small units, find data (or use expert judgment to obtain appropriate estimates) that fit these subdivisions and then recombine them to derive estimates of error probabilities for the tasks performed in NPPs. The recombination process must be such that the outcome is consistent with what we know about the psychology of the whole individual. Until we have data on task error relative frequencies, this type of decomposition and recomposition is often necessary. Even when data become available, models of human performance will aid in the extrapolation of data from one task to another.

For purposes of PRA only, a general descriptive model of human performance in an NPP will be sufficient. It is not necessary to understand fully the underlying dynamics of human behavior in order to estimate the probabili- ties of human error. Any model represents some kind of abstraction, and many facets of human behavior are omitted from our model because they are not necessary for its purposes. It is not necessary (or even possible) to represent every human characteristic in a model for PRA; neither is it necessary to subscribe to any particular theoretical position. If one uses "field theory" terms, "stimulus-response" terms, "cognitive science" terms, or terms from other theoretical arenas, the terms themselves may become ob- stacles to an objective consideration of the model proposed.

3-2

The Human As a System Component

What follows, then, is a general descriptive model of the human as a com- ponent in a closed-loop, man-machine system that can be used as background for the specific human performance models that provide the necessary quan- tification of human actions. The human as part of a closed-loop system is a conventional treatment of the human in a man-machine system that began after World War II and was influenced by the then new field of engineering psychology (Fitts, 1951a and b; Taylor, 1957). Figure 3-1 depicts the human as a system component that receives inputs from signals, signs, and conditions of his job situation (Box A) and produces external results that act on the system (Box E). Boxes A and E represent the man-machine inter- faces and also the man-man interfaces, such as oral or written communica- tion with others. The area bounded by a heavy line between Boxes A and E represents the human, who is supposed to receive the information from the system and act upon it.

The human is an extremely interactive and adaptive component in any man- machine system. Any attempt to depict his role in as simple a drawing as Figure 3-1 will introduce certain discontinuities in the representation of human behavior that really do not exist. (For a more complete representa- tion of the interactive functioning of the human as a system component, see Rasmussen, 1980.) As shown in the figure, the human is part of a closed- loop system, since information about the effects of his outputs to the system (Box D to Box E) is fed back through the displays and-other external indications to become another human input (Box A to Box B). Thus, it is possible for the human to discriminate any significant difference between the actual system status and the desired system status if proper feedback is provided.

As shown in Box B, the human senses, discriminates, and perceives informa-

tion from the outside sources *depicted in Box A, which may be modified by the internal feedback loop, as indicated. In short, he filters, recog- nizes, and organizes information as an active perceiver. Once information has been accepted, it has to be processed, as shown in Box C. This general area is variously called "mediating activities and processes," "information processing," "organismic processes," "cognition," or "cognitive processes." Various other terms have been used in different schools of psychology to describe these processes. We regard the above terms as equivalent to the general term cognition, defined as "the act or process of knowing, includ- ing both awareness and judgment" (Webster, 1975). In PRA, the term knowl- edge-based behavior is Sometimes used to describe operator behavior when he must plan his actions based on an analysis of the functional and physical properties of the system. This analysis is influenced by other cognitive aspects (e.g., the operator's goals and intentions) (Rasmussen, 1981).

As indicated in the words below the dotted line in Box C and by the inter- nal feedback loop, the cognitive processes affect what a person perceives and how he perceives it, the meaning he attaches to it, and what he decides to do about it. His training, especially in regard to the importance of certain goals or functions in his job, will materially affect what he per- ceives and decides to do. The feedback he receives from his peers and his immediate supervisor, and even his emotional state and more permanent

3-3

Figure 3-1

A EXTERNAL INPUTS TO HUMAN VIA SIGNALS, SIGNS, & CONDITIONS

-DIS PL-AYS:.-- VISUAL AUDITORY VIBRATORY OLFACTORY ETC

COMMUNICATION: ORAL WRITTEN

ENVIRONMENTAL CONDITIONS

CORRECT/INCORRECT HUMAN COMPONENTS

APPARENT PATH FOR WELL-PRACTICED ASSOCIATIONS rINTERNAL FEEDBACK LOOP- -

B C D INTERNAL INPUTS MEDIATING RESPONSES TO HUMAN VIA (COGNITIVE) (HUMAN SENSING, ACTIVITIES & OUTPUTS TO DISCRIMINATION, VPROCESSES SYSTEM) AND PERCEIVING

FILTERING, ORGANIZING, & RECOGNIZING VIA: SENSE ORGANS (RECEPTORS) & MEDIATING ACTIVITIES OR PROCESSES

PROCESSING OF PERCEIVED INFORMATION INCLUDING EFFECTS OF MEMORY, GOALS, INTENTIONS, PERSONALITY (ATTITUDES, MOTIVATION, EMOTIONS, ETC), INTERPRETATION, & DECISION- MAKING

NO ACTION; USE OF HAND, FOOT, OR OTHER BODY MEMBER TO CONTROL SOME NPP FUNCTION, OR ISSUANCE OF ORAL OR WRITTEN COMMANDS OR INFORMATION

E EXTERNAL RESULTS ON MAN-MACHINE SYSTEM

CORRECT OR INCORRECT

HUMAN INPUTS TO SYSTEM VIA:

NO ACTION SWITCHES VALVES COMPUTER KEYBOARD CRANE HANDLING EQUIPMENT & TOOLS CONNECTORS INSTRUCTIONS

L --- ---- EXTERNAL FEEDBACK LOOP -

Figure 3-1 A simplified model of the human component in a man-machine system for the purpose of PRA.

3-4

The Human As a System Component

personality characteristics, will partially determine how he interprets some stimulus pattern in his job situation. Conflicting goals (e.g., economic: keep the plant running--versus safety: shut the plant down now) will often have an adverse effect on a person's behavior; he may fail to take appropriate action or he may follow someone else's advice without weighing the consequences (Janis and Mann, 1977).

It is not possible to quantify all of these influences of the cognitive aspects of behavior and the interactions of other PSFs on the behavioral processes depicted in Box C. In Chapter 12, "Diagnosis of Abnormal Events," we present some interim diagnosis models that are limited in scope but that serve the purposes of PRA.

Finally, the human makes a response he judges to be appropriate to the situation (Box D). It may include the use of his hands, feet, voice, etc., to effect some change in the system (Box E). For some extremely well learned activities, it will appear that the operator responds immediately to external signals without going through any filtering, organizing, and cognitive processes. This is depicted by the path labeled "Apparent Path for Well-Practiced Associations." There is no need to get into the con- troversy of human action taking place without any cognition; one need merely note that for some well-learned activities, it is as if no thinking processes were involved.

The term skill-based behavior is often used in the PRA community to refer to this almost automatic form of human activity. Rasmussen (1981) notes that skill-based behavior consists of the performance of more or less subconscious routines governed by stored patterns of behavior (e.g., manual control of fuel rod insertion and withdrawal or operating a crane). He uses the term rule-based behavior to denote behavior that requires a more conscious effort in following stored (or written) rules (e.g., calibrating an instrument). The term knowledge-based behavior is applied to cases in which the situation is, to some extent, unfamiliar--that is, where con- siderably more cognition is involved in the operator's deciding what to do. These terms are useful if one remembers that they are merely convenient names for progressively more complex behavior and that the line of demarca- tion between skill- and rule-based and between rule- and knowledge-based behavior may be fuzzy at times (Goodstein, 1981). Most of the estimated HEPs and the related human performance models in the Handbook deal with rule-based behavior. Since most of the behavior analyzed in a PRA is rule-based, this seem appropriate. The human performance models dealing with knowledge-based behavior are presented in Chapter 12.

For PRA, Box E in Figure 3-1 takes on special importance since it refers to the direct inputs fromn the human to system equipm~ent. These inputs, or lack of inputs, represent the man-machine interfaces that are important to PRA. For example, was a valve restored to its normal position after main- tenance? Was the correct input made to calibrate a pressure sensor in a safety system? Was correct action taken in turning off the high-pressure injection system? Although in PRA one must be concerned with any aspect of human performance that has the potential for reducing system safety or

3-5

The Role of PSFs in HRA

reliability, the primary concern of the PRA team is with the "categories of incorrect human outputs related to human reliability analysis" shown in Table 3-1. Estimates of probabilities of the categories of human error in this table are made and then entered into the part of a PRA for which HRA is employed. To arrive at these estimates, the aspects of human behavior depicted in Figure 3-1 must be evaluated, along with all thq underlying PSFs that could materially affect human reliability.

Some of the human outputs listed in Table 3-1 are continuous in nature (e.g., the time taken to diagnose a transient). For PRA, continuous variables are usually discretized to facilitate relatively simple mathe- matical treatment of them. This simplification can be justified in terms of the scarcity and uncertainty of the data, plus the fact that discrete variables are usually adequate for PRA purposes.

It should be clear from the foregoing that the human component in a system is much more variable, and more difficult to predict, than the equipment component. In everyday language, the human has a mind of his own. Unlike the equipment component, the human will not always do what he has been "programmed" (i.e., trained and instructed) to do. Barring catastrophic failure of the component in question, the same input to an equipment com- ponent will result in very nearly the same output, time after time. This is not so with the human component; the human in the system will process the input, using his intentions, needs, goals, biases, and even his emotions. For these reasons, estimates of probabilities and the degree of uncertainty of failure of people in a system will usually be larger than the estimates assigned to equipment in that system.

The Role of Performance Shaping Factors in HRA

Any factor that influences human performance is designated as a PSF (Swain, 1967a). The quantitative models of human performance in the Handbook center on combinations of PSFs that are the major determiners of the proba- bilities of human errors. The models are based on the manner in which the human perceives, processes, and responds to the inputs he receives from the system. The PSFs largely determine whether his performance will be highly reliable, highly unreliable, or at some level in between. In developing the models, there was no attempt to develop detailed theories of behavior in practical work situations; the current state of theoretical development is described in the informative compendia edited by Sheridan and Johannsen, 1976; Moray, 1979; Rasmussen and Rouse, 1981; and Sheridan et al, 1982.

Table 3-2 presents a listing of those PSFs that we have found useful for HRA/PRA. The PSFs are divided into three classes: (1) the external PSFs-- those outside the individual, (2) the internal PSFs--those that operate within the individual himself, and (3) stressors. If there is a good match between the external PSFs and the internal PSFs, performance of tasks will be more reliable than if there is a mismatch. Figure 3-2 indicates that a match results in stress that is facilitative in nature while a mismatch results in stress that is disruptive. Stress, of course, is an internal

3-6

Table 3-1

Table 3-1 Categories of incorrect human outputs related to human reliability analysis*

ERRORS OF OMISSION

(1) Omits entire task

(2) Omits a step in a task

ERRORS OF COMMISSION

(1) Selection error:

(a) Selects wrong control**

(b) Mispositions control (indludes reversal errors, loose connections, etc.)

(2) Error of sequence

(3) Time error:

(a) Too early

(b) Too late

(4) Qualitative error:

(a) Too little

(b) Too much

As shown in Figure 3-1, the incorrect human outputs (Box D) become incorrect inputs to the system (Box E). Incorrect human outputs may be due to errors in providing correct information to the human (Box A), in his receipt of the information (Box B), in his processing of the information (Box C), in the type of responses he makes (Box D), or any combina- tion of these errors.

** Types of controls are listed in Chapter 13.

3-7

Table 3-2

Table 3-2 Some PSFs in man-machine systems*

EXTERNAL PSFs STRESSOR PSFs INTERNAL PSFs

SITUATIONAL CHARACTERISTICS TASK AND EQUIPMENT PSYCHOLOGICAL STRESSORS: ORGANISMIC FACTORS: CHARACTERISTICS: CHARACTERISTICS OF PEOPLE

THOSE PSFs GENERAL TO ONE OR THOSE PSFs SPECIFIC TO TASKS PSFs WHICH DIRECTLY AFFECT RESULTING FROM INTERNAL & MORE JOBS IN A WORK SITUATION IN A JOB MENTAL STRESS EXTERNAL INFLUENCES

ARCHITECTURAL FEATURES PERCEPTUAL REQUIREMENTS SUDDENNESS OF ONSET PREVIOUS TRAINING/EXPERIENCE QUALITY OF ENVIRONMENT: MOTOR REQUIREMENTS (SPEED, DURATION OF STRESS STATE OF CURRENT PRACTICE

TEMPERATURE, HUMIDITY, STRENGTH, PRECISION) TASK SPEED OR SKILL AIR QUALITY, AND RADIATION CONTROL-DISPLAY RELATIONSHIPS TASK LOAD PERSONALITY AND INTELLIGENCE

LIGHTING ANTICIPATORY REQUIREMENTS HIGH JEOPARDY RISK VARIABLES NOISE AND VIBRATION INTERPRETATION THREATS (OF FAILURE, LOSS OF JOB) MOTIVATION AND ATTITUDES DEGREE OF GENERAL CLEANLINESS DECISION-MAKING MONOTONOUS, DEGRADING, OR EMOTIONAL STATE

WORK HOURS/WORK BREAKS COMPLEXITY (INFORMATION LOAD) MEANINGLESS WORK STRESS (MENTAL OR BODILY SHIFT ROTATION NARROWNESS OF TASK LONG, UNEVENTFUL VIGILANCE TENSION) AVAILABILITY/ADEQUACY OF SPECIAL FREQUENCY AND REPETITIVENESS PERIODS KNOWLEDGE OF REQUIRED

EQUIPMENT, TOOLS. AND SUPPLIES TASK CRITICALITY CONFLICTS OF MOTIVES ABOUT PERFORMANCE STANDARDS MANNING PARAMETERS LONG- AND SHORT-TERM MEMORY JOB PERFORMANCE SEX DIFFERENCES ORGANIZATIONAL STRUCTURE CALCULATIONAL REQUIREMENTS REINFORCEMENT ABSENT OR PHYSICAL CONDITION

(e.g., AUTHORITY, RESPONSIBILITY, FEEDBACK (KNOWLEDGE OF RESULTS) NEGATIVE ATTITUDES BASED ON INFLUENCE COMMUNICATION CHANNELS) DYNAMIC vs. STEP-BY-STEP ACTIVITIES SENSORY DEPRIVATION OF FAMILY AND OTHER OUTSIDE

ACTIONS BY SUPERVISORS, CO- TEAM STRUCTURE AND COMMUNICATION DISTRACTIONS (NOISE, GLARE, PERSONS OR AGENCIES WORKERS, UNION REPRESENTATIVES. MAN-MACHINE INTERFACE FACTORS: MOVEMENT, FLICKER, COLOR) GROUP IDENTIFICATIONS AND REGULATORY PERSONNEL DESIGN OF PRIME EQUIPMENT, INCONSISTENT CUEING

REWARDS, RECOGNITION, BENEFITS TEST EQUIPMENT. MANUFACTURING EQUIPMENT. JOB AIDS. TOOLS, PHYSIOLOGICAL STRESSORS:

JOB AND TASK INSTRUCTIONS: FIXTURES PSFs WHICH DIRECTLY AFFECT

SINGLE MOST IMPORTANT TOOL FOR PHYSICAL STRESS MOST TASKS

DURATION OF STRESS PROCEDURES REQUIRED FATIGUE

(WRITTEN OR NOT WRITTEN) PAIN OR DISCOMFORT WRITTEN OR ORAL COMMUNICATIONS HUNGER OR THIRST CAUTIONS AND WARNINGS TEMPERATURE EXTREMES WORK METHODS RADIATION PLANT POLICIES (SHOP PRACTICES) G-FORCE EXTREMES

ATMOSPHERIC PRESSURE EXTREMES OXYGEN INSUFFICIENCY VIBRATION MOVEMENT CONSTRICTION LACK OF PHYSICAL EXERCISE DISRUPTION OF CIRCADIAN RHYTHM

Some of the tabled PSFs are not encountered in present-day NPPs (e.g., G-force extremes) but are listed for application to other man-machine

systems.

3-8

Figure 3-2

EXTERNAL

SITUATIONAl CHARACTER

TASK & EQL CHARACTEF

JOB TASK INSTRUCTIC

OPTIMUM PERFORMANCE

FACILITATIVE STRESS

PSFs MATCH

L ISTICS INTERI

IIPMENT ORORGANI ISTICS FACTOF

~NS MISMATCH

DISRUPTIVE STRESS

SUBOPTIMUM PERFORMANCE

Figure 3-2 Relationship of external and internal PSFs.

IAL PSFs

ISMIC IS

3-9

Classes of PSFs

characteristic of people, so we prefer to deal with stressors because they can be measured.

The human performance models in the Handbook are based on the use of task analysis (in any of its many forms) to identify the important PSFs affect- ing performance of a task, and the assignment of HEPs and their ranges (called uncertainty bounds [UCBs]), which reflect the estimated effects of certain combinations of these PSFs in typical online U.S. NPPs. Estimates are made to reflect the likely effects of improvements in display tech- nology and other human factors considerations that are being brought into some existing plants or plants soon to be producing commercial power. It is up to the analyst to match the PSFs he determines to be important (in a PRA) to the PSFs in the Handbook and to use the estimates of HEPs and UCBs associated with these PSFs for the HRA.

This approach to the modeling of human performance is only approximate. In the present state of human reliability technology, for example, it is not possible to assign weighting factors to each individual PSF that would be applicable to the specific task and situation in which an analyst is inter- ested. One reason for this limitation is that most of the PSFs interact, and the interaction effects are usually complex. The best we have been able to do in the Handbook is to offer guidelines to identify the most important combinations of PSFs in each task being analyzed. The analyst must use his expertise to match these combinations to the appropriate PSFs in the Handbook and to make the necessary adjustments for task-specific, considerations.

Thus, the human performance models and derived data in the Handbook are based to a large extent on the estimated influence of different combina- tions and levels of PSFs. If the analyst finds that the relevant PSFs are particularly beneficial in a given situation, he may assess a lower HEP than that given in the document. For example, if the HEP is stated as .003 (.001 to .01) (where the numbers in parentheses refer to lower and upper

UCBs), rather than use the nominal (best) estimate of .003, he may use the lower UCB of .001 or some other value less than .003. Conversely, if he judged that the PSFs for the tasks were particularly unfavorable for reliable human performance, he might use the upper bound of .01. In the Handbook, the UCBs are estimated to include the middle 90% of the true HEPs for the task and combination of PSFs identified.

Classes of Performance Shaping Factors

The following sections present some additional information on the three major classes of PSFs and their subdivisions. These general discussions are followed by a more detailed discussion and examples of many of the specific PSFs applied in NPP operations. Our major purpose is to illus- trate how either good or poor implementation of PSFs affects human relia- bility. It is not our intent to present design guidelines, although many of the examples and much of the discussion can be used for that purpose.

3-10

Classes of Performance Shaping Factors

External PSFs

In general, the external PSFs are those that define the work situations of the operators, technicians, maintenance personnel, engineers, clerks, and others who keep the NPP performing reliably and safely. The external PSFs fall into three general categories: Situational Characteristics, Task and Equipment Characteristics, and Job and Task Instructions. Situational Characteristics include PSFs that are often plantwide in influence or that cover many different jobs and tasks in the plant. Task and Equipment Characteristics include PSFs that are restricted to some given job or even to a task within a job. Job and Task Instructions include PSFs connected with the instructional materials used in jobs. Although instructions are really task characteristics, they are singled out because they have been found to be much more important than is generally believed, and they repre- sent an area in which a relatively small investment in time and money can result in substantial improvement in human reliability.

Internal PSFs

It is a truism that every human function in a man-machine system is unique. Each person comes to the job with certain skills, abilities, attitudes, and a host of other human attributes. It would be very convenient if one could select or develop standardized people for work in NPPs. Since this is not possible, an attempt is made to select workers who can develop acceptable levels of performance; the internal PSFs determine the potential level to which the individual can be developed. However, the states of some of the internal PSFs (e.g., "previous training/experience" and "state of current practice or skill") are dependent on the methods employed to train the worker and to maintain or improve his proficiency. As will be seen, there is ample room for improvement in the development and maintenance of skills, especially those skills related to coping with unusual events in NPPs.

Nevertheless, with only some exceptions (such as the lack of practice in coping with certain unusual events), internal PSFs are generally less important than the external PSFs in their estimated influence in a PRA. This may seem surprising, but it should be remembered that NPP personnel represent a highly selected group with generally good (but improvable) training and good motivation. Compared with some unselected and untrained group, many of the internal PSFs for NPP personnel are already very good in terms of their influences on performance; there is not nearly as much room for improvement of the internal PSFs as there is room for improvement of external PSFs. This is a fortunate state of affairs, since the external PSFs are nearly all under the control of utility management and the design organizations that provide the detailed human factors considerations in equipment, procedures, and operations.

Stressors

Because of its importance, stress, an internal PSF, is handled differently than the other internal PSFs. We define stress as bodily or mental tension

3-11

Situational Characteristics

and a stressor as any external or internal force that causes stress. Stress is something that is felt by a person; a stressor is something that produces stress. The stress a person experiences can be psychological or physiological stress. It is difficult at times to differentiate between the two; often the stress experienced is a combination of both. The term stress is a controversial one (Meister, 1981), but it is useful in PRA work. As shown in Figure 3-2, an adverse level of stress can arise when there is some mismatch between the external and internal PSFs. For exam- ple, if the perceptual requirement of a task imposes too many demands on a worker, his performance will usually suffer because of excessive task load- ing, a psychological stressor. On the other hand, without sufficient task loading, the operator's performance may be degraded because there is in- sufficient stress (mental or bodily tension) to keep him alert at the job. A well-designed man-machine system is one in which the task demands are consistent with the worker's capabilities, limitations, and needs. To the extent that this consistence is not achieved, human errors and degraded motivation can be expected. The latter effect can, of course, act to in- crease the frequency of errors, and so on.

The above example illustrates why it is convenient to speak in terms of stressors rather than stresses. We can avoid the obvious circularity in the use of the term "stress" by using the term "stressor" to mean some PSF or combination of PSFs that can be measured, observed, or at least inferred from observation and interviews and that can either facilitate or hinder reliable performance. We do not have to speculate about underlying mental or other unobservable human processes. The concept of stressors is dis- cussed more fully later in this chapter and in Chapter 17 where models of human performance related to certain types of stressors are presented.

Situational Characteristics

Architectural Features

Situational characteristics refer to PSFs that apply to more than one task. The first situational characteristic listed in Table 3-2 is architectural features. By this term we mean the general work area or areas. A control room, a room in which there is equipment to be calibrated, the room housing the turbine and generator--all have certain architectural features that can affect human performance either favorably or adversely.

One familiar example of an architectural feature with both favorable and adverse effects is the large control room that houses the operating panels for two or three reactors. One positive aspect is the fact that in an emergency condition, centralized control is facilitated, and it is more likely that sufficient qualified personnel will be available to cope with the emergency even in its earliest stages. A negative aspect is the higher noise level than a control room with one unit would have, the excessive number of people present at times, and the possibility for confusion.

A problem in many control rooms arises from their sheer size. Operators often have to walk several yards or more to read a display. Because it is

3-12

Situational Characteristics

normal for people to avoid unnecessary effort, they try to read the dis- plays from a distance and consequently make errors in their readings (Seminara et al, 1976, p 4-27).

Outside of the control room, cluttered passageways, stairway- design, and other characteristics that increase the difficulties of rapid and safe passage from one work area to another can have an effect that needs to be considered in an HRA. Since this kind of problem is situation-specific, the Handbook includes no estimates of HEPs or transit time. At one NPP we visited, the emergency control room was accessible only by climbing over large pipes and other obstructions that were often slippery. Had we per- formed an HRA involving the use of the emergency control room, we would have made a subjective estimate of the effect of this design on transit time. In general, estimates of time should be based on exercises in which actual time measurements are taken. These measurements have to be adjusted upward to account for the extra problems associated with a real emergency situation. In some cases, it may not be possible to take actual measure- ments, and reliance must be placed on estimates by personnel familiar with the required operation. Such estimates also must be adjusted upward for the usual tendency of people to underestimate how much time it will take them to do something, as described in Chapter 6, "Sources of Human Per- formance Estimates."1

The addition of administrative control features (such as locks on en- trances) can also increase transit time. This special problem is discussed in Chapter 16, "Management and Administrative Control." Additional archi- tectural problems are discussed in a later section in this chapter entitled "Some Ergonomics Problems in NPPs."1

Quality of the Working Environment

The next four situational characteristics in Table 3-2 (temperature, humid- ity, and air quality; noise and vibration; illumination; and degree of gen- eral cleanliness) refer to the quality of the worker's environment. In general, if the quality of the working environment is very poor, we con- sider this to be a form of stressor and double the estimated HEPs, as dis- cussed in Chapter 17, "Stress."

NPPs generally provide a satisfactory environment, but there are exceptions

regarding the noise level and an excessive number of people in the control rooms. There are certain areas, e.g., the turbine room, where a high noise level is to be expected and ear protectors should be worn. However, a high noise level should not be tolerated in the control room because it can cause irritation and fatigue, which may result in errors. The problem appears to be more psychological than physiological, since noise levels in most control rooms are well below the levels specified under the Occupa- tional Safety and Health Act (OSHA). In an informal, unpublished study, we took measurements with a sound level meter in various areas at Sandia National Laboratories where drafting and related support tasks are done. We found one consistent result: in any work area where the noise level was over 60 decibels, several occupants complained that the noise interfered

3-13

Situational Characteristics

with their concentration. Even though there are no physiological bases for these complaints, the occupants were irritated and distracted by the noise.

Methods for damping noise are well-known and will not be discussed here. In one Swedish NPP, the control room floor was carpeted to reduce the physiological stressor of noise. There were favorable psychological impli- cations as well, since the operating personnel saw this as an example of the company's concern for their well-being.

Lighting for NPP tasks is often a problem. In some control rooms, glare is such a problem that the ambient illumination must be reduced, sometimes to the point that errors in reading displays are very likely. In some areas in a plant, the lighting may be so poor that errors in reading valve labels are likely. In one plant we visited, the problem of glare had been so severe that the operators disconnected many of the lamps, with the result that a flashlight was considered a standard accessory for reading certain difficult-to-read displays such as chart recorders.

A special problem for certain NPP tasks is that of exposure to radioactiv- ity and the requirement for wearing protective clothing when performing certain tasks. (We list radiation as a physiological stressor in Table 3-2, but it could also be listed as a psychological stressor or under quality of the working environment.) Interviews with operating personnel indicate that although they have no concern about the very low levels of radiation, the clothing is uncomfortable, and a primary motivation of per- sonnel in a "rad" environment is to get the job done as quickly as possible and "get out of there." This motivation mitigates against human reliabil- ity. One reviewer of the draft Handbook stated that at one large labora- tory that handles radioactive materials, the facilities engineering group used a factor of 2 to account for the effect of anticontamination clothing on worker efficiency. Time estimates for work ranging from cleaning/paint- ing to piping replacements were based on standard work times and then doubled to allow for the effects of the uncomfortable special clothing. Records kept over an extended period showed that the factor of 2 was a good approximation.

Based on this and our own observations, we consider the requirement to wear protective clothing when working to constitute a moderately high level of stress, and the relevant HEPs in the Handbook are doubled. However, this is only a very general guideline; there are some unusual environments and situations in which the reduction in visibility from wearing face masks might result in a more severe degradation of human reliability. This is an area where the use of expert judgment is often required.

The remaining PSF under the general heading of "Quality of the Working Environment" is the degree of general cleanliness. This PSF has a psycho- logical impact quite apart from any impact on plant equipment. A generally dirty and untidy work environment may suggest to the worker that indiffer- ent work performance is tolerable. Although we cannot quantify the in- fluence of this PSF on human errors, it should be obvious that a clean and tidy work environment is more likely to affect human performance positively

3-14

Situational Characteristics

than the opposite kind of work environment. Although most NPPs maintain good housekeeping practices, there are substantial differences across plants.

Work Hours and Work Breaks

Since World War I, there have been many studies of human performance as a function of work hours and scheduling of work breaks (pauses). For re- views, see Viteles, 1932, Ch. 22; Ghiselli and Brown, 1955, Ch. 10; McGehee and Owen, 1961; Karn, 1966; and more recent introductory texts in indus- trial psychology. Most of these studies are relevant to production jobs or other work where the work rate is paced by the job. Much of the work in NPPs is self-paced except in response to unusual situations, in which case the workers are fully aroused and involved. The usual findings regarding work breaks are not applicable, since most of the NPP operators, techni- cians, and other personnel are relatively autonomous in performing their duties, and their efficiency is not likely to be improved by a rigid schedule of rest periods.

For PRA of NPP work operations, there are two important areas of concern-- the effect on human reliability of work hours that are longer than normal and of rotating shifts, a common practice in NPPs in the U.S. and other countries. This section deals with the number of consecutive hours spent on the job, and the following section deals with the effects of rotating shifts.

Longer than normal work hours are called for fairly often in NPPs, as when someone stays on duty at the end of his shift to fill in for someone on the next shift or when there are plant shutdown and start-up operations that require the presence of certain key personnel throughout. For PRA, we want to know the effects of extra work hours on the probability that an operator will detect unannunciated indications of marginal or out-of-tolerance con- ditions and on any decisions he may be required to make, especially those related to nonroutine conditions. In addition to the effects of extra work hours, there can be time stress on personnel during nonscheduled reactor shutdown activities because everyone is aware of the economic impact of unscheduled hours of plant unavailability. It is known from military studies that the combined effects of stress and fatigue are usually greater than the sum of their separate effects.

We performed a literature search (unpublished) of the effects of work hours on visual detection and decision-making with implications for the schedul- ing of work hours for NPP personnel. There are many studies, but the tasks represented are only marginally related to NPP tasks. For example, in the visual detection area, the studies typically employed a signal rate per hour far in excess of the rate of unannunciated deviant indications in an NPP control room (Colquhoun et al, 1968 a and b and 1969). Nevertheless, one fairly consistent finding is that during work performed in a typical night shift, there is a major impairment of performance (Folkard and Monk, 1980). In a study of Swedish NPP operations, operators on other than the day shift reported greater fatigue during work hours and difficulty in

3-15

Situational Characteristics

sleeping through days, which resulted in fewer hours of sleep (Axelsson and Lundberg, 1975). A review by Trumbull (1968) notes the significant effects of losing even one night's sleep--which is not uncommon in NPPs. Glenville et al (1979) showed that control subjects in a vigilance test detected twice as many signals as did experimental subjects deprived of a night's sleep. These and other studies (Bloom, 1961; Grandjean, 1968) indicate that as fatigue increases, the detection of visual signals deteriorates markedly, choice behavior demands more time and exhibits more error, and reading rate decreases. Sleep deprivation of 50 to 60 hours can lead to difficulties in mental arithmetic, inability to recall names and objects from recent conversation, and even to momentary hallucinations.

Bjerner et al (1980) studied errors made from 1912 to 1931 by three men whose job was to record meter readings in a gas works in Sweden. Some of the recordings required some simple arithmetic; most did not. It was found that when the 6-day work week was cut from 56 to 48 hours, the number of errors was reduced by 57%. For either the 56- or 48-hour work week, the 10 p.m. to 6 a.m. shift had nearly twice as many errors as each of the other two shifts. This study is interesting because long-term records were kept of three men who worked on a weekly rotating shift, so the differences were not due to differences among people.

Other studies (Grant, 1971, on drivers of British Railways locomotives; McFarland, 1971, on aircraft pilots; and Grandjean et al, 1971, on Swiss air controllers) show that fatigue due to long work hours or highly con- centrated work results in less attention to certain types of signals: people develop their own subjective standards of what is important, and as they become more fatigued, they ignore more signals. Bartlett (1951) called this the "phenomenon of the lowered subjective standard."

Although none of the above studies provides an unequivocal answer to the question of how the performance of NPP personnel is related to hours on the job, they indicate that long work hours may result in impaired performance and that the workers may not be aware that their performance is impaired.

In a recent study, the results of an extensive literature review of the effects of long work hours and shift work were used to estimate the per- formance of NPP personnel (Schneider et al, 1982; Kelly et al, 1982). In this study, changes.in human error rates (HERs) were estimated for per- sonnel who switched from 8-hour to 12-hour shifts at the Bruce Heavy Water Plant in Ontario, Canada. The estimates were derived by applying the data from the literature search to a detailed task analysis of operator duties. The investigators concluded that HERs would ". . . increase by approxi- mately 80% to 180% when comparing 24-hour days of 12-hour shifts with 24-hour days of 8-hour shifts under normal operating conditions . . . (Schneider et al, 1982, p 89). They also concluded that these bounds would be changed minimally under emergency conditions at the plant. They further estimated that the HER for the last 4 hours of a 12-hour shift would be 3 to 4 times the HER for the initial 8 hours.

3-16

Situational Characteristics

The study mentions that an examination of actual plant data revealed no evidence of any change in shift crew error associated with the change from 8-hour to 12-hour rotating shifts. However, the investigators stated in a personal communication that this lack of evidence is very likely due to several uncontrolled factors: (1) the automatic error tracking mechanisms in the plant recorded system error rather than only human error, (2) sig- nificant hardware and organizational changes occurred concurrently with the change in shift length, and (3) the Canadian Significant Event Reports from which the reports of human error were obtained have the same limitations as the U.S. Licensee Event Reports (LERs) described in Chapter 6. Therefore, the lack of evidence referred to does not invalidate the estimates of the increases in HERs due to a longer shift.

It is apparent that none of the above studies provides a direct answer to the question of how performance of NPP personnel is affected by longer than usual hours on the job. For purposes of PRA, we make the conservative recommendation that the estimated HEPs in the Handbook for detection of unannunciated signals, mental calculations, clerical work, and inspections and for general errors of omission be doubled for the case in which per- sonnel are working 12-hour shifts or for the occasional cases in which personnel are working beyond their normal 8-hour shifts.

The NRC has recognized the potential for errors in overtime work. A recent policy statement (Federal Register, 1982, p 23836) suggests the following individual overtime restrictions:

(1) No more than 16 consecutive hours (not including shift turnover time) should be worked in any given work period.

(2) No more than 16 hours should be worked in any 24-hour period, more than 24 hours in any 48-hour period, or more than 72 hours in any 7-day period (all exclusive of shift turnover time).

(3) A break of at least 8 hours should be allowed between work periods.

The above statements are less conservative than the suggestions made in NUREG-0737 (1980), which the 1982 reference replaces, and are considerably less conservative than our own recommendations to NRC in 1977:

(1) A maximum work period of 16 consecutive hours, but the second 8 hours should not occur during a person's normal sleeping time.

(2) A maximum of 60 hours in any 7-day period and a maximum of 100 hours in any 14-day period.

(3) A break of at least 12 hours between all work periods.

The studies cited in this and the next sections indicate that alertness and attention to detail decrease markedly after 60 hours in a work week or when normal sleep patterns are disrupted. Therefore, we judge that the new NRC policy materially increases the probability of human error due to overtime

3-17

Situational Characteristics

compared with the earlier suggestions, even though we cannot quantify the effect on the basis of available data.

In any event, we believe that our suggested factor-of-2 increase in esti- mated HEPs represents a conservative application of the intent of the over- time restrictions. We think it unlikely that an occasional extra 8 hours added to the day shift will result in any material degradation in perfor- mance; in this case, there is little, if any, disruption of normal sleep patterns. Major degradation in performance will arise from interference with a person's usual sleeping hours, as when a person on the swing shift or the midnight shift is asked to continue work for another 8 hours. Since one cannot predict the sleep-deprivation status of people working overtime in the event of an emergency, we do not attempt to fine-tune the correction factor of 2 for all possible situations, and we apply it to all cases of people working on overtime.

Shift Rotation and Night Work

In recent years, there have been several studies on the effects of shift rotation, especially on the disruption of the physiological circadian (approximately 24-hour) rhythm caused by having to be awake and working at unusual hours and to sleep during the daytime (Smith et al, 1982; Folkard et al, 1979; Colquhoun, 1970; and Colquhoun et al 1968a, 1968b, 1969). There seems little doubt that the individual circadian cycles of operators do affect their performances. A book edited by Colquhoun (1972) includes the proceedings of a conference on the relationship between human effi- ciency, diurnal (daily) rhythm, and loss of sleep. A more recent book, "Studies in Shiftwork," edited by Colquhoun and Rutenfranz (1980), reviews 39 studies involving measures of work decrement and physiological cost during standard commercial work and in experimental studies. A recent compendium edited by Noro (1982) includes eleven studies on circadian rhythm and shift work.

Another relatively recent concern is the effect on a worker's family life and other sociological costs of night work and shift rotation (Maurice, 1975; Carpentier and Cazamian, 1977) and flexible working hours (Wade, 1973). It is concluded by Kasl (1974, p 177) that poor mental health of workers is related to "fixed afternoon and rotating shifts that affect time-oriented body functions and lead to difficulty in role behavior (that of spouse or parent, for example) particularly for activities that are normally performed during the time of day when the worker is on the shift." Kahn (1974, p 213) notes that "studies of shift work typically (but not always) show significantly lower levels of satisfaction among employees who must work at night, and still lower satisfaction levels for those who must change from one shift to another."

The primary concern in this Handbook is whether any effects on human per- formance have been quantified so that an analyst could use some correction factor for, say, a plant that uses a rotating shift compared with one that does not. In part, this is an academic question; it is standard practice in U.S. and foreign NPPs to employ rotating shifts. However, there is a very practical issue: is there a difference in human reliability as a

3-18

Situational Characteristics

function of the periodicity of shift rotation? If so, an analyst could apply a correction factor. Unfortunately, the evidence is inconclusive. As noted by Carpentier and Cazamian (1977, p 59), some investigators (more recently, Tilley et al, 1982) recommend a "fast shift rotation" in which personnel rotate shifts every 2 to 3 days, while others have noted dif- ficulties in adjustment in biological rhythms in such cases, and they recommend the use of slower rotations (2- to 4-week periodicity).

About all we can conclude is that rotating shifts and night work have some unknown degrading effect. Since rotating shifts are the general rule for NPPs, we regard this as the "norm" for NPP operating personnel and offer no correction factor for shift rotation to apply to our estimated HEPs.

Availability/Adequacy of Special Equipment/Tools and Supplies

The effect of the availability or adequacy of special equipment, tools, and supplies on job performance constitute a PSF often ignored in industry. Generally, it appears that these items are locafed and managed in a way to suit the toolcrib attendants rather than the people who will use them. This reversal of priorities can adversely affect job performance. For example, if much effort is required to obtain certain tools, the worker will tend to make do with what he has, often to the detriment of his work.

A positive application of this PSF has been observed in some NPPs. Those who perform calibration tasks are provided with special carts with all the test equipment, tools, and procedures needed for the job. Because it is convenient to use the correct items, they are used.

There are no data on which to base an adjustment factor for very poor im- plementation of this PSF. If the analyst encounters such a situation, he may estimate an HEP toward the upper end of the UCB for each nominal HEP that applies to the tasks that could be affected by extremely poor availa- bility and/or adequacy of the necessary equipment.

Manning Parameters

The manning parameters refer to how many and what kinds of people perform which types of jobs. Prior to the Three Mile Island (TMI) incident, one of the basic requirements for manning effectiveness was not implemented--the use of a task analysis of all the tasks to be performed by an organization to establish the manning requirements. Without a basic task analysis, jobs tend to be defined in general terms rather than from an analysis of task demands. In U.S. NPPs, there has been a natural tendency to carry over the manning practices from fossil fuel power plants. While much of this carry- over is probably relevant, there was no systematic study of NPP manning to determine the extent to which NPPs have different manning needs.

After the TMI incident, however, the NRC stated that training for each job position would be determined by a "position task analysis, in which the tasks performed by the person in each position are defined, and the train- ing, in conjunction with education and experience, is identified to provide

3-19

Situational Characteristics

assurance that the tasks can be effectively carried out" (NUREG-0660, Task I.A.2-2). Based on the favorable experience in the use of task analysis in determining manning and training requirements for military jobs, over time there should be an improvement in the manning parameters and in the train- ing of NPP personnel.

One major recommendation since TMI, additional control room manning per guidance in NUREG-0737, has been implemented or exceeded. The major changes are an additional licensed reactor operator (RO) as a backup to the RO who operates the control room panels for each reactor unit, the require- ment for a senior reactor operator (SRO) to be in the control room at all times, and the creation of a new position, the shift technical advisor (STA) who is to be on call 24 hours per day. These changes are expected to become a formal NRC regulation soon (Goller and Merschoff, 1982).

With the new manning changes, the minimum staffing for control room opera- tions for the typical two-reactor unit with a single control room is as follows, assuming both units are in other than a cold shutdown condition:

One shift supervisor (SS), an SRO One SRO Three ROs One STA

In addition, we assume that three auxiliary reactor operators (AOs) will be available each shift. Although not a requirement, this is a carryover from the interim manning suggested in NUREG-0737.

Considering the above minimum staffing only, two of the ROs continuously man the control panels, one RO for each reactor unit. The third RO must be available as a backup to either of the ROs, but he does not have to be in the control room area continuously. At least one SRO must also be present in the control room area continuously; he may be the SS or the extra SRO. The SRO in the control room does not act as a regular RO. Thus, at any time both units are operating (i.e., not in a cold shutdown condition), the following minimum number of people must be present in the control room area: one SRO and two ROs (one at each unit's panels). The other person- nel need not be in the control room area continuously. The control room area is protected by administrative locks and includes the central operat- ing area where the primary operating panels are located, other auxiliary panels and components, and (usually) the SS's office, toilet facilities, and eating facilities. Thus, anyone within the control room area has immediate or very quick access to the central operating area.

All but the AOs and the STA must be licensed reactor operators. The AOs typically are unlicensed personnel who are training to become ROs. For purposes of PRA, their most important duties are changing and restoring the state of safety-related components such as valves, pumps, and circuit breakers to permit maintenance, tests, and other routine operations; and, during abnormal events, performing such monitoring and control functions

3-20

Situational Characteristics

inside or outside the control room area as may be required by the ROs or other licensed personnel. The STAs are not required to be licensed ROs; most are not. They do require special training and education, but not a formal university degree. The function of the STA is to provide "advanced technical assistance to the operating shift complement during normal and abnormal operating conditions" (NUREG-0737). An individual may function simultaneously as a shift SRO and as the STA provided that he has a bachelor's degree in engineering, is a licensed SRO, and has the STA training described in NUREG-0737.

The human performance models in Chapter 18, "Staffing and Experience Levels," are based on the assumption of the availability of three licensed control room operators (including the SS) and the STA for a unit experi- encing an abnormal event. The models include assumed times for each person to reach the control room after an unusual event has been noticed by the primary control room operator.

We have observed some similarities and differences in manning and training requirements between the U.S. and other countries. NUREG-0863 (Au et al., 1982) provides a recent summary for countries with operating reactors. Our own observations follow. In the U.S., ROs are often former fossil fuel plant operators or former operators of shipboard reactors from the U.S. Navy. In addition, as is the case in Canada, France, Sweden, Finland, Norway, and Japan, U.S. ROs generally are not university graduates. One notable exception is that at the Zion Pressurized Water Reactor (PWR), the STAs, who have technical university degrees, are also SROs. In the Zion Probabilistic Safety Study (1981), this unusual combination was given greater credit for detection and correction of human errors than for another PWR in which the STAs were not SROs (Indian Point Probabilistic Safety Study, 1982).

In the U.S.S.R., control room operators are graduate engineers or the equivalent, and in Great Britain, the SROs must have university-equivalent backgrounds. There are pluses and minuses to both practices. The use of graduate engineers may have some advantages in coping with unusual events if their expertise is necessary. On the other hand, problems of boredom are accentuated with these personnel if the daily task demands are not up to their capabilities and job expectations.

The use of technically trained noncollege personnel may involve less risk of job dissatisfaction, but some have questioned how well these personnel can cope with highly unusual events. We believe it is less a question of whether a college degree is necessary than of how job-relevant the training is. Training problems are addressed later in this chapter.

In the draft issue of the Handbook, we mentioned some manning problems in Swedish NPPs because of conflicts arising from the use of two different job classifications, reactor engineers and turbine engineers. We have since learned that these problems were limited to one operating station and that the major issue, that of the promotion path to SS, has since been resolved.

3-21

Task and Equipment Characteristics

For a review of Swedish manning practices, see Wirstad and Andersson (1 980).

Organizational Structure and Actions by Others

A plant's organizational structure (e.g., authority, responsibility, and communication channels) and actions by supervisors, coworkers, union repre- sentatives, and regulatory personnel often fall into the sociological realm. Some of these actions have technical consequences and are therefore appropriate to this Handbook. One important area is administrative control of the status of safety systems and components.

When systems or components are removed from their normal operating status for maintenance or other purposes, proper restoration procedures may not be carried out, and certain necessary safety systems or components may not be available because of oversights or other errors. In NPPs, considerable reliance is placed on human actions to avoid such problems. Therefore, the kind of administrative control in a plant is very important. If admini- strative control is tight and the supervisors insist on adherence to plant policies, it will be much less likely that a valve will be left in the wrong position or that a jumper will not be removed after-maintenance. Chapter 16 details the importance of sound administrative control in re- covering from errors in system and component restoration.

Actions by government regulatory personnel, especially those assigned to a plant, can have a substantial effect on plant personnel and practices. For example, plant personnel will usually respond if an onsite federal inspec- tor emphasizes good housekeeping. If the inspector frequently queries operators on how they would respond *to hypothesized unusual events, the operators will tend to think about coping with the unusual. On the other hand, if the inspector spends most of his time checking paperwork, the plant will expend most of its effort to ensure acceptable paperwork. Thus, the emphasis of. the inspector may have a strong influence on the "per- sonality" of the plant.

Rewards, Recognition, and Benefits

These PSFs have at least an indirect influence on the performance of tech- nical jobs. Industrial and social psychology textbooks discuss the impor- tance of these factors. Although they may have an effect on human relia- bility, they are outside the scope of this Handbook. We cannot estimate the influence of these PSFs on HEPs for NPP tasks.

Task and Equipment Characteristics

The task and equipment characteristics are PSFs that are task-specific. The following sections describe in general terms the task and equipment PSFs listed in Table 3-3. A later section, "Some Ergonomic Problems in Operating NPPs,"1 deals with specific findings on these factors in typical light water reactors.

3-22

Table 3-3

Table 3-3* Some task and equipment characteristics

Perceptual requirements

Motor requirements (speed, strength, precision)

Control-display relationships

Anticipatory requirements

Interpretation

Decision-making

Complexity (information load)

Narrowness of task

Frequency and repetitiveness

Task criticality

Long- and short-term memory

Calculational requirements

Feedback (knowledge of results)

Dynamic versus step-by-step activities

Team structure

Man-machine interface factors:

Design of prime equipment, test equipment, manufacturing equipment, job aids, tools, and fixtures

Source: Table 3-2

3-23

Task and Equipment Characteristics

Perceptual Requirements

The perceptual requirements of a task are determined by the task and equip- ment features that convey information to the personnel. Almost all of the perceptual requirements placed on the personnel are visual--reading meters, charts, labels, etc. Auditory requirements are minor, requiring only the ability to hear and recognize various alarms. The crucial PSFs for dis- plays are their ability to reliably convey the essential information to the user and to attract the operator's attention (if prompt response is re- quired). Chapter 11, "Displays," and Chapter 15, "Oral Instructions and Written Procedures," present information on this topic relevant to PRA.

Motor Requirements

Motor requirements refer to control, adjustment, connecting, or other actions, as described in Chapter 13, "Manual Controls," and Chapter 14, "Locally Operated Valves." Speed of movement is rarely a problem in NPPs, although the strength required to operate a control can be a problem in some maintenance work in which "come-alongs" are needed to change the status of large locally operated valves. Generally, demands for muscular strength in NPP operations are well within the capabilities of the fifth percentile female operator (Hertzberg, 1972). Precision of motor response is not a problem in NPPs except for certain operations performed during refueling and rod manipulations. Most of the human factors problems with controls in NPPs are related to the use of unnecessarily large control handles and to the poor location and identification of controls.;

Control-Display Relationships

The relationships between controls and displays refer to the compatibility of displayed information with the required movement of controls. Certain displays lead to expectancies as to how a control should be moved. If the displays and their associated controls violate these expectancies, the probability of human error will be very high, especially under stress (Fitts and Posner, 1967, p 25). In our human performance models, viola- tions of such expectancies are associated with very hiah estimates of HEPs.

Anticipatory Requirements

Anticipatory requirements occur in tasks in which a person has to be alert

for some signal while performing another activity that also requires atten- tion. Humans have definite limitations in this area. Man is essentially a single-channel mechanism, and he can pay attention to only one thing at a time (Fitts, 1951a and b). With practice, he can rapidly switch his atten- tion among several stimuli, and it may appear that he is attending to several things simultaneously (Gabriel and Burrows, 1968). Still, at any moment, he is attending to just one stimulus.

The skilled control room operator ordinarily has ample capacity for the tasks at hand. Under unusual conditions, however, he may be overloaded-- there may be so many visual and auditory signals competing for his atten- tion that he is unable to divide his attention among them in the most

3-24

Task and Equipment Characteristics

effective manner. He will ignore some signals either because he does not perceive them or because he has had to select the more important ones. The performance model for annunciated displays reflects this effect of signal loading on operators (Chapter 11.).

Interpretation Requirements

Interpretation requirements in NPPs are related to situations in which the presented information requires some mental processing. The course of action implied by the information is not obvious, and interpretation of the data is required. The more interpretation required, the longer the re- sponse time and the greater the probability of a time error and error of interpretation. Ease of interpretation, or minimization of the need for interpretation, is achieved by an optimum combination of operator training, design of displays, and design of written procedures. Less-than-adequate design implementation of these PSFs will increase interpretation demands and thus increase human errors. Chapter 12, "Diagnosis of Abnormal Events," presents a treatment of interpretation for purposes of PRA.

Decision-Making

The need to make decisions in a job can help keep the job interesting and challenging. Without any requirement for decision-making, most people become bored. Therefore, the best-designed jobs and tasks include the need for a person to use his decision-making ability. Errors occur when the information presented to the decision-maker does not adequately support the kinds of decisions he needs to make. If this happens, the person making an incorrect decision is likely to be faulted even though he has responded logically to the information he had. For example, in aircraft crash investigations, a verdict of "pilot error" may be attributed to the pilot without any attempt to uncover the factors that led to the wrong decision (Pressey, 1976). Chapter 12 includes a discussion of our treatment of decision-making.

Complexity (Information Load)

The complexity of a job is a function of the amount of information the worker must process and the amount of abstract reasoning or visualization required. obviously, errors will be frequent if the job is too complex. Tasks in an NPP ordinarily are well within the capabilities of the workers. The experienced operator understands the working of the plant, and he pro- cesses information at a self-determined pace. However, in some plants, the emergency procedures introduce complexities that exceed the capabilities even of highly skilled operators. For an example, see Problem 3, "Case Study of a LO0CA Procedure," in Chapter 21.

As the information load-on an operator increases, a point may be reached at which he can no longer process information. As described later in the dis- cussion on stress, he has several ways of compensating for overload, some of which can result in error. Some years ago, it was believed that the concept of information theory (as described by Wiener, 1948; and Shannon

3-25

Task and Equipment Characteristics

and weaver, 1949;,and extended to psychology by Attneave, 1959; Garner, 1962; Sheridan and Ferrell, 1974; and others) could be used to quantify information load. In application to practical situations, however, the utility of information theory has been disappointing. One reason is that the objective measure of information in tasks or task elements cannot as- sess the meaning that the individual attaches to each signal. For example, when shift, change approaches, signals related to this event are more com- pelling to some operators than are other signals. Analyses of LERs show that a disproportionate number of errors of oversight occurs within an hour in each direction from shift change.

Frequency and Repetitiveness

The frequency and repetitiveness of human actions are PSFs that have a dual relationship to human performance. Although the ability to perform relia- bly obviously increases with the frequency and repetitiveness of a task, highly repetitive tasks become boring and few workers will do their best on such jobs. The optimal trade-off between reliability and boredom in the design of jobs remains unsolved in many industries, including nuclear power. Some calibration tasks, for example, involve so much repetition that it is a tribute to the patience of the technicians that so few errors are made.

At the opposite extreme, if a task that is performed infrequently must be

performed correctly, frequent practice is necessary, as with fire drills or the procedure for cardiopulmonary resuscitation. NPP emergency procedures 'should be practiced periodically.

Task Criticality

The criticality of a task as perceived by plant personnel will affect how much attention they devote to the task. This is especially true during periods of time stress or fatigue when one does not have the time or the energy to perform all tasks or task elements. A person at work will naturally devote more of his attention to those tasks or task elements that he considers most critical to the job (Bartlett, 1951 ). A person's percep- tion of what is critical is obviously influenced by instruction from his supervisor and by the "old hands" with whom he works. For example, the operators' overriding concern with keeping the pressurizer from going solid (filling completely) in the TMI incident is thought to have led to their failure to detect other, more important conditions (Kemeny, 1979). This concern (their-perception of the importance of this task) stemmed from the particular training and instruction they had received.

Long- and Short-Term Memory Requirements

These requirements can often degrade human performance. Although long-term memory for facts is not one of man's outstanding capabilities, he does have a good capacity for remembering principles, strategies, contingencies, and other rules and their applications--provided that he has been properly taught and has had a chance to practice them occasionally (Fitts, 1951a).

3-26

Task and Equipment Characteristics

Short-term memory is notoriously unreliable (Welford, 1976, p 101), yet many jobs place unrealistic demands for short-term memory on personnel. Short-term memory is less reliable than long-term memory because it lacks the long learning time and rehearsal associated with the latter. An exam- ple is that of looking up a new telephone number in the directory and keep- ing it in mind for dialing purposes. Data from the Bell System indicate that under these circumstances, a probability of error of 5% can be ex- pected with the circular dial. If a person has to remember digits (or other meaningless information) for more than a few seconds, errors of 1% to 5% can be expected. For reliable performance, people must be supplied with properly designed forms on which to record data. Chapter 15, "Oral In- structions and Written Procedures," provides estimated HEPs for those cases in which people are performing tasks using short-term memory instead of a written list.

Calculational Requirements

The performance of even simple arithmetic accounts for many errors in tech- nical work. In one study, highly skilled inspectors routinely calculating familiar arithmetic problems involving simple addition, subtraction, and division made slightly more than 1% errors using paper and pencil (Rigby and Edelman, 1968b). Our experience in industry shows that the relatively high frequency of calculational errors is always a surprise to those who design work operations. The typical "solution" to these problems is to tell those who make calculational errors to be more careful, an approach that rarely works for long.

Chapter 13 provides estimates of HEPs for calculational tasks with and without the use of hand calculators.

Feedback

Feedback, a term borrowed from engineering technology, refers to the knowl- edge of results that a person receives about the status or adequacy of his outputs. Without the feedback loop shown in Figure 3-1, a worker operates as an open-loop system and cannot perform complicated activities reliably.

It is recognized that feedback has motivational as-well as informational aspects; the motivational aspects are outside the scope of this manual. The informational content of feedback provides a person with objective information on what he is supposed to do and whether he does it correctly and with detailed information on when and how he failed to do it correctly. Feedback must always be timely. In some cases, delays of even a few seconds can seriously degrade performance (Miller, 1953a, Section IV). Chapters 11 and 13 treat this aspect of feedback.

Dynamic Versus Step-by-Step Activities

In our human performance models, we distinguish between dynamic and stR by-step tasks as one characteristic of different levels of task complexity

3-27

Job and Task Instructions

(or information load). A dynamic task is one that requires a higher degree of man-machine interaction than is required by routine, procedurally guided tasks. Dynamic tasks may require decision-making, keeping track of several functions, controlling several functions, or any combination of these. These requirements are the basis of the distinction between dynamic tasks, such as may be involved in responding to a transient, and step-by-step tasks, such as carrying out written calibration procedures.

Team Structure

This term refers to the combination of people performing work that must be done by two or more people. In this Handbook, we are not concerned with the sociological aspects of team makeup, only with the technical aspects of people working in teams. One major technical aspect has to do with the recovery factors made possible by having one person in a -position to ob- serve another's work either during or after completion of the work. The effects of these recovery factors are discussed more fully in Chapter 19.

Another technical aspect has to do with dividing tasks among different people to reduce task overload. An example is given in Chapter 21 in which the addition of a second operator to control rooms in certain NPPs is estimated to result in a substantial gain in human reliability for a task to be performed under abnormal conditions. (See the fourth problem in Chapter 21.)

Man-Machine Interface Factors

This final task and equipment PSF is a catchall category covering all points of contact between the human and the hardware that were not covered in the other categories. It includes test equipment, tools, handling equipment, etc. Although these items all affect performance, they are not treated individually since the general principles discussed in Chapters 11, 13, and 14 apply in most instances. Some of the problems in the design of the man-machine interface are in a later section, "Some Ergonomic Problems in Operating NPPs," and provide some of the qualitative rationale for the human performance models and derived data presented in those chapters.

Job and Task Instructions

The PSF, "Job and Task Instructions," includes directives such as written

or nonwritten procedures, written or oral communications, cautions and warnings, work methods, and plant policies (sometimes called shop prac- tices). Although all of these are important for reliable human perfor- mance, our comments here are directed mainly towards written procedures, work methods, and plant policies. Oral instructions are described in Chapter 15.

One of the most important work methods is the correct use of written pro- cedures and checklists. If any task is performed without step-by-step reference to written procedures, errors of omission are much more likely to occur. Also, if a checklist is used improperly, as when someone first inspects several items of equipment for proper status and then checks items

3-28

Job and Task Instructions

off the checklist all at once, errors of omission are again very likely. Chapter 16, "Management and Administrative Control," provides quantitative estimates of the probabilities of these improper work methods.

Earlier, in discussing "Organizational Structure and Actions by Others," we stressed the importance of a proper and enforced administrative control system for the restoration of equipment after maintenance. This falls under the PSF of situational characteristics. In our opinion, the most imiportant plant policy related to reliable restoration activities is the requirement to tag all valves removed from normal operating status and to remove all tags when the valves are restored. In view of the importance of proper restoration of equipment, the implementation of this policy involves a system of record keeping that could be improved materially in most NPPs.

As described in Chapter 16, there are different levels of implementation of tagging policy. In terms of manpower expenditure and other expenses, the cost of the best level of tagging is small when compared with the losses prevented. A plant that uses a tightly controlled tag inventory is less likely to experience unavailabilities because of failures to restore components to their normal positions after maintenance. Effective plant discipline, enforced by supervision and management, is necessary for proper functioning of this best level of tagging procedure.

The performance models in Part III of the Handbook show that estimated probabilities of unrecovered human errors can differ by factors of 100'or greater, depending upon the type of written procedures and the related work methods followed. Experience in many industries shows that well-written procedures and good work methods can often compensate for less-than- adequate human factors engineering of equipment. However, as stated in WASH-1400 (p 111-64), "The written instructions [in NPPs) do not conform to established principles of good writing; they are more typical of military maintenance procedures of approximately 20 years ago."

Written procedures that are difficult to read, difficult to locate, or inconvenient to use are seldom used. At some plants, emergency operating procedures are not easily distinguishable from the other procedures; and, once located, a specific emergency procedure is difficult to find because there are no tabs or other indexing methods to assist the operator. Finally, the format and content of the typical NPP procedures are not conducive to easy use.

Reading has been studied intensively by the National Institute of Educa- tion, which states that a substantial proportion of the U.S. population does not read well enough to function in society. Studies sponsored by that institute report that "some 12 million people 14 years of age and older cannot read as well as the average fourth-grader, yet seventh grade reading ability is required to perform such skilled or semiskilled jobs *as machinist or cook" (Eaton, 1974). To reduce errors in using written pro- cedures, the writing style should place minimal demands on reading ability. The writing style of the typical NPP procedures requires about a Grade 12 reading level, whereas a Grade 4 reading level would be better for maximum

3-29

Job and Task Instructions

reading comprehension.* This is particularly important for highly stress- ful conditions, during which difficulty in comprehension can result in disorganized behavior as well as in specific errors.

It is not difficult to improve the readability of written procedures, even to write to the Grade 4 reading level. Pyrczak and Roth (1976) studied the readability of directions on nonprescription drugs and found that the required reading level was usually Grade 11 or higher. They showed how some simple rewriting could improve the readability to the Grade 4 level. The following statement on the bottle of a nonprescription drug is at the Grade 11 or 12 reading level: "WARNING: Keep this and all medicines out of children's reach. In case of accidental overdose, consult a physician immediately." Their Grade 4 version of this warning would be: "WARNING: Keep this and all medicines out of children's reach. If someone takes too much by accident, talk to a doctor right away." This example shows that writing at the Grade 4 level does not have to be insulting to the reader, a common misconception. The important thing is to communicate reliably, even under abnormal conditions.

Technical written procedures can be improved easily. In one unpublished study by a Department of Energy contractor, the procedures for weapons assembly were revised so that the required reading grade level was reduced from Grade 11 to Grade 8. This was done by using photographs and drawings, by reducing the number of words by about one-half, and by reducing the average number of words per sentence from 16 to 8.5. Although the new procedures were about 50% costlier than the standard procedures, this increase was far outweighed by the cost savings resulting from the reduc- tion in human-initiated defects.

It has been known for some years that a columnar type of format for tech- nical instructions is superior to a narrative format. In a study by Haney (1969), experienced technicians familiar with the usual narrative type of format were tested with a columnar type of format after a short practice session. They made one-third fewer errors with the latter format even though it was unfamiliar. With this study in mind, we rewrote part of the loss-of-coolant-accident (LOCA) procedures of one plant in a columnar for- mat (Swain, 1975). Figure 3-3 shows part of the procedures for immediate manual actions. The operating personnel and supervisors at the NPP liked the idea and adopted it. The parenthetical notating, e.g., (Ann-7, Yellow 3) in Step 1, refer to a type of location aid presented in Figures 3-11, 3-12, and 3-13. This format is discussed further in Chapter 15. Note that the format in Figure 3-3 could be improved by adding a column to tell the operator what to do if the anticipated result or feedback in the last column fails to materialize.

There are several advantages to the columnar format. First, many words can be omitted (conjunctions, articles, etc.), resulting in a substantial gain in the signal-to-noise ratio. Second, important information is easy to

This Handbook is written at the college level since it conveys concepts and principles rather than simple instructions.

3-30

Figure 3-3

Step Check

0 0 0

Indication or Item Manipulated

SI Initiation

REACTOR COOLANT PUMP IA, 1B, 1C, 1D (U-Grey 84-87)

GROUP A MONITOR LIGHTS (U-Blue-3)

Activity

Verify

TRIP 4 Switches

Result/Feedback

SAFETY INJECTION ACTIVATED

(Ann-7 Yellow 3)

TURBINE TRIP REACTOR TRIP (Ann-6 Grey 1)

Dark except: (1) N2-ACC 1880

(Yellow 4) (2) RHR HX2

1-0807 (White 3)

Verify

MOTE: This format is based on Swain (1975). Some revisions have been made for simplification.

Figure 3-3 Steps from columnar style format for NPP written procedures.

3-31

Job and Task Instructions

find--not buried in a sentence or paragraph where it might be overlooked. Third, the columnar format forces the writer to concentrate on what indica- tions are presented to the user, what decisions he has to make, and what control actions he has to take. Fourth, with provision for checking off each item as completed, errors of omission are much less likely. Fifth, since such procedures are more convenient to use, it is more likely that they will be used.

Apart from problems in content and format, one of the most serious problems with NPP emergency procedures is that often there are too many instructions that are not safety-relevant. Some of this safety-irrelevant information concerns the reduction of monetary loss. We observed a talk-through of emergency procedures at one plant by a highly skilled and experienced SS. He performed under ideal conditions--no real stress present and no deci- sion-making required, yet he barely managed to get through the procedures on a timely basis despite his exceptionally high skill level; there were too many tasks to perform within the allowed time. The money-saving in- structions could have been put in a later section of the procedures, so that all of his initial effort could be devoted to the really critical issue--the safety of the plant.

Since TMI, a number of reports have been written on the improvement of both normal and emergency procedures. Some of these reports are listed in Chap- ter 15, and a discussion is included on a new type of emergency operating procedure (EOP) called symptom-oriented EOPs, also called function-oriented EOPs. These procedures are keyed to symptoms resulting from an abnormal event rather than being keyed to systems, like the usual system-oriented EOPs, also called event-based EOPs. The latter is the usual type of emer- gency procedure found in a plant and is compatible with the type of train- ing that is oriented toward providing operators with an understanding of the various systems in an NPP. A problem with this type of procedure is that to select the correct system-oriented procedure, the operator must perform a certain amount of diagnosis, a task often performed poorly under the duress of an abnormal event. If the wrong diagnosis is made, the operator is likely to select and follow the wrong procedure, and critical time may be wasted before such an error is finally recognized.

In the symptom-oriented procedure, the idea is to direct operator actions to mitigate the consequences of an abnormal event. Special written in- structions are used, sometimes employing event trees or other graphic aids that make use of the patterns of control room stimuli associated with an abnormal event. The emphasis in these procedures is to ensure that safety- related operations are carried out on a timely basis.

As in other new approaches, there can be disadvantages as well as advan- tages. Symptom-oriented procedures can become quite detailed and perhaps unnecessarily so. One approach suggested by French investigators is to combine both types of procedures. That is, when an abnormal event occurs, the operator would immediately pick up a symptom-oriented master procedure that, through the use of a binary decision tree, would guide him to the correct set of system-oriented procedures. This approach appears to have

3-32 0

Stressors

the advantage of maintaining the use of the type of procedures the operator is used to, yet ensuring that he will select the correct set.

Two reports from the Idaho National Engineering Laboratory provide examples of the symptom-oriented procedure approach as applied to Westinghouse plants (vonHerrmann, 1983, and vonHerrmann, et al, 1983). The former re- port (p 4-4) lists some of the potential problems with the new approach, e.g., "If the critical functions are too general and/or the number of key symptoms too few, there is increased danger that the specified actions will not always be appropriate or that the operator will not be able to clearly determine the correct set of actions. . . . Thus, just as the pre-TMI event-oriented procedures were prone to problems of misdiagnosis or incor- rect response due to their over-specific nature, the alternative function- or symptom-oriented guidance is susceptible to the same problems in their attempts to generalize."

Since the concept of symptom-oriented procedures is still new, there is insufficient plant experience with them to estimate how the HEPs in Chapter 15 might change if such procedures became common. Our estimated HEPs re- lated to the use of written procedures are based on currently used system- oriented procedures.

Stressors

Stress can be psychological, physiological, or both. Often, it is not possible to differentiate between the two (Meister, 1981). We define a stressor as "any external or internal force that causes bodily or mental tension." This definition allows an optimum level of stress as well as nonoptimum levels. This concept of stress is different from the one given by Weiford (1974), who states that "stress appears to arise whenever there is a departure from optimum conditions which the organism is unable, or not easily able, to correct."

Our reaction to a stressor is the stress we feel. Stress per se is not undesirable. As we will show later, unless there is some stress, nothing is likely to be accomplished in a work situation. Through common usage, the word "stress" has acquired a negative connotation because we tend to think of situations with high, incapacitating levels of stress. This is the kind of stress we wish to avoid in NPP operations, whether it is psychological or physiological.

Psychological Stressors

Table 3-4 lists some psychological stressors. Some of these are clearly undesirable, but many are acceptable or even desirable in some limited amount.

Depending upon the level, psychological stress can be either disruptive or facilitative. Disruptive stress is the result of any stressor that threatens us, frightens us, worries us, angers us, or makes us uncertain,

3-33

Table 3-4

Table 3-4* Some psychological stressors

Suddenness of onset

Duration of stress

Task speed

Task load

High jeopardy risk

Threats (of failure, loss of job)

Monotonous, degrading, or meaningless work

Long, uneventful vigilance periods

Conflicts of motives about job performance

Reinforcement absent or negative

Sensory deprivation

Distractions (noise, glare, movement, flicker, color)

Inconsistent cueing

Source: Table 3-2

3-34 0

Stressors

so that our performance is worse than it would be normally. Because of the great differences among individuals, a stressor that causes disruptive stress in some people may not do so in others; even the same individual reacts differently to the same stressor at different times.

We can use the word "arousal" for facilitative stress--the result of any stressor that alerts us, prods us to action, thrills us, or makes us eager. As with the response to disruptive stressors, there are great individual differences in what is felt as facilitative stress. If a normally facili- tive stressor becomes too strong, it can have a disruptive effect. Also, a work situation that provides sufficient arousal for some people is seen by others as dull and monotonous. Other things being equal, the higher the levels of education and technical skills a person brings to a job, the more arousal he requires. There is no "exciting and challenging" work per se, and there is no "dull and unchallenging" work per se; these are the judg- ments of people who differ in their perceptions.

Dealing with stress, or even getting people to agree on what stress is, is not easy. Figure 3-4 shows that when one plots stress level against per- formance effectiveness, the plot is not a linear one. With extremely high levels of stress (as exemplified by life-threatening emergencies), the performance of most people will deteriorate drastically, especially if the onset of the stressor is sudden and the stressing situation-persists for long periods (Berkun et al, 1962). Even when an escape route is obvious, some people will freeze up. A few people, like Audie Murphy (the most decorated American soldier in World War II), will behave in an exemplary manner and do the right things at the right times. Regrettably, the Audie Murphy type of behavior is not universal under highly stressful situations (Berkun, 1964; and Ronan, 1953).

Figure 3-4 also indicates that at very low levels of stress, performance will not be optimum. There is not enough arousal to keep a person suffi- ciently alert to do a good job. Under these conditions, some people tend to drowse on the job, or their level of attention and job involvement is materially reduced.

The curve also shows that there is a level of stress at which performance is optimum. This optimum level of stress is difficult to define--it varies for different tasks and for different people. All we know is that the shape of the curve as shown is generally correct (Appley and Trumbull, 1967). This means that the tasks assigned to NPP personnel should be neither too boring nor so demanding that serious human errors are inevita- ble. With good ergonomics in the design of the plant and with properly skilled and practiced personnel, one has the best chance of avoiding both ends of the stress curve.

One of the difficulties in generating the stress models in Chapter 18 is that it is difficult to know just where on the stress curve certain unusual events in NPPs will fit. In our modeling of stress, we assume that a

3-35

Figure 3-4

HIGH

LOW I VERY MODERATELY EXTREMELY LOW HIGH HIGH

OPTIMUM

STRESS LEVEL

Figure 3-4 Hypothetical relationship of psychological stress and performance effectiveness.

_00

3-3r-

Stressors

design-basis LOCA* is near the far right of the curve. Such a large LOCA would meet several requirements for classification as a severe stressor. First, based on their training, operators recognize the possible conse- quences to the public and to the plant of an uncontrolled large LOCA. Second, it would be totally unexpected. Interviews with plant personnel and with NRC personnel indicate that no one thinks a design-basis LOCA will ever occur. If this well-founded opinion proved false and a design-basis LOCA did occur, the most likely reaction of the operators would be one of sheer disbelief. We call this the incredulity response. It has been observed in other work situations. For example, in one refinery, the first indication the control room operator had of a serious fire was that many alarms occurred and many instruments behaved abnormally. This operator's first response was to run upstairs and demand of an instrumentation tech- nician, "What's wrong with my instruments?" By the time he returned to the control room, it was too late to take action that might have reduced the loss due to the fire.

Finally, operators rarely practice responses to simulated large LOCAs (or other unusual events) after their formal training. Yet the only way to minimize the incredulity response is to provide frequent drills so that the operator wil he well practiced in responding to low-probability events. Unfortunately, this internal PSF, "State of Current Practice or Skill," is not at an optimum level for, most NPP personnel for dealing with unusual events.

We judge,-that unusual events that are less threatening to NPP personnel than a large LOCA should be placed around the moderately high stress part of the curve in Figure 3-4. Examples might include certain hot shutdown activities and other tasks that place time-stress on a person, but without disrupting factors such as fear, anger, and uncertainty.

There are two important problems for human reliability under high levels of stress: (1) man tends to revert to his populational stereotype and (2) he tends to perseverate among a very few response alternatives. When we are in an emergency and are experiencing stress, as evidenced by tightening of stomach and sphincter muscles, pounding of the heart, dryness of the mouth, etc., we tend to do "what comes naturally" or revert to our populational stereotype. This means that we will see things as we customarily see them and will respond in the way we are accustomed to responding. If some man-machine interface violates these ingrained habits of perception and response (e.g., in U.S. plants, an emergency switch that must be flipped up

A design-basis LOCA is a large LOCA in which one or more large coolant pipes suddenly experiences a guillotine-type break. It is also generally assumed that this break occurs when the emergency coolant inventory is at the lowest operating level allowed by NRC technical specifications. This combination of events has never occurred in the approximately 1,800 reac- tor-years of large NPPs in the combined experience of all large commercial and production reactors in the noncommunist world. In fact, a large LOCA has never occurred.

3-37

Stressors

for "off" or a manual valve that must be turned counterclockwise for the closed position), the probability of inappropriate action is extremely high. We estimate that, under highly stressful conditions, and even despite extensive training, the probability of human error in such cases ranges from .1 to 1.0 if the equipment violates a person's populational stereotype.

Whereas the problem of populational stereotyping can be solved by appro- priate human factors engineering, the problem of response perseveration can be solved only by a combination of good design, training, and practice. Response perseveration is the term for the tendency to make some incorrect response repeatedly (or to restrict one's behavior to a very limited number of responses even when they result in inappropriate system outcomes). This may be in response to some unusual but not especially stressful event, as when a motorist (even an engineer!) repeatedly pumps the gas pedal when trying to start a car with a flooded carburetor.

Perseverative behavior has been observed in people under the severe stress of combat (Grinker and Spiegel, 1963), under realistic experimental condi- tions (Berkun et al, 1962, p 27), under the much less stressful condition of trying to troubleshoot electronic equipment under time pressure (Bryan et al, 1956; and Bond, 1970), and under other conditions in which the correct path of behavior is not clear-cut. Ambiguity resulting in response perseveration can arise from inadequate presentation of information (a design problem), from lack of skills to process adequate information (a training problem), or from inability to recall and use the appropriate skills because of lack of continuing practice (also a training problem).

The low end of the stress curve (see Figure 3-4) has important implications for monitoring tasks. If a control room operator is not sufficiently aroused, he is less likely to detect deviations from normal before they result in some annunciated indications. If an operator's first indication of some unusual plant condition is an annunciated signal, he may not always be able to correct the situation on a timely basis (Seminara et al, 1976). This is in part a design problem, but it is also a problem of ineffective monitoring that develops when the operator is not experiencing enough signals to maintain arousal or alertness. This loss of alertness is called the vigilance effect (Figure 3-5). This phenomenon was noted in World War II by the British, who discovered that the maximum time a shipboard lookout could be kept on duty effectively was about one-half hour. After that, the probability of his detecting an enemy submarine or aircraft was unaccept- ably low even though his own life and those of his shipmates were at stake. Later research verified the vigilance effect and found that it applied also to some industrial inspection tasks in which the involvement of the in- spector was essentially passive, such as in looking for defects when the number of actual defects was very low (one or fewer defects per 100 items) (Harris and Chaney, 1967, 1969; McCornack, 1961; and Fox, 1975).

In WASH-1400, we stated that the level of activity in a control room was usually such that the vigilance effect, or the low end of the stress surve, did not apply. However, subsequent information we have gathered in obser- vations of U.S. and European NPPs and from the Electric Power Research

3-38

Figure 3-5

100%

z w

U- U- w

II I I

0 1/2 1 1-1/2 2

TIME (hours)

Figure 3-5 Vigilance effect for passive tasks with low signal rate.

3-39

Stressors

Institute (EPRI) Review (Seminara et al, 1976, p 18-6 to 18-9) indicates that, at least in some plants and especially during the evening and night shifts, operators often consider their work to be very dull, monotonous, and unchallenging. At one European plant, there was even a request to install a television set so that the operators could watch television programs when not otherwise busy. (The request was turned down.) Our modeling considers the effects of nonarousal in the low estimates of proba- bilities of detection of unannunciated deviant indications.

In summary, the effect of psychological stress in NPPs is a serious prob- lem. It can be addressed effectively through a combination of sound equip- ment design, frequent practice, and responsible supervision.

Physiological Stressors

Table 3-5 lists some physiological stressors. As stated, all of these stressors would be disruptive. Earlier in this chapter, we addressed the effects of fatigue and disruption of circadian rhythm ("Work Hours and Work breaks" and "Shift Rotation and Night Work"). The special problem of working in a radiation environment is discussed in Chapter 17, "Stress." Few of the other stressors constitute serious problems in NPP operations. However, discomfort can be a highly disruptive PSF for certain maintenance tasks in which awkward positions must be assumed for access to components. Errors, especially errors of omission, can be expected to increase, partic- ularly if such discomfort is combined with temperature extremes, as is sometimes the case.

Movement constriction and lack of physical exercise is a problem primarily in the control room. However, it is common practice for operators to walk around frequently not only to monitor displays but probably also just to get up and move around. Some designers of NPPs have misconstrued this small problem of movement constriction and have argued that a standing operator is more effective than a seated operator. A sitdown console concept was changed to a standing one because of this mistaken belief. What the designers failed to consider were the PSFs of fatigue and discom- fort. Furthermore, when operators need to sit, they will sit, even if this means sitting on consoles or other places where inadvertent manipulation of controls could result.

With emphasis on the use of cathode-ray tubes (CRTs) as primary displays in NPP control rooms (Chapter 11), movement constriction is a potential but controllable problem. Studies of workers at video display terminals show that their constrained postures often result in physical impairments that can be presumed to reduce performance effectiveness (Huenting et al, 1982). In our opinion, a workplace design that comforms to anthropomorphic stan- dards as stated in MIL-STD-1472C can prevent physical tension, and with the new manning requirements for NPP control rooms described earlier, there is no need for one person to sit in front of visual display units for periods of time that could result in visual or other physical fatigue.

One final physiological topic is mentioned only because many people ask about it. This is the idea that one's "biorhythm" affects performance and

3-40

Table 3-5

Table 3-5. Some physiological stressors

Duration of stress

Fatigue

Pain or discomfort

Hunger or thirst

Temperature extremes

Radiation

G-force extremes

Atmospheric pressure extremes

Oxygen insufficiency

Vibration

Movement constriction

Lack of physical exercise

Disruption of circadian rhythm

Source: Table 3-2

3-41

Internal PSFs

that each operator's biorhythm should be determined so that he is not assigned to critical or dangerous work on biorhythmically critical days. Extensive reviews of biorhythm theory (Wolcott et al, 1977; McConnell, 1978; and others) indicate that while there are certain psychophysiological rhythms, the 23-day physical cycle, 28-day emotional or sensitivity cycle, and 33-day intellectual cycle postulated by this theory are not supported by any reliable evidence. However, as stated earlier in this chapter, there is evidence to suggest that the individual circadian cycles of opera- tors do affect their performances. Humans require from 4 days to a week to adjust to shift changes that materially disrupt their established sleep schedules. The usual practice in NPPs is to rotate shift changes on a weekly basis; this practice does not allow sufficient time for recovery.

Summary of Human Reaction to Stress

When overburdened by a situation, people respond in one or more of the ways listed below (Edwards and Lees, 1973, p 20):

Queueing - delaying some responses during overload, with the intention of responding at a later time.

Omission - ignoring information or actions that are considered rela- tively unimportant.

Gross Discrimination - responding to gross aspects of signals and ignoring finer aspects; e.g., noting that the water level in the sump has risen but not noting the extent of the change.

Errors - processing information incorrectly.

Escape from Task - physical or mental withdrawal.

As can readily be seen, some of these responses are more detrimental than others in their consequences for a man-machine system.

Internal PSFs

Table 3-6 lists some of the internal factors of the individual in a man- machine system. Some of these PSFs are outside the control of supervision and management, but most are either the direct responsibility of the util- ity or can be positively influenced by utility policy.

In WASH-1400 (p 111-64) we judged that the level of training of NPP per- sonnel was outstanding. Based on our subsequent studies and on the EPRI Review (pp 18-9 to 18-14), it is apparent that this earlier judgment should be modified. We still believe that the training of NPP control room oper- ators is generally good, but there is much room for improvement (Kemeny, 1979). Moreover, another EPRI report indicates that the training of main- tenance personnel is quite deficient (Seminara, Parsons, et al, 1979). As was the case in the training of military electronics personnel in the

3-42

Table 3-6

Table 3-6. Some individual (organismic) factors

Previous training/experience

State of current practice or skill

Personality and intelligence variables

Motivation and attitudes

Knowledge of required performance standards

Sex differences

Physical condition

Attitudes based on influence of family and other outside persons or agencies

Group identifications

Source: Table 3-2

3-43

Internal PSFs

1950s, some NPP training courses include much theory that may not be neces- sary for plant personnel who perform operator, maintenance, or other hands- on activities. With limited amounts of time for training, and with costs S between $100,000 and $200,000 to train each operator, the elimination of job-irrelevant training from the syllabus would allow more time for opera- tionally oriented content. It is apparent from the EPRI reports that the training of NPP personnel needs a thorough reevaluation.

As mentioned earlier in this chapter ("Manning Parameters"), the NRC has taken the position that the training for NPP personnel must be based on a task analysis (NUREG-0660, 1980, Task I.A.2-2). As this requirement be- comes implemented, the job-relevance of training course material for NPP operators will increase, resulting in higher levels of useful skills and less instruction time being taken up with unnecessary material. A recent report describes the purpose of the task analysis of U.S. PWR licensed operators: ". . . to produce detailed task information for industry use in developing valid, performance-based training programs" (Analysis & Tech- nology, 1982). For more detailed information on the application of task analysis techniques to determining NPP training course content, see, Andersson et al (1979) and Wirstad and Andersson (1980).

The training an operator receives includes practice in a training simulator of responses to transients, LOCAs, and other abnormal events. This train- ing is very valuable, but such initial practice of simulated emergencies is not sufficient to maintain the operator's skills in coping with unusual events. He needs to have sufficient continuing practice in safety-related tasks for adequate implementation of the PSF of "state of current practice or skill" of these tasks. In Figure 3-6, we postulate the general shape of the curve for loss of ability to cope with emergencies in the absence of

practice after the initial training an operator received (the solid line) compared with continuing improvement that takes place with periodic prac- tice (the dotted line). The time intervals for periodic practice by NPP personnel should be determined empirically, and the ratio of time periods spent in training simulators on requalification exercises to the time spent in other kinds of on-site exercises should also be determined. Periodic practice should include coping with abnormal events without all normally displayed information or controls being available. As noted in Chapter 12, this type of partial panel practice will be especially necessary for future plants in which operators will place primary reliance on computer-driven displays and keyboards.

In the draft Handbook, we noted definite concern about continuing practice provisions for safety-related tasks. Since then, there has been a major improvement. Requalification of reactor operators occurs annually rather than every 2 years, and the requalification exercises typically include some practice of transients and LOCAs in training simulators. However, there are still problems, as voiced by operators we have interviewed re- cently. One problem is that for some operators, the training simulator panels may be so different from the panels in their plants that a major part of their practice time may be spent just in learning how to operate the simulator. Another problem is that there is time for practice on only

3-44

Figure 3-6

END OF FORMAL TRAINING ASSUMING PRACTICE OF EMERGENCIES

HIGH

EFFECTIVE COPING WITH EMERGENCIES

LOW

- .j WITH PRACTICE OF SIMULATED EMERGENCIES

NO FURTHER /- PRACTICE

TIME

VERTICAL ARROWS REPRESENT PRACTICE SESSIONS:

SIMULATOR

DRILLS ON SITE TALK-THROUGHS

CHALLENGES ("WHAT IF?")

Figure 3-6 Hypothetical effects of practice and no practice on maintenance of emergency skills.

3-45

Internal PSFs

a relatively few abnormal events. In our opinion, a third major problem is that annual practice on a training simulator (even if it represented a sizeable sample of unusual events) is not sufficient to keep an operator's skill at a very high level. We think the current situation is better than the solid line in Figure 3-6, but it falls far short of the dotted line in the figure.

Because of the considerable expense of simulator practice and the extra manning required because of the annual requalification requirement, a less expensive and more convenient form of "simulation" is needed to supplement the training simulator exercises. This simulation could consist in large part of talk-throughs and walk-throughs of emergencies and other unusual events (see Chapter 4 for a description of these techniques). As noted in WASH-1400 (p 111-64), we made an informal test using talk-throughs, and "it was found that operators interviewed could explain in general terms what they should do in postulated emergency situations, but they did not always appear to be sure of the locations of switches and readings on displays relevant to manual backup actions required in the event of failure of automatic safeguards systems. . .. the lack of ability to 'talk through! appropriate procedures without hesitation or indecision potentially indi- cates lack of a clear plan of action should such emergency situations occur. Based on the above findings, relatively high error rates were consequently assigned to operator actions required soon after the onset of a major emergency such as a large LOCA."

Our conservative estimates of operator ability to respond properly under highly stressful conditions could be modified upward if talk-throughs of these tasks were practiced frequently. Periodic testing by the onsite NRC inspectors using the talk-through method would provide sufficient motiva- tion for a plant to ensure that its personnel would practice coping with emergencies.

Personality and intelligence variables obviously influence human reliabil- ity in NPPs, but it is not possible at this time to quantify these effects. Although the level of intelligence is known to affect performance, it has yet to be demonstrated in studies of NPP personnel what levels are required for various jobs. Presumably, if a person can complete the required train- ing curriculum successfully, he has sufficient intelligence to do the job. The training courses in themselves should eliminate those who do not have sufficient background to perform NPP tasks.

A different problem exists concerning the variability of personality. It would be desirable if tests and interview formats were available that could eliminate those who could not perform well under the stress of an abnormal event, those who might be subject to bribery or coercion to perform un- authorized actions, and those whose personality traits would have a nega- tive impact on the cohesiveness and cooperation of personnel who must work together. At this time, these are only goals. There are many tests of emotional stability (as cataloged by Matarazzo, 1972), but none has been validated in the NPP environment. Frank et al (1981) describe some of the practical problems in identifying persons who might manifest undesirable

3-46

Internal PSFs

personality characteristics in the NPP environment. In general, it can be said that personality assessment tests have very severe limitations for NPP use.

Although personality and intelligence variables do influence human relia-

bility, it is our judgment that they are not nearly as important as other internal PSFs, especially those related to practice of skills. In part, this judgment is based on the process of selection and elimination that occurs in the NPP training schools and work environment. We believe that the most effort should be made to improve those PSFs over which control is possible and which have been demonstrated to have major effects on human performance.

The motivation and attitudes of the individual in an NPP obviously have considerable influence on how well he performs. From experience, it is known that an ergonomically engineered work situation plays an important role in the workers' acceptance of and enthusiasm for their work. Thus, application of sound human factors practices to NPP design and work opera- tions should have a substantial beneficial effect on operator motivation and attitudes. Another aspect of motivation and attitudes is the training program. If the course material is truly relevant to job performance, these personnel will be more likely to have confidence in the tasks assigned them and to feel that the time and effort they have spent was worthwhile. But if the training includes advanced nuclear theory that is not necessary for job performance, they may feel that their skills and abilities are not being used. This could have a negative effect on their motivation.

The internal PSFs of stress and the knowledge of required performance standards listed in Table 3-6 have been covered earlier. Sex differences are of current interest in the U.S. because women are being trained as ROs. We have already mentioned that at least for the kinds of tasks done by AOs, the lesser average physical strength of females does not seem to pose a problem. One major concern that has been expressed in our interviews with U.S. and foreign NPP personnel (all male) is whether women can perform adequately under high levels of stress. Some strongly believe that it is "a well-known fact" that women will tend to "fall apart and cry when the going gets tough." However, we have found no documentation to support such beliefs; certainly there have been no scientific studies to support these convictions. It is more likely that if women, as well as men, are given appropriate practice in coping with emergency situations, both in simula- tors and in plant drills and talk-throughs, no measurable sex differences will be found.

The last three PSFs from Table 3-6, the operator's physical condition, his

attitudes based on outside influences, and his identification or affilia- tion with various groups, deal with influences that are not under the control of a utility but are listed to show that the responsibility of a utility for the performance of its personnel does have limitations.

3-47

Ergonomics Problems in NPPs

Some Ergonomics Problems in Operating NPPs

To illustrate some effects of ergonomic design on the reliability of human actions in NPPs, some examples of poor design are given along with sugges- tions for design improvements that might increase human reliability. The suggested design improvements are not intended to be prescriptive; better design options for any given plant may be available. Our intent is to show that, in at least some cases, relatively simple improvements in the design of man-machine interfaces in NPPs can result in material improvement in human reliability. We also mention some other design options being studied for future plants.

Seminara et al (1976), Raudenbush (1971, 1973), and Swain (1975) present detailed expositions of human factors problems in the design of control rooms. Most of the examples in this section are based on our observations and interviews in several U.S. and foreign plants. Others are taken from the references cited.

Some of the ergonomics problems observed were these:

(1) Poor design and layout of controls and displays.

(2) Poor labeling of controls and displays in the control room.

(3) Inadequate indications of plant status.

(4) Presentation of nonessential information

(5) Inadequate labeling and status indications outside the control room.

Poor Design and Layout of Controls and Displays

Typically, the design and layout of controls and displays in online NPPs are not in accord with recognized human factors practices. Controls and displays are not always grouped by function, nor are the existing func- tional groups always clearly delineated. At many plants, the operators have applied colored tape to the panels in an attempt to delineate func- tional groups. (For a formal method of using tapes on existing panels, see Seminara, Eckert, et al, 1979; and Seminara et al, 1980.) At other plants, the operators have been prevented from applying such markings by management policies that place aesthetics above the needs of the operators (Seminara et al, 1977). The lack of functional grouping may partially explain the tendency of operators to rely on one display instead of cross-checking with other instruments that might indicate whether the first display is operat- ing properly. There are several LERs involving operators relying on a chart recorder indication and performing inappropriate actions because of a stuck pen. In some incidents, operators apparently relied exclusively on one display when other displays would have provided information pertaining to the appropriate actions.

3-48

Erogonomics Problems in NPPs

In many U.S. plants, the control room is roughly divided into three areas: Engineered Safety Feature (ESF) panels, Reactor Control panels, and Balance of Plant panels. This does not mean that all the displays and controls used in coping with safety-related incidents are located on one panel. For example, in coping with a LOCA, the use of all three panel areas is re- quired. The layout of controls and displays only partially supports the required operator activities. In one PWR, rod control is hindered by the location of the display far off to one side of its related control. This forces the operator to assume an awkward stance to read the display while trying to operate the control.

Several informal link analyses of control room activities have been per- formed (Swain, 1975; "Human Reliability Analysis," Section 6.1, WASH-1400, 1975; Seminara et al, 1977; and Seminara, Eckert, et al, 1979). It is clear that the layout of NPP control rooms does not match the operating needs of control room personnel very well. Figure 3-7 shows a formal link analysis for a boiling water reactor (BWR) control room operator for a 23-minute period, and Figure 3-8 shows an expanded link analysis for an 8-hour shift for the same operator. The report from which these figures are taken (Axelsson and Lundberg, 1975) notes that the layout of the con- trol room makes for considerable travel time and some reluctance on the part of operators to perform all display checking functions on a timely basis.

Examples from U.S. NPPs also show a general architectural layout that increases the probabilities of errors of both omission and commission. In several plants, free-standing consoles block the operators' views of ver- tical panels. In one plant, some important indications have been mounted on back panels that are not in the operator's immediate line of sight. A mirror has been installed to permit the operator to check indicator lights on a back panel without leaving the main control area. At many plants, important indicators (e.g., chart recorders) are mounted next to the floor. The possibility for reading errors is increased by this type of design, as well as reluctance of the operators to read the indicators. Important switches are often found at the edges of sloping panels and can be inadver- tently manipulated by operators. Chapter 13 discusses this problem and offers some solutions.

Perhaps the most serious deviation from accepted ergonomics practices in the design of NPPs is the use of mirror-imaging of panels for a two-reactor control room. This practice consists of reversing the layout of the dis- plays and controls from one reactor to the other. (The mirror-imaging does not go all the way and reverse the components within a display or control.) Mirror-imaging aggravates the problems of inadequate panel layouts. Even highly experienced operators reported moments of confusion in this kind of work situation, though they are fully trained and experienced on both layouts.

Except for the mirror-imaging, the above deviations from good ergonomics practices are typical of operating U.S. plants, and the estimated HEPs and human performance models in the Handbook are based on current design. With

3-49

Figure 3-7

I I I I I I I I I I I I

2,A

I I I I 1l l l l l l l

I~~1 2,6I I

yo 11Th /

I I I I

ACTIVITY KEY:

0. NO ACTIVITY 1. MONITORING 2. MANUAL ACTIVITY 3. WRITING OR CALCULATING 4. STUDYING WRITTEN INSTRUCTIONS 5. CANCELING SOUND 6. CANCELING BLINKING OF LIGHT 7. TALKING (FACE-TO-FACE DISCUSSION) 8. PHONE CALLS (ARROW INDICATES

INCOMING OR OUTGOING)

9. GO AND GET INSTRUCTIONS, OR PUT THEM BACK

TIME KEY:

6:47 - 6:52

6:52 - 6:55 6:55 - 6:57 6:57 - 6:60 7:00 - 7:10

LETTER KEY:

A - SHIFT SUPERVISOR T - WALK K - REACTOR ENGINEER

Figure 3-7 A 23-minute link analysis of reactor engineer

activities in a Swedish BWR.

3-50

Figure 3-8

1 01 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 1

I I' II I I I

Figure 3-8 An 8-hour link analysis of reactor engineer activities in a Swedish BWR.

3-51

Ergonomics Problems in NPPs

better delineation of functional groups using a "paint, label, and tape" approach (Coley, 1982, p 145), decreases in probabilities of selection errors can be expected. (See Chapter 13 for some differences in HEPs with and without such grouping.) In general, there are no data to show the extent of improvement that could be realized with good ergonomics prac- tices, but there is little doubt that improvement would result. Until such data are available, for PRA, an analyst may select HEPs smaller than the nominal HEPs in this Handbook when he finds a design that conforms to the ergonomics practices described in NUREG-0700. If a plant conforms to all the practices recommended in that document, that plant is better than the industry average, on which our nominal HEPs are based. On the other hand, for tasks that would be affected by extremely poor ergonomics design prac- tice, such as mirror-imaging of panels, the analyst should select HEPs higher than the nominal HEPs in the Handbook for the tasks in question.

Poor Labeling of Controls and Displays in the Control Room

The labels and legends used in NPPs are not always clear. In some plants, labels on controls and displays are taken from construction drawings with no attempt to develop labels useful to operating personnel. For example, consider the labels of the switches on the ESF portion of the panel shown in Figure 3-9. The switch labels describe their functions, which is help- ful to the operator. However, the numbers differ so little that confusion among them is quite possible. The five switches immediately above these two rows (not shown in the figure) are numbered MOV-1864A, MOV-1885A, MOV-1885C, MOV-1885B, and MOV-1864B. Obviously, the numbers assigned to the switches provide no cues as to their locations on the panels.

In other plants, the possibilities for confusion and difficulty in locating controls and displays relevant to safety activities are even greater. Figure 3-10 represents the lower ESF panel at one PWR. Note the labels and how difficult it would be to find the correct switch if it were not used frequently. We observed a highly skilled operator go through the LOCA procedures on the dynamic simulator that had panels identical to these NPP panels. Even this skilled operator had trouble locating certain switches, and he described at length the greater problems experienced by less skilled operators.

These difficulties prompted us to devise location aids similar to the matrix used on road maps (Swain, 1975). Figure 3-11 shows how this loca- tion aid would be applied to the lower (L) ESF panel shown in Figure 3-10. Figure 3-12 shows the same scheme for the upper (U) ESF panel, and Figure 3-13 shows the location aid applied to an annunciator (ANN) panel. (This annunciator panel is labeled 1; the others would be labeled 2, 3, etc.) The five colors chosen can be discriminated even by persons with poor color vision (Baker and Grether, 1954). They are yellow (33538), blue (35231), white (37875), black (37038), and gray (36173). The five-digit numbers are the Federal Standard 595A (1974) identification numbers for the preferred shades of lusterless paint (used to avoid glare).

3-52

Figure 3-9

@00( ( 000ý0 00 00

L-J BORON INS TK OUTLET BORON INS TIC INLET BORON INS TIC OUTLET

RECIRC ISO WA RECIRC ISO WC RECIRC ISO WR

LO HEAD SI PUMP A LO HEAD SI PUMP B

I - S1 - P -IA TV-1884A TV-1884C TV-1884B I -l- P-B

0 0 0 0 0 0 0

L0 HEAD SI PP B DISC ISO VV LO HEAD SI PP A SUMP SUCT VV LO HEAD SI PP A DISC ISO VV LO HEAD SI PP A.B RWST SUCT VV MO - 1863B LO HEAD SI PP B S MP SUCT VW

MOV - 1860A MOV- 1863A MOV 1862 ISO DISC FROM HI HEAD SUCT MOV- 1860B

KEY: The letters G. Y. and R refer to indicator lamps with GREEN, YELLOW, and RED filters, respectively.

Figure 3-9 MOV switches on part of the ESF panel at the PWR used in the WASH-1400 Study. (The sketch is based on Figure III 6-2 from WASH-1400.)

3-53

Figure 3-10

MS LINE MS LINE MS LINE MS LINE SI PMP IA SI PUMP IB SI PMPS ISOL - - ISOL- BLDN VLV BLDN VLV SUCT FROM SUCTFROM RWST RECIRC

TEST I-P A TEST LIP B TEST LP C TEST IP D Sj j] SGRWT IARWST STOP 0~G 0 ()OA E 00 G 0 0 0 0 l0o loo 0_0 0_o U

CLOS F SET

MAIN STM MAIN STM MAIN STM MAIN STM BLDN VLV BLDN VLV SI PMP IA TO S1 PMP 18 TO SI PMPS 'LINE ORN LINE RN *LINE RN LINE ORNS IBL L L PABC & LI A 8 C & D COLD LEGS

LP A0Ps LP8 IC LP0[i I SG 181 COLD LEG ICOLD LEG DISCH ISOI. 0 0 Q0 6Pb 0 0 0o0 0 l 0 0i

S1 PMP IA TO SI PMP 18 TO RHRP IA FW LINE FW LINE. FW LINE FW LINE- BLDN VLV 8LDN VLV r-LP A & D. LP A & D. TEST LINE

ISOL LP A ISOL LP 8 SOL LP C ISO1 P D ISG IC SGIC HOT LEGi HOT LEG ISOL 0 00 0~ 0 0 0 00 0 0 0 01 0 0 0 _0H HH L~-KEY LOCK- KEY 100 AUX FW AUX FW AUX FW LP A LP DCOL LEG[]MHR 'T O [TM [ DR- BLON VLV BLDN VI-V COLD LEG COLD LEG ESTD LIEG

STOP IA STOP I0 STOP SG ID H I TESTISOL TES I EST LINEL

Problem: Quickly find Switch 8923B

Figure 3-10 Part of the lower ESF panel in a PWR.

3-54

Figure 3-11,

2 3 4 5 6 79 .... 42

MS LINE MS LINE MS LINE MS LINE SI IMP EU SI PUMP 10 SI PMPSBLON VLV ALDN Vl DUCT FROM SUC R0DM RWS RECIRCTEST LPA TCSTCI JP8 EST LPC TEST LP IGD S ARS

0 00 0 0 0 00 00 00 00 00 YELLOW N* rM~ SA M~ S3 OSS 508 A2A 820~ YELLOWCLOSE ( )RESE T

MAIN STM MAIN SIM MAIN STM MAIN STM SLDN VLV LDN VLV 5S PMP IA TO 51 PMP 1l TO S1 PMPSLN A LI NPA O &A LPAAC&D COLD LEGS

BLUE r MOD SIT MO"• •I[ rSOT2 850002-SIIDA AD BLUE

S1 PMP IA TO SI PMP IB TO RHP 1A FW LINE W LE W LINE N W LINE IN VLS L LDN VLV LI A AS 0 PALS ET LINEISOL LP A ISL LP ISOL LP C ISOL LPI SUIC SGIC HOT LE OF LEG ISOL

WHITE -* IM ] 1 EE- lrS-Al I050-0l -- • I--1 1 000 -• WHITE

AUX FW AUX FW AUX FW LP A LPD CO C jSM S __So ADS SM AD SLoNoALAoALSo AL COLDo LoED C LD LED

92 3 4 5 6 7 8 .... 42

Which is easier to find?

(1) Switch 8923B. or

(2) Switch 8923B (L-Yellow 8) (Note: L = Lower Panel)

Figure 3-11 Location aids for lower ESF panel in a PWR.

3-55

Figure 3-12

.1 2 3 4 5 6 7 8 9 .... 42

YELLOW -

BLUE-

WHITE

BLACK

GRAY -

1 2 3 4 5 6 7 .9 42

Problem: Quickly find FW I80L

(U-Black-4) (Note: U = Upper Panel)

Figure 3-12 Location aids for upper ESF panel in a PWR.

3-56

Figure 3-13

YELLOW

BLUE

WHITE -1"-

BLACK --

GRAY--

2 3 4 5

o 0

ACCUMULATOR TANK IC LEVEL

HIGH LOW

0 0 0 0 0 0 0 0

<--YELLOW

BLUE

-- WHITE

-- BLACK

-'i--GRAY

ICB12/1

Problem: Quickly find ANN-1 White 4 when it may be one of several blinking annunciator (ANN) indicators on several ANN panels.

Figure 3-13 Location aids for annunciator panel #1.

3-57

Ergonomics Problems in NPPs

As an example of how the location aids would be used, assume that the operator had to locate Switch 8923B. Figure 3-10 shows how time-consuming such a search can be (only one-fourth of the panel is included in the drawing). For Figure 3-11, the written instructions would tell the opera- tor to "turn on 8923B (L-Yellow 8)." (The L stands for the "Lower" panel.) One can readily see how this system would help in locating controls. See Figure 3-3 for an example of a written format that incorporates this type of location aid.

In some plants, commonly accepted labeling conventions are violated. For example, the line drawing in Figure 3-14 shows that the labeling of the six monitor panels on the ESF board in one PWR is not in accordance with the reading habits of the users: most of us read left to right and top to bottom. The figure shows the existing labeling, with the suggested label- ing in parentheses. Correction of this type of poor labeling would involve minimal expense.

In another plant, on one panel there-are four different label abbreviations used for "Pump." They are PU, PP, PMP, and PUMP. This may seem minor, but when one multiplies this one source of confusion by many, the information- handling load on the operators increases materially.

Populational stereotypes are sometimes violated in the design of labels for displays. For example, in one PWR, there are side-by-side displays showing Refueling Water Storage Tank (RWST) level and Containment Sump Level (Fig- ure 3-15). The indication for RWST level agrees with our populational stereotype: as the water level falls in the RWST, the pointer drops and shows a decreasing level. However, for the containment sump level display, 0 the lowest level of water in the sump (4 inches) is shown by the top indi-

cator and the highest level (205 inches) by the bottom indicator. This indication is the reverse of what one would expect.

An extreme example of mislabeling occurs in one plant in the labeling of switches for the primary coolant loops on the reactor control board (Figure 3-16). Switches for controlling valves associated with primary coolant loop A (labeled RC A HL, RC A BYP, and RC A CL) are logically labeled A (e.g., 1-8001A), but switches for loop B are marked D (e.g., 1-8001D), those for loop C are marked B, and those for D are marked C. The operators have attempted to cope with this by memorizing a mnemonic aid: All Dogs Bite Cats.

The major effects of inadequate labeling are increases in the perceptual and interpretational requirements of the job, with the result that more errors occur and longer times are required to do the right things. This is one area in which substantial improvement in human reliability could be achieved in existing plants with relatively little expense. There are no data to enable us to estimate precisely the HEPs associated with the above kinds of poor ergonomic design practices for displays. Chapter 11 presents some guidelines. In general, for displays that clearly violate accepted ergonomic practices (e.g., those in NUREG-0700), the analyst may select HEPs higher than the nominal HEPs listed in the Handbook for these dis- plays. ___

3-58

Figure 3-14

B (A) A (B) G (E)

I I I I I I I I I I I i i

C D

Note: Spacing between panels proportional but not to scaleH M I I I I I I II I I I Ib ntoscl

Monitor Panels: Current and (Suggested) Labeling

Figure 3-14 Violation of reading stereotype in labeling of monitor panels.

3-59

Figure 3-15

-35

-30

-25

F -20 TWC -15

-10

-5

-0 205 in1

RWST LEVEL

1 LI-920

CONTAINMENT SUMP LEVEL

I 1LI-940

Figure 3-15 Two adjacent displays with conflicting representations of the real world.

3-60

Figure 3-16

RC A HL STOP VLV 0 0 1i-8001A.

RC A BYP STOP VLV 0 0 1-8003A

RC A CL STOP VLV 0 0

RCBHL STOP VLV 0 0 11-8001D,

RC BBYP STOP VLV 0 0 [1-8003DJ

RC B CL STOP VLV 0 0 1-S002D

RC C HL STOP VLV 0 0 11-8001B

RC C BYP STOP VLV 0 0 [1-8003B

RC C CL STOP VLV 00 11-8002B

RC D HL STOP VLV 0 0 [1-8001CI

RC D BYP STOP VLV 0 0 11-8003C

RC D CL STOP VLV 0 0 *1-8002C

Green light is at left; red light at right.

Figure 3-16 Reactor control switches for the four primary

coolant loops.

3-61

Ergonomics Problems in NPPs

Inadequate Indications of Plant Status

Often the information conveyed by indicators of plant status requires excessive interpretation by the operators. At one plant, monitor panels were designed with the excellent intent that all the indicators on a panel would be illuminated or none would be illuminated when the monitored func- tions were in their proper modes so that the operator could quickly detect a deviant state of any individual function. However, changes were made, and the operators had to learn that for condition so-and-so, all the indi- cators on one panel should be illuminated--with a few exceptions--and all the indicators on another panel should be dark--again, with a few excep- tions. These exceptions would change for different conditions. The memorization task for the operators proved impossible. Interviews with these operators established that they temporarily memorize all the indi- cations to pass certification or recertification examinations and then forget them.

The 1979 TMI incident illustrates several problems related to the require- ment for excessive operator interpretation (Kemeny, 1979). One such prob- lem is related to the closing of the pressure-operated relief valve on the pressurizer. The problem occurred because an amber light in the control room comes on when an automatic signal is sent to a solenoid to cause it to close the valve. The light does not monitor the position of the valve. The operator is supposed to remember that the amber light may not mean that the valve has closed (Rubinstein, 1979). Another problem occurred because there was no direct indication of flow in the auxiliary feedwater system. "The indicator the operator was supposed to watch to see if there was auxfeed was a pressure gauge on the discharge header. This is midleading if the auxfeed valve is closed because the pressure would be high and yet the flow is zero" (Sugarman, 1979). Again, too much interpretation is required for high levels of human reliability.

Considerable use is made of color coding in NPPs, including the use of the colors red and green. An obvious potential problem stems from the most common form of so-called color blindness, the red-green confusion experi- enced by about 7% of American males and somewhat less than 1% of American females (Hilgard et al, 1971, p 119). However, this problem is recognized, and utilities have color-blindness tests to screen for persons with this type of visual limitation. To the extent that these tests are properly employed, we can conclude that color vision weakness is not a material problem in NPPs. For other process industries that may not employ such tests, forms of coding not relying on color discrimination would be useful.

There are, however, other problems related to the use of color coding in NPPs. For example, in most U.S. NPPs, the color coding of status lamps that follows the convention established in fossil fuel plants rather than ergonomics guidelines. In connection with NPP valves and switches, the color red routinely indicates the open status of a valve or the closed position of a switch; i.e., in both cases red indicates flow. Green in- dicates the closed position of a valve and the open position of a switch. In the early years of fossil fuel plant development, these colors were

3-62

Ergonomics Problems in NPPs

selected because red was supposed to indicate the potential for danger when a switch was closed or a valve was open, and green indicated a safe con- dition. The use of these colors today is contrary to our populational stereotype of green for go and red for no-go.

Other colors are also used in NPPs but without uniformity. In some cases, yellow is used for cautionary indications; sometimes blue is used for caution. Sometimes the same color is used for different indications within the same plant. At one plant, a blue lamp was substituted for a red lamp to indicate an open valve. Added to these problems is poor maintenance of color quality; at one plant formerly red or green filters had become so faded that they looked white, a color that had an entirely different mean- ing at that plant. In one case we observed a formerly red lamp that now appeared dull green, providing a serious potential for error. The long- term acceptance by operating personnel of this defect illustrated their lack of understanding of human reliability, especially under stressful conditions.

These examples show that there are problems even with the simple use of color for status indications. More central to the operator's problem is the fact that the typical NPP status lamp does not indicate what the status of the- monitored item should be when the plant is in the power-generating mode. This lack of information materially reduces the probability that an operator will detect unannunciated deviations from normal operating condi- tions.

A very practical method of presenting both the actual and the normal status of a control is used on shipboard reactors. The status lamps are shape- coded as well as color-coded. Two shapes are used: bars and circles. A pair of horizontal bars indicates that a valve is closed or a switch is open (the no-flow position). A circle indicates that a valve is open or a switch is closed (the flow position). These indicators are backlighted in either green or red, with green indicating the normal state and red the nonnormal state. Thus, if a valve is open or a switch is closed and this is the normal state for that item, the associated circle is illuminated green. For the abnormal state, the circle would be illuminated in red. In most cases, a red indication would be the equivalent of presently used tags.

This shipboard philosophy is used at one BWR we have visited. When the reactor is in the normal power-generating mode, a red indication means that the status of the indicator is not normal for that condition of the reac- tor, and a green indicator means that the status is normal. The bars and circles give the actual status. With this system, the operators know that the red and green cues pertain only to the normal power-generating mode; for other conditions of the reactor, they have to depend on the bars and circles only and ignore the color cues. With new plants, even this limita- tion could be overcome with computer technology. Theoretically, it would be possible for a computer-based system to sense the plant's state, along with that of any given valve, switch, etc., and to indicate via the red or green light whether the state of the component were the appropriate state

3-63

Ergonomics Problems in NPPs

for the present operating conditions of the plant. Of course, such a system could be advocated only if all possible abnormal states could be anticipated and programmed. The superiority in terms of human reliability of this design over the standard type of display (red means open and green means closed) is obvious. In a typical reactor, there are several hundred dual-lamp indicators for status of motor-operated valves (MOVs), pump motors, and other equipment. Some of this equipment should normally be off or closed, and some should normally be on or open. The probability that an operator will detect nonnormal status is extremely low unless some other indication has directed the operator's attention to a specific pair of lamps.

In military and space systems, a green-board philosophy is used where possible. In such a system, all indications for a given subsystem or function must be "green for go." Thus, any nongreen indication stands out and is much more likely to be detected. As far as we know, this excellent practice has not been followed in any U.S. commercial power plant. Such a practice, combined with functional groupings of displays, would materially improve the probability of detection of unannunciated deviant indications.

At one plant we visited, the typical NPP color coding is used (red = open valve or closed switch; green = the opposite), but the operators are pro- vided a special color cue to use in case initiation of safety injection has occurred. The labels associated with indicator lamps on the "emergency systems" panel are colored either green or red to indicate whether the green or red lamp should be illuminated once safety injection has occurred. This provides a simple way for the operator to check on the correct status

of functions appropriate to coping with a serious abnormal event. He does0 not have to memorize the correct status of the components; he must only match the illuminated lamp with its label and see if there is agreement.

At the same plant, there is a status check made of switches and indications at the beginning of each shift. An operator uses a checklist that lists the correct status of each item to be checked. He does not use the colored labels because for some items their states for both normal and post-safety- injection situations should be the same; whereas, for other items, they should have different states.

An additional problem in status indications is the lag in display feedback

to an operator performing rod control or chemical concentration adjust- ments. Because of the lag in system response, the operator must anticipate the response on the basis of his familiarity with it. While highly skilled personnel can perform this kind of operation most of the time, errors do occur. Instrumentation techniques are available that can predict system response so that the operator can tell what the terminal system state will be when he neutralizes his controls (Frost, 1972). The requirement for the operator to compensate for system lags is thus eliminated.

As noted earlier, the PSFs described in this chapter, and in general throughout the Handbook, deal primarily with the design and operation of online NPPs. There are new design concepts currently under investigation

eA0 .3-0

Ergonomics Problems in NPPs

that promise to provide the operators in the control room with better organized and displayed information so that they can note the development of an abnormal event early and be better able to carry out the required actions to control and mitigate the effects of the event. The new concepts center about displays on CRTs controlled by computer software.

One approach, designated as the Safety Parameter Display System (SPDS), incorporates a system to display a minimum set of parameters that define the safety status of the plant through continuous indication of direct and derived variables as necessary to assess plant safety status (NUREG-0737, paragraph I.D.2-1). NUREG-0835 further defines this approach, and NUREG-0696 includes a "minimum list of plant safety parameters for SPDS": reactivity control, reactor core cooling and heat removal from the primary system, reactor coolant system integrity, radioactivity control, and con- tainment integrity.

Another approach incorporates Disturbance Analysis and Surveillance Systems (DASS), "computer-based information systems which take in plant process and control data, process the data, and display results to the nuclear plant operating crew in a prioritized order of importance such that plant safety and availability are improved" (Long et al, 1980). Foreign organizations are also studying DASS concepts and technology; for a review of this ef- fort, see Bastl and Felkel (1981).

As this Handbook is written, all we can say is that the above and similar approaches are promising. What they can do has yet to be demonstrated, and, as with any other new technology, it is reasonable to expect some problems along with the gains in performance. One such problem, the main- tenance of operator diagnostic skills, is described in Chapter 12. To the extent to which DASS and SPDS incorporate diagnostic aids, this problem of maintenance of skills increases. The new technology can alleviate or eliminate many of the display problems described earlier, but even then it may be necessary to maintain operator skill in using non-CRT displays to back up those cases in which the new displays are not available. Chapter 11 describes some of the implications of the new display technology for PRA.

In general, we cannot be specific about the potential effect of SPDS, DASS, or similar approaches on operator reliability. This is a completely new area that needs further study before a handbook such as this one can incor- porate estimated changes to HEPs resulting from implementation of the technology.

Presentation of Nonessential Information

The presentation of too much information, in the form of excessive annun- ciated indications, is a problem in the operation of NPPs. There are hundreds of annunciator indicator panels in a typical control room, and the operators complain about the constant clamor of auditory alarms, most of which convey no real emergency message.

3-65

Ergonomics Problems in NPPs

In discussions with design personnel, one gets the impression that each designer of a particular subsystem insists that his is so important that it

must be annunciated. The result is an ineffective signal-to-noise ratio in the control room. Operators have been known to cancel both the auditory and blinking lights of an annunciated signal without even looking up at the annunciator lamp. Costly errors have occurred as a result. The reason for this behavior is as familiar and predictable as Aesop' s fable about the boy who cried "wolf" once too often.

If the significance of each annunciated display in a plant were reviewed, many of them would be classified as not requiring immediate attention. Usually the blinking of the display, without an auditory alarm, is adequate to attract the operator's attention in time. Reducing the number of audi- tory alarms would lessen the. tendency of operators to use the "kill switch" without attending to the display.

Our Annunciator Response Model in Chapter 11 is based on the current design in online plants in which there are many auditory alarms competing for the operator's attention. In some plants being designed, e.g., the Paluel 1300 MW PWR in France, software techniques are being used to suppress nuisance alarms. For example, if there is a turbine trip, it should not be neces- sary for the operator to respond to an alarm that shows that there is a change in temperature of the oil that cools the turbine bearings. This type of alarm is A nuisance for this situation (although not for other situations) and will be suppressed in the Paluel design.

Inadequate Labeling and Status Indications Outside the Control Room

Poor labeling can lead to errors in any aspect of NPP operations, but the

most serious consequences for the availability of safety systems are often associated with -locally operated valves, pumps, and other safety-related components outside the control room. To permit maintenance or testing, some components must be changed to nonnormal states and then be restored to their normal states after the maintenance or testing has been completed. There have been cases in which an operator closed a valve in the wrong reactor system because the labeling did not clearly indicate the relation- ship of the valves to the reactors. Errors such as this occur because the labeling may be difficult to read, inconveniently located, or missing altogether.

In French NPPs, a remedy for this problem has been implemented. The con-

ventional labels on locally operated valves are the usual, nearly illegible manufacturer's labels. Letters and numbers are etched into a metal plate, providing very little brightness contrast between the numbers/letters and their background. The Electricit'e de France recognized this serious prob- lem and has supplemented the manufacturer's label with a well-designed blue plastic label with white-filled letters and numbers. Furthermore, the first number on the label is the unit number of the reactor. This is important since at least one complex consists of four reactor units in the same area. The label also includes the function of the component, its identifying numbers, and other useful information.

3-66

Ergonomics Problems in NPPS

In addition to problems of inadequate labeling, a source of uncertainty in valving operations is the lack of indication regarding the normal position of the valve, that is, its correct position for the power-generating mode of the plant. If each valve had some indication of its normal operating position, errors in deciding whether the valve is in the proper position would be materially reduced. There are several ways of indicating normal operating position. One obvious method is to inscribe N-0 or N-C for Normally Open or Normally Closed on the component label. This type of labeling is used in some control rooms for switches that govern the action of pumps and valves outside the control room. A more eye-catching design is possible for rising-stem valves. One could place on each valve a sketch of the valve stem in its normal position so that the operator could see if the valve stem matched the sketch. This inexpensive feature would materially reduce errors of commission.

At one U.S. plant, an inexpensive coding scheme used standard NPP meanings for the colors red and green. Green plastic tags were fastened to the handles of locally operated valves that were normally closed. Red plastic tags were used on locally operated valves that were normally open. This coding scheme eliminated the requirement for long-term memory of what the normal operating status is for each such valve. These plastic tags are permanently attached and clearly different from the temporarily attached yellow cardboard tags that signify valves taken out of service. This method of tagging suggests a procedure that is an extension of the "green board"1 philosophy. Four types of tags would be prepared, with green or red circles and green or red bars, to signify the actual state and the normal state of a valve. Thus, if a valve is open, and this is its normal state, a tag with a green circle would be placed on it. If it had to be closed for some reason, a tag with red bars would be substituted. In the course of the most casual inspection, the presence of red on a tag would alert the viewer that the valve is in a nonnormal state.

Ideally, safety-related components outside the control room and the switches that govern them inside the control room would have the following indications readily visible to the operating personnel:

(1) Which reactor unit the component applies to (if there is more than one such unit in the plant)

(2) An identifying number for the component

(3) The function of the component

(4) Its present position (for some valves, whether they are fully open or fully closed is not always obvious)

(5) The position of the component in the power generation mode of the plant (so that its present position can be compared to its normal position)

For future designs and retrofittings of NPPs, more use will be made of computer systems. Valve Position sensors tied in with computer programs

3-67

Most Important Classes of PSFs

and control room displays could warn operating personnel when any disparity exists between valve state and the momentary demands of the plant. (The same system can be used for other safety-related components.) This is an example of the use of equipment to perform monitoring activities that equipment can perform with much greater reliability then humans. For the time being, however, the derived HEPs in the Handbook are based on current, manual restoration procedures.

Most Important Classes of PSFs Related to System Safety

Some PSFs are more important than others in their influence on the human error potential for contributing to the unavailability of systems and equipment or to the failure of the man-machine system to respond appropri- ately to an abnormal event. Based primarily on our experience in conduct- ing and reviewing PRAs (WASH-1400, Arkansas Nuclear One Unit #1, Zion, Indian Point, and other PRAs), we list in Table 3-7 what we judge to be the most important classes of PSFs related to system safety. The table in- cludes reference to the Handbook chapters that provide models and/or esti- mates of HEPs related to each PSF class.

Our experience indicates that the most important PSFs will usually differ according to whether one is analyzing errors that can occur in normal or abnormal plant operating modes. Accordingly, the table is divided into two sets of PSFs. Obviously, there are many other PSFs, but those listed in the table appear to have the greatest influence on human errors that con- tribute materially to the results of a PRA. It can be noted that this "most important" listing does not include certain design features of dis- plays and controls that deviate from the best ergonomic standards such as those found in MIL-STD-1472C or in NUREG-0700. One should not infer from this omission that adherence to the best ergonomics principles related to such factors as illumination, stroke height-to-width ratio of letters and numbers, operating friction of switches, and so on, is not important. All of these factors found in standard ergonomics references should be adhered to for optimum human reliability, and most of them can be incorporated into the design of NPPs for little or no additional cost if done early in the design phase of a plant. However, these "bread and butter" ergonomics design features do not appear to be as important in terms of their in- fluence on potential safety consequences as the classes of PSFs listed in the table.

The following comments apply to the eight items in the table:

I-I: Poorly designed written materials and the reliance on memory for restoration of components contribute heavily to estimated HEPs in restoration tasks.

1-2: At some plants, there are few recovery factors for restoration errors; if a person makes an error, it has a much lower probability of being immediately detected than if there are good recovery fac- tors. Some newer plants have special control room displays that provide an additional recovery factor for errors of omission in restoration tasks.

3-68

Table 3-7

Table 3-7 Most important PSF classes related to system safety*

I - Normal Operating Conditions**

1. Type of work operations used to restore valves, pumps, circuit breakers, and other components after calibration, maintenance, tests, or other operations in which safety-related equipment or systems must be made temporarily unavailable - Sequences of activities (Ch. 15) - Written procedures (Ch. 15) - Checklists (Ch. 15) - Tagging and related paperwork (Ch. 16)

2. Recovery factors for restoration errors above (Ch. 19) - Human redundancy (i.e., use of a second person as a checker) - Special displays in control room

3. Administrative control for items 1 and 2 above - How well items 1 and 2 are carried out (Ch. 16) - Dependence among and within persons performing items 1 and 2 above

(Ch. 10, 18)

4. Design of labels and other displays outside the control room related to restoration of safety components (Ch. 3, 13, 14)

II - After an Abnormal Event (e.g., transient or LOCA)**

1. Extent to which needed information is directly presented on control room displays and in associated written materials to reduce information processing, interpretation, and decision-making by control room personnel (Ch. 11, 12)

2. Type and frequency of onsite practice and experience that control room personnel have had that is directly related to the abnormal event in question Ch. 3, 17, 18) - Drills - Challenges ("What if . . .7" - Talk-throughs

3. Working relationships (especially dependence) among control room per- sonnel on duty when the abnormal event occurs (Ch. 18) - Roles of individuals and how well they are understood by all - Influence of supervisor (degree of authoritarianism) - Communication provisions and use

4. Level of psychological stress resulting from the abnormal event (Ch. 17) - Poor implementation of the above three PSFs - Uncertainty as to the nature of the event (insufficient or conflict-

ing information) - Indecision about what to do (insufficient or conflicting

instructions) - Extent to which the safety systems perform as required

It is assumed that all personnel have had adequate training prior to their job assignments and have had 6 months experience in the job in which they are working.

The numbers 1 through 4 do not imply an order of importance.

3-69

Effects of Good Ergonomics Practices

1-3: Administrative control refers to how well the specified procedures and other activities are carried-out by plant personnel. Are short- cuts taken in place of strict adherence to methods devised to reduce errors of omission? This applies especially to the following of checklists and other written procedures.'

1-4: If labeling of locally operated valves and other safety-related components outside the control room does not provide all the in- formation necessary for reliable operation and provide it so that reading errors are unlikely, both errors of omission and commission will be relatively frequent.

II-1: Under high stress, many people (even experienced operators) tend to lose some of their ability to integrate sources of information that are separated in time or space, and they also lose some of their ability to discard an incorrect hypothesis about the nature-of the problem they face so that they could try some other hypothesis. For this reason, the ability to initiate and complete successfully the tasks related to an abnormal event is heavily influenced by the manner and type of information displayed. Written procedures, especially those using simple binary logic diagrams to facilitate the operators' activities, could provide much help to the operators.

11-2: If operating personnel have had little practice in thinking about and talking through their responses to an abnormal event, their probability of responding correctly will be greatly reduced.

11-3: To what extent do the shift control room personnel operate as a team when an abnormal event occurs? This is obviously related to item 11-2 above. Are they well-disciplined yet have sufficient flexibil- ity and communication to permit discussion of other hypotheses and options in a new situation?

11-4: A well-designed control room, well-written procedures, frequent practice of response to abnormal events, and a good working rela- tionship among the shift personnel can do much to reduce uncer- tainty, indecision, and other aspects of behavior that can increase stress to debilitating levels in the response to an abnormal event. If the practice includes "partial panel" operation, the personnel are more likely to be able to cope with the failure of certain safety components.

Effects of Good Ergonomics Practices on Estimated HEPs

The estimated HEPs in this Handbook reflect our evaluation of the effects of current design practices in NPPs on human performance. In our survey of NPPs before and during the WASH-1400 study, and in subsequent' visits to other NPPs, it became apparent that no systematic consideration of human factors technology was incorporated in the design of man-machine inter- faces, written procedures, or operational practices for online NPPs.

3-70

Effects of Good Ergonomics Practices

Violations of conventional human factors practices (as outlined in MIL- STD-1472C and NUREG-0700) are the general rule rather than occasional occurrences.

For several years prior to the TMI incident, human factors experts at the American Institutes for Research, Human Performance Technologies, the Lockheed Missiles and Space Co., LUTAB, Riso National Laboratory, Sandia National Laboratories, and other institutions have pointed out that the design of man-machine interfaces in online NPPs is not fully compatible with the capabilities, limitations, and needs of the personnel who function at those interfaces. There is no doubt that the incorporation of good human factors practices in the design of NPPs and related human operations and procedures could effect substantial improvements in human reliability.

Just how much improvement could be effected is obviously situation- specific. In Table 3-8, we have developed some conservative, highly specu- lative estimates of the benefits that would result from an across-the-board application of good human factors practices in NPPs. These estimated factors are not additive.

3-71

Table 3-8

Table 3-8 Estimated decreases in HEPs resulting from application of good ergonomics practices to nuclear power plants

Resulting Decrease If done: in HEPs (Factors*):

Good human factors engineering practices in design of controls and displays** 2 to 10

Use of well-designed written procedures and checklists to replace typical narrative style procedurest 3 to 10

Redesign of displays or controls that violate strong populational stereotypes ) 10

Redesign of valve labeling to indicate its function (including a clear indication of the system with which a valve is associated) and also to show clearly its normal operating status '-5

Frequent practice of the appropriate responses to potential emergencies or other abnormal situations (practice includes periodic recertification in dynamic simulators and talk-throughs conducted at least once per month for the major potential problems) 2 to 10

These estimated factors are not additive. **

No evaluation is made of new display technology using CRTs and computer software.

t No evaluation is made of symptom-oriented procedures.

3-72

Part II - Methods

PART II. METHODS FOR ANALYSIS AND QUANTIFICATION OF HUMAN PERFORMANCE

To consider the impact of human errors in a system, several methods are required for probabilistic risk assessment (PRA). There must be methods to identify the potential for errors, to estimate the probabilities of the errors, and to assess their impact on system goals and functions. Part II presents methods for analysis and quantification of human performance in the following six chapters:

Chapter 4 - Man-Machine Systems Analysis. This is the basic approach used by human factors personnel to identify the potential for human errors in a system and to make some qualitative judgment as to the relative importance of each error in the system. The approach includes task analysis, the method for analyzing each task to identify the important performance shap- ing factors (PSFs) that increase the potential for error.

Chapter 5 - A Technique for Human Reliability Analysis. The human relia- bility analysis (HRA) technique described in this chapter has been used in PRAs of nuclear power plant (NPP) operations in the U.S. and in Europe. It is based on a level of task analysis suitable for the PRA being performed. This technique can be used either as a design tool (for trade-off studies) or as a part of PRA per se. This HRA technique is a subset of man-machine systems analysis.

Chapter 6 - Sources of Human Error Probability Data. This chapter de- scribes the major problem for HRA: the relatively small amount of human error data that can be used to estimate human error probabilities (HEPs) for NPP tasks. The chapter tells how the HEPs in the Handbook were derived.

Chapter 7 - Distrioution of Human Performance and Uncertainty Bounds. For some PRAs, it is desirable to base the analyses on distributions of equip- ment failure and HEPs rather than deal with single-point estimates for each system event or human task. This chapter presents some interim methods for treatment of distributions and uncertainty bounds of HEPs until data-based models can be derived.

Chapter 8 - Use of Expert Opinion in Probabilistic Risk Assessment. Some of the problems in using expert judgment in place of the missing data on HEPs are presented in this chapter. Guidance is provided in selecting the appropriate method for psychological scaling in using expert judgment for this purpose.

Chapter 9 - Unavailability Calculations. This chapter discusses the use of HEPs to estimate component unavailabilities resulting from human error and provides some examples of unavailability calculations. Other examples are found in Chapters 11 and 19.

II-1

Part II - Methods

After studying this part of the Handbook, we suggest that the reader study the companion volume to the Handbook, NUREG/CR-2254 (Bell and Swain, 1983). An example of HRA is given in the latter document, beginning with the association of HRA with the total PRA, obtaining required information from a plant, and continuing through the task analysis, including sensitivity analysis, to the provision of estimated HEPs to the appropriate blocks or locations in system fault or system event trees. In addition, practical exercises in HRA, including unavailability calculations and bounding analysis, are given in the appendix to NUREG/CR-2254.

11-2

Ch. 4 - MMSA Overview

CHAPTER 4. MAN-MACHINE SYSTEMS ANALYSIS

Overview

This chapter describes the basic analytical methods used in the human factors community to identify and evaluate existing or potential human performance problems in man-machine systems. These methods can be used as design tools for any man-machine system, and they also furnish the raw material for a human reliability analysis (HRA), described in Chapter 5. The level of detail for application of these methods will vary with the purposes of the analysis.

The general method for the analysis and improvement of human performance consists of the following steps:

(1) Identify all the interactions of people with systems and components, i.e., the man-machine interfaces.

(2) Analyze these interfaces to see if the performance shaping factors (PSFs) are adequate to support the tasks that people have to perform.

(3) Identify potential problem areas in equipment design, written proce- dures, plant policy and practice, people skills, and other factors likely to result in human error.

(4) Decide which problems have sufficient potential impact on the system to warrant changes.

(5) Develop candidate solutions for the problems.

(6) Evaluate the estimated consequences of these changes to ensure that they will improve system reliability and availability of safety systems and that no additional serious problems will result from them.

This general method, which has been in use for some time in the human factors community, is called man-machine systems analysis (MMSA). MMSA can be used as a qualitative or a quantitative analysis. The qualitative part is based on a descriptive and analytical technique known as task analysis. The quantitative part uses a human reliability technique to develop esti- mates of the effects of human performance on system criteria such as relia- bility and safety.

MMSA, whether its application is qualitative or quantitative, is based on a thorough analysis of the operator's* tasks in the context of the

. In this chapter, the term "operator" is used to designate anyone who per- forms NPP tasks.

4-1

Step 1 - System Goals & Functions

application, e.g., the nuclear power plant (NPP). Tasks performed by NPP personnel are analyzed to identify actual or potential sources for human error--error-likely situations (ELS) or accident-prone situations (APS). Techniques for identifying ELSs and APSs in complex man-machine systems were developed in the early 1950s by Dr. Robert B. Miller and his associ- ates at the American Institutes for Research. These techniques, collec- tively titled A Method for Man-Machine Task Analysis (Miller,- 1953b), have been refined and expanded for application to HRA. See especially the task analysis in NUREG/CR-2254.

The remainder of this chapter discusses the 10 iterative steps in MMSA listed in Table 4-1. Much of the discussion in this chapter is on the qualitative application of MMSA as a design tool; Chapter 5 treats the quantitative aspect. Whether the intended application of MMSA is as a design tool or as a part of probabilistic risk assessment (PRA), it is important to initiate the method as early as possible in the life history of a system. For a more detailed discussion of task analysis as a design tool, see Kidd and Van Cott (1972) and Woodson (1981, Chapter 1).

Step 1 - Describe the System Goals and Functions

The purpose of this step is to see where people fit in with system goals and functions. What are people supposed to do to accomplish various system functions? Where are the points of interaction between the system and the people? In a PRA, these points are defined as the interfaces between equipment and people, e.g., locally operated valves, switches for motor.- operated valves, displays to be read, provisions for calibrating setpoints, etc. System analysts will usually identify the interfaces that are impor- tant to a PRA. The human reliability analyst may identify additional interfaces that may have an adverse impact in the PRA or that provide error recovery factors. In general, the number of interfaces to be considered in MMSA will be larger when this method is being used as a design tool than for PRA.

It is important to understand the assumptions about people that are in- herent in the design of each system. Usually these assumptions will not be stated and must be inferred. Identification early in the design phase of unrealistic demands on people by system design characteristics can elimi- nate the need for costly retrofits once equipment has been built and tested for reliable human performance.

When using MMSA, one should not unquestioningly accept the definition and division of jobs and tasks as they exist in the plant or in a plant being designed. For each system function, one must determine whether there is a reasonable division between those tasks that are accomplished by equipment, those by people, or those that are accomplished by an interaction of both. Too frequently, this allocation seems to have developed historically by trial and error rather than through systematic analyses. Sources of in- formation for this step include design requirements, planning documents, proposals, schematics, flow diagrams, written procedures, and interviews with system planners and people with experience in the operation of similar

4-2

Table 4-1

Table 4-1 Steps in Man-Machine Systems Analysis (MMSA)

1. Describe the system goals and functions of interest.

2. Describe the situational characteristics.

3. Describe the characteristics required of the personnel.

4. Describe the jobs and tasks performed by the personnel.

5. Analyze the jobs and tasks to identify error-likely situations (ELSs) and other problems.

6. Estimate the likelihood of each potential error.

7. Estimate the likelihood that each error will be undetected (or uncorrected).

8. Estimate the consequences of each undetected (or uncorrected) error.

9. Suggest changes to the system.

10. Evaluate the suggested changes (repeat Steps 1 through 9).

4-3

Step 2 - Situational Characteristics Step 3 - Characteristics of Personnel

systems. For the evaluation of an existing plant, the information should be checked by visits to the plant.

Flow charts with a time baseline may be used to show the system functions for each major area in the plant. Flow charts can show how people fit into the system and what the design constraints are. (For preparation of flow charts, see Edwards and Lees, 1973 and 1974.)

Step 2 - Describe the Situational Characteristics

Situational characteristics of interest are those PSFs under which the tasks will be performed. Examples are air quality, general cleanliness, lighting, accessibility, union restrictions, and other PSFs listed in Table 3-1. Some PSFs may vary 'from job to job in a plant; others will be essen- tially the same for several jobs. Sources of information include the documentation listed in HMSA Step 1, but the best sources will be observa- tions at a plant visit, and interviews with management, supervisors, and the people working in a plant. The primary interest is in those situa- tional characteristics likely to have adverse effects on human performance.

Step 3 - Describe the Characteristics of the Personnel

In this step, one identifies the skills, experience, training, and motiva- tion of the personnel who will operate, calibrate, and maintain the plant systems. The capabilities and limitations of the people in a system must be understood so that they can be compared with the demands the system makes upon them. Any mismatch between these two sets of factors will increase the potential for errors and will be important in estimating the human error probabilities (HEPs) needed for a PRA. If the MMSA is being used as a design tool, a mismatch will require a change in man-machine interfaces, procedures, or modification of personnel characteristics through training and/or selection.

One important requirement of this step is the evaluation people's past experience with other systems to identify the potential for transfer of habits that would interfere with reliable performance in the new system. At present, there is no standardization of ergonomic considerations in NPPs, although some standardization of certain control room features will evolve through the application of NUREG-0700 (1981) and NUREG-0801 (1981). Possibilities for the transfer of old habits with negative impact on a new system must therefore be evaluated when personnel are assigned who may have worked in other plants or trained on simulators where the PSFs differed materially from those in the plant in question.

For safety-related operations, it is important to evaluate the provisions for continued practice by each operator in responding to low-probability events such as a loss-of-coolant accident (LOCA) or an anticipated transi- ent. Without practice, the readiness to handle such events will decrease, as explained in Chapter 3 (see Figure 3-6).

4-4

Step 4 - Task Analysis - Descriptive

Step 4 - Describe the Jobs and Tasks the Personnel Perform

Steps 4 and 5 of the MMSA jointly constitute task analysis. Task analysis is an analytical process for determining the specific behaviors required of the human components in a man-machine system. The individual tasks, or steps in the tasks, become the limbs in the HRA event trees. A task is a level of job behavior that describes the performance of a meaningful job function. It is any unit of behavior that contributes to the accomplish- ment of some system goal or function.

Task analysis is applicable to any kind of a task. Examples of the appli- cation of task analysis to both dynamic and step-by-step tasks are found in Miller (1953b), Meister and Rabideau (1965), and Edwards and Lees (1974). The examples include the analysis of continuous tasks such as the in-flight functions of an aircraft pilot, the tracking employed in air-to-air flexi- ble gunnery, and the tracking tasks in operating a continuous strip mill. In such cases, the continuous nature of these tasks is often described as a series of discrete task elements. This type of abstraction is true of human performance modeling in general and is used here. Although contin- uous or dynamic tasks can be used directly as entries in task analysis or performance modeling, the solution methods are cumbersome and are much simplified if the variables are treated asdiscrete values--the differences in accuracy are negligible for practical work. Examples of continous-type NPP tasks that we have analyzed using the above approach are refueling and reactivity control. In modern plants, however, because of the employment of digital control, it is stretching a point to classify refueling as a "tracking task."

Dynamic tasks often involve interpretation and decision-making aspects in

the diagnosis of some abnormal event. These aspects of cognitive behavior are more difficult to represent in a task analysis. However, in some plants, actions to be carried out in response to certain patterns of annun- ciators and other displays are so well rehearsed that the diagnosis aspect can be considered negligible. For example, in most plants when annuncia- tors indicate a call for a reactor/turbine trip, operators are trained to push in the manual trip buttons without even thinking about it. Such behavior is "second nature," and for PRA purposes, it can be assumed that there is no interpretation or decision-making involved. However, if the reactor still fails to trip, then the analyst must evaluate the various facets of diagnosis discussed in Chapter 12.

Task analysis for PRA purposes generally is not done in as much detail as task analysis done as a design tool. In either case, the personnel who perform the tasks are one of the best sources of information for a task analysis. Often the operating, calibration, test, or other written proce- dures are available and can be used as the basic source document for a task analysis. Even when written procedures are available, however, the analyst must confer. with NPP personnel to determine any differences between what the written procedures say and what the people actually do and to under- stand fully the relevant PSFs. Methods for involving the operator in the task analysis are discussed in Step 5 and in NUREG/CR-2254.

4-5

Step 4 - Task Analysis - Descriptive

Task analysis consists of description and analysis. The descriptive part involves setting down what leads to what, that is, what the job performer is supposed to do. The analysis part involves figuring out what could go wrong and why. This step in the MMSA deals with the descriptive part and considers those PSFs related to (1) task and equipment characteristics and (2) job and task instructions. With the situational characteristics from Step 2 of the MMSA, they describe the demands that each job places on the personnel.

There are many different formats for task analysis. The particular format used is unimportant; the important thing is to describe and analyze each task as necessary and to identify ELSs and APSs. Figure 4-1 shows a for- mat used in some early military studies that illustrates the kinds of factors to be considered in a detailed task analysis. Note that the format includes a descriptive part related to Step 4. of the MMSA and an analytical part related to Step 5 of the MMSA.

There are five columns in the descriptive part of the format. In the first column, "Task or Step," one uses numbers to indicate the sequence of peL- formance. In the second column, "Instrument or Control," one lists each item that displays information to the operator or that must be manipulated. In the control room, for example, the annunciators, meters, chart record- ers, and other items display information. The controls are mainly switches on the control panels and typewriter keys for the computer. For calibra- tion tasks, the "Instrument" includes meters for measuring setpoints, and the "Controls" include potentiometers for adjusting setpoints. Controls include connectors and tools. In all cases, the labels on the equipment are used and are capitalized if that is the way they appear on the equipment.

In the third column, "Activity," one sets down action verbs describing the actions to be carried out on the items in the second column. The action verbs should help identify the kind of display or control used, and the position to which a control is to be set, or some other indication of response adequacy. For example, if a toggle switch is used, the words "Flip up" or "Flip down" are preferred over less specific words such as "Manipulate."

The fourth column, "Cue for initiation or completion of activity," is used to indicate the cue that tells the operator when to begin a step or the cue that tells him when he has successfully completed a step. In most cases, this information is found in the "Activity" column, but it is used in this format to remind the analyst to pay particular attention to these cues. Errors can result if the equipment or procedures do not provide good cues. Misleading or incomplete cues can result in discontinuities in carrying out a task, with the result that some step in a procedure may be omitted.

The last column, "Remarks," is used for relevant information not covered in the other four columns. The completed task description provides the in- formation for the analytical part in which the identification of ELSs and APSs is made. 0

4-6

Figure 4-1

Job: Task: Page of Pages

Subtask: Conditions: Task Analyst:

TASK BEHAVIORS (Description) TASK COMPONENTS (Analytical)

-- M -- - E - -

0 o0 c' o. 2. . 'oC h . - (."A•"-, , " _L E E 0 E

CL 0 . C. C 0-oo I C I o. I _ E a. 0 0: 0. s-b .V4WU4-C9M C

0 0 1,.~ .9, Oh 4- 4) 2L 0.0

0f EO Cl, U)

•.• JReduced from 2 pages

*Anything that displays information

t Anything that Is manipulated

Figure 4-1 A job-task analysis format used for

military system studies (Swain, 1962).

4-7

Step 4 - Task Analysis - Descriptive

The task analysis format in Figure 4-1 is more detailed than experienced analysts will find necessary. In some HRAs done for the NRC, a simpler task analysis format was used when analyzing actions by control room opera- tors (NUREG/CR-2254, Figure 6). This format used the following tabular headings: Step, Equipment, Action, Indication, Location, Notes, and Errors. The important thing is that the format must permit the analyst to identify potential sources of error. When the task analysis is intended as permanent documentation for a PRA, additional columns may be added to indicate the estimated HEPs, their source (e.g., the relevant table and item from chapter 20), and modifiers such as level of stress, dependence, skill, etc. (NUREG/CR-2254, Figure 20). Documentation is important if others will need to evaluate the HRA.

The task description will be fairly gross at first. Identification of the tasks is all that need be done initially, i.e., a task listing. This will enable the analyst to relate that which people have to do to the various functions defined in Step I in the MMSA. It may be useful to key the tasks to flow charts developed from Step 1. When the task analysis is done early in the design and development stage of a system, the task listing will depend heavily on the analyst's understanding of similar NPPs that are operational. More detail in the task description will be possible as the system design becomes more definitive. When all of the procedural steps have been recorded, they can serve as entries to procedures manuals and training courses when the task analysis is being used as a design tool. Since the Three Mile Island (TMI) incident, the U.S. Nuclear Regulatory Commission (NRC) has taken the position that a task analysis should be used as one important basis for the design of procedures manuals and training course content (NUREG-0660, 1980, Vol. 1). Task analyses for these pur- poses will usually have much more detail than task analyses used for PRA purposes only.

One useful aid to. task description in NPP studies is link analysis. This task analysis technique is often used in laying out work places and job operations and can be used to study the interactions among people in an existing plant. A link analysis depicts the pathways among different parts of a system as they are generated by people walking about and communicating.

Figures 3-7 and 3-8 show a link analysis for the reactor engineer for a boiling water reactor (Axelsson and Lundberg, 1975). Additional link analyses from the same study showed that the reactor. engineer*, turbine engineer*, and shift supervisor had to spend much time looking at displays not visually accessible from their normal working positions. This kind of analysis can suggest improvements for future designs, and, for HRA pur- poses, can provide an understanding of the difficulties and inconveniences that influence human performance. Procedures for performing link analyses are described in Chapanis. (1959, pp 52-61) and McCormick and Sanders (1982, pp 352-355).

The duties of these positions are performed by reactor operators in U.S. plants.

4-8

Step 5 - Task Analysis - Analytical

Another useful technique for outlining operating time and personnel inter- actions is called the operational sequence diagram (Brooks, 1960; Kurke, 1961; and Kidd and Van Cott, 1972, pp 11-12). The operational sequence diagram displays information-decision-action sequences in a man-machine system. It can be used in preparing time-sequence process charts or spa- tial flow charts. This technique involves some symbolic shorthand, but the number of symbols to be learned is not excessive. The main advantage of these diagrams is that they outline essential interactions among operators, work stations, items of equipment, and time. For an application of task analysis of nuclear fuel handling using operational sequence diagrams, see Hottman et al, 1982.

Step 5 - Analyze the Jobs and Tasks To Identify Error-Likely Situations (ELSs) and Other Problems

In the analytic part of the task analysis, each action identified in Step 4 of the MMSA is analyzed to identify ELSs arising from equipment design features, methods of use, methods of training, and the skill levels of the people in the system. There are no hard-and-fast rules for making these determinations. The validity of the task analydis will depend upon the skill of the analyst in assuming the role of the operator so that he can understand the actual and potential problems in each task. Our reviews of some PRAs indicate that task analyses were not done by persons with train- ing and skill in this qualitative technique. This deficiency possibly accounts for some overly optimistic assumptions about NPP personnel perfor- mance. We wish to emphasize that unless a PRA team includes someone who fully understands how to do a task analysis, the HRA portions of the PRA will not be as accurate as they could be. For MMSA used as a design tool, the task analysis must include personnel skilled in the above technology.

Even the best analyst cannot identify all possible modes of human response. No one can predict unlikely extraneous acts by plant personnel, such as the use of a candle to check leaks in the negative pressure containment build- ing (the Brown's Ferry Incident). Still, given sufficient time, a skilled analyst can identify most of the important tasks to be performed in a system and most of the ways in which errors are likely to be committed.

The "Analytical" half of the format in Figure 4-1 indicates the kinds of factors to consider in identifying an ELS. The factors are listed under four broad headings: (1) Scanning, perceptual, and anticipatory require- ments, (2) Recall requirements (long- or short-term memory) and initiating cue (present, absent, or poor), (3) Interpreting requirements, and (4) Manipulative problems. The terms are self-explanatory and may be used without modification for NPP analyses. The column headings should be re- garded as suggestions only--any unique problem should be listed regardless of the column headings presented here. In general, the analyst is at- tempting to identify the underlying PSFs that are related to Boxes B, C, and D from Figure 4-2.

In use, the analyst makes entries in this half of the form only when he identifies an ELS. For example, assume that he finds an ELS in the third

4-9

Figure 4-2

A EXTERNAL INPUTS TO HUMAN VIA SIGNALS, SIGNS, & CONDITIONS

DISPLAYS: VISUAL AUDITORY VIBRATORY OLFACTORY ETC

COMMUNICATION: ORAL WRITTEN

ENVIRONMENTAL CONDITIONS

CORRECT/INCORRECT HUMAN COMPONENTS

APPARENT PATH FOR WELL-PRACTICED ASSOCIATIONS S--INTERNAL FEEDBACK LOOP--I

B C~f D~J INTERNAL INPUT MEDIATING RESPONSES TO HUMAN VIA (COGNITIVE) (HUMAN

AN PERCEIVINGSENSING, "]ACTIVITIES & OUTPUTS TO DISCRIMINATION, PROCESSES SYSTEM) AND PERCEIVING

FILTERING, ORGANIZING, & RECOGNIZING VIA: SENSE ORGANS (RECEPTORS) & MEDIATING ACTIVITIES OR PROCESSES

PROCESSING OF PERCEIVED INFORMATION INCLUDING EFFECTS OF MEMORY, GOALS, INTENTIONS, PERSONALITY (ATTITUDES, MOTIVATION, EMOTIONS, ETC), INTERPRETATION & DECISION- MAKING

NO ACTION; USE OF HAND, FOOT, OR OTHER BODY MEMBER TO CONTROL SOME NPP FUNCTION, OR ISSUANCE OF ORAL OR WRITTEN COMMANDS OR INFORMATION

E -EXTERNAL

RESULTS ON MAN-MACHINE SYSTEM

CORRECT OR INCORRECT

HUMAN INPUTS TO SYSTEM VIA:

NO ACTION SWITCHES VALVES COMPUTER KEYBOARD CRANE HANDLING EQUIPMENT & TOOLS CONNECTORS INSTRUCTIONS

. . S I I

_- - EXTERNAL FEEDBACK LOOP - -

Figure 4-2 A simplified model of the human component in a man-machine system for the purpose of probabilistic risk assessment. (Note: This is a repeat of Figure 3-1.)

4-10

Step 5 - Task Analysis - Analytical

step of a set of actions NPP personnel are to perform. In the analytical portion, he will write "3" in the "Step" column, and he will note the basis for his judgment of an error-likely task. Referring to the factors in the columns, an ELS exists when the discriminating, recalling, interpreting, inferring, decision-making, or manipulating processes demanded of the operator are likely to exceed his capacity. The potential errors can be any of the errors of omission or commission listed in Chapter 2, or they can be errors in any one of the underlying internal inputs (Box B from Figure 4-2) or mediating processes (Box C).

The analysis is done for each task or step in a task to determine those PSFs that seem likely to result in errors. It must be determined whether there are any conflicts between the external PSFs and the internal PSFs since such conflicts can be expected to result in errors. Chapter 3 lists examples of external PSFs that are not compatible with various human at- tributes and therefore result in lowered human reliablity. As shown in Figure 3-2, conflicts between the external and internal PSFs can cause psychological or physiological stresses. If there is a high stress level, the performance of a person in the system will probably deteriorate. If the level of stress is too low (as with monotonous work), alertness may be degraded and signals may not be noticed on a timely basis.

In summary, we define error-likeliness in terms of those PSFs in a task that are incompatible with the capabilities and limitations of the intended performer of the task. The task analysis will indicate if human reliabil- ity can be improved by changing any PSF.

Whether it is important enough to the system to warrant changing the design is another matter. The object of task analysis is to identify potential sources of error regardless of their impacts on the system. Step 8 in the MMSA takes the consequences of error into account.

Several publications are available to assist in identifying ELSs in NPP operations. Two NRC documents (NUREG-0700, 1981, and NUREG-0801, 1981) provide information to enable an analyst to judge less-than-optimum human factors engineering (ergonomics design) in NPP control rooms. The review of NPP control rooms sponsored by the Electric Power Research Institute (EPRI) illustrates some of the poor human factors engineering features in some online plants (Seminara et al, 1976). Another EPRI report describes problems in maintenance tasks (Seminara et al, 1980; Seminara and Parsons, 1981). The three-volume study of human factors problems at the TMI-2 plant by Malone et al (1980) includes examples of poor ergonomics in NPPs and describes the kinds of studies necessary to identify such problems. In the area of NPP written procedures, the documents referenced in Chapter 15 can help the analyst identify less-than-adequate procedures.

More general documents (for human factors not restricted to NPP operations) should be in every task analyst's library. The most concise document is MIL-STD-1472C, Military Standard, Human Engineering Design Criteria for Military Systems, Equipment and Facilities, U.S. Dept of Defense, Washing- ton, D.C., 2 May 1981. This set of standards was developed by practicing

4-11

Step 5 - Task Analysis - Analytical

ergonomists in the U.S. Army and in U.S. industry and adopted by all of the U.S. military services. These standards are not absolute, but their accep- tance in the design of NPPs would materially improve human reliability. They represent sound human factors engineering practices to be followed unless specifically contraindicated by other aspects of the system. A companion document, MIL-H-46855B, Military Specification, Human Engineering Requirements for Military Systems, Equipment and Facilities, U.S. Dept. of Defense, Washington, DC, 31 January 1979, defines the general requirements for incorporating ergonomics considerations in the design of systems. Other documents useful in identifying ELSs are the Human Engineering Guide to Equipment Design (Van Cott and Kinkade, 1972), the Human Engineering Guide for Equipment Designers (Woodson and Conover, 1964), the Human Fac- tors Design Handbook (Woodson, 1981), and two textbooks: Human Factors in Engineering and Design (McCormick and Sanders, 1982) and Ergonomics: Man in His Working Environment (Murrell, 1969). These documents provide much of the rationale and data behind the standards in MIL-STD-1472C. Two volumes by Edwards and Lees (1973 and 1974) constitute a useful description of operator roles in complex industrial processes analogous to those in NPPs.

Although the above documentation will be useful, the best way for the analyst to determine which human processes and actions will be employed in performing each task is to perform the tasks himself, in fact or in simula- tion, using the available written procedures. Then he should observe and interview operators who perform the tasks. Since highly skilled operators can make even poor designs look good, it is necessary to include talk-- throughs or walk-throughs in the observation of the operators at their jobs. This technique involves pauses in the actions while the operator explains what he is doing and his mental processes. The analyst should observe the operations being performed at their normal speed until he develops a sense of familiarity. Then he should ask the operator to slow down his work activities and explain what he is doing and why. As he performs the tasks himself and interacts with the operators, the analyst will develop familiarity with the system hardware and procedures. This is the period when he will obtain most of the data for the analytical half of the task analysis format. Table 4-2 is a general checklist that can be used during this period.

Another useful technique is to have the operator talk through hypothetical, but realistic, emergency problems. In the WASH-1400 and subsequent PRAs, this technique was employed to find out how much operators knew about responding to certain emergency conditions and what provisions had been made for these responses in the design of the equipment and the written procedures. Talk-throughs can also reveal the mental model the operator has of the plant and its processes. This is useful when employing task analysis as a design tool either to decide what new types of displays are needed, to check on the adequacy of newly installed safety parameter dis- play systems (SPDS), or to obtain information on an existing plant to use in the design of a new plant. For PRA, talk-throughs are imperative. Unannounced talk-throughs of transients that are of interest to a PRA can reveal strengths and weaknesses and estimated operating times of the staff

4-12

Table 4-2

Table 4-2 A checklist for evaluating task error-likeliness during observation or performance of tasks

1. The cue or sign that tells the operator to begin each task and each activity in a task is simple and unambiguous:

a. No inconsistencies, gaps, or misleading information that could result in errors of judgment

b. No competing activities that could interfere with perceiving the cue or with responding in a timely manner

2. The cue or sign points to the correct task or activity only.

3. The task or activity is easy to do:

a. No special problems in the scanning, anticipatory, or other percep- tual requirements; in long- or short-term memory requirements; in interpreting and decision-making requirements; or in manipulative requirements

b. No special problems with competing activities or past experience

4. The completion cue or sign for the task or activity is simple and unambiguous:

a. No misleading feedback to the operator b. No special problems with competing activities

5. The completion cue or sign for one activity in a task cannot be misin- terpreted as signaling the completion of the entire task

4-13

Step 5 - Task Analysis - Analytical

in coping with abnormal events. The abnormal events included in the talk- throughs should include transients with more than one problem if this is of interest in a PRA (e.g., anticipated transient without scram (ATWS) coupled with a loss of feedwater). If enough talk-throughs are done to include a cross section of control room skills, it may be possible to obtain a better estimate of the range of performance of unusual tasks than is provided by the Handbook.

Experienced people on a job can rightly be regarded as subject-matter ex- perts. They know more about the intricacies and difficulties of their tasks than anyone else. The importance of interviewing such experienced operators is that they are invaluable in identifying problem areas in tasks. They can describe errors they have made or have seen others make and can offer opinions on the underlying PSFs related to these errors. These subject-matter experts can also describe close calls and, in general, what errors are possible and likely. In the interviews, these experts should be encouraged to report all errors, even those that did not result in any actual safety or reliability problems. Such errors are called no-cost errors and can reveal the potential for serious problems. If each respondent's anonymity can be assured, more useful information will be obtained.

One formal method of soliciting operator opinion is the critical incident technique (CIT). As defined by Flanagan (1954), "The critical incident technique consists of a set of procedures for collecting direct observa- tions of human behavior in such a way as to facilitate their potential usefulness in solving practical problems and developing broad psychological principles." An incident is defined by Fivars and Gosnell (1966) as "any observable bit of human behavior sufficiently complete in itself to permit inferences to be made about the person performing the act." In applying this definition to human factors problem identification, Swain (1980a) added "or about the work situation or circumstances related to the act." Fivars and Gosnell further state that for an incident to be critical, "an incident must make a significant difference in the outcome of the beha- viors; it must contribute either positively or negatively to the accom- plishment of the aim of the activity." Traditionally, a negative incident has been called a red incident and a positive incident a blue incident.

An application of the CIT, described in Chapter 17, is used as data in support of our extremely high stress model. The CIT has also been applied to the identification of significant radiological hazards in a series of studies conducted by Idaho National Engineering Laboratory beginning in the late 1960s (Eicher, 1976, and Johnson, 1980, pp 386-7). This study illus- trates a typical application of the CIT. Every 6 months, an investigator from this laboratory would ask workers in areas with potential radiological hazards to identify negative and positive critical incidents they had experienced in the last 6 months. Special forms were provided and time was set aside for each participant to respond. The investigator was present to assist in questions and to encourage the participants to think of positive as well as negative incidents. The participants' suggestions for correc- tive action were solicited. All participation was confidential. A similar

4-14

Step 5 - Task Analysis - Analytical

application of the CIT was used in the Swedish logging industry (Lagerl~f, 1977). The investigators trained the onsite safety representatives to query each worker to recall any close calls (near-accidents) he had had in the last 6 months, and the details of equipment, work operations, and work situation involved. It was found that the percentage of different types of near-accidents very closely paralleled the percentage of the same types of actual accidents (where personal injury was sustained). The CIT represents a useful technique for identifying ELSs in NPP operations, especially when employed so that each individual's anonymity is assured.

In performing a task analysis, the underlying behavioral processes required in each task must be identified. As shown in Figure 4-2, it is convenient to think of these processes as consisting of internal input variables, mediating variables, and output variables, a conventional trichotomy in psychology. One must remember, however, that the trichotomy is an abstraction and that for a complete task analysis, one must consider those PSFs related to the organizational structure, actions by others in his work situation, and the organismic factors such as the training and experience of the worker.

Referring to' Figure 4-2 (Box A), signals, signs, and conditions feed infor- mation to the senses of the operator (Box B) where it is filtered, orga- nized, and recognized. The operator's discriminations are determined both by the features of the task to be sensed and by his individual characteris- tics--his sense organs, past training and experience, any ongoing activi- ties that compete for his attention, his emotional state, and so on (Box C). In our post-WASH-1400 studies, we have found that much reliance is placed on these internal PSFs to make up for inadequate human factors engi- neering of the job situation. In a well-designed system, the equipment, procedures, etc, do not place undue demands or reliance on operator charac- teristics, that is, the design associated with the system elements in Box A is such that the internal processing of information by the operator is well within his capabilities and limitations.

The operator's responses (Box D) are his outputs to the system by which he affects the system (Box E). (His responses may or may not be appropriate and include taking no action at all.) When using the HRA event trees described in the next chapter, each task element is treated as either a success or a failure in terms of system requirements. When this either/or distinction is not appropriate, different degrees of success or failure can be treated as different events. For example, assume that there is some probability that an operator will leave a locally operated valve in some position between open and closed. Although there are an infinite number of in- between positions, for any practical application, this number can be reduced to one (or a very few) in terms of system effects and each of these positions treated as an event. Such reduction is necessary to keep the analysis manageable and is adequate for PRA purposes.

The mediating processes (Figure 4-2, Box C) are internal activities of the operator, such as thinking, deciding, and worrying. These processes con- stitute the bridge between the internal inputs to the operator (Box B) and

4-15

Step 5 - Task Analysis - Analytical

his outputs (Box D,). Although not directly observable, the processes can be inferred by attention to the physical features of the task and the known needs, capabilities, limitations, and motivations of the operator. To understand and identify these processes, interviews with the operators are the best source of data. If you want to know what is going on in the mind of an operator, ask him. Obviously, this technique is subjective; the operator may not really know why he does certain things, or he may wish to deceive the analyst deliberately. Finally, thinking about what he does may actually change what he does. Still, invaluable information can be ob- tained by interviewing operators as they work. (For an example of inter- viewing to study the mental processes involved in electronics troubleshoot- ing, see Rasmussen and Jensen, 1974.)

Figure 4-2 shows that the operator's outputs (Box D) produce system results (Box E) and that information about these results is fed back to the opera- tor's sense organs via displays (Box A). Thus, the man-machine system is a negative feedback system in that information about the system can be com- pared to some standard (i.e., a desired output), and the operator can take corrective action to minimize the system error.

The most difficult part of a task analysis is to identify the possible unplanned modes of operator response. It is not difficult to set down in sequence the tasks and steps in these tasks for well-defined jobs with more or less invariant sequences of actions. Thus, a task description of a calibration technician's job is straightforward. One can readily set down what leads to what, and the identification of potential errors is not difficult.

In other jobs in the NPP, tasks may not be as well-defined, and variable

sequences may be common. Even in "routine" tasks, one must consider how the operator might deviate from the anticipated routine. In less- structured tasks, such as those involved in responding to transients or troubleshooting, the job of the analyst is more difficult. He must iden- tify where and how the operator's responses to unusual events might creatq more demands on the system instead of correcting the situation. He must also identify areas in which operator intentions deviate from written procedures or plant policy. For example, an emergency operating procedure ý(EOP) may be quite specific as to the point in a transient at which the operator should introduce boron-into the primary cooling system of a reactor. Yet, private conversations with operators may indicate that because of economic consequences, this action will not be taken until much later in the transient event. Such information can result in a sizeable change to estimates of the effectiveness of system-required human actions. This part of task analysis is best accomplished by an analyst who has a high level of knowledge of the job he is describing and analyzing and who is well versed in human performance technology, including the psychology and sociology of human behavior. In this effort, he should use the exper- tise of the personnel who perform the tasks.

4-16

Step 6 - Estimate Error Likelihood

Step 6 - Estimate the Likelihood of Each Potential Error

Steps 6, 7, and 8 of the MMSA provide an estimate of the importance of each potential error identified in the task analysis. The importance of an error is a function of its frequency, the efficiency of the recovery fac- tors for the error, the likelihood and severity of the potential conse- quences of the error, and the costs of fixing the underlying ELS.

In the most quantitative form of HRA, the estimated frequency of human error is expressed as a probability estimate. Chapter 20 summarizes the available HEPs that are relevant to NPP operations, and Chapter 5 shows how these HEPs are handled in the analysis. Less quantitative approaches are discussed later in this chapter.

The context of any human behavior must be considered in order to estimate its probabilities. For human activities, interaction (dependence) is the rule rather than the exception. The task analysis for a PRA involves identification of task and people dependencies; these are quantified using procedures that are part of HRA, as discussed in Chapters 5 and 10. The analyst uses the information from the task anal)sis to determine the PSFs that materially affect the probability estimates for each task or subtask. The examples of HRA event trees throughout this Handbook illustrate the types of judgments used to derive these estimates.

The purpose of breaking down a task into inputs, mediating processes, and outputs is to obtain smaller bits or elements of behavior that can more readily be conpared with available data. This task decomposition makes it easier to identify all of the PSFs that influence the reliability of a human task and to evaluate the adequacy of available data for assessing task reliability.

When estimates of HEPs for individual behavioral units have been obtained, they can be combined into estimates of HEPs for larger units of behavior corresponding to entire tasks or groups of tasks. In this recomposition, the estimated HEPs for small behavioral units often must be modified in consideration of their interdependencies. Usually, the estimate of an HEP for a large unit of behavior is not merely the addition of a group of esti- mated HEPs for smaller units of behavior. Before this combination can take place, the interdependencies must be considered and the error contributions modified as described in Chapter 10.

Extrapolations are the rule rather than the exception. Occasionally, if error probability data exist for tasks similar to the tasks of interest, the decomposition of tasks into smaller bits of behavior may be unneces- sary. However, the decomposition is often worthwhile to ensure that all major possibilities for error have been assessed.

4-17

Step 7 - Recovery Factors

Step 8 - Estimate Error Consequences

Step 7 - Estimate the Likelihood That Each Error Will Be Undetected (or Uncorrected)

Other things being equal, the smaller the likelihood that some error will be detected before it causes undesirable consequences, the more important the error is for a PRA. If an error is detected before it causes unde- sirable consequences, we call it a recovered error, on the assumption that detection will result in corrective action. Some errors will be detected by the person who makes them; other errors may be detected by inspectors or by subsequent testing and use of system components. These are all examples of recovered errors. Some errors may not be detected until unwanted conse- quences to the system have occurred; those are unrecovered errors. In an HRA, recovery factors are used to modify the HEPs, since the interest in HRA is in estimating the joint probability that an error will be made and will not be recovered. Chapter 19 treats the topic of recovery factors for HRA.

Step 8 - Estimate the Consequences of Each Undetected (or Uncorrected) Error

The consequences of an error define another aspect of the error's impor- tance. In a well-designed system, a single uncorrected human error does not often cause serious degradation. Although there have been such cases, normally there is sufficient redundancy in the system that these errors will not result in serious consequences. For example, although a calibra- tion technician may miscalibrate a single setpoint for some temperature sensor, there will be other sensors that will indicate a disagreement.

The usual procedure in HRAs is to perform a separate analysis for each system consequence of interest. Generally, quantification of the relative importance of each consequence is not part of the HRA. For example, sepa- rate HRAs would normally be done for the influence of human errors on the risk of personnel injury and on the risk of some economic loss. One would not place these two system consequences on a single continuum of loss.

In PRAs or design trade-off studies, it is often of interest to learn how the probability of failure of a system involving many human tasks would change if different estimated HEPs were used for the individual tasks. Such an assessment is called a sensitivity analysis. As described in Chapter 5, this method is very useful in HRA because estimates of HEPs are ordinarily made with uncertainties larger than those assigned to estimates of failure probabilities of other system components. Sensitivity analysis was used in some of the early analyses in WASH-1400 and also in evaluation of subsequent. PRAs (Kolb et al, 1982b; Berry et al, 1983). It was dis- covered that the probabilities of failure of some large subsystems in NPPs were insensitive to substantial variations in estimated HEPs as well as to variations in assumed distributions of HEPs.

4-18

Step 9 - Suggest System Changes

Step 9 - Suggest Changes to the System

This step is related to the use of MMSA as a design tool. Most considera- tion is given to those potential errors with a high combined probability of (1) occurring, (2) going undetected or uncorrected, (3) causing an un- acceptable system consequence, and (4) being inexpensive and easy to cor- rect. Thus, a task with a high error probability may be less important to system success than some other task with a lower error probability. For example, if the first task has good recovery factors and the second one does not, the latter task may have more potential for degrading the system.

One approach for evaluating the above system considerations as a design tool is represented by Table 4-3, which shows an importance profile for APSs. This profile was designed as a semiquantitative method for risk evaluation to assist management in high-risk industries in allocating funds and other resources available for accident reduction objectively (Swain, 1980a, Appendix B). This profile uses five-point rating scales rather than probabilities. For many industrial purposes, this semiquantitative ap- proach is sufficient for estimating the risk and the cost of fixing any particular APS. The particular profile in Table 4-3 was used in a large petrochemical company as part of a safety improvement program.

The first scale in the profile lists the consequences if an accident oc- curs. The entries for this scale are taken from Table 4-4, which was pre- pared for tasks involving nuclear materials. The definitions of the five consequence categories could be modified for different kinds of potential accidents as well as for changes in monetary values in the past 10 years since the development of the scale.

The second scale, "likelihood of accident," reflects categories of fre- quencies that can be modified for different industries. The frequencies in Table 4-3 were prepared for jobs in the petrochemical industry.

The third scale, "fiscal attractiveness for corrective action," was designed so that the top letter of the scale would represent the worst condition and the bottom letter the best condition. This arrangement is consistent with the other two scales.

In practice, the first and third scales have been used by technical per- sonnel in a plant. The second scale has been used by those who perform the work being rated for potential safety problems. Very little training is required for personnel to understand and use the scales (Swain, 1980a, appendices).

The final result of the scale is a profile of three letters. Since the scales represent three different dimensions, letters were used instead of numbers to discourage the tendency to add or multiply scale values to de- rive a single score. However, within each of the three scales, the scores from individual raters can be combined using conventional techniques as referenced in Chapter 8.

4-19

Table 4-3

Table 4-3 Sample scales for profile of importance of accident-prone situations

Scale #1: Scale #2: Scale #3: Consequences, Likelihood of Fiscal attractiveness

i.e., severity accident for corrective action

A - Category A from Table 4-4

B - Category B from

Table 4-4

C - Category C from Table 4-4

:DI Division of Category D from Table 4-4 into two scale

E values

A - Very likely (>3 times yearly)

B - Possibly three

times yearly

C - Probably twice yearly

D - Perhaps once a year

E - Much less often than once yearly

A - Most attractive

(least cost, e.g., <$100)

B - $100 to <$1000

C - $1000 to <$10,000

D - $10,000 to <$100,000

E - Least attractive: $100,000 or more to correct APS

4-20

Table 4-4

Table 4-4 Consequence categories*

HIGH POTENTIAL CONSEQUENCES*

CATEGORY A--Those situations or occurrences that can or have led to:

Fatal or total permanent disability of more than one person, or to disabling injury or disabling occupational illness of five or more from one occurrence;

Estimated loss, damage, or costs of $100,000 or more;

Incidents involving nuclear devices or more than 2,000 gallons of liquid oxygen, liquid fuels, metallic alkalis, pyrophorics, hyper- golics, propellants, primary explosives, or initiators; injury or damage to private property;

" Loss or theft of radioactive or fatally toxic materials that could constitute a hazard to health;

* Whole body exposure to 25 rem or higher release of radiation or other relatively dangerous hazards to life;

" Pressure systems with a potential of more than 300,000 foot pounds of energy.

" CATEGORY B--Those situations or occurrences that can or have led to:

" Loss or damage of $25,000 to $99,000;

* Vehicle fire or accident while transporting radioactive materials or more than 2,000 gallons of pyrophorics, explosives, or imme- diately toxic materials, or when such vehicle is contaminated or its shipment is damaged;

" A fatality or imminent fatality or total permanent disability;

* Disabling injury or disabling occupational illness of two or more from one occurrence, loss of sight in one eye, loss of hand, leg, foot, or total disability potential for more than one year from any occupational exposure or occupationally related exposure;

Pressure systems with a potential of more than 150,000 foot pounds of energy.

CATEGORY C--Any other situation or occurrence that meets the definition of a High Poten- tial, including any disabling injury or occupational illness that will or apparently may result in loss of any member, any sight, or permanent loss of use of a member, or in total disability of over two weeks' duration;

" Any damage or loss over $50 from a fire or explosion or more than $1,000 from any other cause;

* Any pressure system with a potential energy of more than 1,500 foot pounds of force.

NON-HIGH POTENTIAL CONSEQUENCES

" CATEGORY D--Any damage or loss of less than $50 from fire or explosion or between $500 and $999 from any other cause--providing these do not otherwise meet the definition of High Potential;

" Workmen's Compensation case, disability, lost time, minor injury, error, or loss that does not meet the definition of High Potential.

Whole body exposure to 5 rem or other rela- tively high exposures to radioactive or dan- gerously toxic materials or other relatively serious hazards to health, body, and life;

Note: Categories are determined by the highest category in which they fit.

Taken from Allison (1973) who defines a high-potential situation as one in which "near-misses, unsafe practices, conditions delayed treatment cases and accidents that did or could under slightly different circumstances potentially result in serious injury or damage."

4-21

Step 10 - Repeat MMSA

Decisions to incorporate ergonomics changes in a system often require trade-offs of various criteria and costs. Although the issue of such trade-offs is outside the purview of this Handbook, Table 4-5 lists some important system criteria. (These criteria are not listed in order of importance.) It is clear that costly changes should not be recommended solely because a design may deviate slightly from the optimum. However, gross deviations from optimum ergonomics design principles have resulted in errors in large numbers and varieties of man-machine systems. Kemeny (1979) and Sugarman (1979) describe several such deviations that contrib- uted to human errors in the TMI accident. Based on experience with other complex man-machine systems, the incorporation of standard ergonomics principles in the early design stages of TMI-2 would not have been costly.

A useful rule is to follow optimal ergonomics principles in designing and building a plant unless there are overwhelming reasons to disregard them. The consideration of such principles early in the design phases of a system will usually allow for the incorporation of optimum human factors design at minimal cost.

Using HRA, suitable design changes can be developed and evaluated to reduce the probability of system degradation. A candidate design change may reduce the probability of an error (e.g., by reducing the number of oppor- tunities to make the error), it may increase the likelihood that an error will be detected or corrected (e.g., by providing better feedback to the operator or by adding an inspection step), or it might involve some provi- sion for the system to tolerate an error.

The design changes may address any of the PSFs associated with the poten- tial error. Sometimes the design change is as simple as changing the scale on a meter, the color of an indicator lamp, or the size of a handle. Even for a plant in operation, such changes may not be very expensive (Seminara et al, 1980). At other times the design change might be more costly, such as a rearrangement of controls, displays, or panels. At times, a desired design change may not be feasible for some reason, and the only alternative may be to provide human redundancy by assigning an extra operator when certain procedures are to be carried out, e.g., assigning a second control room operator who is dedicated to maintaining adequate levels of coolant in the primary and secondary loops, a practice initiated by some plants after the TMI accident.

Obviously, the decisions are not always as simple and clear-cut as the above examples suggest. The final decision will have to be made on the basis of acceptable risk and cost-effectiveness.

Step 10 - Evaluate the Suggested Changes (Repeat Steps 1 through 9)

Finally, when using MMSA as a design tool, each suggested change to the system must be reevaluated by repeating most of the above steps. Thus, MMSA is iterative in its design application--the steps are repeated until the estimated human error contribution to system degradation has been reduced to some tolerable level in terms of system effects and costs of the

4-22

Table 4-5

Table 4-5 System criteria for trade-off considerations*

1. Speed (mean and variability) 2. Accuracy (constant error) and precision (variable error) 3. Dependability (maintainability and reliability) 4. Adaptability (of equipment to changes in requirements, equipment

design, or operating conditions) 5. Mobility (including dispersal requirements) 6. Graceful degradation (ability to continue to operate although at sub-

standard levels of performance) 7. Equipment test life (need to avoid adverse effects of confidence or

other testing) 8. Completeness or exhaustiveness (the proportion of system parameters

that must be measured) 9. Personal involvement (extent to which personnel identify themselves

with their tasks or are aware of system operation) 10. Personnel hazard and risk of equipment damage 11. Delivery schedule (time for system to become operational) 12. Equipment weight and/or volume 13. Training costs (personnel, time, facilities) 14. Manning level and quality 15. Development costs 16. Logistics costs and policy (pipeline and spares provisioning policies) 17. Equipment unit cost in production (including spares) 18. System environment (ability to operate under various climatic, ter-

rain, sociopsychological, political, and other conditions) 19. Selling costs (including advertising) 20. Aesthetic attractiveness 21. Effects on environment (ecological considerations) 22. Costs of employee dissatisfaction (indifferent and slow work, absen-

teeism, turnover, grievances, strikes, sabotage)

*Modified from Swain and Wohl, 1961.

4-23

Step 10 - Repeat MMSA

changes. The contribution of human error may be reduced either directly, by improvements made to reduce error frequency or to increase the proba- bility and error recovery, or indirectly, by design changes that will tolerate human errors. The best solution is obviously situation-specific.

4-24

Ch. 5. A Technique for HRA Overview; Background of THERP

CHAPTER 5. A TECHNIQUE FOR HUMAN RELIABILITY ANALYSIS

Overview

This chapter describes a technique for human reliability analysis (HRA) that has the acronym THERP (Technique for Human Error Rate Prediction) and includes examples from probabilistic risk assessments (PRAs) of nuclear power plants (NPPs) in which this HRA technique has been employed. This chapter is restricted to a discussion of the use of single-point estimates of basic and conditional human error probabilities (HEPs); Chapter 7 shows how distributions of HEPs for a task are handled in a PRA. A graphic method of task analysis called the HRA event tree is presented as a means of diagramming correct and incorrect human actions. For PRA use, the out- puts of the HRA event trees are fed into system or functional event trees or system fault trees. For design trade-off use, the HRA outputs are en- tered into appropriate levels of system or comp6nent reliability analyses.

Estimates of basic and conditional HEPs to be used in THERP are compiled in Chapter 20, based on the models and data presented in Part III of the Hand- book. Chapter 6 describes how such data have been derived. This chapter presents the mechanics of incorporating the estimated HEPs into the HRA event trees. For this purpose, an example of an HRA is provided. The incorporation of HRA in design trade-off studies and in PRA is discussed, including the use of sensitivity analysis when data on HEPs are lacking or are speculative. The general form of an HRA for PRA is described, with the note that there are possibilities for different levels of incorporation of HRA results in the PRA. Finally, some comments are made on the validity of HRA using THERP.

Background of THERP

In Chapter 2, we defined human reliability as the probability of successful performance of only those human activities necessary for either a reliable or an available system. HRA is a method by which human reliability is estimated. The HRA method described in this chapter is an extension of human reliability studies made at Sandia National Laboratories (SNL) in the early 1950s in which reliability analysts estimated the quantitative influ- ence of first-order human failure terms on the reliability of military systems and components. In the early 1960s, the method used in these early HRA studies was expanded and refined to permit more detailed consideration of the human component in system reliability. Subsequent development of the method has included its applications to a large variety of military systems, to the U.S. Nuclear Regulatory Commission's (NRC's) Reactor Safety Study (WASH-1400), and to subsequent human reliability problems in NRC- and utility-supported PRAs of a number of NPPs (Dougherty, 1983; Kolb et al, 1982a; Carlson et al, 1983).

5-1

Description of THERP

Most of the applications of THERP have involved estimates of the probabil- ities that system-required tasks will be executed correctly and, in some cases, within specified time limits. Nearly all of the applications have assessed the probability that individuals or teams would carry out speci- fied procedures correctly under varying degrees of stress.. The types of tasks include assembly of equipment in manufacturing processes, calibrating and testing equipment, air crews performing military tasks (sometimes under great stress), and the various NPP activities assessed in WASH-1400 and subsequent PRAs of NPP operations. It is only with the most recent PRAs that the so-called cognitive aspects of behavior have been addressed (Chapters 3 and 12).

Since the use of THERP in WASH-1400, there have been some refinements in this method and the data it uses:

(1) The development of several human performance models:

(a) a model of positive dependence

(b) several models for the perceptual and response aspects of human activities in NPPs

(c) an interim model to assess cumulative probability over time of correct diagnosis of transients and other abornmal events in NPPs (i.e., the "cognitive" aspect)

(d) expansion of the extremely high stress model to include several other stress levels

(e) a model for assessing team interaction in NPP control rooms

(2) Use of estimated distributions to permit employment of a Monte Carlo or equivalent procedure for handling distributions of HEPs.

(3) A method of estimating and propagating uncertainty bounds in an HRA.

(4) Partial verification of some predictions of rule-based behavior.

(5) The Handbook itself, which brings together the latest developments.

Description of THERP

At the Sixth Annual Meeting of the Human Factors Society in November 1962, the acronym THERP was first used to designate the human reliability method developed at SNL (Swain, 1963a). Until publication of the draft version of this Handbook, we used the expression human error rate (HER) interchange- ably with human error probability (HEP). For reasons stated in Chapter 2, we have dropped the term HER in favor of HEP. However, since the acronym THERP is now well established, we retain it despite its use of the term "human error rate."

5-2

Description of THERP

The following is a revised definition of this human reliability technique:

THERP (Technique for Human Error Rate Prediction) is a method to predict human error probabilities and to evaluate the degradation of a man-machine system likely to be caused by human errors alone or in connection with equipment functioning, operational procedures and practices, or other system and human characteristics that influence system behavior.

The method uses conventional reliability technology with modifications ap- propriate to the greater variability, unpredictability, and interdependence of human performance as compared with that of equipment performance. The steps in THERP are similar to those in conventional reliability analysis, except that human activities are substituted for equipment outputs. The steps are:

(1) Define the system failures of interest. These pertain to system func- tions that may be influenced by human errors and for which error probabilities are to be estimated.

(2) List and analyze the related human operations. This step is the task analysis described in Chapter 4.

(3) Estimate the relevant error probabilities.

(4) Estimate the effects of human errors on the system failure events. This step usually involves integration of the HRA with a system relia- bility analysis.

(5) Recommend changes to the system and recalculate the system failure probabilities. (The procedure is iterative.)

The above five steps typify the use of HRA as a tool in system design. For assessments only, Step 5 is not required, and not all tasks will be ana- lyzed and included in the HRA. For PRA purposes, the HRA will include only those tasks that will have a material effect in the system event or fault trees.

In an HRA, the primary interest is in estimating the following parameters:

(1) Task Reliability--Task reliability is defined as 1.0 minus the esti- mated probability of task failure. For each task to be analyzed, we use Chapter 20 or other data sources to estimate the probability that it will be completed successfully (within some period of time if time is a requirement).

(2) Recovery Factors--The probability of detecting and correcting incor- rect task performance in time to avoid undesirable consequences con- stitutes the recovery factor. In any man-machine system, there are usually several recovery factors, e.g., inspections that increase the probability of detecting errors before they affect the system. Recov- ery factors are discussed in Chapter 19.

5-3

Model Versus Method

(3) Task Effects--This is the probability that unrecovered errors will result in undesirable consequences to a system. A separate calcula- tion is made for each system consequence of interest. Therefore, one may estimate the effects of the same human error on more than one system outcome. These calculations are usually a joint effort by system and human reliability analysts. For PRA, the system analysts will perform these calculations.

(4) Importance of Effects--This is the importance of each set of unde- sirable effects to a system in terms of cost or other criteria. Generally, no attempt is made to quantify this parameter; it is often a value judgment made by persons in authority. In PRAs of NPP opera- tions, a frequently used criterion is avoidance of a core meltdown.

THERP is used to generate quantitative estimates of the first two param- eters based on estimates of human reliability, on the dependences among human performance, equipment performance, other system events, and outside influences. The operator's perception or anticipation of the effects of his tasks (parameter 3) is a potent PSF and is included in formulating the estimates of HEPs in the first two parameters.

Model Versus Method

In the draft issue of this Handbook, we described THERP as a human relia- bility model. We now think it preferable to restrict the term model to the kinds of human performance models described in Part III of the Handbook. To clarify our position, the following is a quote from an earlier report (Swain and Guttmann, 1975):

The term "model" has several meanings and is used loosely in the behavioral sciences. For our purposes in describing THERP ... , we will use the narrow meaning of "model," defined as "a set of rela- tions and operating principles." Although this may slight some who are interested in theory development, THERP was developed as an applied technique, for use as a fast and relatively simple method of providing data and recommendations to system designers and analysts who require a quantitative estimate of the effects of human errors on system performance.

THERP is not a model in the usual sense of a hypothetical analogy, although it is regarded as a modest form of mathematical (Boolean) modeling, representing behavior variability by simple equations deal- ing with equipment parameters, human redundancy, training, stress, etc. When using this method, we are cognizant of its limitations, but for practical reliability work, these limitations are sur- prisingly few. The only limitation of consequence is the dearth of data on human error rates in many areas. However, the variables affecting human reliability are fairly well understood, and we are expanding the data base for human error rates and the effects of the pertinent performance shaping factors. The method of assessing the effects of errors is well established, both in logic and experience,

5-4

HRA Event Trees

and, while refinements will continue to be made, it does not appear that any major changes are indicated. Therefore, we suggest that THERP be regarded as a practical method of predicting human reliabil- ity rather than as a hypothetical model.

HRA Event Trees

The basic tool of THERP is a form of event tree called the probability tree diagram (Figure 5-1) in use at SNL since the 1950s (MUller, 1964). In this issue of the Handbook, we use the new term HRA Event Tree to distinguish our event trees from other kinds of event trees used in PRA. In the HRA event tree, the limbs represent a binary decision process, i.e., correct or incorrect performance are the only choices. Thus, at every binary branch- ing, the probabilities of the events must sum to 1.0. Limbs in the HRA event tree show different human activities as well as different conditions or influences upon these activities. The values assigned to all human activities depicted by the tree limbs (except those in the first branching) are conditional probabilities. The first limbs may also be conditional probabilities if they represent a carry-over from some other tree. That is, the first branching in an HRA event tree may represent the outputs of another tree.

Table 5-1 presents the symbology used with the limbs of the HRA event tree. Note that a letter can have more than one meaning depending on whether it is in quotes, capitalized, or lower case. In some cases, it is useful to provide short descriptions of system events or human actions along with the symbols.

We use capital letters in quotes to denote tasks and events. A capital Roman letter in quotes denotes a human task, and a capital Greek letter in quotes denotes a system or component event or state. We use Roman letters not in quotes to denote the success or failure of a human task and Greek letters not in quotes to denote the desirable or undesirable state of a system or component. Capital letters not in quotes indicate failures or undesirable states, or their probabilities. Small letters indicate suc- cesses or desirable states, or their probabilities. For example, Task "A" in Figure 5-1 might represent the task of a calibration technician setting up his test equipment before calibrating some sensor. The lower case letter a represents the fact that the technician has correctly set up his test equipment and also stands for the probability of correct performance. The capital letter A represents the fact that the technician has incor- rectly set up the test equipment and also stands for the probability of incorrect performance.

The letter b represents the fact of correctly performing Task "B," the second task performed, and b also stands for the probability of correct performance. Task "B" might be the calibration of the sensor mentioned above. The letter B stands for the incorrect performance as well as for the probability of incorrect performance. The dependences between Tasks "A" and "B" are represented by the symbols bja, Bia, blA, and BIA. Nor- mally the conditional relationships are understood, and the letters b and B are written without the conditional qualifications.

5-5

Figure 5-1

a A

bla BIA

SERIES S F F F PARALLEL S S S F

TASK "A = THE FIRST TASK

TASK B" = THE SECOND TASK

a = PROBABILITY OF SUCCESSFUL PERFORMANCE OF TASK 'A'

A = PROBABILITY OF UNSUCCESSFUL PERFORMANCE OF TASK "A'

bla = PROBABILITY OF SUCCESSFUL PERFORMANCE OF TASK "B" GIVEN a

Bla = PROBABILITY OF UNSUCCESSFUL PERFORMANCE OF TASK "B" GIVEN a

bIA = PROBABILITY OF SUCCESSFUL PERFORMANCE OF TASK "B" GIVEN A

BIA = PROBABILITY OF UNSUCCESSFUL PERFORMANCE OF TASK "B" GIVEN A

FOR THE SERIES SYSTEM:

Pr[S] = a(bla)

Pr[F] = 1 - a(bla) = a(Bla) + A(bIA) + A(BIA)

FOR THE PARALLEL SYSTEM:

Pr[S] = I - A(BIA) = a(bla) + a(Bla) + A(bIA) Pr[F] = A(BIA)

Figure 5-1 HRA event tree for series or parallel system.

5-6

Table 5-1

Table 5-1. symbology used in THERP method of HRA

Symbol Meaninq

Capital Roman letter in quotes, e.g., "A"

Capital Roman letter not in quotes, e.g., A (except F and S)

Capital letter a path through

F at the end of an HRA event tree

Capital letter S at the end of

Lower case Roman letter, e.g., a, b, etc. (except i and r)

1. The human action itself; e.g., Task "A." Note: The expected or required sequence of tasks is designated by the alphabetical ordering; e.g., Task "A" is the first task, Task "B" is the second task, etc.

1. The estimated probability of incor- rect performance of a human action; e.g., A stands for the probability of incorrect performance of Task "A." This is the usual meaning of A.

2. Incorrect performance of a human action; e.g., A stands for the in- correct performance of Task "A."

1. The end point or end-failure proba- bility for a failure path in the tree.

1. The end point or end-success proba- bility for a success path in the tree.

1. The estimated probability of success- ful performance of a human action; e.g., a stands for the probability of correct performance of Task A. This is the usual meaning of a.

2. Successful performance of a human action; e.g., a stands for the correct performance of Task "A."

1. ith and rth task.

1. The number of events or tasks in a sequence. Not to be confused with an n, which indicates the success (or probability of success) on Task "N."

1. The system event itself, or the equipment state, e.g., Event "A."

1. The estimated probability of equip- ment failure or of nonoccurrence of some system event (not a human activity).

1. The estimated probability of equip- ment failure or of occurrence of some system event (not a human activity).

Lower case Roman letters, i and r

Lower case underlined Roman letter, n

Capital Greek letter in quotes, e.g., "A" (delta)

Capital Greek letters not in quotes, A

Lower case Greek letters, e.go, 6

5-7

HRA Event Trees

Note that the above symbology is presented as an aid only, since it is the one that we have been using. Any symbology can be used as long as it makes the required distinctions. Most analysts will evolve their own "shorthand" as they prepare their HRA event trees; rigid consistency is not essential. When convenient, we have used other symbols, as in the example in Figure 10-6 near the end of Chapter 10. In this example, we use a lower case Roman e for equipment success (the capital Roman E is the same as the capital Greek epsilon). No problems will occur as long as the meaning of the symbols is understood by the users.

In a system, sometimes all of some set of human activities must be per- formed correctly for system success to be achieved. We call this appli- cation a series system. In other, cases, a system will fail only if all of the human activities in a set are performed incorrectly. We call this application a parallel system.

In Figure 5-1, if we are dealing with a series system, only the complete- success path, i.e., a(bla), is designated as ending with S, which stands for a success path. All the other paths end in F, which stands for fail- ure. Thus, for a series system, the only success path is one in which both tasks are correctly done, i.e., the calibration technician correctly sets up his test equipment and also correctly calibrates the sensor. For other problems, the interest might be in performing either task correctly, i.e., a parallel system, and any path other than A(BIA), the complete-failure path, would be considered a success path, as indicated in the figure. For example, Tasks "A" and "B" might represent the restoration of two blocking valves that are in parallel. As long as the operator remembers to restore at least one of the two valves, the system will function properly. It is the system application that determines which paths through the tree are considered success paths or failure paths.

Since the limbs in the tree represent a binary decision process, a + A in

Figure 5-1 must equal 1.0, and bja + Bla as well as bjA + BIA must equal 1.0. At every binary branch, the probabilities of the two limbs sum to 1.0. In other event trees there could be more than two limbs in a branch- ing to represent different conditions or events, or different levels of correctness or incorrectness, but in all cases, the probabilities of the limbs in any one branching must sum to 1.0. In our event trees, we use binary branching even if there are three states to be modeled (e.g., three stress levels: very low, optimal, and moderately high). In this example, one could show a binary branching to indicate (1) the probability of very low stress and (2) the probability of other than very low stress. The limb designated as "other than very low stress" would then be subject to a further binary branching to indicate the probabilities of (1) optimal and (2) moderately high stress levels. The use of binary branching facilitates computer usage. As with any HRA event tree, the sum of the probabilities at the terminals of all paths also must sum to 1.0. Thus, in Figure 5-1, (a x bja) + (a x Bla) + -(A x b1A) + (A x BJA) = 1.0.

The limbs in an HRA event tree may represent correct or incorrect human outputs, errors of discrimination, or mediating processes related to those

5-8

Event Trees Versus Fault Trees

human outputs. Any limb in the tree may also represent correctness or incorrectness of some system-required task or activity or it may denote some other human task or activity that is not system-required but that has potential for reducing the probability of achieving desired system func- tions. In the HRAs done for WASH-1400, for example, besides depicting the selection of certain correct switches, tree limbs depicted the selection of nearby switches that could cause serious problems if inadvertently selected (see Problem 2, Chapter 21). Thus, limbs in HRA event trees can be used to represent plausible and important extraneous actions. Of course, not all

extraneous actions can be identified in advance. Although most such ac- tions are unimportant in terms of system consequences, it is always possi- ble that some important extraneous action will be overlooked in an HRA. The more detailed the task analysis behind the HRA event tree, the greater

the likelihood of identifying the important, plausible extraneous actions. The identification of extraneous acts resulting from incorrect interpreta- tion of signals is very difficult and speculative. For PRA purposes, how- ever, it is often sufficient to estimate the probability that some small set of correct actions will be taken and to designate any other action as system failure. Therefore, although the prediction of errors of inter- pretation is often speculative, it is often the case that considerable latitude is allowed the analyst.

Once the HIRA event tree is properly drawn, the mathematics are simple. When the estimates of the conditional probabilities of success or failure of each limb in the tree have been determined, the probability of each path

through the tree is calculated by multiplying the probabilities of all limbs in the path. This does not correspond to the simple multiplicative model, that is, the multiplication of task probabilities without the as- sumption of any task dependence. The use of conditional probabilities takes into account the interdependence between the limbs in the tree, and no errors will result from the use of this simple mathematical approach.

Errors arise from incorrectly estimating the HEPs for the tasks represented by a limb.

Event Trees Versus Fault Trees

The HRA event tree starts with any convenient point in a system procedure

and works forward in time. The level of detail depends on the purpose of the analysis; for PRA, the level of detail in an HRA is generally less than when the HRA is being done as part of design trade-off studies. The top-

down procedure enables the user to analyze what leads to what and allows him to identify the important events affecting human performance. It follows the same sequence in time as the underlying task analysis and facilitates showing in graphic form those parts of the task analysis rele-

vant to the HRA.

In the fault tree approach, one starts with some fault and works backwards

in time (Vesely et al, 1981). As with fault trees, boxes with short de- scriptions of events may be used in addition to symbols in an HRA event

tree to assist the reader. Also, as with fault trees, one limb in an HRA event tree may stand for another entire tree. Nielsen (1971 and 1974) has

5-9

Nominal, Basic, Conditional, and Joint Probabilities

developed a combination of event trees and fault trees called Cause/Conse- quence Diagrams; this combination has some of the advantages of both sche- matic techniques.

It is more difficult to represent dependence among human actions in a fault tree than in an event tree. In the latter, the conditional probabilities are shown directly following the events or actions that generated the conditionality. In fault tree modeling, the dependence may show up in different boxes of the tree or in different fault trees altogether. How- ever, if great care is taken, human dependence can be represented in fault trees, and one should get the same answer to a human reliability problem regardless of the graphic method employed.

The event tree format presented in this chapter is the one we used for the HRA in WASH-1400 and in the Arkansas Nuclear One Unit 1 PRA (Kolb et al, 1982a). In WASH-1400, the system analysts used a different format for their event trees, in which the progression is from left to right, and in each branching, the lower limb is the failure limb (Figure 5-2). Varia- tions of this event tree format have been used by Dougherty (1983); by Harrington (in Cook et al, 1981), whose format is called the Operator Key Action Event Tree; by Brown et al (1982) and Brinsfield et al (1983), who use the term Operator Action Event Tree; and by Wreathall (1981, 1982) (see also Hall et al, 1982), whose format is known as the Operator Action Tree System (OATS). Full descriptions of these alternative graphic methods are found in the referenced articles. Which format is used for an event tree is a matter of convenience for the analyst. If used correctly, any format will provide the correct answer.

Nominal, Basic, Conditional, and Joint Probabilities

Use of the HRA event tree is based on the assumption that the estimates associated with the limbs are conditional probabilities. Considerable error in estimating the probabilities of paths through the tree can occur unless conditional probabilities are used. If independence of acts is mistakenly assumed, the failure contributions of people to systems can be significantly underestimated.

For convenience, we repeat some definitions of different kinds of HEPs from Chapter 2. The nominal HEP is the probability of a given human error when the effects of plant-specific PSFs have not yet been considered. The basic HEP (BHEP) is the probability of human error without considering the condi- tional influence of other tasks. The conditional HEP (CHEP) is a modifica- tion of the BHEP to account for influences of other tasks or events. The joint HEP (JHEP) is the probability of human error on all tasks that must be performed correctly to achieve some end result. There are also basic, conditional, and joint human success probabilities (HSPs), but usually we speak only of the HEPs. To avoid excessive use of abbreviations, we will generally write out the words "conditional" and "joint" in association with HEPs, but BHEP will be used frequently.

To illustrate these HEP terms, let us take the tree in Figure 5-1 and assume a parallel system, i.e., one in which all of the human activities

5-10

Figure 5-2

I A a I o D EPIPE ELECTRIC FISSIONBREAK POWER ECCS PRODUCT i ,NTEGRITYi REMOVAL

BASIC TREE

FAILS PAx PE1 PE1

FAILS r PD1

SUCCEEDS PD 1 PE A x PD1 X PE2

PA x PC1

LPAX PC8 X PE3PE3 "PC1 I PA x PC1 x PD2

INITIATING PD PA X PC1 X PD2 X PE4EVENT PD2

PA PA x PB

PE5 xA x PEB X PE 1.PA x P13 x PD3

FAJLS PD3 PG PAx P13 x PO3 x PE6

PB PA x P S x PC2

p - PA x PB x PC 2 x PE7 PC2 PA x PB x PC2 X PD4

PD4 L E PA x PB3 x PC2 X PD4 X PE8

REDUCED TREE

e I PA X PE1I

PA x PD 1

PPAX PD 1 X PE 2INITIATING PlPE2 1 2

EVENT PA X PC1

PA PC, t2 PAx XC 1 x PD2 PD2P

PB PA X PS

NOTE - SINCE THE PROBABILITY OF FAILURE, P. IS GENERALLY LESS THAN 0.1, THE PROBABILITY OF SUCCESS (1 - P) IS ALWAYS CLOSE TO 1. THUS, THE PROBABILITY ASSOCIATED WITH THE UPPER (SUCCESS) BRANCHES IN THE TREE IS ASSUMED TO BE 1.

Figure 5-2 Simplified event trees for a large LOCA (from Figure 4-4 of the Main Report to WASH-1400).

5-11

Example of an HRA

must be performed incorrectly for the system to fail. The only failure path is the joint HEP, A(BIA), i.e., failure occurs only if Tasks "A" and "B" are both performed incorrectly. All the other paths lead to system success. Assume that Tasks "A" and "B" are the opening of two valves after maintenance and that the system will function properly if either valve is opened. Further assume that the valves are next to each other, so that if the operator forgets to open one valve, he is 100% sure to forget to open the other valve, i.e., there is complete dependence between the two acts. Also assume that the BHEP of forgetting to open a valve of this type is 10-2. If we mistakenly assume independence between the opening operations for these two valves, we will get an unrealistically low joint HEP estimate of A x B = 10-2 x 10- 2 = 10-4. However, if we correctly assign the conditional HEP of 1.0 to BjA, our estimate of the joint HEP is A(BJA) = 102 x 1.0 = 10-. This example illustrates the importance of incorporat- ing dependencies among the events in an HRA event tree. Chapter 10 presents the dependence model in detail.

An Example of a Human Reliability Analysis

To illustrate the use of HRA event trees, Figure 5-3 is based on a case study described by Bell (1983) in which she identifies the major human activities and errors in the establishment of the appropriate steam genera- tor feed-and-bleed procedure following a loss of both normal and emergency feed to the steam generators in a pressurized water reactor (PWR). The loss of all feedwater results in failure to maintain cooling of the nuclear reactor core unless the operating crew establishes feed and bleed. The tree represents the activities of a team of three licensed reactor opera- tors, one of whom is the shift supervisor.

In this chapter, we use the tree merely to illustrate the mechanics of the logic and calculation of the overall system failure probability based on single-point estimates of HEPs. In Appendix A, we expand this tree and add uncertainty bounds for each HEP to illustrate the handling of distributions and a method of propagating uncertainty bounds through an HRA. A similar tree is also used in Problem 1 in Chapter 21 to provide a complete example that ties together models for dependence, diagnosis, and other human per- formance considerations, including the use of the tables of estimated HEPs in Chapter 20.

For this example, we have arbitrarily omitted some recovery factors and have defined six end-failure terms in the tree. In the real world, some failure paths would be insignificant because of system recovery factors. The reader is asked to accept the estimated HEPs without question since the example is given for purposes of illustrating the HRA event tree structure only. In Chapter 21, the derivation of each estimated HEP is explained fully.

The sequence of events is presented at the bottom of the figure. The letter A stands for the joint HEP of all three people failing to initiate some kind of action after turning off the auditory and flashing signals from some annunciators (ANNs). An estimate of .00008 is assigned to this

0 5-12

Figure 5-3

a = .99992 A =.00008

F 1 b = .99

C= .9997_ \ .00015

d =.9984 D =.0016 F2

F3 e =.9984 E =.0016

g=.99999 G =.00001

h =.9984 H =.0016

g=.99999 G =.00001

F 5 k =.9999 K =.0001

ST F 6

JNEPs EVENTS JOINT HEP* FOR 3 OPERATORS

A FAIL TO INITIATE ACTION TO ANNs* .00008

B MISDIAGNOSIS .01

C FAIL TO INITIATE ACTION TO ANN .00015

0 OMIT STEP 2.4 .0016

E OMIT STEP 2.5 .0016

G FAIL TO INITIATE ACTION TO ANN .00001

N OMIT STEP 2.8 .0016

K FAIL TO INITIATE HPI** .0001

*ANN = ANNUNCIATOR S* HPI = HIGH- PRESSURE INJECTION

Figure 5-3 HRA event tree for loss of steam generator feed.

5-13

Example of an HRA

joint HEP. As shown in the tree, if A occurs, the system will fail. There are no recovery factors for this joint error.

At this point, a word of explanation is in order regarding such a small number as .00008. Obviously, the data for HRA are not so accurate that the number 8 x 10 5 can be regarded as "right on." We could round this number to 10 4. However, the practice we follow is to postpone rounding until the final answer to a problem is obtained.

B stands for the joint HEP of .01 of misdiagnosis by the three operators of the pattern of annunciators. Note that there is a recovery factor (Task "C") of a special ANN to show that the margin to saturation in the steam generator has exceeded tolerable limits. C stands for the failure of all three to be cued properly by this ANN. If B and C happen, the system will fail.

However, if c occurs, the crew has been cued and will proceed to the next sequence of activities. Note that in the tree, the end of the path a-B-c leads back to the main success path in the tree. Thus, in this tree, the path a-B-c is equivalent in terms of success, not of probability, to the path a-b. One could repeat the tree beginning at the branching for Task "D" so that it would begin following c. This repetition is unnecessary; the dotted line means the same thing.

D stands for the omission of an important step in the written procedures, which we designate as a first-order human failure term. E is a similar error of omission, but there is a recovery factor, Task "G,, consisting of the cueing by an ANN of the loss of feed. If g occurs, the dotted line shows that the path E-g is equivalent in terms of success to the limb e. H stands for another error of omitting a critical step in the procedure, with the same recovery factor as for E. Finally, K stands for failure to initiate high-pressure injection (HPI) by the use of the proper feed- and-bleed method.

To arrive at the overall failure probability, the exact failure equation involves summing the probabilities of all failure paths in the tree. There are 17 failure paths, which are listed below with their probabilities:

A .00008

aBC .0000015

aBcD .000016

abD .0015839

abdEG -I0-8

abdEgHG -10-11

abdEghK _1o-7

abdEgHgK -I0-10

abdeHG -10-8

5-14

Example of an HRA

abdeHgK -10 '

abdehK .0001

aBcdEG _I0-I0

aBcdEgHG -I0-13

aBcdEgHgK 10- 12

aBcdeHG -10-10

aBcdeHgK -10-9

aBcdehK _10-6

=.00178 .002 (rounded)

When all the HEPs are .01 or smaller, the exact failure equation can be approximated by summing only the primary failure paths, ignoring all the success limbs. There are six primary failure paths. Thus, an approxima- tion for the total failure term, FT, given each HEP 4 .01, is

FTIHEPs 4 .01 A + BC + D + EG + HG + K

.00008 + (.01 x .00015) + .0016 + (.0016 x .00001)

+ (.0016 x .00001) + .0001 = .0018 = .002 (rounded)

The rounded values of the approximate and the exact equations are the same. The accuracy of the approximation decreases as the number of terms or the values of the HEPs increase.

Another way of working the problem is to use the exact success equation:

S = a(b +Bc)d(e + Eg)(h +Hg)kT

= .99992 x [.99 + (.01 x .99985)] x .9984 x [.9984 + (.0016 x .99999)]

x [.9984 + (.0016 x .99999)] x .9999 = .9982188 = .998

As a final check on the logic and arithmetic,

F = 1.0 - .9982188 = .0018 = .002T

The above HRA event tree and the calculations are typical of the use of this approach for single-point estimates. However, for most PRA work, each human activity is characterized by a distribution of HEPs. The HRA event tree will be the same, but the total-failure term will be expressed as a distribution of HEPs. This method is described in Chapter 7 and Appendix A and provides an example of these points, using an expansion of this tree. Examples of HRA event trees to illustrate dependence and errors of diag- nosis are presented in Chapters 10 and 12.

5-15

Use of HRA Outputs

Use of HRA Outputs in System Reliability Studies

The outputs from the HRA will consist of estimates of success and failure probabilities of human actions or tasks that can be used in design trade- off studies or in PRAs. Application of HRA to these two areas and the use of sensitivity analysis for both are described below.

Design Trade-Off Studies

In systems under design and development, there are often reliability goals set by system reliability specialists. "Tolerable failure probabilities" are set for different subsystems or functions so that their sum will not exceed 1 minus the total system reliability goal. A particular subsystem may consist of several items of equipment, each of which is allocated a tolerable failure probability. This process is called reliability alloca- tion. The human contribution to the tolerable failure probabilities may occur at the component level or at some higher- level. Calculation of the estimated human error contribution at any level can be used to determine if that contribution itself is tolerable. If the human contribution to the failure probability is too large, some change is necessary. This applica- tion of HRA falls within the latter steps in the man-machine systems analy- sis (MMSA) described in Chapter 4. As noted in step 9 of the NMSA (Table 4-1), changes are proposed, and the estimated effects of these changes are analyzed in an iteration of the MMSA (Step 10). Historically, most appli- cations of THERP preceding the WASH-1400 safety study were in this area of application.

Probabilistic Risk Assessment

In a PRA, the HRA is used to estimate the human error contribution to the failure of system components or functions. If the contribution is too large, presumably someone will be directed to do something about it. If PRA is done as part of the NRC licensing process and if the PRA yields results that are not tolerable, corrective actions are appropriate if the plant is to continue in operation. In Chapter 21, Problem 3 describes an informal study in which it was judged that the probability of plant personnel coping successfully with a design-basis loss-of-coolant accident (LOCA) was not high enough, and changes in written procedures and training were necessary for the plant to continue in operation.

In a PRA of an NPP system, the estimates of human reliability can be included in system or functional event trees or in system fault trees, pro- vided that the dependencies between human tasks are also incorporated. Thus, the outputs of the HRA are estimates of the human component reliabil- ity and can be handled just like estimates of the reliability of other system components.

In WASH-1400, for example, the fault tree shown as Figure II 5-45, p 11-303 in Appendix II, indicates that an operator might forget to open a

5-16

Use of HRA Outputs

motor-operated valve (MOV-1866E) after a monthly flush of the High Pressure Injection System (HPIS) discharge lines. The estimated probability of an unrecovered human error is 3 x 10-4. If that human error occurs, or if any one of three mechanical failures occurs, the HPIS will fail to deliver sufficient water through the HPIS discharge line.

To arrive at the 3 x 10-4 error probability estimate in the fault tree, we could have constructed an HRA event tree to ensure consideration of all the events related to the failure to open MOV-1866E. However, as often occurs, this failure event was so simple that a tree was not necessary. For the error to occur, an operator has to fail to initiate the task of reopening the MOV or fail to operate the MOV switch correctly, and someone else has to fail to inspect the switch or fail to note that it is closed when he makes the inspection.

The estimated 3 x 10- 4 unrecovered error probability was-derived by using the 3 x 10 3 estimated probability of an error of omission for items that are embedded in a procedure (Table III 6-1 in WASH-1400). This estimate was modified by assuming a 10-1 probability that.either the inspection would not be carried out or the inspector would fail to notice that the switch was in the incorrect position. Thus, Pr[F] = 3 x 10 3 x 10-1 = 3 x 10 4. The 10-1 estimate included the dependence between the operator and the inspector. For the WASH-1400 analysis, this gross estimate was adequate.

For plant-specific analyses, more precise estimates can be obtained by performing a detailed task analysis of each specific task, constructing one or more HRA event trees to indicate the success paths and the plausible failure paths, and assigning the appropriate estimates of conditional probabilities of success and failure to the branches.

The most obvious sources of information for the HRA event trees for the NPP tasks to be performed are the written procedures for the tasks themselves and the plant's operating rules for verifying the correctness of the tasks performed. However, basing the tree solely on written procedures and plant operating rules may not result in a complete and accurate picture of the human actions. It is necessary to follow up the study of the written material with observations and interviews, as described in the preceding chapter.

Once an HRA event tree has been constructed, any potential system failures resulting from a single human failure will be obvious. For example, if an auxiliary operator has to reopen a critical valve, if no one checks his work, and if there are no other recovery factors in the system, that error alone could result in system failure. Such errors represent first-order human failure terms. An advantage of the HRA event tree is that these system-critical errors become obvious when the tree is drawn.

The HRA event trees also help the analyst identify possible common-cause failure events where the common-cause is the human error. An example is

5-17

Use of HRA Outputs

the common-cause failure potentiality in the misadjustment of the test equipment used for calibration of sensors, described in the discussion of Figure 5-1.

Sensitivity Analysis

Because of the uncertainty associated with most of the data on human per- formance that is expressed in probabilities, it is often useful to perform a sensitivity analysis in which the estimated HEPs, dependence levels, stress levels, or other indices of human performance are systematically varied to determine the effects of such variation on system outcomes. In the performance of an HRA, insufficient information may be available for one to be certain whether the estimated HEP for a certain system-critical task is .003 or .01. One approach to this problem is to perform relevant parts of the HRA twice, using both estimates, and to note if it makes any difference to the overall system analysis. If it does not, it will be adequate to use the .01 estimate. If it does make a difference, additional information should be obtained to enable a more accurate estimate to be made of the HEP.

Two types of sensitivity analysis have been used in HRA by the authors. One is the familiar worst-case analysis, in which highly pessimistic esti- mates of HEPs are used in assessing the impact of human errors in a system. In design trade-off work, we have used a 10-1 HEP for all tasks involved in some assembly or test procedure to see if the system could tolerate a consistently high HEP. If the system has sufficient redundancy and re- covery factors to tolerate the worst case, no detailed HRA may be required; the design may be judged adequate without further analysis.

The opposite case is the best-case analysis, in which highly optimistic

estimates of HEPs are used in assessing the impact of human errors on a system. In one such application, we wanted to find out if the addition of new equipment to a system would be much better than merely retraining operators in the existing system and providing better procedures. In the existing system, several tasks were accomplished by clerical personnel. Even though their error rates were low, those uncaught errors that did occur were very costly. It was possible to purchase a modification to the system that would accomplish the same functions performed by humans, but the modification involved a substantial one-tkie cost. After a task analy- sis of the operations performed manually, we determined that some improve- ment could indeed be made but without considerable study, data collection, and analysis, it would be difficult to arrive at accurate estimates of how much improvement could be achieved by retraining and the use of better designed procedures.

We therefore decided to use a best-case analysis in which we assumed that

retraining and new procedures would result in an HEP of 10 4 for each relevant task. There are human activities that have HEPs that low or lower, but it would be unrealistic to think that all activities in the job could be performed with such accuracy.

5-18

General Form of an HRA for PRA

An HRA event tree was drawn using 10-4 for each failure limb, and the total failure probability was calculated. The result was clear. Even assuming superaccurate humans, the old system would be much less reliable than the modified system. The cost of the new system was justified. Had the com- parison gone the other way, we would have had to perform additional analy- sis to derive more realistic estimates of HEPs.

Another form of sensitivity analysis is called a bounding analysis, in which both the worst-case and the best-case analyses are used to establish boundaries of the estimated influence of human performance in a system. The same HRA event tree is used, but two sets of total failure probabili- ties are calculated using two different sets of HEPs, one based on the worst-case HEPs and the other on the best-case HEPs. The former is said to define the upper bound of the total failure probability and the latter the lower bound. This type of bounding analysis will usually result in ex- tremely wide bounds. A different type of bounding analysis based on characteristics of distributions is described in Chapter 7.

A type of worst-case analysis often used early in a PRA reduces the amount of analysis necessary. Upper bounds or high values for failure probabili- ties are assigned to equipment and human events for initial screening. In the HRA, for example, rather than use the best estimate for each HEP, one uses a screening value that may be the best estimate multiplied by a factor of 2 or more. The idea is to see if a task with a high HEP will affect the system analysis. If it does not, the task does not have to be analyzed further. The major caution to be observed in using screening values in a PRA is to ensure that all task dependences are taken into account. Screen- ing HEPs for rule-based tasks and for cognitive tasks are discussed in Chapter 12.

General Form of an HRA for PRA

In the PRAs of NPP operations that have been done to date, the general approach used for HRA has been to identify, analyze, and to estimate HEPs for human tasks that system analysts and human reliability analysts deter- mine could have major impact on system criteria of interest. This section describes the approach to HRA used by SNL human reliability analysts as part of the Interim Reliability Evaluation Program (IREP). The SNL HRA is based on the use of the Handbook as the major source document and on THERP (including task analysis) as the major analytical tool. Except for minor changes to figure numbers and references and the underlining of some terms, the remainder of this section is a quote from Bell (1983).

Figure 5-4 shows the four phases of an HRA. In the first phase, familiarization, the human reliability analyst is in the process of gathering plant- specific and event-specific information. The former he obtains from actually visiting the plant (if he is not already quite familiar with it) and becoming aware of the PSFs peculiar to it. He should at this time evaluate the plant's admin- istrative control system and its test, maintenance, and restoration policies. The latter (event-specific) information can be obtained

5-19

Figure 5-4

FAMILIARIZATION

* INFORMATION GATHERING " PLANT VISIT " REVIEW OF PROCEDURES/INFORMATION FROM SYSTEM ANALYSTS

QUALITATIVE ASSESSMENT

9 DETERMINE PERFORMANCE REQUIREMENTS 9 EVALUATE PERFORMANCE SITUATION * SPECIFY PERFORMANCE OBJECTIVES * IDENTIFY POTENTIAL HUMAN ERRORS * MODEL HUMAN PERFORMANCE

QUANTITATIVE ASSESSMENT

" DETERMINE PROBABILITIES OF HUMAN ERRORS " IDENTIFY FACTORS/INTERACTIONS AFFECTING HUMAN PERFORMANCE " QUANTIFY EFFECTS OF FACTORS/INTERACTIONS " ACCOUNT FOR PROBABILITIES OF RECOVERY FROM ERRORS " CALCULATE HUMAN ERROR CONTRIBUTION TO PROBABILITY OF SYSTEM

FAILURE

INCORPORATION

* PERFORM SENSITIVITY ANALYSIS * INPUT RESULTS TO SYSTEM ANALYSIS

Figure 5-4 Four phases of human reliability analysis.

0 5-20

General Form of an HRA for PRA

at this time by the human reliability analyst if the scenarios to be evaluated have already been identified by the system analysts. If such is the case, the human reliability analyst is able to conduct on-the-spot task analyses of the events in question (part of phase 2). If this is not the case, he can gather information sufficient to allow him to complete his analyses at a later date by referring to them.

The second phase, qualitative assessment, involves preliminary analyses of the performance situations to be included in the HRA. portion of the PRA. The human reliability analyst performs a rough task analysis of the actions required of the operators in responding to each situation. That is, he breaks down the operator actions into tasks and subtasks based on the action characterizations found in NUREG/CR-1278. It is not necessary for him to determine the motivation behind every operator action, but rather for him to be able to identify the most likely operator activities.

For each of these activities or tasks that the human reliability analyst identifies, possible human errors must be pinpointed. In assessing the actual performance-situation, the human reliability analyst decides which potential errors are likely. Each of these errors is included in the model of human response that is then developed. This model is a representation of.the quality of the human/system interface in that from it certain system deficiencies can be detected.

The third phase of HRA, the quantitative assessment, involves devel- oping a probabilistic statement of the likelihood of each system outcome as defined in the model constructed in phase 2. In other words, the human reliability analyst makes estimates of the proba- bilities of the human errors listed in'the model. These estimates could be based on data stores, expert prediction, or extrapolation of existing data from similar actions. The effects of any factors acting on the situation that might alter the probabilities of the identified human errors must also be quantified at this time, along with the probabilities of any possible recovery factors. Finally, the results of the quantified model can be applied to the definition of system failure as established by the system analysts to determine the contribution of the probability of human error to the probabil- ity of system failure.

The fourth phase, incorporation, deals with the use of the results of the HRA. Sensitivity analysis can be performed in which the assumptions made in the original analysis are modified, and the analysis itself is repeated to see what impact these assumptions had on the probability of system failure. The results themselves are then input to the PRA.

The level at which incorporation occurs is a point of controversy in the current state of PRA. If this incorporation is performed cor- rectly, there should be no numerical difference in the resulting

5-21

"Validity" of HRA Using THERP

probability of system failure (see Figure 5-5). However, there are more subtle implications of the level of incorporation chosen. It may be that the "higher" in the system model the HRA appears (the closer it is input to the system rather than to the component level), the less likely it is that important human-caused system interactions will be overlooked. For example, dependence between operators may link their manipulations of items of equipment from operationally dissimilar systems. This dependence may not be in- cluded in the system models as represented by the system analysts because of this operational dissimilarity. The controversy over the preferred level of incorporation of HRA into PRA continues. One method of incorporation is discussed in Bell and Carlson (1981). Resolution of the problem should not materially affect the perfor- mance of the actual HRA, but rather it should affect the emphasis placed on various parts of it.

Figure 5-6 shows a block diagram of one possible ordering of the basic steps of HRA as specified by THERP. This figure is reproduced from the December 1981 draft of NUREG/CR-2254 (Bell and Swain, 1981a). It depicts the procedure followed for some of the IREP PRAs. The procedure is by no means static nor is it rigid. There are in practice several feedback loops that operate during the course of the HRA. These are not reproduced here in order that the simplicity of the procedure may be more easily explained. The steps do not necessarily have to be performed in this exact sequence, although performing them out of sequence may lead, at least for the novice human reliability analyst, to inadvertent deletion of impor- tant aspects of the analysis. The steps in the procedure are dis- cussed at length in NUREG/CR-2254 and summarized in Bell and Swain (1981b); this discussion will not be repeated here.

"Validity" of HRA Using THERP

The quotation marks around "validity" are deliberate. There has been no real validity test of our predictions using THERP. One's predictions are valid if they are borne out by real-world events. The majority of the applications of THERP since 1961 have been to military systems, often as part of system reliability studies that estimate probabilities of extremely improbable events or of events that would occur'only in combat. Conse- quently, evidence to support our predictions in such studies is usually unavailable.

There have been a few occasions on which our estimates of HEPs could be verified up to a point. Two of these are described. Both occasions dealt with the estimation of probabilities of error in assembling components for military systems.

One HRA involved predicting the probability that a system would fail because of unrecovered human errors during the production process (Swain, 1980b, pp 104-5). One of the authors of the Handbook visited the production plant and performed the usual task analysis. During the visit,

5-22

Figure 5-5

FUNCTION 2 FUNCTION 1 (S)I

COMPONENT 1I UNAVAILABLE I

COMPONENT 2 UNAVAILABLE

HUMAN ERROR]

(THUMAN ERROR' CONTRIBUTION|

TO COMPONENT UNAVAILABILITY

Figure 5-5 Two options for incorporation of HRA into PRA.

5-23

Figure 5-6

0 PLANT VISIT

PHASE 1:

REVIEW INFORMATION FROM FAMILIARi SYSTEM ANALYSTS

TALK- O

IZATION

TALK-'OR WALK-THROUGH

TASK ANALYSIS

DEVELOP HRA EVENTTREES

1PHASE 2: j

IASSIGN NOMINAL HEP.1

I ESTIMATE THE RELATIVE

EFFECTS OF.PERFORMANCE SHAPING FACTORS

7ASSESS DEPENDENCEJ

QUALITATIVE ASSESSMENT

PHASE 3:

QUANTITATIVE ASSESSMENT

PHASE 4:

INCORPORATION

IDETERMINE SUCCESS ANDIFAILURE PROBABILITIES I ,IDETERMINE THE EFFECTS]OF RECOVERY FACTORS

I-

I PERFORM A SENSITIVITY ANALYSIS, IF WARRANTED

SUPPLY INFORMATION TO SYSTEM ANALYSTS

Figure 5-6 Outline of a THERP procedure for HRA.

5-24

"Validity" of HRA Using THERP

he interviewed workers, inspectors, foremen, and manufacturing engineers and observed all of the operations being performed. Using the information obtained by this analyst, the available paper work, and a working model of the equipment to be produced, both authors independently performed an HRA using the HRA event tree method.

There was considerable agreement between the two analysts in most of the HEP estimates. The few disagreements that occurred were resolved by asking a third member of the human reliability staff to provide an independent estimate. In general, a voting concensus was used. On this basis, a composite estimate of field failure due to production errors was made.

The system was produced in quantity and field tested. Although failures occurred, none could be ascribed to unrecovered production errors. Our estimate of production errors was within the 90% confidence interval.

The other occasion for verifying our estimates of HEPs has been reported (Swain, 1982a). Two hundred and eighty-four electronic components were installed in major systems. During a random check of one of the major sys- tems, it was discovered that one of the two connectors in the component had a defect that resulted in a significant loss of reliability for that sys- tem. One of the authors of the Handbook visited the plant, performed the task analysis (including an interview and talk-through with the person whose error resulted in the defect), and generated the HRA event tree. He then showed the tree (without the estimated HEPs) to two other SNL human reliability analysts and described the results of his interview and talk- through. The three analysts then independently evaluated the tree and assigned the appropriate HEPs. The estimated end-failure terms of the three analysts differed by less than a factor of 2. After discussion, the consensus estimate of the production error probability per component was .004.

Subsequent to the analysis, 64 additional major systems were disassembled to check for similar errors. None were found. The analysis in Swain's paper shows that if the true probability of the error per component is .004, the probability that a sample of 65 will have at least 1 bad compo- nent is .23. There is bias in the statistical test since the one failed unit was used in calculating the probability of at least one bad component. However, the bias is toward rejection of the assessment, which was not done, so the conclusion of an assessment of .004 is valid.

Although the verifications described above do not constitute a complete validity check, they do provide some confidence in our estimates of HEPs when using THERP and the data in this Handbook to perform an HRA of procedure-based routine human performance in a well-structured manufactur- ing complex. In both instances, the estimates of the different analysts

5-25

"Validity" of HRA Using THERP

were in close agreement, and the composite estimates were not significantly different from the data that were observed subsequent to the estimates.*

The above examples illustrate another aspect of THERP that has not been discussed thus far. There is room for a certain amount of subjectivity in arriving at error probability estimates, and it is valuable to obtain independent estimates when available. The most difficult part of the job is the gathering of the basic information--the interviewing, the observing, and the filtering of all the available information. This "legwork" re- quires a degree of acumen that develops only with experience in the actual conduct of onsite task analyses.

Those interested in the history and use of THERP can refer to Rook, 1962, 1963, and 1964; Swain, 1963a and b, 1964a and b, 1967a and b, 1969a and b, 1971, 1974a and b, 1976, and 1977a and b, 1980b (Ch. VIII); Rigby, 1967; Rigby and Edelman, 1968a; Rigby and Swain, 1968; Swain and Guttmann, 1975; and "Human Reliability Analysis," Section 6.1, Appendix III - Failure Data, 1975, WASH-1400; Bell and Swain, 1981a and 1982; Bell and Carlson, 1981; Kolb et al, 1982a; Bell, 1983; and Bell and Kolb, 1983.

It is perhaps fitting, or inevitable, that a paper on the "validity" of a technique to estimate human errors would include a major human error. The equation for the significance level (SL) in Swain's paper left out the combinatorial term. The correct equation is as follows:

nSL= n (n)px(l _Pn-x SL = -x=1x

=1 -(1- n)

where n = sample size, and p = assessed failure probability.

5-26

Ch. 6. Data Sources Overview: Data Problem

CHAPTER 6. SOURCES OF HUMAN PERFORMANCE ESTIMATES

Overview

This chapter explains how the estimated human error probabilities (HEPs) and performance times were derived. The discussion is restricted to point-estimates; Chapter 7 deals with the derivation of estimates of dis- tribution characteristics, including uncertainty bounds. It is important for the user of our estimated HEPs and performance times to keep in mind that our estimates are for tasks or actions that the performers consider to be important. Furthermore, there may be alternate ways of performing some important function even though a written procedure or plant policy may specify only one method. In a probabilistic risk assessment (PRA), alter- nate methods should not be considered as errors unless defined so by system analysts.

The paucity of actual data on human performance continues to be a major problem for estimating HEPs and performance times in nuclear power plant (NPP) tasks. The analyst must therefore turn to various other sources of information to derive such estimates for human reliability analysis (HRA) purposes. For estimates of HEPs, extrapolation from these sources combined with expert judgment, or the use of expert judgment only, are the usual sources. The estimation of performance times poses a more difficult prob- lem because the task times taken from other than NPP tasks cannot be used for accurate estimates of performance times for NPP tasks. The situations tend to be so different as to make extrapolation very risky. The best approach is to take measurements of actual or simulated task times. A second-best approach is to combine the estimates of those who perform the tasks.

Sections in this chapter treat the data problem for HRA, information sources used to derive the estimated HEPs and performance times, the usual discretizing of continuous variables for. HRA, the manner in which we derived some of the HEPs in Chapter 20, and some limitations and precau- tions to be observed in applying our estimates in HRAs. The rationale for each performance time is discussed, but not all of the background infor- mation used in deriving the HEPs is presented in this chapter; some is presented in other chapters.

The Data Problem for HRA

There is general agreement that the major problem for HRA is the scarcity of data on human performance that can be used to estimate HEPs for tasks. For HEPs, one would like to know the number of errors that occurred for each action and how often these actions were performed. We define the error relative frequency as the ratio of the number of errors to the number of attempts. The ideal situation for PRA would be one in which error relative frequencies were collected on large numbers of people performing

6-1

Categories of HEP Sources

NPP tasks so that distributions of these frequencies could be determined determined for each action for each individual. Then, estimates of central tendency and distribution parameters could be calculated directly.

This ideal situation is impractical for several reasons. First, error probabilities for many tasks are very small. Therefore, in any reasonable time period, not enough errors will occur to permit the fitting of a mean- ingful distribution to the observed error relative frequencies. Second, usually there is some penalty associated with the commission of an error, which discourages the identification of individuals who commit the errors. If such identification were insisted upon, there would be some unknown, but sizeable, number of errors not reported. Third, the administrative prob- lems and costs of recording and analyzing errors by person would probably be unacceptable either to the utilities or to the Nuclear Regulatory Com- mission (NRC).

The next best solution for obtaining data from NPPs is to pool errors across individuals and record error relative frequencies and performance times by task and other descriptors. There are still a number of sizeable practical problems, such as assuring the anonymity of people making errors and obtaining the cooperation of the utilities, but the concept of this type of data recording seems feasible. NRC has funded Sandia National Laboratories (SNL) to develop a program plan and perform a tryout of a human performance data bank to collect such data on NPP tasks (Comer et al, 1983). If a human performance data bank can be successfully established and maintained, future HRA analysts will be aided materially. The raw data will include ratios of incorrect to total responses for each human action recorded, i.e., the number of errors divided by the number of opportunities for each type of error to occur. Performance time data will also be recorded, e.g., how long it took a team of operators to diagnose a par- ticular transient or other abnormal event. The raw data will be codified into a task taxonomy designed to fit the needs of PRA.

Until such time as data from NPPs become available in sufficient quantity, other sources of information are required from which one can derive esti- mates of HEPs and performance times for PRAs. The next section describes the categories of sources for HEPs presently available.

Categories of Sources Used for Derived HEPs

This section presents a discussion of the estimated error probabilities, which are extrapolated from a variety of sources. The nature of these sources determines the qualitative confidence we have in these derived HEPs. The following source categories were used in deriving the HEPs:

(1) Nuclear power plants.

(2) Dynamic simulators of NPPs.

(3) Process industries.

6-2

Categories of HEP Sources

(4) Job situations in other industries and in military situations that are psychologically similar to NPP tasks.

(5) Experiments and field studies using real-world tasks of interest,

e.g., experimental comparisons of measurement techniques in an in- dustrial setting, performance records of industrial workers, etc.

(6) Experiments using artificial tasks, e.g., typical university psychol- ogy studies, which have limited application to real-world tasks.

The above listing orders the sources by their relevance to NPP tasks. Un- fortunately, the availability of HEP data is almost exactly the inverse of this order. Available human performance data relevant to estimates of HEPs for NPP tasks are discussed below.

As part of the peer review of the draft Handbook, we asked experts in the fields of human factors, psychology, and PRA to provide us with any data they knew of in any of the above categories (Miller, 1983). Some useful data. on performance times were provided by investigators from Oak Ridge National Laboratories (Haas and Bott, 1979; Bott, Hutto et al, 1981; Bott, Kozinsky et al, 1981; Barks et al, 1982; Beare, Crow et al, 1982; and Beare, Dorris et al, 1982, 1983a). No one.' however, could provide us with data on HEPs related to NPP operations. This negative response came from the 29 experts in human factors and/or PRA (and others) who were asked to solve some practical problems using the draft Handbook as the primary source book (Brune et al, 1983). In general, these experts noted the sub- jectivity of our estimated HEPs in the Handbook, but they had no substi- tutes to offer. This is not intended as a criticism of these experts; they were merely reflecting the major problem for HRA: the lack of actuarial data from which HEPs can be determined.

With NRC funding, SNL also contracted with General Physics Corporation to survey existing data banks for relevance to HRA (Topmiller et al, 1982 and 1983). Their report is essentially negative; except for three derived data banks dating back to the early 1960s, very little HEP data could be found, and serious limitations were noted even for this data. The three data banks are the AIR Data Store (Munger et al, 1962), the Aerojet-General Data Bank (Irwin et al, 1964a and b), and the Bunker-Ramo Data Bank (Meister, 1967). These out-of-print data banks have been reprinted in Topmiller et al (1982).

Nuclear Power Plants

Hardly any HEPs for NPP tasks have been recorded. In WASH-1400, we used some data collected by the United Kingdom Atomic Energy Authority (Ablitt, 1969). To date there has been no systematic program to collect HEP data in operating NPPs in the United States. The only formal record of errors in NPPs consists of the Licensee Event Reports (LERs), which do not yield probability data in the sense of number of errors made per opportunity for error. In some cases, it may seem possible to derive a probability from an LER. For example, if an error is made on a task and it is known that the task is performed, say, once a day every day, we have the denominator for

6-3

Categories of HEP Sources

that task, and it seems possible to calculate an HEP for that task. How- ever, the HEP obtained will be the unrecovered error probability, which will have to be adjusted for the unknown number of times that an error is made that does not result in a reportable event, e.g., when recovery fac- tors compensate for the error. This adjusted number should constitute the actual numerator of the HEP.

Several analyses of LER data have been done by personnel at Brookhaven National Laboratory. In one study, LERs from 1976 through 1978 were evalu- ated for evidence of any human errors in the operation, maintenance, and testing of motor-operated valves (MOVs), manual Valves, and pumps (Luckas and Hall, 1981). In general, MOVs and pumps are remotely operated by switches in the control room, and manual valves are locally operated at the valve site. The investigators counted the number of errors in the category "Personnel Error" in the LER summaries prepared by Oak Ridge National Laboratory. They estimated the appropriate denominators (number of valve or switch manipulations in the 3-year time period) based on plant records and on interviews with operating personnel. Dividing the number of errors by the estimated number of opportunities, they derived what they called "generic human error rates." Following is a summary of their derived error rates:

Reactor Safety

System Interfacing Reactor Type Component PWR BWR Pooled

MOVs .0003 .006 .0004

Manual valves .0015 .0014 .0015

Pumps .0012 .0006 .001

These rates are overall rates; they include those cases in which there were no recovery factors for an error and also those cases in which a recovery factor existed but failed to recover the error, as when an inspector failed to notice that someone had made an error. We do not know the proportions of the two cases that entered into the overall rates. The observed error rate for cases without recovery factors would be higher than the rate for cases with recovery factors, by some unknown amount. In addition, the investigators noted that the LER data underestimate the numerators of the rates because not all errors are reported or even required to be reported and because licensees often classify a failure as "component failure" even when human error is involved.

In another Brookhaven study, the same kind of generic human error rates for instrumentation and control system components were derived (Luckas et al, 1982). The reported rates range from 6 x 10- to 8 x 10-3. The low rate was for reversing the input/output of the recorder and for offsetting the test meter (resulting in bad calibrations) for the average power range monitor. The high rate was for inadvertent jumpering of the entire logic

6-4

Categories of HEP Sources

for a source range monitor. Most of the rates were 2 x 10-3. Again, these are unrecovered rates with some unknown underestimation of the numerator.

Recognizing these problems, the Brookhaven investigators checked the origi- nal plant data related to any report of a valve failure listed in the LER summaries (Speaker et al, 1982). That is, they did not restrict their investigation to "Personnel Errors" as classified by the LER summaries. They found that for every LER summary of a valve failure classified as in- volving personnel error, there were from five to seven additional report- able events dealing with valve failures that involved human error. This analysis illustrates a limitation of the LER summary classifications.

If one multiplies the generic human error rates in the Brookhaven reports by a factor of 6, the resultant rates are generally close to the estimated HEPs in this Handbook. However, due to the limitations of data recording of human errors in the present LER system, any agreement or disagreement with the estimates in the Handbook is difficult to interpret.

In another study of LER data (Joos et al, 1979), "gross human error proba- bilities" were derived by counting the number of each type of error re- ported in the LERs submitted by operating U.S. commercial NPPs over a 25-month period. These numbers were then divided by the total number of months that the plants had been operating during the 25-month period to arrive at "the number of errors per plant month." Although such numbers do indicate the relative frequency of different types of reported events, and thus provide useful information, they do not yield the type of error proba- bilities required for an HRA. We have no way of estimating appropriate factors by which to adjust the above data to obtain estimates of HEPs comparable to those in the Handbook.

In summary, data on human errors currently collected from NPP operations are not the type necessary for HRAs. What is needed is the ratio of the number of errors to the number of opportunities for each type of error of interest in a PRA. Until valid error relative frequency data can be col- lected in NPPs, other sources of data from which to estimate HEPs will be required. The following sections describe these sources.

Dynamic Simulators of NPPs

Dynamic simulators have been used extensively--almost exclusively--for training and recertifying NPP operators. Until recently, there have been no extensive programs to gather HEP data, although the simulators are readily adaptable for this function. The first program to systematically gather HEP data in simulators was completed recently by Beare, Dorris et al (1983b) under contract to SNL. This study involved gathering HEP data from teams of operators undergoing training or requalification in a variety of training scenarios in two simulators (a SWR and a PWR). Much useful data was obtained relative to errors of omission and for several types of errors of commission, as well as on recovery rates and on the effects of a number of performance shaping factors. Although the study was not completed in time for incorporation of its results in the Handbook, it was found that

6-5

Categories of HEP Sources

many of the findings on errors of commission are in close accord with the Handbook HEPs. The observed error rate for errors of omission was higher than the Handbook estimates; this was tentatively attributed to the casual manner in which written procedures are used in a simulator, as compared with the manner in which they are used in an actual plant.

The study by Beare, Dorris et al (1983b) represents the first attempt to systematically gather empirical HEP data and constitutes a landmark effort for HRA. The report is an example of objective data-gathering carried out in conjunction with an ongoing training program and also reveals the dif- ficulties of data-analysis even when the data are gathered under relatively well-controlled conditions.

Although a simulator is not "the real thing," it is expected that HEPs obtained from advanced trainees and from experienced operators undergoing recertification will be very valuable. Swain (1967a) has described a technique for combining real-world data with simulator data in such a way that real-world data serve as "anchor points," permitting valid extrapola- tions of the simulator data to the real-world situation. The HEP data collected in the above study will have to be calibrated in some manner to allow for the differences between the simulator and the real world.

Other General Physics Corporation simulator studies, funded by NRC and managed by Oak Ridge National Laboratory, have provided some performance time data that we have used in the Handbook. In these studies, it was found that operators tend to underestimate decision and action times by about a factor of 2 (Haas and Bott, 1979).

In an unpublished study of simulator exercises involving shipboard nuclear reactor operators, 80 requalification trials yielded HEPs for individual operators that averaged to within a factor of 2 of comparable HEPs in the Handbook. At this time, no further information is available to us from this study.

Process Industries

Human tasks in the operation of process industries, such as chemical plants, refineries, etc., are very similar to those in NPPs. Some data have been collected in various process industries, many of which are directly applicable to NPP operations (e.g., Kletz, 1979; Kletz and Whitaker, 1973; and Edwards and Lees, 1974). Extensive use has been made of these data, in some cases with modifications to allow for differences in task details.

Industrial and Military Data

Many job situations in the nuclear weapons industry, in the military proce- dures for the handling of weapons, and in other military tasks are similar to tasks performed in an NPP. In some of these areas, reliable error data were available that were directly applicable to NPP tasks. For example,

6-6

Categories of HEP Sources

errors in radar maintenance were used in estimating some maintenance errors in NPPs, and errors in measurement tasks in the nuclear weapons field were used in estimating calibration errors in NPPs.,

Data on human errors in the production and testing of military systems and components were analyzed to derive HEPs of NPP tasks that are similar in terms of the human actions involved. These data include those reported in Rook, 1962; Rigby and Edelman, 1968a and b; and Rigby and Swain, 1968 and 1975.

Several reviewers of the draft Handbook stated that we should obtain the error data from flight simulators and various kinds of military simulators. As part of an SNL contract, Beare, Dorris et al (1983b) conducted a survey of all relevant simulation facilities in the U.S., which included the nuclear power industry, the U.S. Air Force, the Army and the Navy, the National Aeronautics and Space Administration, the Federal Aviation Admin- istration, and commercial air carriers. None of these agencies maintained records that could provide HEPs. Simulator facility personnel reported that their simulators were used almost exclusive-ly for training purposes and that no records of human errors were kept. This had been the case with NPP simulators as well, prior to the studies currently under way.

Field Studies

Field studies and experiments in industrial settings yielded some reliable data directly applicable to NPPs. Many of these studies are cited through- out the Handbook. A problem with most experimental studies is that the very fact that an experiment is being conducted tends to influence the performance of the workers. Some allowance has to be made for this effect. Data gathered in field studies are less susceptible to this effect, but these data are harder to obtain.

Experiments Using Artificial Tasks

Data obtained in university laboratories usually deal with artificial tasks but are very precise because all pertinent variables can be tightly con- trolled and there is ample time to gather a large amount of data. The studies carried out in university laboratories usually answer specific aca- demic questions, and the findings may be applicable to real-world situa- tions only after appropriate modifications are made.

A feature of academic studies is that the limits of acceptable performance are often very narrow. Also, the situation is often arranged to be arti- ficially difficult so that the distribution of responses will be "forced" into some desired form (usually approaching the normal distribution). The reported performance of subjects under these conditions could lead to a pessimistic evaluation of human capabilities in real-world situations; therefore, the unmodified data are not used. Allowances are made for the broader tolerances in an industrial situation. As an example, Payne and Altman (1962) multiplied HEPs from laboratory experiments by a factor of .008 to derive HEPs for certain maintenance tasks. Use of this weighting factor yielded values similar to those observed in field operations.

6-7

Use of Expert Opinion

Many academic experiments are comparative studies to determine how basic performance is affected by different conditions. In such studies, we can often use the comparative performance data directly since we are interested in the ratio of performance under differing conditions rather than in the absolute levels of performance.

Use of Expert Opinion

All the data sources listed above yield numbers based on some kind of documented records of performance. In addition to estimates of HEPs based on such "hard" data, estimates are prepared on the basis of expert judg- ment. For example, in those instances in which error terms from a particu- lar task are modified to apply to some task in an NPP, the modification is necessarily based on a judgment of the similarities and differences between the tasks and the extent to which these would affect the error probabil- ities. The uncertainty of the judgments is reflected in the size of the uncertainty bounds assigned to the estimated HEPs.

Error probability modifications based on judgment may be made informally or formally. Informal evaluations of estimated HEPs are usually obtained from just a few experts who are thoroughly familiar with the tasks and the rele- vant performance shaping factors (PSFs). Their opinions are elicited informally and pooled to arrive at the extrapolated error probability. Formal evaluations use methods known as psychological scaling techniques, as discussed in Chapter 8. A larger number of judges is required and standardized procedures and evaluation forms are used. Although the formal judges may not have the thorough task knowledge that the informal judges have, their pooled evaluations can provide useful estimates of error proba- bility modifications. In this Handbook, when using expert judgment, we

have relied on informal judges because of their greater familiarity with the tasks in question and their ready availability.

In many instances, because of the paucity of relevant "hard" data, judg-

ments were the only source of error probability estimates. In such cases, the judgments were based on information from tasks that most nearly re- semble the task in question, and the magnitude of the uncertainty bounds was adjusted in accordance with the judged similarities or differences between the tasks.

We have endeavored to ensure that our derived H.EPs reflect the appropriate

ordinal differences even though there may be considerable uncertainty as to absolute magnitudes. For example, check-reading errors for an analog meter with well-defined limit marks will be fewer than for an analog meter with-

out limit marks, although the extent of the difference may not be well defined. To the extent possible, we also endeavored to have our derived HEPs reflect appropriate interval differences. as well.

For many years, an HEP of .01 was routinely assigned to all human actions

by reliability analysts in weapon systems for lack of more specific data. It was recognized that .01 was often a pessimistic estimate of HEPs, but

6-C

Discretizing of Continuous Variables

this conservative estimate was used in view of the nature of the systems. In applying HRAs to other fields, more realistic HEPs were desired, and the figures in the Handbook were developed for application to the specific actions with which they are associated.

To the reader not familiar with HRA, most of the HEPs in the Handbook may

seem to be too conservative. It must be stressed that these values repre- sent the probability that the error will be made at all, not the probabil-

ity that the error will remain uncorrected. For example, say that there is an HEP of .01 that a person will attempt to insert a key into a lock upside down. Thus, if a person starts his car three times a day, about once a month he will try to insert his key in the ignition switch incorrectly. This does not mean that the error will be uncorrected. In this particular situation, the error is always self-corrected, yet the HEP is .01 (a fairly high probability). Similarly, most of the the HEPs in the Handbook are exclusive of recovery factors. In an HRA, recovery factors are assessed so

that overly pessimistic estimates of the effects of human errors can be avoided.

Discretizing of Continuous Variables

Many of the PSFs that are evaluated in a task analysis are continuous

variables, e.g., age, skill level, stress, etc. In an HRA, such continuous variables are usually represented by a finite number of points. Our stress model, for example, represents the entire continuum of stress by four

points: very low, optimum, moderately high, and extremely high. For PRA

purposes, this discretizing of a continuous variable is entirely adequate

and, considering the scarcity of HEP data, necessary. This approach en- ables the user of the HRA event tree to assign limbs in the tree to repre- sent the various discrete values that are taken to represent the continuous variable. For example, in an HRA problem, the analyst may judge that 70% of the performances of a task may occur under the PSF of a moderately high level of stress while the remaining 30% occur under extremely-high stress.

The top branching of the HRA event tree can be used to illustrate this division of the PSF of stress as follows: the left limb can stand for the .7 probability of the lower stress level and the right limb for the .3 probability of the higher stress level. Subsequent limbs of the tree are

used to represent the performance of task activities under each stress assumption, with the assignment of the appropriate conditional error and

success probabilities.

Not all continuous variables are discretized, e.g., in Chapter 12, note the

cumulative probabilities of failure to interpret transients correctly as a function of the time after the operating team recognizes that something is amiss. In this case, the variable is shown as a continuum, and the analyst reads off the probability of failure for whichever time he wishes to consider.

The distribution of HEPs for a given task reflects individual differences

in task performance and uncertainties in the estimations of the HEPs.

Treatment of HEP distributions in a PRA is described in Chapter 7.

6-9

Estimates of Performance Times

Estimates of Performance Times for HRA

There are three different categories of performance times that may be of interest in an HRA done as part of PRA:

(1) How long it takes to begin a task.

a. Reaction time--usually for tasks in which diagnosis is not a material problem.

b. Correct task initiation (response time)--includes diagnosis.

(2) How long it takes to do a task correctly.

(3) The fixed time interval within which a task must be performed correctly.

Performance times tend to be very plant- and situation-specific. The estimated times in the Handbook are, of course, generic and if available, actual measurements of the times of interest should be used. Often we must rely upon the operating personnel to estimate performance times, but expe- rience indicates that people tend to underestimate the time necessary for task performance, especially when diagnosis is involved. An approach we recommend is to ask plant personnel to estimate performance times for tasks for which we have time data along with tasks for which there are no time data. A comparison of actual times with estimated times can be used to derive a calibration factor for those time estimates for which there are no data (Swain, 1967a).

Data sources and comments on each of the above categories of performance time are presented below followed by a summary of the use of estimated performance times in the Handbook.

Time Taken To Begin a Task

Data on reaction times for many different kinds of experimental tasks are found in the psychological literature. As defined by English and English (1958), "reaction time is the interval between application of a stimulus and the beginning of the subject's response." There are several different kinds of reaction time, according to these authors. "In simple reaction time, a predetermined response is made as quickly as possible to a pre- arranged signal. In discrimination reaction time, response is made to one, but not to others, of two or more expected signals. In 'cognitive reaction time, response is not to be made until the stimulus is recognized. In choice reaction time, a different response is specified for each kind of stimulus. All of these except for the first are called complex or compound reaction times." As noted earlier, the types of tasks employed in typical university experiments are usually highly artificial but can be extrapo- lated to NPP situations if calibrated with plant data. Our own HRAs have not included reaction time estimates since this has not been important in PRA considerations. However, we do include the equivalent of "cognitive reaction time," but we consider it an aspect of response time rather than

6-10

Estimates of Performance Times

reaction time since the latter is associated with tasks that are relatively simple as compared with those tasks in the NPP situation that require

cognitive behavior, such as diagnosis of stimulus patterns associated with transients and other unusual events.

Response time is often important in a PRA, as when plant personnel must respond to unusual events and determine what the problem is. In many, perhaps most, cases, the problem is obvious. With unanticipated problems, the time required to diagnose the problem and to begin the correct task responses may be substantial. Some data for this category of performance time are available from the Oak Ridge National Laboratory studies cited earlier.

Time Taken To Do a Task Correctly

The time it takes to perform maintenance or testing tasks is often required in a PRA in estimating the unavailability of some system. Such information is obtained from plant records or from interviews of plant personnel. Given the tendency to underestimate task completion times (Haas and Bott, 1979, 1982), it is preferable to obtain actual maintenance or testing time

data from records if they are available in a plant. Such data can also be

used to derive a calibration factor for correcting estimates of performance time given by operating personnel.

For cases in which it is necessary to estimate how long it takes to achieve

control after the occurrence of abnormal events, plant information may not

be available. For example, when considering plant response to a loss of offsite power, it is necessary to estimate how long it will take plant personnel to restore offsite power. In the Zion PRA (1981), the analysts

broke down the complicated procedures into successive elements (initial diagnosis, travel time, additional diagnosis, more travel time, and so on). Then, operating personnel were asked to estimate times and probabilities of successful completion of certain task elements. For other task elements,

time data from operating experience were used.

The use of expert estimation is often necessary but should not be substi-

tuted for actual time measurements when the latter are possible. For example, rather than ask operating personnel how long they think it will

take them to travel from the control room to the diesel room and from the diesel room to the relay room, it would be better to take actual time measurements from a simulated exercise. When estimating the time required for successful diagnostic tasks, again it is preferable to use actual data.

In both cases, upward adjustments of the measured times must be made to account for differences between the simulated exercises and the hypothe-

sized real-world events. If simulator data can be obtained, they should be used, but with appropriate calibration for differences between plant and

simulator conditions, as discussed earlier. If simulator data are not available, estimates of diagnostic times can be approximated through the

use of the talk-through method (Chapter 5). If this method is used, the pattern of stimuli associated with each event to be diagnosed should be represented as completely as possible with pictures or words, including the

6-11

Estimates of Performance Times

"noise" stimuli that ordinarily would be associated with the event. In a talk-through, it would naturally be impossible to present all of the nones- sential and ambiguous stimuli, but the analyst should attempt to avoid presenting the operator with only the essential information since the diagnostic task would be unrealistically easy and the obtained diagnosis times would be unrealistically small.

Fixed Time Intervals for Correct Task Performance

In a PRA, it may be determined that in response to some transient, there is a certain fixed time for one or more tasks to be performed correctly. As long as this time is not exceeded, the correct human actions are designated as system success inputs. A common example is the time for switchover from the injection to the recirculation phase after a loss-of-coolant accident (LOCA). If the time available is perceived to be short, it is presumed that the stress level is relatively high and that the HEPs are relatively high. If the time period is perceived to be more generous, it is presumed that the stress level is lower, and the HEPs are lower. Furthermore, with more time available, recovery factors can operate. No plant data exist to verify these presumptions, but they make good psychological sense in most cases. The uncertainties are large, and the, estimated HEPs are estimates based on consensus.

For cases in which the plant dynamics change materially during an abnormal event, the estimated HEPs must take such changes and their effects on people into account. Regrettably, there are no directly relevant data; it is a matter of judgment by the analysts. When there are large uncertain- ties, the use of several analysts and the psychological scaling techniques in Chapter 8 are suggested for an HRA.

Estimates of Performance Times in the Handbook

Following is a summary of the sources of estimated performance times for each chapter in Part III.

In Chapter 11, Figure 11-1 presents a nominal "display-scanning model" that hypothesizes an exponential curve representing probability of detection effectiveness as a function of time of unannunciated deviant displays at and following the initial scanning of control boards made near the begin- ning of a shift, with the assumption that no written checklists are used. The form of the curve is based on the experimental literature. The assump- tion of hourly scans is based on interviews with reactor operators at several plants. The assignment of decreasing probabilities of detection per hourly scan in Tables 11-8 and 11-10 for meters and chart recorders, with and without limit marks, is based on our judgment; no data were avail- able. For unannunciated display scanning tasks in which the HEPs are very large, a different model is used. For example, in the case of the scanning efficiency of indicator lamps with an HEP = .99, no change in HEPs is assumed across hourly scans. Our rationale is that HEPs of about .99 are already so high that there is no point in attempting to estimate the influ- ence of time. In this case, time would act to increase errors, as dis- cussed in the general display scanning model in Chapter 11.

6-12

Data - Use of Displays

Also in Chapter 11, we had to rely on our judgment for the assumption of a factor of 10 increase in the probability of failing to respond to a can- celled annunciator (i.e., one in which the annunciator indicator light is a steady-on white in most plants) after having been interrupted by some other signal or signals (Figure 11-2). Again, no data were available, and future simulator data could revise this assumption.

In Chapter 12, the values in the initial screening model (Table 12-2) depicting the decreasing HEPs over time for diagnosis of a single abnormal .event were obtained in part from a consensus of PRA experts (Oswald et al, 1982). We modified the workshop values slightly to render the joint HEPs for a team of three operators compatible with our dependence model, as explained in Chapter 12. We added a constant of 10 minutes for time to diagnose a second abnormal event (of two simultaneous abnormal events), based on simulator data in a study sponsored by the Electric Power Research Institute (Woods et al, 1982). In Table 12-4, the values for the nominal model for diagnosis of a single abnormal event represent one time step up from the equivalent times in the initial screening model and also represent our judgment. Also, we used judgment in taking the values from the initial-screening model for the diagnosis of a first abnormal event and using them for the nominal model for the case in which a second abnormal event must be diagnosed. All of these models are speculative and resulting estimates of HEPs have large uncertainty bounds; They were developed to fill a need for present PRAs until data for better estimates become available.

In Chapter 17 on stress, the Large LOCA curve (Figure 17-'2) is based on our

judgment, but the estimated HEP of .25 if highly stressful conditions per- sist represents an extrapolation from military studies described in the chapter (Ronan, 1953; Berkun, 1964; Berkun et al, 1962). The "doubling rule" to express the rapid disintegration of performance under time-stress was originally developed as an hypothesis for military HRA studies (Swain, 1963b and 1974b). Subsequent field studies by Siegel and Wolf (1969) supported the hypothesis.

Some Data Related to Use of Displays

The derived HEPs in this Handbook were based on the sources described above. As an example of the manner in which error probabilities were derived, the following section outlines the derivation of estimates as- sociated with displays. HEPs were derived for the following:

Quantitative reading errors on analog displays Check-reading errors on analog displays

Quantitative reading errors on digital displays Recording errors Errors in the use of CRTs

6-13

Data - Use of Displays

Data on Quantitative Reading of Analog Displays

in one study of. errors in reading and orally reporting two-digit numbers from analog displays (Chapanis and Scarpa, 1967), a response was scored as an error if it was incorrect by a single digit, i.e., if either the first or second digit was incorrect. Under these conditions, the error proba- bility was .05. In another study of errors in reading two- or three-digit numbers from standard dials (Murrell et al, 1958), the subjects were al- lowed a tolerance of one digit for the leas t-signif icant (last) digit. Thus, if the required number was 148 and the subject read 147 or 149, the response was counted as correct. Any response < 147 or > 149 was counted as an error. In this study, the error probability was .024. It is obvious that the error probability declines rapidly as the tolerance is increased, which is what we would expect. In a real-world situation, an analog dis- play rarely has to be read to an accuracy of 1 digit on a scale of 100 or more. The accuracy of the display itself is usually only 3% of the full- scale reading, so that errors of the last digit are of little consequence. Although we do not have experimental data on the frequency of gross errors in reading analog displays, in most cases it is not important that an operator obtain an exact reading, so the error probability reported in laboratory studies can be modified on the basis of industrial requirements.

The Murrell study cited above was a realistic study of analog display read- ing. A group of British naval chief petty officers served as subjects and read a variety of standard power meters at various distances. The distance of interest to us, 30 inches, is typical of the distance at which operators would read a meter in an NPP. At 30 inches, there were 60 errors in 2,520 trials, yielding the HEP of .024. (About half the readings were two-digit numbers and half were three-digit numbers.)

The report does not indicate the magnitude of the errors that did occur, so we do not know the distribution of such errors, but a related study allows us to form an estimate (Guttman and Finley,. 1970). In a study of visual interpolation in which subjects were required to report map coordinates to the nearest tenth of a grid cell, the distribution of errors, based on 1,613 readings, was as follows:

Magnitude of Error (Tenths) Number___

N one 1415 87.72 One 190 11.78

.Two 8 .50

In this study, one-tenth of a scale division corresponded to one last-place digit. The map-reading task was much more difficult than a dial-reading task, so the higher error scores are not unexpected. The significant find- ing is that when the error tolerance is increased by a factor of 2 (from one-tenth to two-tenths) the number of errors is reduced by a little more than an order of magnitude. Similar findings were obtained in another study (Kappauf and Smith, 1948).

6-14

Data - Use of Displays

If we apply these findings to the Murrell data and assume an order-of- magnitude reduction in error for a one-digit increase in error tolerance for the last digit (as would be the case in a practical situation), we obtain an HEP of .0024, which is very close to the .003 we used in WASH-1400 and which we have used here (Table 11-3, item 1). We are using the larger error probability to allow for those cases in which people read meters from distances greater than the 30 inches assumed to be typical.

Data on Check-Reading of Analog Displays

The HEP for check-reading of analog displays has also been studied in laboratory settings. Check-reading involves the determination that a display parameter is within certain limits, e.g., the display indicator is within the green area and does not normally require that a quantitative value be read. Check-reading should not be confused with the use of a checklist, which may require both check-readings and quantitative readings of the user.

For the conventional single-pointer display with limit marks, the probabil- ity of a check-reading error is assessed as .001 (Table 11-4, item 2). This estimate is based on the AIR Data Store (Munger et al, 1962). A study by Kurke (1956) indicates that check-reading errors using meters without limit marks occur more than twice as frequently as with meters that have limit marks. Therefore, we would expect check-reading errors of analog displays without limit marks to occur with a probability > .002. Because the check-reading conditions in an NPP are not as well-structured as those in the Kurke study, we assumed an HEP of .003 to allow for various PSFs that could affect reliability (Table 11-4, item 4).

Data on Quantitative Reading of Digital Displays

Unlike the case with analog diaplays, for quantitative reading of digital displays, no tolerances are allowed since no interpolation or alignment of scales and pointers is involved. Digital displays are read more accurately than analog displays. In a study of digital-clock reading (Zeff, 1965), 4 errors were made in 800 trials, an unmodified error probability of .005. This figure includes errors of reading, memory, and recording. Subjects read the four-digit digital clocks as rapidly as they could, and the dis- play was removed from view as soon as they started writing so that there was no opportunity to review the display. In a practical NPP situation, a person would have adequate time to read his display and could recheck his readings as desired. We estimate that the probability of error on the reading task alone (excluding writing errors) is no greater than .001 in a realistic situation (Table 11-3, item 2).

As noted in Table 11-4, item 1, the operator does not really check-read digital readouts; he must actually read the value. Therefore, whenever he is checking a digital readout, we use the estimated .001 HEP for quantita- tive reading of this type of display. Inaccuracies in remembering what digital value is appropriate or in reading the desired value from a written procedure are not included in this .001 HEP.

6-15

Data - Use of Displays

Errors of Recording

As noted in Chapter 11, the HEP for entering data from a display into a log is actually a composite of at least two actions: reading a display and recording the reading. Sometimes a third action, communicating, is involved, as when the reader states the reading to the writer, but this activity will be disregarded. Thus, there are two primary opportunities for error when a quantitative display indication is to be recorded. .In one study, skilled machinists copied six-digit numbers from their own hand- writing onto a special form. Their HEP for reading and recording was .004 (Rigby and Edelman, 1968b).

If we assume the reading HEP to be .001 (see previous heading) and subtract this from .004, the HEP for recording per se is .003. Actually, this value is pessimistic because the errors occurred in the course of performing a series of calculations, and the subjects were not paying particular atten- tion to recording individual groups of numbers accurately.. In a realistic NPP situation, in which a person reads a display and records it on a pre- pared form as an individual task, without time pressure and with opportu- nity to recheck his readings, recording errors will occur with a probabil- ity estimated to be no greater than .001 per symbol for each of more than three symbols (Table 11-3, item 9). Thus, the combined error probability of reading and recording a digital display of four digits is estimated as .002.

Errors in the Use of CRTs

Cathode-ray tubes (CRTs) are the most versatile type of display currently in wide use, and their capabilities are too well known to require descrip- tion here. As in the case with any display device, attention must be paid to the usual human factors principles, such as symbol size, brightness, stability, freedom from glare, etc. Such requirements are readily accommo- dated by present-day equipment.

In addition to the capability of presenting any information available via conventional displays, the CRT display can be sequenced. Any information desired by the operator can be "called up" almost instantaneously. The most significant advantage of the sequencing capability is that the opera- tor is more likely to read a display that is directly in front of him than one that requires gross physical movement. In present-day plants, it is common practice for displays that require periodic scanning to be located at various distances from the operator's station, requiring him to walk to the far end of the control room, or behind several equipment racks, to read the required displays. It is inevitable that those displays that are not conveniently accessible will not be read as often or as accurately as those displays that are easily "reached." The CRT makes it possible to present the information from all displays in one place, conveniently in front of the operator, where he is unlikely to overlook it. The CRT also enables us to present the information in the most meaningful format for the situation; mimic diagrams of the plant, or any subsystem, can be generated almost as readily as ordinary alphanumeric symbols.

6-16

HEPs - Valving Operations Data - Oral Instructions/Written Procedures

The introduction of CRTs at the operator's station should make available all plant information to the operator, as required or as desired. Alarm functions are readily incorporated in the displays. Although we do not as yet have enough experience with CRTs in NPPs in this country to develop estimates of HEPs for different situations, it appears that many of our listed HEPs relevant to conventional displays will be reduced appreciably when the same information is presented via CRTs at the operator's station.

Estimates of HEPs Related to Valving Operations

Almost all of the derived HEPs in Chapter 14, "Locally Operated Valves," are based on judgment. For example, in Table 14-1, item 1, we assessed an HEP of .001 for an error of selection of locally operated valves in which good ergonomics is employed: the valves are clearly and umambiguously labeled and set apart from valves that are similar in all of the following: size and shape, state (open or closed), and presence of tags. The other entries in the table were derived by incorporating judgment for the effects of progressively worse ergonomics. We are confident that the estimated HEPs truly reflect increasing difficulty (i.e., more selection errors). The absolute values assigned need to be verified by actuarial data.

Estimated HEPs for valve restoration by operators are affected strongly by the type of tagging procedures and related paperwork described in Chapter 16. Again, no error relative frequencies have been recorded on which to base HEPs for the different tagging possibilities, so the Table 16-2 modi- fications to nominal HEPs are based on judgment, with care taken to ensure that the relative differences in these HEPs make good psychological sense.

Some Data Related to Oral Instructions and Written Procedures

In deriving the HEPs for Chapter 15, "Oral Instructions and Written Procedures," most were based on judgment and extrapolation from experi- mental data. For example, Table 15-1 gives estimated probabilities of errors in recalling special instructions given orally. The available experimental studies on retention typically test the retention of rela- tively meaningless materials such as numbers, syllables, or unrelated words (cf. Chapter X, "Retention and Forgetting," in McGeoch and Irion, 1952). But oral instructions in an NPP deal with meaningful material. Lacking relevant experimental data, in Table 15-1, item la, we estimate a token .001 probability of forgetting a single oral instruction when it is the only one given--due to the possibility of task interruption and other factors discussed in Chapter 15. For the retention of more than one item to be remembered, we use a factor of 3 increase in the HEP for an error of omission for each item beyond the first. This factor was selected because we judged it provided a reasonable modification of laboratory data for the retention of meaningless items (of. Woodworth, 1938, p 18).

In Table 15-3, item 4, a value of .01 is assessed as the nominal HEP for omitting a step when using a written procedure in which there are over 10 separate steps in the procedure and there is no checkoff provision for the

6-17

Administrative Control Data - Manual Controls

step being evaluated. This value is derived from industrial experience,0

records of inspection, and field studies in military and industrial set- tings and was modified on the basis of applicable PSFs to account for whether the list is a long one (W1 items) or a short one (410 items), whether or not a checkoff provision is used properly, and whethier the task is done from memory without reliance on the written procedure. This model- ing is another example of representing a complex set of PSFs with only a few points; there are no data to support a more detailed model. Although the absolute values for each entry in Table 15-3 are based on judgment, the relative differences in estimated HEPs seem psychologically sound.

Administrative Control

In Chapter 16, "Administrative Control" refers to the checking of human performance mandated in a plant and the extent to which this and other plant policies are carried out and monitored, including the use of inven- tory systems. No hard data exist to show the effects of different kinds and levels of administrative controls. Table 16-1, item 1, lists an esti- mated probability of .01 that plant policies will be ignored. This esti- mate is based on industrial experience-- it reflects the occasional lapse from conscientious performance that occurs in any normal, properly moti- vated group. Such lapses may occur because of time stress, fatigue, or other causes. The figure does not consider chronic violators of plant policies--it is assumed that chronic violators will not be tolerated.

The estimated .01 probability that plant procedures or policies will be

ignored does not apply given the proper use of checklists. The use of0 checklists is a unique case in that people may use a checklist incorrectly without violating plant policies. (This could occur if the plant training program does not stress the proper use of checklists.) Also, the checklist user often regards "shortcuts" in the use of checklists as a step toward improved efficiency rather than as a violation of plant policies. The estimate that a checklist will be used properly only one-half of the time (Table 16-1, item 8) is based on observations in a number of industrial settings. Some reviewers of the draft Handbook have stated that the esti- mate of correct use of the checklist 50% of the time is too optimistic for job operations with which they are familiar. Although this may be true in certain cases, the estimate of 50% is reasonable as a generic figure. For any plant, the-analyst should attempt to derive specific estimates.

Some Data Related to Manual Controls

Some of the HEPs in Chapter 13, "Manual Controls," are based on other data stores and have been modified for NPP situations. The basic situation is that of selecting a control within a group of similar-appearing controls identified only by labels. The AIR Data Store (Munger et al, 1962) sug- gests an error probability of .002 for such a situation. Our REP of .003 (Table 13-3, item 2) allows for the difficulty of relating controls to their functions in a typical NPP. The error probability is judged to be reduced by a factor of 3 if the similar-appearing controls are functionally

6-18

Models - Response to Abnormal Events Use of Ch. 20 Nominal HEPs

grouped (Table 13-3, item 3) and by another factor of 2 if clearly drawn mimic lines are provided (Table 13-3, item 4). This is consistent with the improvements expected as PSFs are improved (Table 3-8).

If the desired control is selected, the only remaining errors of conse- quence are the direction of operation of the control and the final control setting. For errors in direction of operation (Table 13-3, items 5-8), the estimated HEPs are derived from a review of the available data banks: the Bunker-Ramo Data Bank (Meister, 1967), the Aerojet-General Data Bank (Irwin et al, 1964a and b), and the AIR Data Store (Munger et al, 1962).

For errors in setting a multiposition switch (Table 13-3, item 9), there are numerous relevant variables that can affect the accuracy of the setting--thus the wide uncertainty bounds associated with the nominal HEP.

Models of Response to Abnormal Events

For purposes of PRA, errors in responding to abnQrmal events are divided into perceptual, cognitive, and response aspects of human behavior in accordance with Figure 3-1. For reasons expressed in Chapter 12, diagnosis errors are the only cognitive aspect of human behavior considered. Chapter 12 presents two human performance models that plot estimated HEPs for diagnosis against time after some compelling indication of an offnormal situation. The models are based entirely on speculation. One model, an initial-screening model, is based primarily on a consensus of PRA special- ists (Figure 12-3 and Table 12-2). The other model, a nominal model, is based in part on an adjustment to the initial-screening model and in part on our judgment (Figure 12-4 and Table 12-4). Both models incorporate very wide uncertainty bounds and are presented as interim models until in-plant and simulator data can provide the basis for data- based models.

Use of the Nominal HEPs from Chapter 20

In performing an HRA, the HRA event trees require estimates to be made of HEPs and human success probabilities (HSPs) to represent the failure and success limbs in each branching. A major source of these estimates is Chapter 20, which includes a procedure for using its tables. The tabled HEPs are nominal HEPs, i.e., estimates of HEPs for tasks or activities prior to consideration of plant-specific PSFs. Unless otherwise specified, the nominal HEPs are for one person and must be modified, using our depen- dence model, if more than one person is involved. These nominal HEPs are modified, using directions in Chapter 20, to account for PSFs such as stress, skill level, administrative control, and so on. Within each table, the items pertaining to a certain class of error, e.g., errors of omission in valve restoration, reflect more specific PSFs such as equipment display features, size of valve handle, and so on. Using plant-specific informa- tion, the analyst selects the appropriate item or items from a given table.

In the tables, each nominal HEP is given as a single-point estimate with an error factor (EF) designating the uncertainty bounds. Thus, an HEP of .003

6-19

A Final Comment on Data Sources

with an EF of 3 means that the lower uncertainty bound is .003 divided by 3 = .001, and the upper uncertainty bound is .003 x 3 = .01. When the analyst has plant-specific information, he may use the nominal estimate of .003, or any other value within the stated uncertainty bounds, as his esti- mate of the item for the plant he is analyzing. Chapter 7 discusses uncer- tainty bounding, its use in PRA, and provides a rationale for considering that each nominal HEP be treated as the median of a lognormal distribution of HEPs in which the lower bound is the 5th percentile HEP and the upper bound the 95th percentile HEP.

Based on our experience in the plants listed in the Foreword, we have set the nominal HEPs to be representative of the nuclear power industry. Therefore, if the analyst believes that the PSFs in the plant he is analyz- ing are average for the industry, he should use the nominal HEP as his best estimate. We are not able to provide an objective definition of what is average. Analysts who have visited a large number of NPPs develop a feel- ing of what is an above-average plant, an average plant, and a below- average plant for the application of any PSF. In general, if a plant adheres remarkably well to standard ergonomics principles (such as those listed in MIL-STD-1472C or NUREG-0700), if it has provided excellent recov- ery factors to catch human errors, if it is clear that the personnel really do follow the plant rules and procedures fully, the plant is designated as above average. This judgment obviously requires a good deal of digging; an analyst cannot form an accurate judgment in one brief visit to a plant. Although it is necessary to consult with plant management (as well as with the working crews) in evaluating a plant, it is unrealistic to expect that plant management will make a completely objective assessment of their own plant. In the authors' experience, most plant management personnel believe theirs is among the better plants if not the best. We have noted cases in which plants with serious human factors deficiencies have been considered by their management to fall into the above-average category.

It will be apparent from the foregoing that considerable judgment is re- quired to assess HEPs that accurately reflect the performance of tasks in a given plant. As a body of data on HEPs accumulates, the requirement for subjectivity on the part of the human reliability analyst will lessen. In the interim, the requirement for judgments by the analyst will be very frequent. For this reason, it is preferable that more than one analyst be employed in performing an HRA and that expertite in human behavior technol- ogy be represented on the HRA team.

A Final Comment on Data Sources

The preceding describes the manner in which the HEPs and performance times in this Handbook were derived. It is not exhaustive (not all HEPs are dis- cussed); rather, it describes the approach that was followed. We expect that experience will reveal any gross inaccuracies in our estimates, and we fully expect inaccuracies to be discovered. To the extent of our knowl- edge, we have attempted to err on the conservative side, so our estimates will probably be somewhat pessimistic. In view of the potential conse- quences of errors, we felt that this was a prudent approach. When suffi- cient data have been gathered to provide more reliable estimates of the

6-20

A Final Comment on Data Sources

HEPs, we will revise them. Our position is very clear; judgment of experts is necessary in performing an HRA, but this judgment will be much more reliable if actuarial data on the tasks of interest are available to aid the judgment process. Extrapolation from highly relevant data should nearly always provide better estimates of HEPs and performance times than judgment based even on superior expertise.

6-21

Ch. 7. Distributions and UCBs Overview; Human Variability

CHAPTER 7. DISTRIBUTION OF HUMAN PERFORMANCE AND UNCERTAINTY BOUNDS

Overview

In most probabilistic risk assessments (PRAs), it is necessary to propagate distributions of failure probabilities through the system analysis rather than propagate single-point estimates of each failure event. This chapter provides background information and guidance on the propagation of distri- butions of human error probabilities (HEPs) in the human reliability analy- sis (HRA). The characteristic for which a distribution is proposed is the HEP for a randomly selected individual performing a specified task under defined conditions.

Because of the lack of data on distributions of HEPs for nuclear power plant (NPP) tasks, it has been necessary for us to propose distributions for PRA purposes. The proposed distributions are based on some data on individual differences for tasks outside the NPP regime, but mostly they are speculative. It is suggested that the lognormal distribution be employed for PRA purposes, that the single-point estimates of HEPs in the Handbook be regarded as medians of this distribution, that the error fac- tors (EFs) listed in the Handbook be used to designate the range of HEPs symmetrical on a logarithmic scale about the median, and that this range be assumed to include the middle 90% of the HEPs in that distribution. If the analyst has reason to question the usefulness of these suggestions for a given application, he can employ sensitivity analysis to check the sensi- tivity of the PRA to other assumptions he believes may be more appropriate.

This chapter treats three related topics in the following order:

(1) We present some background data on the distribution of human perfor- mance in a variety of settings and offer some hypotheses about the distributions of HEPs in NPP tasks.

(2) We present guidelines for establishing uncertainty bounds (UCBs) around point estimates of HEPs in NPP tasks. The estimated UCBs are wider than the presumed variability among NPP personnel so as to include other sources of uncertainty when assigning the nominal HEP for a task.

(3) Based on the above two topics, we present some guidelines and methods on the propagation in an HRA of UCBs for HEPs. One of these methods is a UCBs propagation method for HRA presented in Appendix A.

Data and Hypotheses about Human Variability

Intra- and Interindividual Variability

All human performance is characterized by variability within the same in- dividual and across individuals. Generally, variability within individuals is small compared with variability among different persons in an NPP. Moreover, it is impossible to predict a specific individual's day-to-day

7-1

Human Variability

variability. In any event, it is assumed that our estimates of variability among individuals are large enough to account for the effects of variabil- ity within individuals. Therefore, our primary concern is with estimating the effects of interindividual variability--the variability among properly trained and experienced operating personnel.

Despite variability among people, managerial influences in most technical jobs tend to restrict variability under normal operating conditions. If a person consistently performs far below the average for his group, he usually will be retrained, reassigned, or terminated. If he consistently performs in an exemplary manner, he usually will be promoted or transferred to a more challenging and responsible position (Peter and Hull, 1969).

The Normal Distribution

Few human traits and abilities conform to the normal (Gaussian) distribu- tion. As pointed out by Wechsler (1952), the only human distributions that approximate the normal curve are those that pertain to the linear measure- ments of people, such as stature, lengths of extremities, the various diam- eters of the skull, and certain ratios of bodily measurements. Even among these, there is often a considerable departure from the symmetry of the normal distribution.

Wechsler, a psychologist well known for his development of scales to mea- sure adult intelligence, measured the abilities of many people in such disparate tasks as running the 100-yard dash and assembling radio tubes in a factory. He noted that the usual measure of human variability, the standard deviation (SD), was highly sensitive to the shape of the distribu- tion. He therefore suggested a different measure of variability, which he called the total range ratio. He defined this as the ratio of the highest score to the lowest score of a group of people homogeneous with respect to what is being measured* but excluding the extreme scores, i.e., the lowest and highest one-tenths of 1% of the population (Wechsler, 1952, p 46). He discovered that when the extreme scores were discarded, the ratio of the scores of the best performers to the scores of the worst performers was generally on the order of 3:1 to 5:1. He further discovered that if only production tasks were considered, the typical range ratio was rarely over 3:1 and more typically approached 2:1 (Table 7-1). He noted that this ratio was probably influenced by unwritten agreements among workers. This peer restriction of performance has long been recognized in production work, even with piecework systems in which workers are paid in direct relation to their output.

By homogeneous, Wechsler means that all the individuals considered were subject to the factors that produced the characteristics that influence what is being measured. Therefore, one would not compare persons in running speed when only some of them were especially trained in this ability or when some of them were physically impaired.

7-2

0. Table 7-1 Range ratio in production operations

Unit or Subjects

Operation Criterion (Adults) Mean SD Extremes Ratio*

Filament mounting Avg number 100 104.9 13.5 132, 75 1.73:1 (elec. bulbs) per hour

Assembling radio Number of tubes 65 99.5 16.9 132, 61 2.00:1 tubes done per hour

Manipulating Avg output 120 71.8 6.85 104, 51 2.04:1 automatic lathe per day

Card punching Avg no. cards 113 232.2 29.8 340, 160 2.10:1 punched per hour on day

shift

Checking, posting, Percent of 34 102.1 20.7 150, 65 2.30:1

and listing avg females

Checking grocery Time in 46 365.0 -- 575, 225 2.53:1

orders seconds

Machine sewing Units per hour 101 72.4 14.3 112, 43 2.55:1 (5-day avg) females

Typing Number of words 616 53.4 9.57 85, 30 2.83:1

per minute

Card punching Avg no. cards 121 198.5 31.1 275, 115 3.00:1

punched per hour on night shift

-J1

(

extreme values in the previous column.*The ratio is computed by discarding the

Source: Wechsler, 1952, p 68.

Human Variability

Wechsler's data pertain to quantity of output, not generally a concern in NPP operations. Wechsler states, "Of the two basic characteristics of production, namely, amount and accuracy, only that of amount can be treated unequivocally in studying range. . . . If errors (or accuracy) alone are taken as a base, they are likely to prove misleading" (p 65).

To study the validity of this limitation, L. W. Rook, then of Sandia National Laboratories, analyzed 6 months of production defect data from a large industrial plant that manufactures electronic and electromechanical components. (This unpublished 1962 study is described briefly in Swain, 1980b, pp 64 and 69.) Rook had the following hypothesis:

"In an industry where [quantity of] production is more important than errors, the production process will tend to weed out those workers who produce far less than the average worker and will tend to promote those who produce far more; this process would tend to generate range ratios of quantity of production of less than 3:1. But in an industry such as the nuclear weapons industry where pro- duction quantity is not nearly so important as the number of de- fects, the production process will tend to weed out those workers who generate more defects than the average worker and will tend to promote those who generate far fewer; this process would tend to generate a range ratio of defect frequencies of less than 3:1."

When Rook tabulated individual differences for each of several types of defects, he found range ratios approximating 2:1 for numbers of defects, and stated, "It has been observed many times that, in typical situations, the best workers are almost never more than three times better than the worst workers. In fact, a ratio of two to one is more typical" (Rook, 1965). This ratio, known as Wechsler's Range Ratio, can be stated as follows:

Given a population that is homogeneous with respect to some ability that can be measured with a ratio scale, if the extreme scores are discarded (i.e., the lowest and highest tenths of 1%), the highest score in the distribution will rarely be more than five times the smallest score and usually not more than three times the smallest score.

Based on the above, if we have a good estimate of the mean of some ability, and if we assume a roughly normal curve, we can construct an estimated distribution by assigning a range ratio to the ±3 SD points. For example, if we assume a range ratio of 3:1, the ±3 SD points in the distribution correspond approximately to values 50% greater and less than the mean. The lower bound would be the point estimate of the HEP divided by 2 and the upper bound the HEP times 1.5. We have used this method to derive esti- mated extremes of performance in HRA in weapon systems.

7-4

Human Variability

The Lognormal Distribution

As noted earlier, the assumption of a normal curve often is not valid. When data are collected in experiments of human performance, different shapes and types of distributions occur. Normal, lognormal, Weibull, Gamma, exponential, or other distributions may fit the empirical distribu- tions more closely (see Green and Bourne, 1972, Chapter 7, for a discussion of various distributions).

We believe that the performance of skilled persons tends to bunch up toward the low HEPs on a distribution of HEPs, as found, for example, in the ranges of individual differences in errors among keypunch operators and proof machine operators (Klemmer and Lockhead, 1962). This is quite unlike the case in the typical laboratory experiment in which the task is made deliberately difficult so that a high HEP can be obtained for statistical convenience. In such cases, one is more likely to obtain a raw data curve approximating the normal distribution. However, in the typical industrial setting, especially with highly skilled performers, it is reasonable to presume that most HEPs fall near the low end of the error distribution. In the absence of actuarial data from which actual distributions could be derived, it is appropriate, for PRA purposes, to select some nonsymmetric distribution of HEPs that reflects the above presumed characteristics of skilled performers. We have selected the lognormal distribution rather than the Beta, Gamma, Weibull, or other nonsymmetric distributions because the lognormal distribution appears to provide an adequate fit to human performance data and is computationally tractable. For example, the para- meters of a lognormal distribution can readily be determined by specifying two of its percentiles.

Most of the data that support the use of a lognormal distribution for the performance of skilled people are for response times. For example, a lognormal distribution was reported in a British study of the time taken to respond to a simulated alarm signal superimposed on normal tasks in an NPP (Green, 1969). In a follow-up study in Danish research reactors, similar results were found by Jens Rasmussen and his coworkers at the Ris0 National Laboratory (Platz et al, 1982). Several studies in NPP simulators have shown distributions of response time versus cumulative probability that are approximately lognormal (Haas and Bott, 1979; Bott, Hutto et al, 1981; Bott, Kozinsky et al, 1981; Beare, Crowe et al, 1982; Beare, Dorris et al, 1982; Kozinsky, 1983b). Finally, in a recent survey by Fragola of studies in which distributions of performance time were reported, all of them showed distributions that approximated the lognormal (reported in Hall et al, 1982).

The above studies support the assumption of a lognormal distribution for response time. We have not found a large amount of data to support the assumption of any particular distribution of HEPs. However, error distri- butions in keyboard entry of experienced bank proof operators, bank proof trainees, experienced keypunch operators, and first-week keypunch school attendees appeared to be closely approximated by the lognormal. In addi- tion, we performed an analysis of HEP data that revealed lognormal-type

7-5

Human Variability

distributions for simple tasks and slightly skewed distributions approach- ing the normal for more complicated tasks (Swain, 1967b). Furthermore, it makes good psychological sense to presume that the performance of skilled personnel will bunch up toward the good end of a distribution of perfor- mance whether the measure be performance time or human error frequencies.

Until we can obtain additional data on distributions of response times and HEPs for individual performers of NPP tasks, we are limited to proposing distributions based on the above information and on our understanding of the psychological principles of behavior. As explained in Chapter 6, it is doubtful that appropriate data can be collected in NPPs, but if error frequencies and response times can be obtained from plant records, it will be possible to pool these data across individuals and record error frequen- cies and response times by task and other descriptors, regardless of who made the errors. Such data, combined with data from NPP training simula- tors, will assist in the derivation of distributions, of human performance in which we will have greater confidence than in the derived distributions presented in the Handbook. In the meantime, we 'suggest the lognormal distribution be employed for PRA studies.

The parameters of the applicable lognormal distribution are, of course,

speculative. We hypothesize that for most NPP tasks, a lognormal probabil- ity density function (pdf) with an SD of 0.42 would provide a suitable fit (Figure 7-1). This SD of 0.42 was obtained by assuming a 4:1 range ratio between the 95th and 5th percentiles of error probabilities for tasks performed under routine conditions. This range ratio is more conservative than Wechsler's Range Ratio and seems appropriate in view of the current lack of data on individual differences in the performance of NPP tasks. For tasks performed under high stress, the entire distribution will be shifted to the right and may be skewed to the left rather than to the right. The rationale is that under high stress, even trained personnel will tend to group at the poor-performance end of the distribution. We also expect the range of individual differences to increase under high stress. Until more data can be collected on the performance of NPP person- nel, such hypothesized distributions are merely suggestive. As described in the section on "Uncertainty Bounds," for PRA purposes we recommend the use of wider UCBs than would be defined by individual differences per se.

Relative Unimportance of Assumed Distributions

Although we would like to have data clearly showing the distributions of human performance for various NPP tasks, there is ample evidence that the outcomes of HRAs are relatively insensitive to assumptions about such distributions. One example is described in Appendix A8 to the draft Hand- book (Swain and Guttmann, 1980).

Another example is illustrated by an experimental study in which Mills and Hatfield (1974) collected distribution data on task completion times for several realistic tasks. One significant finding was that errors resulting from assuming an incorrect form of the pdf of task times were small and of little practical significance. The distributions, all unimodal, were best

7-6

Figure 7-1

0.ft CL zi

LOW HIGH PROBABILITY OF ERROR (HEP)

Figure 7-1 Hypothesized lognormal probability density function of HEPs for nuclear power plant personnel.

7-7

Human Variability

described by various Weibull functions, but the authors state, "From a practical point of view, . . . the error produced by assuming normality may not be large enough to warrant using a better-fitting pdf."

During the preparation of WASH-1400, a sensitivity analysis was performed by Dr. W. E. Vesely using a Monte Carlo procedure to see if the assumption of different pdfs for the human error estimates would materially affect the predicted availability of various safety-related subsystems. It was found that the predicted availability did not differ materially when different distributions were assumed.

We can conclude that the assumption of normal, lognormal, or other similar distributions usually will make no material difference in the results of HRA analyses for NPP operations. In some cases, this insensitivity may result from a well-designed system that has so many recovery factors that the effect of any one human error on the system is not substantial. If very different distributions such as a bimodal type, an exponential, or an extreme value were used, it is possible that different results could be obtained. For computational convenience, one might wish to assume the same distribution for probabilities of human failure as the one used for proba- bilities of equipment failure, as was used in WASH-1400. A sensitivity analysis will reveal whether any significant differences will be obtained with different assumptions.

In the estimation of data distributions for cognitive aspects of tasks, such as the time taken to diagnose a transient correctly, it is possible that different diagnoses might well result in a bimodal or multimodal distribution, that is, the distribution of performance time to diagnose a transient correctly, given Diagnosis A, may differ substantially from the time distribution to make the correct diagnosis, given Diagnosis B. The two distributions of time may show very little overlap. This could readily be the case if both diagnoses are plausible but one of them is incorrect. If the analyst believes that this is a likely possibility, he can simplify his analysis by estimating the percentage of operator teams that would employ each possible diagnosis and treat each different diagnosis as a separate branching in his HRA event tree. For the time estimates associ- ated with each diagnosis, the analyst could employ the lognormal distribu- tion of performance times. A sensitivity analysis can be used to determine if such a detailed analysis is worthwhile.

Definition of Nominal HEPs

In Chapter 6, we described the derived HEPs in the Handbook and the tables of HEPs in Chapter 20 as nominal HEPs. A nominal HEP is the probability of a given human error when the effects of plant-specific performance shaping factors (PSFs) have not been considered. These nominal HEPs are single- point estimates and represent the median of a lognormal distribution. The median HEP of the lognormal distribution is a good estimate if one does not know which person will perform a task and one does not know his particular capability, motivation, and other internal PSFs that affect his performance.

7-8

Uncertainty Bounds

The mean (or expected value) of a lognormal distribution of HEPs is not a parameter that has intuitive psychological meaning. Inasmuch as consider-, able judgment has been used in deriving the nominal HEPs in the Handbook, it was reasonable to use medians as the point estimates.

Uncertainty Bounds

Need for and Definitions of Measures of Uncertainty

Each estimate of an HEP or a performance time related to a task or activity is associated with some degree of uncertainty. As noted in Chapter 12 of NUREG/CR-2300 (1983), the term "uncertainty" has historically been used to mean both imperfect knowledge and stochastic variability. We use the term in the same sense and do not attempt to differentiate between these two classes of uncertainty; that is, in our estimates of uncertainty for HRA, we include both the stochastic (random) variability of individuals and the presumed uncertainty of the analyst in the HRA process.

Generally, the use of single-point estimates of task reliability are ade- quate in design trade-off studies. In such work, we simply ignore uncer- tainty in the estimate. However, distributions are required for PRAs. Because of the lack of data, we derive or assume distributions for each of the nominal HEPs in the Handbook. These distributions are based on our judgment and are intended to include the normal variability of individuals performing tasks as well as other sources of uncertainty described below. When we use the term uncertainty bounds (UCBs), we mean estimates of the spread of HEPs associated with a lognormal distribution, as discussed later. There is a lower and an upper UCB. The ratio of the upper to the lower bound is the range ratio. The square root of this ratio is known as the error factor (EF). We attempt to select UCBs that have a very high likelihood of including the actual measure of human performance that would be obtained in any given performance. In our opinion, the UCBs will include at least 90% of the true HEPs for a given task or activity. It should be clear that these UCBs are not the same as statistical confidence limits. If the analyst wants to express greater uncertainty, he can desig- nate our UCBs as conforming to wider distribution, e.g., to the 10th and 90th percentiles.

Elements of Uncertainty in an HRA*

This chapter treats the uncertainties for estimates of HEPs only, because the estimates related to response times are expressed in the form of HEPs at selected times after some stimulus for operator action. Uncertainty in estimates of human performance is due to the following elements (among others):

(1) The dearth of data on human performance in NPPs that is in a form suitable for HRA,

Some of the material in this section is taken from material prepared by B. J. Bell and A. D. Swain for Chapter 4 of NUREG/CR-2300.

7-9

Uncertainty Bounds

(2) The inexactness of models of human performance that purport to describe how people act in various situations and conditions,

(3) Inadequate identification of all relevant PSFs and their interactions and effects,

(4) Limitations in the skill and knowledge of the human reliability analyst, and

(5) The variability in performance within an individual and among the performances of different individuals.

The first element of uncertainty for estimates of HEPs, the shortage of NPP-specific data on human performance, is the most critical and has been discussed earlier in the present chapter and in more detail in Chapter 6.

The second element of uncertainty is the modeling of human performance. The state of the art of HRA is such that modeling of human behavior can qualitatively account for its variability and for discrepancies in response situations, but there are definite limitations in quantifying such models. There are many models of human performance, but few can be used to estimate the probability of correct or incorrect performance in applied situations. Furthermore, all models, even those that can be applied to HRA, e.g., the models in this Handbook, are themselves abstractions of real-world circum- stances. As such, they only partially represent the situations they simu- late. In some cases, experimental data have provided strong support for the general form of the models, e.g., the usual curvilinear form of the performance-under-stress curve (Figure 3-4), but in others, the forms are still speculative although based on sound psychological concepts.

The third element of uncertainty, inadequate identification of PSFs associ- ated with a task, is due to unfamiliarity of the analyst with the task. Without a good knowledge of the PSFs for a plant-specific HRA, the analyst is not likely to use the appropriate nominal HEPs from this Handbook, and he will not have enough information to adjust these HEPs for plant-specific applications. This is probably the biggest source of error in extrapolat- ing data from other sources for application to the NPP situation. Unless analyses of the tasks in both situations are done in sufficient detail, data from an outside source may be misapplied to NPP tasks. For example, a valve restoration task in a chemical processing plant may be superficially similar to an equivalent task in an NPP, but the HEP from the former plant may be based on people using well-designed checklists, and the valve re- storation procedures carried out in the NPP may be done from memory only. Use of the HEP from the former situation to estimate the HEP for the latter situation would obviously result in a gross underestimate of the true HEP.

The above difficulties will be exacerbated if the level of interaction between the human reliability analyst and other members of the PRA team is low. Unless the HRA analyst is a real working member of the team, he may not become aware of important plant-specific effects of certain PSFs, and his estimates of HEPs may be much too low or even much too high. In such

7-10

Uncertainty Bounds

cases, the assignment of large UCBs will not compensate for his lack of knowledge.

The analyst himself is the fourth element of uncertainty. That is, if he

is not fully qualified, he may not be able to perform the necessary extrap- olations or to use the human performance models in this Handbook correctly. The less he knows about the operations and human activities in the plant, and the less he knows about the underlying psychology of human behavior in general, the less accurate his estimates of HEPs will be. That is obvi- ously a source of uncertainty that the untutored analyst may not recognize.

Finally, there is the uncertainty that results from the inherent variabil- ity of human performance due to individual differences, both within and between the people whose performances are being assessed in the HRA, as discussed earlier in this chapter. Even if one had a large amount of excellent human performance data on all NPP tasks, there would still be variability from this source contributing to the uncertainty in an HRA. In HRA, one does not attempt to estimate the performance of one particular person; instead, the estimates have to allow for the fact that any task may be performed by any one of many individuals, each of whom will vary some- what in his reliability from day to day and even within a day.

Derivation of Uncertainty Bounds

Earlier sections indicated the range of variability that may be expected in routine performance of well-defined tasks such as factory production work. In estimating the range of variability to be expected in NPPs, we have to make allowances for the greater variety of tasks, for the less routine nature of much of the work, and for a large number of unknowns, such as the relevant PSFs, administrative practices, and other such variables discussed in other chapters. Consequently, the range ratios used in reliability analyses of NPP tasks will be considerably wider than the nominal 4:1 ratio described in Figure 7-1, since there is uncertainty in the nominal HEPs assigned to the tasks as well as in the variability of the task performers. In consideration of the above sources of uncertainty, we have extended the UCBs around the estimated HEPs to limits that we feel confident include the true HEPs. For example, for an estimated HEP of .01 (.003 to .03), the numbers in parentheses represent a lower and an upper bound of uncertainty. These bounds reflect our judgment regarding the likelihood of various values of HEPs. We believe that the bounds include most of the range of HEPs resulting from individual differences and other unspecified sources of uncertainty. It is difficult to associate an exact probability statement with the bounds. However, it is our intention that the lower bound corre- spond to the 5th percentile and the upper bound to the 95th percentile of this distribution of HEPs. If the reader has better estimates of the distribution for some given application, he should use them.

For convenience., we usually state each set of upper and lower UCBs as an EF that corresponds to the square root of the ratio of the upper to lower bound. In the above example, .01 (.003 to .03) is usually restated as .01 (EF = 3). The EF of 3 is obtained by dividing .03 by .003, taking the

7-11

Uncertainty Bounds

square root of the results and rounding to 3. Conversely, .01 (EF = 3) is the equivalent of setting the lower bound = .01/3 = .003 and the upper bound = .01 x 3 = .03. This use of an EF assumes UCBs that are symmetrical on a logarithmic scale about the nominal HEP. Recall that we recommended that a lognormal distribution of HEPs be assumed for each task and that the nominal HEP be regarded as the median of the lognormal distribution.

The draft version of this Handbook did not always employ symmetrical UCBs. Given the speculative nature of the UCBs, the use of asymmetrical bounds was an unnecessary refinement. Furthermore, if one defines the upper bound as representing the 95th percentile HEP and the lower bound as representing the 5th precentile HEP on a lognormal distribution, the use of asymmetrical bounds is not possible. In this issue of the Handbook, we have employed symmetrical UCBs for most of the nominal HEPs listed. One exception ap- plies to large HEPs (those of about .25 or higher), where symmetrical UCBs are not possible because of the truncation of the distribution at the upper bound of 1.0. In such cases, a smaller range ratio will be designated to include the middle 90% of the true HEPs. For example, assume a nominal HEP of .25 with EF = 5. The lower bound is .05, and the upper bound is trun- cated at 1.0.

To permit the assumption of a lognormal distribution with a median that is very small, it is necessary to define a lower bound that may be smaller than 5 x 10-5. It is unlikely that such small HEPs would exist. However, in order to facilitate the method for propagating the UCBs, the exception- ally small values will be used as a lower 5th percentile. The only data we have seen with HEPs that small were collected in weapons production, where the lowest HEP for very small units of behavior (e.g., omitting a component from a circuit board in a well-designed assembly process) was about 3 x 10 5 (Rook, 1962; Rigby and Swain, 1968). Since the units of behavior in the Handbook are larger, and usually less controlled than for those in weagons production, we regard an estimate of a median HEP lower than 5 x 10 as one that indicates further analysis is needed.

Table 7-2 offers some general guidelines in assigning UCBs. The EFs in the table are generic; in several cases in the tables of estimated HEPs throughout Part III, the UCBs we have assigned to tasks are different from the generic bounds designated by the EFs in Table 7-2. In some situations, the user may wish to assign larger EFs than those in the table because he may have more uncertainty than that designated by the error factors in the table.

Table 7-2 shows that the larger EFs are used for HEPs smaller than .001 to reflect the greater uncertainties associated with infrequently occurring events. HEPs in the range of .001 to .01 generally apply to routine tasks involving rule-based behavior. HEPs greater than .01 are uncommon and are associated with tasks performed under conditions conducive to error, such as performance under high stress levels or checking the status of items that provide no indication of their correct status. Uncommon cases should be evaluated individually to determine appropriate UCBs, as we have done in developing the bounds for many tasks cited throughout the Handbook.

7-12

Table 7-2

Table 7-2 General guidelines for estimating uncertainty bounds for estimated HEPs*

t Item Task and HEP Guidelines** EF

Task consists of performance of step-by-step procedure con- ducted under routine circumstances (e.g., a test, maintenance, or calibration task); stress level is optimal:

(1) Estimated HEP < .001 10

(2) Estimated HEP .001 to .01 3

(3) Estimated HEP > .01 5

Task consists of performance of step-by-step procedure but carried out in nonroutine circumstances such as those involving a potential turbine/reactor trip; stress level is moderately high:

(4) Estimated HEP < .001 10

(5) Estimated HEP ) .001 5

Task consists of relatively dynamic interplay between operator and system indications, under routine conditions, e.g., increas- ing or reducing power; stress level is optimal

(6) Estimated HEP < .001 10

(7) Estimated HEP ) .001 5

(8) Task consists of relatively dynamic interplay between operator 10 and system indications but carried out in nonroutine circum- stances; stress level is moderately high

(9) Any task performed under extremely high stress conditions, 5 e.g., large LOCA; conditions in which the status of ESFs is not perfectly clear; or conditions in which the initial operator responses have proved to be inadequate and now severe time pressure is felt (see text for rationale for EF = 5)

The estimates in this table apply to experienced personnel. The perfor- mance of novices is discussed in Chapter 18.

For UCBs for HEPs based on the dependence model, see Table 7-3. tThe highest upper UCB is 1.0.

See Appendix A to calculate the UCBs for Pr[FT 1, the total-failure term of an HRA event tree.

*t SoaSee Table 18-1 for definitions of step-by-step and dynamic procedures.

7-13

Guidelines - Use of UCBs

The UCBs in Table 7-2 are also addressed to differences in the nature of the task itself (whether it is step-by-step or dynamic and whether it is routine or not) and to the level of stress that is likely to be associated with performance of a task (optimal, moderately high, or extremely high). (These task descriptors are defined in Chapter 2.) It will be noted that an EF of 10 is listed for the performance of dynamic tasks under moderately high stress, while an EF of 5 is employed for extremely high stress. Our rationale for the latter is that the estimated HEP for this level of stress will be high and the upper bound is already truncated at 1.0. Since it is desirable to have a more conservative (i.e., higher) lower bound for tasks performed under extremely high stress, the smaller EF is applied to raise the value of the lower UCB.

The calculation of UBCs for conditional HEPs (CHEPs) derived on the basis of our dependence model in Chapter 10 is a special case, and guidelines in Table 7-2 do not apply. The UCBs for CHEPs based on the dependence model are determined by the "Dependence UCBs Assessment Method" described in Appendix A. Table 7-3 shows the approximate values of nominal CHEPs and related UCBs for several basic HEPs (BHEPs) for each of the five levels of dependence. Chapter 10 describes this modeland the five dependence levels in detail, For most applications, the approximate values in Table 7-3 will be adequate. Most of the UCBs in Table 7-3 are not symmetrical. Some of the asymmetry is due to rounding, and some is due to the truncation of the upper bound at 1.0.

Guidelines for Use of HEP Uncertainty Bounds

In Chapter 5, we showed how single-point estimates of HEPs can be propa-

gated through an HRA event tree to arrive at an estimated total failure probability for the human actions represented in the tree (Figure 5-3). In this section, we discuss the use of UCBs and distributions rather than single-point estimates in the HRA event tree.

Bounding Analysis

Perhaps the simplest method of using HEP uncertainty bounds is the bounding

analysis, a procedure for determining a highly conservative estimate of the spread between the upper and lower failure bounds for some system, func- tion, or part of a system. In Chapter 5, we described the use of a best- case analysis and a worst-case analysis to establish the boundaries of the estimated influence of human performance in a system. We noted that for some HRAs, we have propagated a 10-1 estimated HEP through the HRA event tree to establish an upper boundary and a 10-4 HEP to establish a lower boundary.

An alternative approach is to propagate the upper UCBs for each HEP (and

lower bounds of each human success probability [HSP]) in the tree to derive the final upper bound and to propagate the lower UCBs for each HEP (and upper bounds of each HSP) in the tree to derive the final lower bound. To illustrate this approach, consider the two-task HRA event tree in Figure

7-2, and assume that A = .01, Bla = .001, and BIA = .05, and that the EFs

7-14

Table 7-3

Table 7-3 Approximate CHEPs and their UCBs for dependence levels* given FAILURE on the preceding task

Levels of

Dependence BHEPs

Item (a) (b) (c)

(1) ZD** 4 .01 .05 (EF=5) .1 (EF=5)

(d) (e) (f)

.15 (EF=5) .2 (EF=5) .25 (EF=5)

Levels of

Dependence Nominal CHEPs and (Lower to Upper UCBs)%

Item (a) (b) (c)

(2) LD .05 (.015 to .15) .1 (.04 to .25) .15 (.05 to .5)

(3) MD .15 (.04 to .5) .19 (.07 to .53) .23 (.1 to .55)

(4) HD .5 (.25 to 1.0) .53 (.28 to 1.0) .55 (.3 to 1.0)

(5) CD 1.0 (.5 to 1.0) 1.0 (.53 to 1.0) 1.0 (.55 to 1.0)

(d) (e) (f)

(2) LD .19 (.05 to .75) .24 (.06 to 1.0) .29 (.08 to 1.0)

(3) MD .27 (.1 to .75) .31 (.1 to 1.0) .36 (.13 to 1.0)

(4) HD .58 (.34 to 1.0) .6 (.36 to 1.0) .63 (.4 to 1.0)

(5) CD 1.0 (.58 to 1.0) 1.0 (.6 to 1.0) 1.0 (.63 to 1.0)

Values are rounded from calculations based on Appendix A. All values are based on skilled personnel (i.e., those with >6 months experience on the tasks being analyzed.

ZD = BHEP. EFs for BHEPs should be based on Table 7r-2.

Linear interpolation between stated CHEPs (and UCBs) for values of BHEPs between those listed is adequate for most PRA studies.

7-15

Figure 7-2

bla BIA

SERIES S F F F PARALLEL S S S F

TASK A'A= THE FIRST TASK

TASK 'B" = THE SECOND TASK

a = PROBABILITY OF SUCCESSFUL PERFORMANCE OF TASK A

A = PROBABILITY OF UNSUCCESSFUL PERFORMANCE OF TASK 'A'

bla = PROBABILITY OF SUCCESSFUL PERFORMANCE OF TASK "B" GIVEN a

Bla =PROBABILITY OF UNSUCCESSFUL PERFORMANCE OF TASK "B" GIVEN a

bIA = PROBABILITY OF SUCCESSFUL PERFORMANCE OF TASK "B" GIVEN A

BIA = PROBABILITY OF UNSUCCESSFUL PERFORMANCE OF TASK -B" GIVEN A

FOR THE SERIES SYSTEM:

Pr[S] = a(bla)

Pr[F] = 1 - a(bla) = a(Bla) + A(bIA) + A(BIA)

FOR THE PARALLEL SYSTEM:

Pr[S] = I - A(BIA) = a(bla) + a(Bla) + AWbIA)

Pr[F] = A(BIA)

Figure 7-2 HRA event tree for series or parallel system. (This is a repeat of Figure 5-1.)

7-16

Guidelines - Use of UCBs

for A and Bia are 3, and for BIA, it is 5 (Table 7-2, items 2 and 3). The UCBs for HEPs are calculated using the EFs as described earlier; the UCBs for HSPs are obtained by subtracting the UCBs of the HEPs from 1.0. Thus, the nominal probabilities and lower and upper UCBs for the limbs are as follows:

a = .99 (.97 to .997)

A = .01 (.003 to .03)

bla = .999 (.997 to .9997)

Bla = .001 (.0003 to .003)

bIA = .95 (.75 to .99)

BIA = .05 (.01 to .25)

Using the failure system,

equations given in Figure 7-2, and assuming a series

Pr[F] = 1 - a(bla)

If we propagate the nominal probabilities, then

Pr[F] = 1 - .99(.999) = 1 - .989 = .011

Using the upper bounds of the success probabilities, becomes

the lower bound

Pr[F] = 1 - .997(.9997) = I - .9967 = .0033

bounds of the success probabilities, the upper boundUsing the lower becomes

Pr[F] = 1 - .97(.997) = 1 - .967 = .033

Thus, for the series system, one could express the end failure probability as .011 (.0033 to .033), the range in parentheses representing the results of the bounding analysis.

Assuming a parallel system, the failure equation is

Pr[F] = A(BIA)

Using the same approach, the total failure probability can be expressed as .0005 (.00003 to .0075), the range in parentheses representing the results of the bounding analysis.

It should be apparent that this type of bounding analysis will give highly optimistic lower failure bounds and highly pessimistic upper failure

7-17

Guidelines - Use of UCBs

bounds, but these boundaries may be useful for some PRA purposes. A more complicated example of bounding analysis is given in Figure A-13 in NUREG/CR-2254 in which, unlike the above example, both HEPs and HSPs are used in all failure paths through the HRA event tree.

Propagation of Uncertainty Bounds in an HRA*

There are several general approaches that have been used in HRAs to propa- gate UCBs. Some of the propagation methods are described in Chapter 12 of NUREG/CR-2300. Others involve the propagation of single-point estimates through the HRA portion of a PRA and the assignment of UCBs about the final point estimate, i.e., the total failure term for each portion of the HRA. These methods can result in UCBs that are quite different, and it is up to the PRA team to select and justify the method it employs.

Of approaches described in NUREG/CR-2300, we will mention briefly three methods: the Monte Carlo Simulation, the Discrete Probability Distribution Simulation, and the Method of Moments. Based in part on the last method, we have prepared a UCBs propagation method for HRA, presented in Appendix A.

A commonly accepted method of propagating UCBs is the use of a Monte Carlo procedure to sample values from the distribution of each error probability in the analysis.** Generally, in applying a Monte Carlo procedure, random sampling from each distribution in the analysis is used. This procedure will not reflect the true response situation if there is a correlation be- tween the performances of several tasks by the same person. If an oper- ator's skill level is fairly constant with respect to those of other opera- tors for any of the tasks he undertakes, his error probabilities are likely to fall close to the same relative position on each of the distributions in the analysis. Therefore, if the same operator performs each of the tasks being analyzed, there is very little likelihood that his performance will correspond to a set of randomly sampled HEPs derived by the above Monte Carlo procedure. To avoid this problem, one could set up a sampling proce- dure to reflect the above or other sources of dependence. For example, in an HRA one could randomly sample from the distribution of HEPs for the

Some of the material in this section is taken from Chapter 4 of NUREG/CR-2300.

"The modern use of the word (simulation] traces its origin to the work of von Neumann and Ulam in the late 1940s when they coined the term 'Monte Carlo analysis' to apply to a mathematical technique they used to solve certain nuclear-shielding problems that were either too expensive for experimental solution or too complicated for analytical treatment. Monte Carlo analysis involved the solution of a nonprobabilistic mathematical problem by simulating a stochastic process that has moments or probabil- ity distributions satisfying the mathematical relations of the nonproba- bilistic problem" (Naylor et al, 1966). A summary of the Monte Carlo method is found in Martin (1968, pp 31-35), and a detailed treatment of this method is the subject of Hammersley and Handscomb (1964).

7-18

Guidelines - Use of UCBs

first event modeled and determine the percentile for the HEP selected. Ignoring the relatively small influence of intraindividual differences, this same percentile would then be applied to all the other distributions of HEPs for the events analyzed. Thus, if the first percentile were 25, representing a lesser skilled person, the HEPs for all the remaining events in the sample would correspond to that 25th percentile. In a large number of samples, this procedure would avoid the above bias. This procedure has the advantage of more closely representing variability within individuals and is less expensive than conventional Monte Carlo runs.

An alternative method to the Monte Carlo procedure is the Discrete Proba- bility Distribution method, in which one graphs the distribution of each HEP as a discrete histogram. In essence, this method represents each con- tinuous distribution with some finite number of points. To evaluate the uncertainty associated with combinations of human actions and other events, histographs representing the distributions of each can be combined by simulation or, in some cases, convolution to derive an uncertainty dis- tribution associated with the combined failure probabilities of interest.

NUREG/CR-2300 also discusses the Method of Moments and notes that this method "is applicable when input uncertainties are characterized by param- eter variability. If sufficient information is available to generate an estimate of the first few moments of the input variables, the method of moments provides the mechanics of combining this information to generate the corresponding moments for the output variable." Based in part on the Method of Moments, Appendix A presents a method of propagating UCBs for HEPs that has been developed for lognormal distributions of HEPs but which can be modified to fit other distributions. This UCBs propagation method has been demonstrated to provide results in close agreement to those obtained using a Monte Carlo procedure for the HEPs typically used in PRA and may be used in place of a Monte Carlo method when computer facilities are not available. Sample calculations illustrating the method are pre- sented in Appendix A, using an HRA event tree that is an expansion of the tree in Figure 5-3 and the sample HRA worked out in Chapter 5.

If the detail of the above methods is deemed unnecessary or inappropriate in view of the lack of actual data on human error distributions in the per- formance of NPP tasks, simpler approaches to handling uncertainty analysis can be used. Use of these approaches avoids the necessity of propagating UCBs through the HRA portion of a PRA. Instead, UCBs are assigned to the total failure term obtained from each HRA. The simplest approach is to assign some arbitrary set of UCBs to the total failure probability from the HRA event tree. In one PRA (Wreathall, 1982, p 105), once this total fail- ure probability was determined as a point estimate, an EF of 10 was em- ployed. This EF of 10 was larger than the typical EFs for the individual HEPs that were used in this PRA to calculate the total failure probability. For a lengthy and interactive HRA event tree, especially one that repre- sents the performance of more than one person, an EF of 10 might not be considered sufficiently conservative. However, in the example in Appendix A (based on Figure A-2), which includes personnel interaction and some HEPs

7-19

Guidelines - Use of UCBs

with EF = 10, the EF for the total failure probability was only about 5.5, and this EF is equivalent to the use of a Monte Carlo procedure.

Another application of this approach of assigning UCBs to the total failure term of an HRA event tree is to apply the largest EF of the individual HEPs to the total failure term. This application is made on the assumption that the uncertainty associated with the entire HRA is no greater than that associated with the most uncertain element of the analysis. In some cases, even this assumption might not be considered sufficiently conservative. However, if we apply this method to the example in Appendix A, we can note that the largest individual EF for an HEP was indeed 10. Therefore, at least for the example in Appendix A, this second simple method provides the same answer as the first simple method, and both are more conservative than the results of the method employed in Appendix A or by a Monte Carlo pro- cedure.

In view of the different opinions in the PRA field as to how uncertainties should be propagated in a PRA (Chapter 12 in NUREG/CR-2300), no recommenda- tion will be made here as to the best method to use in an HRA. Further- more, because most UCBs for individual HEPs are not determined from data collected in NPPs, the method employed may not be very critical in a PRA so long as the UCBs for terms entered into the system analysis are not un- realistically narrow. It is obvious that in HRA, sensitivity analysis can be very useful to ascertain the impact on the system analysis of assuming different UCBs for the error terms to be incorporated into the system eVent or fault trees.

0 7-20

Ch. 8. Expert Opinion in PRA Overview; Need for Formal Procedures

CHAPTER 8. USE OF EXPERT OPINION IN PROBABILISTIC RISK ASSESSMENT

By L. M. Weston, Statistics, Computing, and Human Factors Division, Sandia National Laboratories.

Overview

As noted in Chapter 6, the major problem for human reliability analysis (HRA) in probabilistic risk assessment (PRA) is the scarcity of actuarial data for estimating human error probabilities (HEPs) and performance times and associated uncertainty bounds (UCBs) in nuclear power plant (NPP) tasks. Data in the Handbook are primarily based upon extrapolations from similar tasks or upon informal evaluations of the expert opinions of the authors.

Formal evaluations of expert opinion, referred to as psychological scaling and probability estimation procedures, appear to be promising methods for obtaining the needed estimates of HEPs and task performance times in a cost-effective and timely manner. Although these procedures have not been frequently tested and validated in applied settings, their potential use- fulness in this application is suggested by psychological theory and empirical research.

In this chapter, the possible applications of these formal judgment proce- dures, general requirements for their use, general descriptions and overall evaluations of the procedures, and recommendations will be presented. Two reports resulting from a psychological scaling research project conducted by Decision Science Consortium (DSC) and managed by Sandia National Labora- tories (SNL) with funding from the Nuclear Regulatory Commission (NRC) form the basis for this chapter (Seaver and Stillwell, 1983; Stillwell et al, 1982).

Need for the Use of Formal Judgment Procedures

The types of data required to perform an HRA in PRA are HEPs and task per- formance times (e.g., time taken to diagnose an abnormal event correctly). Actuarial data for estimating HEPs and task performance times in NPP oper- ations are scarce, thus posing a major problem for HRA. Because of this deficiency, estimates in the Handbook are based upon other sources of in- formation. As explained in Chapter 6, estimates were obtained by extrapo- lations from partially related tasks and from informal evaluations of expert opinion.

Several NRC-funded programs, managed by SNL, were begun in an attempt to alleviate the deficient data problem. One such program is the development of a data bank program plan conducted by the General Physics Corporation (GPC). The objective is to collect human performance data on NPP tasks and to classify the data via a taxonomy suitable for PRA (Comer et al, 1983).

8-1

Potential Applications

Inputs to the data bank will include error relative frequencies (ratios of numbers of errors to number of attempts) and task performance times. Such a data bank would be invaluable to HRA specialists. However, few data are available that are suitable for input to the data bank at present.

A second program is a simulator research program also being conducted by GPC. Error relative frequencies and task performance times were collected in an NPP simulator during training and recertification sessions (Beare et al, 1983). There is no doubt that NPP simulator data are important sources for deriving human performance estimates for HRA. However, these data are both time consuming and costly to collect. Additionally, simulator data require calibration due to potential differences between simulator and real-world conditions. Swain (1967a) reports a technique in which real- world data points are used as calibration points for the simulator data. However, real-world data must be available or estimates must be obtained in some manner for such calibration. Thus, additional time and resources may be required for calibration of simulator data. Without such calibration, one cannot be sure how well simulator data represent real-world data.

The use of formal judgment procedures (psychological scaling and probabil- ity estimation procedures) appears to be a promising method for obtaining currently needed estimates of HEPs and task performance times in a cost- effective and timely manner. Formal judgment procedures may be defined as systematic methods for using expert opinion to assign numbers to objects, events, or their attributes so that the numbers represent relationships among the scaled items. Estimates of HEPs derived from judgment procedures could be used on an. interim basis until actuarial or calibrated simulation data become available.

A third NRC-funded program, managed by SNL, was a psychological scaling

research program that was conducted by DSC. One report resulting from this program was a literature review of psychological scaling and probability assessment studies (Stillwell et al, 1982). A major conclusion from this review was that relatively consistent and valid probability estimates can be obtained from judgment procedures if the procedures are not used in an unplanned and unsystematic mannner. The most promising techniques in terms of among-judge consistency and validity of estimates were identified in the literature review. In the second report, step-by-step instructions for the use of recommended procedures, strengths and weaknesses of each procedure, and requirements for the use of the procedures were explained (Seaver and Stillwell, 1983).

Potential Applications of Judgment Procedures

The types of data that can be obtained with judgment procedures are as follows:

(1) Point estimates of HEPs and performance times and associated UCBs for single actions in NPP operations (e.g., positioning a valve)

8-2

General Requirements

(2) Point estimates of HEPs and performance times and associated UCBs for series of actions in NPP operations (e.g., setting up calibration equipment)

Although the research conducted by DSC addressed only the application of judgment procedures to HEP estimation, it appears that the same procedures can be used to obtain estimates of task performance times. However, some additional research into the problems of this application should be done. For example, Haas and Bott (1982) report that NPP operators underestimated the time it would take to initiate action for an abnormal event by a factor of 2 or 3. The causes of these underestimates and possible solutions should be identified.

Estimates of HEPs and UCBs could be obtained for a plant-specific HRA by using information specific to a particular NPP when defining the events to be judged. Estimates of HEPs and UCBs could be obtained for a non-plant- specific HRA (e.g., all boiling water reactor NPPs) by using information characteristic of a generic set of NPPs when defining the events to be judged.

The objective of the PRA will, of course, determine the level of detail to be used in definition of the events. It should be noted, however, that generally defined events will be more difficult to judge than specifically defined events, and the among-judge variability may be greater with gene- rally defined events than with specifically defined events.

When possible, actual data from NPP tasks or data from simulated NPP tasks should be used for HRA. Judgment data should be employed on an interim basis until actual data or data from simulations are available.

General Requirements for the Use of Judgment Procedures

One critical requirement for the use of judgment procedures is complete definition of the events to be judged. Incomplete definition of the events could lead to discrepancies among experts in their interpretations of the events and lowered among-expert consistency. A major step in defining the events is the identification of important performance shaping factors (PSFs). PSFs consist of situational characteristics, job and task instruc- tions, task and equipment characteristics, psychological stressors, physio- logical stressors, and organismic or individual characteristics (Chapter 3). Seaver and Stillwell (1983) also suggest definition of the subset to which each event belongs and identification of causes of the events. The level of specificity used in event definition will depend upon the objec- tive of the PRA. For example, more general event definitions will be required for a non-plant-specific HRA than for a plant-specific HRA. The events should be defined as completely as possible, given PRA constraints.

A second critical requirement is the use of the appropriate types of judges. The literature review of judgment procedures revealed that indi- viduals should not be asked to make judgments that are not within their

General Requirements

area of expertise. Use of nonexpert populations results in less accurate estimates. Therefore, it is imperative that all judges have a knowledge of NPP operations or other industries whose operations are similar to those in NPPs. Seaver and Stillwell (1983) suggest the following types of experts:

(1) Human factors specialists with knowledge of NPP operations

(2) NPP operators

(3) NPP supervisory personnel

(4) Nuclear and system engineers

(5) Human factors specialists, operating personnel, supervisory personnel, and system engineering personnel with knowledge of industrial opera- tions similar to NPP operations (e.g., chemical processing)

It is also advisable to use the widest range of expertise available. Individuals who all have knowledge of the subject matter and also have training in different disciplines (e.g., human factors versus engineering) should be considered. The review of the judgment literature suggested that the validity of estimates will increase as the judgment group becomes more diverse. However, the number of judges with critical expertise (e.g., human factors specialists with knowledge of NPP operations) should not be reduced in order to include judges with lesser expertise (e.g., engineers with knowledge of chemical processing plant operations-) in the group.

A third requirement is that upper and lower error bounds be obtained along

with the point estimates for events. Error bounds are computed in a manner similar to the calculation of confidence limits and indicate the extent to which the judges agree or disagree with the point estimates. They are not identical to confidence limits or UCBs. Specification of error bounds is important for determining the usefulness of the estimates. One would expect wider error bounds for more generally defined events than for more specifically defined events.

It is also important to obtain estimates of UCBs from the judges when possible. These are estimates of the upper and lower values that an HEP can take due to variations in PSFs And uncertainty in the individual's point estimates.

A final requirement is determination of the among-expert consistency of judgments. If the judgments are too inconsistent, as determined by statis- tical procedures, then the validity of the point estimates may be at question.. Related to the among-expert consistency is the problem of sta- tistical outliers. It is important to examine individual data to determine if there are any experts (or subsets of experts) whose estimates are highly discrepant from most other estimates. The presence of such statistical outliers could affect the validity and usefulness of the point estimates. However, outlying judgments should not be routinely discarded. Potential explanations for the discrepancies should be sought by interviewing the experts before such action is taken.

8-4

Recommended Procedures

General Descriptions of Recommended Judgment Procedures

There are four judgment procedures that have been recommended to obtain currently needed human performance data. These procedures are paired comparisons, ranking/rating, direct numerical estimation, and indirect numerical estimation. In this section, the types of judgments required, the number of experts needed, and the strengths and weaknesses of each of these procedures will be outlined. Details concerning implementation of the procedures (calculation of point estimates, among-expert consistency, error bounds and UCBs) may be found in Seaver and Stillwell (1983).

Paired Comparisons Procedure

In the paired comparisons procedure, events are first arranged in pairs, and then the expert is presented with one pair at a time. To obtain esti- mates of HEPs, the expert is asked to judge which event i-n the pair is more likely or which event in the pair has more of the dimension in question, e.g., error-likeliness. To obtain estimates of UCBs, the expert is asked to judge which event or error in the pair has greater uncertainty (or wider bounds). The expert is not allowed to make the judgment that the events are tied in terms of the amount of the dimension being scaled.

Reasonably consistent judgments can be obtained with 10 experts or fewer. The specific number of experts required is dependent upon the number of events being judged and the acceptable width of the error bounds. Guide- lines for determining the needed number of experts may be found in Seaver and Stillwell (1983).

There are several advantages of using the paired comparisons procedure. Of most importance, research suggests that this procedure yields the highest quality (most accurate) estimates. With paired comparisons, only qualita- tive judgments are required rather than the numerical judgments needed with other procedures (direct and indirect numerical estimation). The litera- ture review suggested that individuals are much more capable of making qualitative than quantitative judgments, especially when judging low- probability events, as will often be the case with HEPs. Furthermore, paired comparisons require the simplest qualitative judgments, since only two events are compared simultaneously. Other procedures requiring quali- tative judgments, ranking/rating, call for simultaneous judgments of more than two events or judgment of events in view of multiple categories. A second advantage of paired comparisons is that the judgments are less subject to various types of bias in human judgment than are numerical estimation procedures (see Stillwell et al, 1982, for a discussion of these biases). Third, paired comparisons require less specifically defined events than the numerical estimation procedures, since qualitative rather than quantitative estimates are needed.

In view of the advantages noted above, the paired comparisons procedure is highly recommended for obtaining estimates of human performance data. However, this procedure presents the most problems in application. A major problem is that a large number of judgments per expert are needed if all

8-5

Recommended Procedures

possible comparisons are made. The number of pairs of k events is equal to k(k - 1)/2. For example, with 10 events there are 45 possible pairs, with 20 events there are 190 possible pairs, and with 30 events there are 435 possible pairs. There are methods, discussed in Seaver and Stillwell (1983), for reducing the number of comparisons; however, a reduction in the number of comparisons results in an increase in the number of experts needed. Thus, if there are a large number of judgment events, use of paired comparisons will demand either a large amount of time per expert or a substantial number of experts. A second problem posed by the paired comparisons procedure is that the judgment process leads to an interval scale rather than an absolute scale. An interval scale is one in which distances between events in scale units are known, but there is no true

zero point; thus absolute values are unknown (D'Amato, 1970). For example, to obtain HEP estimates, the interval scale must be calibrated to obtain a probability scale. For scale calibration, the probabilities for at least two events in the judgment set must be known, or probability estimates for two events must be obtained. The direct estimation procedure (to be dis- cussed) can be used to obtain two HEP estimates when actuarial data are not available. The two events with known or estimated probabilities should be located at the extremes of the probability continuum for the set of judg- ment events. To improve accuracy, more than two events should be used for scale calibration if possible. In addition, one must assume some fixed relationship between the interval scale and probabilities before the inter- val scale values can be transformed into a probability scale. For example, in the case where two known or estimated probabilities at the extremes of the probability continuum are available, one must still specify the rela- tionship between the scale values and the probabilities to derive real- world probabilities from the remaining scale values. The appropriate transformation could be a logarithmic, power, or some other function. Although there are few empirical data available for making this decision, strongest support has been found for use of a logarithmic relationship (Seaver and Stillwell, 1983). Third, paired comparisons require more experts than the numerical estimation procedures. Fourth, problems are presented when there is complete agreement among judges that one event is more likely or has more of the dimension in question than another; some among-expert variability is needed for the use of paired comparisons. Several authors have suggested ignoring cases of total agreement in deter- mining scale values, but this results in a loss of important information. However, Seaver and Stillwell (1983) describe solutions to this problem that do not result in any loss of information. Therefore, this problem is not considered to be as serious as others discussed.

Ranking/Rating Procedure

Ranking and rating procedures will be discussed together, since data from these procedures are analyzed in the same way. To obtain HEP estimates from the ranking procedure, the, expert must rank-order the events with respect to the dimension being scaled, e.g., error-likeliness. To estimate UCBs using this procedure, the expert must rank-order the events with

respect to uncertainty (width of bounds) in the point estimates.

8-6

Recommended Procedures

In the rating procedure, HEP estimates are obtained by asking the expert to rate each event on a scale (e.g., a 10-point scale with endpoints of the scale being extremely unlikely and extremely likely). To estimate UCBs using this procedure, the expert must rate the events on a scale of uncer- tainty (width of bounds) in the point estimates.

As with the paired comparisons procedure, the specific number of experts required is dependent upon the number of events being judged and the acceptable width of the error bounds (Seaver and Stillwell, 1983). Generally, reasonably consistent judgments can be obtained with 10 or fewer experts.

Many of the advantages of the paired comparisons procedure also apply to

ranking/rating. Of most importance, research reported in the literature review indicates that ranking/rating procedures yield higher quality esti- mates than do the numerical estimation procedures, since qualitative rather than quantitative decisions are required, and this is especially important when estimating low-probability events. As noted in the section on paired comparisons, ranking/rating procedures are thought to result in lower quality estimates than paired comparisons, since ranking/rating techniques call for more complicated qualitative judgments than paired comparisons. In common-with paired comparisons, ranking/ratings procedures are less subject to biases in human judgment than the numerical estimation proce- dures, and event definition can be less specific with ranking/rating than with the numerical estimation procedures. One practical advantage of ranking/rating over paired comparisons is that substantially less time is needed to obtain the judgments, since fewer judgments are required.

Three of the disadvantages of paired comparisons are also relevant to ranking/rating procedures:

(1) Ranking/rating judgments lead to interval scales, which must be converted to absolute scales in the same manner as with paired comparisons.

(2) Problems arise when there is complete agreement among the judges that one event is more likely or has more of the dimension in question than another. Solutions to this problem are the same as for paired comparisons.

(3) Ranking/rating procedures require more experts than do the numerical estimation procedures.

Direct Numerical Estimation Procedure

To estimate HEPs using the direct numerical estimation procedure, the expert must provide a numerical estimate of event likelihood, e.g., the probability of an error in performing a task. Seaver and Stillwell (1983) recommend using a logarithmically spaced scale of odds and asking the expert to mark the scale at the point that represents the odds of the event occurring. For example, the judge might be asked what the odds are that an operator would fail to respond to an annunciated legend light (one of one).

8-7

Recommended Procedures

The judge would indicate his decision by marking the appropriate point on a logarithmically spaced scale. If the judge marked 1:100,' this would mean that the operator would be 100 times more likely to respond than not to respond to an annunicated legend light. To obtain estimates of UCBs using this procedure, the expert must indicate upper and lower bounds about the estimate on the scale, such that the expert is 95% certain that the true HEP lies within these bounds.

In the direct numerical estimation procedure, the judgments from the experts can be combined statistically with no personal interaction among experts, or personal interaction can be allowed. A highly popular inter- action method is the Delphi technique (Dalkey and Helmer, 1963). In the Delphi technique, the experts remain anonymous. Each expert makes esti- mates privately and then receives feedback about the overall estimates of the expert group. This process is repeated several times. If a consensus is not reached after several iterations, the estimates are statistically combined. This technique is not recommended, since the literature indi- cates that Delphi produces estimates that are equally valid or less valid than estimates from other interaction methods and statistical combination techniques with no interaction. If interaction is to be allowed, Seaver and Stillwell (1983) recommend use of a structured form of interaction called a modified Nominal Group Technique (NGT). In this technique, each expert makes private estimates in the presence of all experts. Each expert then presents his estimates to the group without discussion. Under the direction of a group leader, all estimates are then discussed. A major role of the group leader is to prevent domination of the group by members with strong personalities. Following this discussion, each expert recon- siders his estimates. Subsequently, the estimates are statistically com- bined. The modified NGT tries to control the influence of strong personal- ities who attempt to dominate group members. However, it should be noted that since discussion is permitted, the NGT may reduce but not eliminate the dominant personality problem. Research on direct numerical estimation indicates that about six experts are needed for the use of this procedure (see Seaver and Stillwell, 1983).

One major advantage of the direct numerical estimation procedure is that

the judgments result in an absolute scale. Therefore, one avoids the difficult problem of scale transformation present with the other proce- dures. Second, direct numerical estimation requires substantially less time than the paired comparisons procedure since fewer judgments are required. Third, direct numerical estimation calls for fewer experts than the paired comparisons and ranking/rating procedures.

A major disadvantage of direct numerical estimation is that this procedure leads to poorer quality estimates than all other recommended procedures. Individuals are less able to make quantitative than qualitative judgments, particularly when dealing with low-probability events. Furthermore, direct numerical estimation requires exact numerical judgments, which are more difficult than the relative numerical judgments needed for the indirect

8-8

Recommended Procedures

numerical estimation procedure, which will be discussed in the next sec- tion. Other problems presented by direct numerical estimation when com- pared with the paired comparisons and ranking/rating procedures are that the experts are more likely to be affected by biases in human judgment, and this procedure requires more specifically defined events due to the exact- ness of the numerical judgments.

Indirect Numerical Estimation Procedure

In the indirect numerical estimation procedure, the events must first be arranged in pairs, and the expert is then presented with one pair at a. time. To obtain HEP estimates, the expert is asked to make ratio judgments as to how much more likely one event is than another. For example, the judge might be asked to state which of the following two errors is more likely and how much more likely is one error than another:

(1) Select wrong control from a group of identical controls marked with labels only.

(2) Select wrong control from a group of identical controls that are functionally grouped.

The judge might decide that the first error is three times more likely than the second error and would mark the ratio scale at the 3:1 point. To estimate UCBs using this procedure, the expert must make ratio judgments as to how much more uncertain he is about one event than another. For n events, n - 1 judgments are required. For example, with three events (a, b, and c), a is paired with b, and b is paired with c.

As with direct numerical estimation, the judgments from the experts can either be statistically combined with no personal interaction, or personal interaction can be permitted. If interaction is permitted, use of the NGT is recommended. About six experts are needed for the use of the indirect numerical estimation procedure.

One advantage of this procedure is that it requires relative quantitative judgments rather than the precise numerical judgment needed for direct numerical estimation. Since relative numerical judgments are easier to make than precise numerical judgments, one can expect higher quality esti- mates from indirect than direct numerical estimation. Other advantages of this procedure are that it requires substantially less time than paired comparisons, since fewer judgments are needed, and fewer experts than for the paired comparisons and ranking/rating procedures.

A major disadvantage of the indirect numerical estimation procedure is that this procedure leads to poorer quality judgments than the paired compari- sons and ranking/rating procedures. This is due to the requirement of quantitative rather than qualitative judgments, which is especially impor- tant when estimating low-probability events. Second, in common with the paired comparisons and ranking/rating procedures, the scale must be trans- formed. For example, to convert the ratios into HEPs, at least one event

8-9

Evaluation of Recommended Procedures

with known probability must be available, or the probability must be obtained through direct numerical estimation (Seaver and Stillwell, 1983). Third, in common with direct numerical estimation, this procedure requires more specifically defined events than the paired comparisons and ranking/ rating procedures due to the requirement for numerical rather than qualita- tive judgments and is more subject to biases in human judgment than the paired comparisons and ranking/rating procedures.

Overall Evaluations of Judgment Procedures

In the Seaver and Stillwell (1983) report, the judgment procedures were rank-ordered from best to worst with respect to six evaluation criteria. Rank-orderings of the procedures on the three most important criteria, along with reasons for rank-ordering, will be presented in this section.

The three criteria that will be discussed are quality of judgment, diffi- culty of data collection, and empirical support. Quality of judgment is defined as the accuracy of the estimates resulting from the judgment pro- cedure. Difficulty of data collection refers to practical concerns in application of the procedures and includes factors relevant to the time and effort required of the experts and data coll'ectors in obtaining the esti- mates. Empirical support refers to the presence of empirical data on the usefulness of the procedure in obtaining probability estimates. The rank- orderings of the four judgment procedures on the three evaluation criteria are shown in Table 8-1.

Quality of Judgments

As shown in Table 8-1, the procedures that require qualitative judgments (paired comparisons and ranking/rating) were ranked as better than the two numerical procedures on this criterion. Research has shown that individ- uals are much more capable of making qualitative than quantitative judg- ments, especially when dealing with low-probability events (e.g., events with error probabilities of 10 4), as will frequently occur with HEPs. Among the procedures where qualitative judgments are needed, paired com- parisons was ranked as better, since simpler qualitative judgments are re- quired for paired comparisons than for ranking/rating. Among the numerical estimation procedures, the indirect is rated as better than the direct pro- cedure. The literature review indicated that individuals are more capable of making relative than absolute numerical judgments.

Difficulty of Data Collection

The paired comparisons procedure was ranked poorest on this criterion due to the relatively large number of judgments needed for its general use. The other procedures require relatively few judgments per expert. The numerical estimation procedures were ranked poorer than ranking/rating; this is because the numerical estimation procedures are more subject to biases in human judgment than ranking/rating. Thus, training and/or instructions with reference to these biases are recommended when using the numerical estimation procedures.

8-10

Table 8-1

Table 8-1 Rank orderings of four judgment procedures on three evaluation criteria*

Type of Judgment Procedure

Direct Indirect Paired Ranking/ Numerical Numerical

Evaluation Criterion Comparisons Rating Estimation Estimation

Quality of judgments 1 2 4 3

Difficulty of data collection 4 1 2 3

Empirical support 3 4 1 1

Source: Seaver and Stillwell (1983)

A rank-ordering of 1 indicates best, and a rank-ordering of 4 indicates worst.

NOTE: Numbers in this table represent rank orderings, not ratings. For example, a rank ordering of I indicates that the procedure is better on the criterion than any of the other procedures considered. A rank ordering of 1 does not mean that the procedure is flawless with respect to the criterion.

8-11

Summary Recommendations

Empirical Support

The numerical estimation procedures have been heavily tested for the pur- pose of obtaining probability estimates. Therefore, these procedures were rated as better than paired comparisons and ranking/rating on this crite- rion. However, the numerical estimation procedures have not been fre- quently employed for the purpose of estimating probabilities for low- probability events. Thus, their use in this application must be considered carefully. Empirical data on the use of paired comparisons and ranking/ rating for estimating probabilities are few, with this situation being worse for ranking/rating than for paired comparisons. A major difficulty with both of these procedures involves the selection of a relationship for transformation of scale values when correlating scale values with real- world values. Several authors recommend a logarithmic relationship, but this recommendation is based upon few data.

Recommendations in View of Practical Considerations and Type of Events To Be Judged

Many practical factors must be considered when selecting the most appropri- ate judgment procedure for a particular application. Several of these factors are related to the characteristics of the population of judges. The number of experts available and the time available per expert are important factors, especially when the set of events to be judged is large. It may not be possible to use qualitative procedures (paired comparisons and ranking/rating) when there are restrictions on the number of experts or time per expert, since the qualitative procedures require more experts and/or time per expert than the numerical procedures. Second, the back- ground of the experts will play a role. If experts unfamiliar with proba- bility concepts are used (e.g., control room personnel), training in proba- bility will be needed if the numerical estimation procedures are employed. For this reason, the qualitative procedures may be preferred with this type of expert.

Availability of events with known or independently estimated HEPs and UCBs is another important factor. Two estimates of HEPs and UCBs are needed for paired comparisons and ranking/rating and one estimate for indirect numeri- cal estimation. If satisfactory data are not available, then the direct numerical estimation procedure must be used.

Seaver and Stillwell (1983) note that procedure selection will be affected by the nature of the judgment events also. If the events are too dissimi- lar, then paired comparisons judgments will be very difficult. For use of the paired comparisons procedure in this situation, grouping of events by similarity is recommended. Second, there is a problem with the use of paired comparisons when the events being judged are the same except for one factor, such as stress level. Paired comparisons are not recommended in this instance, since violations of assumptions of the psychological model may occur that could result in misleading data. Third, when the events to be judged are low-probability events, biases in judgment could occur with numerical procedures.

8-12

Summary Recommendations

In conclusion, if at all possible, the paired comparisons or ranking/rating procedures should be utilized to provide the highest quality estimates. These procedures, however, are often the most demanding ones in terms of the above considerations. Lack of resources will make use of the numerical estimation procedures necessary. Even though some loss in accuracy will result from use of direct and indirect numerical estimation, these proce- dures are still among those recommended for present use.

8-13

Ch. 9. Unavailability Overview; Equations

CHAPTER 9. UNAVAILABILITY

Overview

In probabilistic risk assessment (PRA), it is often necessary to incor- porate human error probabilities (HEPs) in the estimation of component unavailabilities. This occurs when an analysis deals with maintenance, test, calibration, or other errors that can render a component unavailable for some interval of time. In such cases, the component's unavailability is a function of the probability of some human failure event, the proba- bility of nonrecovery, and the average time that the component is in a failed condition before being restored.

This chapter presents some mathematical expressions that are used with the human performance models and HEPs in the Handbook to estimate the unavail- ability of nuclear power plant (NPP) systems and components due to human error. A more general treatment of unavailability in NPPs may be found in Chapter XI of the Fault Tree Handbook (Vesely et al, 1981).

Unavailability Equations*

A few symbols must be defined before presenting the unavailability equations:

A - Availability, the probability that a component or system is operating or will operate satisfactorily if called on (Welker, 1966, p 1-7)

U - Unavailability, the probability that a component or system is inopera- tive or will not operate if called on (U = 1 - A)

p - The probability of an unrecovered human error that results in a com- ponent being in the failed condition

- Mean downtime, the average time the component or system is unable to operate within a given time period, given that a human error has induced a failed condition

S- The time period of interest when estimating unavailability

u - Mean uptime, the average time that a component or system is operating or able to operate within a given time period; u = T - d

Using the above symbols, a common index of unavailability due to human error is

In this chapter, the symbols and notation used are different from those used in the rest of this Handbook.

9-1

Unavailability Equations

an =2 (9-1)

A =1 - U = u + d (1 -p) (9-2)

This expression for unavailability is appropriate only when the probability of being in a failed state is independent of time, the steady-state condi- tion, as in the examples given in this chapter. This assumption of time- independence may not apply in all situations. For example, if unavailabil- ity results frequently from the same source and its consequences are severe enough to call for corrective action, the learning curve associated with taking such action repeatedly will cause the probability of error to be a function of time. In this case, availability should increase because plant personnel will be more aware of the human actions that led to the severe consequences, and they will take more care-in performing the related tasks. In most cases, the improvement in operator performance will be immediate. For example, assume that within a 4-month period, a plant discovers that there were two occasions on which plant personnel forgot to place a safety- related pump back in the automatic-start mode subsequent to a test. The pump would have been unavailable had a safety injection signal occurred. These constitute reportable events under the License Event Reporting Sys- tern. One can surmise that, after the first occasion, plant personnel were told to be more careful (a frequent management response to human errors), and it is reasonable to expect that there was indeed some decrease in the probability of the errors related to the reportable event. Nevertheless, the oversight occurred a second time. Now it can be expected that some procedural changes would occur to ensure a much lower probability of this error. Over time, then, the unavailability of the pump due to this par- ticular human error changed. The size of the reduction in the estimated error probabilities for both such cases has to be estimated. There are no data that bear directly on such cases. Nevertheless, the example illus- trates the need to ensure that the unavailability model presented here be applied only to the maximum time period for which the analyst can reason- ably assume unchanging conditions. When time-dependent unavailability is of concern, more elaborate treatment is required, as described in Vesely and Goldberg (1977). In this chapter, the time-dependent situation will be disregarded since the "steady-state" assumption is assumed for the time periods of interest.

To evaluate the human error contribution to unavailability, we use the same equations that are used for component and system failure occurrences (human errors simply being a particular cause of failure).

To illustrate the application of Equation 9-1 , consider a human action that is performed periodically with an average time, T, between actions. Assume that for any action there is a probability, p, that a human error will occur and not be recovered at or about the time the .error is made. In this

9-2

Unavailability Equations

situation, where p is the probability of the event that generates the down- time, d is the mean downtime, given the error occurrence, and T is the sum of the average uptime and the average downtime (i.e., u + d). To calculate U, we need p, d, and T. In this chapter, we are concerned with human errors as the conditions affecting U, so the probability p is obtainable from this Handbook or other sources of HEPs. d and T are determined by system variables. For convenience, p is sometimes divided into two factors: an error probability, E, and a nonrecovery probability, R:

p = ER (9-3)

where E is the probability of the error and R is the probability of failing to recover from the error at or about the time it is committed. For these unavailability calculations, a delay of an hour or so for the potential recovery operation is ignored. This slight delay will not result, in a significantly altered estimate of unavailability.

Let us assume a system consisting of components that will be tested period- ically--that is, a demand will be placed on the components to operate at specified time intervals. Checks of the operability of the components will also be performed at intervals, although these may not be equal to the test intervals. In Equation 9-1, T is normally taken to be the time between tests (or demands) on a component, and it is usually assumed that any error made on the first test will be. discovered during the second test. Thus, if a valve is left in the wrong position after a monthly test, it is assumed this error will always be discovered on the next monthly test, and T = 720 (the number of hours in an average month, 24 x 30). In this case, T = d. Whenever T = d, the unavailability will equal the unrecovered human error probability, p.

If it is possible to detect the error and restore the component between the tests (if checks are made), the average downtime will be smaller than T since the component may be placed in an available state following detection of the error and before the next demand. In Equation 9-1,_d equals T if there is no checking between tests. If checking is done,, d , the total d, is the sum of the d values for each time period. For example, if there is a midmonth check between two monthly tests, d = d, + d 2 , where d, is the average downtime period between the first monthly test and (but not includ- ing) the midmonth check, and d2 is the downtime period between the midmonth check and (but not including) the second monthly test.

The general equation for calculating d when checks are made between tests is t

dt = h1 + C1h2 + C1C2h3 + 0°- + C1C2 "0* Cm-1 h (9-4)

where m is the number of checking intervals between the two tests; hl, h2, h 3 , and h are, respectively, the number of hours (or any other time unit) between t~e first test and the first check, the first and the second checks, the second and third checks, and the last check and the next test; and C1 , C2 , and Cm_ are, respectively, the probabilities of nondetection

9-3

Unavailability Equations

of the error at the first, second, and last checks performed between the two tests.

Figure 9-1 shows a time line of a segment of equipment life with tests and checks indicated. The equipment is tested at 5-month intervals and checked monthly between tests. Assume that the checks are identical and indepen- dent and that the probability of failing to detect an unavailable equipment state during a check is .01 for each check. dt can now be calculated as:

dt = 720 + (.01 x 720) + (.01 x .01 x 720) + (.01 x .01 x .01 x 720) + (.01 x .01 x .01 x .01 x 720) = 727.27

Assume that the probability of leaving the equipment unavailable after the first test is p = .05. This would also equal the unavailability for the entire 5-month period if there were no checks (if d = T). Unavailability for the case shown in Figure 9-1 is

pd .05 x 727.27 = .01 i 3600

The intervening checks have decreased unavailability by a factor of 5.

Recall that Equation 9-1 applies to the customary steady-state conditions and should not be used if there is time-dependence. Unavailabilities resulting from human actions are used in system safety or reliability analyses in the same way that hardware unavailabilities are used. If the human-caused unavailabilities are combined with hardware unavailabilities, one should check whether the hardware unavailabilities already include human error contributions, to avoid counting them twice. For situations not covered by the equations given here, other unavailability expressions must be developed using the usual reliability theory approaches, as de- scribed in Barlow and Proschan (1965).

In treating unavailabilities resulting from human actions, it is generally assumed that the detection of an error does not influence the probability of committing or detecting future errors; that is, the probabilities of failure on the various checks are assumed to be independent. Although it is true that when an error is found, operators may take extra precautions, it is difficult to estimate this influence and how long the influence will last. Therefore, we usually disregard this possibility for the sake of conservatism. Of course, if some detected unavailability results in a restructuring of the job with resultant increases in human reliability, new estimates of unavailability should be made based on the estimated decreases in HEPs.

It is also generally assumed in unavailability calculations that the non- detection of some faults does not influence the probability of detection of that same fault should more opportunities for detection occur. When this assumption is not appropriate, adjustments must be made in calculating these conditional HEPs. For example, in the basic walk-around inspection model

9-4

Figure 9-1

T= 3600

hI = 720 h 2 = 720 h 3 =720 h 4 = 720 h5 =720

I I , I I TEST CHECK CHECK CHECK CHECK TEST

#1 #2 #3 #4

Figure 9-1 Time line of equipment life to illustrate unavailability.

9-5

Applications, of Equations

described in Chapter 19, the estimated probability of nondetection of some fault increases with every failure to detect that fault.

Applications of the Unavailability Equations

To illustrate the use of the above equations, two examples are given.

Example No.1 - Unavailability of a Diesel Generator

Assume that we wish to estimate the human-error-caused unavailability of a backup diesel generator. Each Monday, a technician takes the diesel off line and tests it. Thus, the time between tests is T = 168 hours. If the technician makes the error of failing to place the diesel back on line, the diesel will be unavailable until the next time it is tested or until the error is detected'during some intermediate inspection.

For this example, assume that the probability that he will forget to place the diesel back on line is estimated as E = .01. Suppose that immediately after the technician performs his task, a checker following a written procedure checks that the diesel is back on line. The checker could fail to notice that the diesel is off line. Assume that the probability of nonrecovery of the error is R = .1. Thus, p = ER = .01 x .1 = .001. This is the probability of an unrecovered error at the time of the test (the short time between the actions of the technician and those of the checker is regarded as having a negligible effect). If no further checking is done, and if there are no other recovery factors, the diesel will be un- available for 1 week, i.e., until the next test. Thus, d = 168 hours. The unavailability is calculated from Equation 9-1 as:

.01 x .1 x 168 U = = .001168

Note that, in this particular example, T and d are the same because no checking occurs between the tests, and U equals p. If checking does occur between the tests, d is calculated using Equation 9-4. If a check occurs at the beginning of the fifth day (i.e., 96 hours after the test), h, will equal 96 hours. Since there is only one check between the tests, h2 = 72 and T = 168 hours. Assume that a check is scheduled at the beginning of h2 to ascertain if the diesel is available. If the probability of the check- ing error C1 is .05,

d = 96 + (.05 x 72) = 99.6 hourst

and using Equation 9-2,

99.6 U = .01 x .1 x- 99.6 .0006168

Example No. 2 - Unavailability of a Primary Source of Emergency Coolant

Assume that a test of the primary water supply in the Emergency Core Cool- ing System at a plant is performed quarterly, and that the only human error

9-6

Applications of Equations

of consequence for the unavailability of this coolant is that of leaving the main isolation valve, in the wrong position after the test. For this error, assume E = .01 and R = .1, as in the first example. Assume that there are two scheduled monthly checks between tests, with C1 = C2 = .05. In this case, hI = h 2 = h 3 = 720 hours and T = 2160 hours. Using Equation 9-4,

d= 720 + (.05 x 720) + (.05 x .05 x 720) = 757.8 hours t

and using Equation 9-2,

757.8 U = .01 x .1' x 2160 .0004

The importance of the two checks between the tests can be appreciated by

calculating U without these tests. In this case, an unrecovered error (ER) at the time of the first test will remain undiscovered until the next test. The probability that the main isolation valve will be unavailable is simply the probability of the unrecovered error, and U = p = ER = .01 x .1 = .001. Thus, the two checks between the tests reduce the unavailability of the primary source of emergency coolant by a factor of 2.5 (i.e., .001/.0004).

9-7

Part III. Models and HEPs

PART III. HUMAN PERFORMANCE MODELS AND ESTIMATED HUMAN ERROR PROBABILITIES

Chapters 10 through 19 present human performance models and estimated human error probabilities (HEPs) with their estimates of uncertainty [error factors (EFs) or uncertainty bounds (UCBs)] to be used in performing human reliability analyses (HRAs) for probabilistic risk assessments (PRAs). Our goal in modeling human performance for PRA is to develop descriptive models to predict (within wide limits) how well people will perform what they are supposed to do in normal and abnormal situations in nuclear power plant (NPP) operations. It is NOT our goal to explain human behavior in general. Our limited goals and modeling are more fully explained in the first part of Chapter 3.

The human performance models in Part III represent our hypotheses about the performance of NPP personnel under a variety of tasks and conditions. For plant-specific HRAs, the analyst will have to adjust these models and associated HEPs according to the performance shaping factors (PSFs) he has identified in his task analyses. Because of the lack of objective data on human performance under controlled conditions in NPPs or NPP simulators, our models and estimated hEPs are based in part on the following sources:

- Our experience since 1961 in HRA and since 1972 in NPP operations, including considerable interviewing and observation of NPP personnel in U.S. and foreign plants

- Our background in experimental psychology as applied to many complex systems since the 1940s

- The shared experiences of the authors' colleagues and other workers in the field of human performance in complex systems

- An extensive literature search in quantitative data, models, and theory related to solving practical problems in human performance

- Reported incidents in NPP operations such as those in the Nuclear Regulatory Commission's (NRC's) Licensee Event Report system

Whenever possible, the estimated HEPs are based on relevant data from laboratory studies or more applied settings. When no documented data were available, judgments were used to estimate HEPs. Chapter 6 describes the supporting data for the estimated HEPs. Each HEP is listed with its estimated EFs or UCBs; the rationale underlying the UCBs is discussed in Chapter 7. The UCBs are intended to correspond to the 5th and 95th per- centiles of the conjectured distribution of an HEP.

Information from the above sources was evaluated in the context of accepted theories of human learning and performance (e.g., Stevens, 1951; McGeoch, 1942; McGeoch and Irion, 1952; Berelson and Steiner, 1964; Chapanis et al, 1949; McCormick and Sanders, 1982, and earlier editions; and Woodworth,

III-1

Models and HEPs

1938). We have been eclectic in the formulation of our human performance models. While we have obviously been influenced by the learning theories of Professors McGeoch, Hull, Spence, Osgood., Tolman, Lewin, and others, we found that rigid adherence to any one theoretical position was of very little value in formulating models to estimate human performance in real systems. In fact, we find that there is much agreement between the various behavioral theorists and that much of the disagreement seems to be due to the use of different terminology to describe the same behavior variables.

As we have said before, ". . . there is still a very large gap between the needs for a central data bank of human performance data and what is avail- able" (Swain, 1969b). Until such a bank can be developed, HRA will depend to a large extent on the analyst's judgment. We make no apology for the fact that much of the quantification and models in the Handbook are based on judgment and expertise; such is the state of the technology today. The establishment of a central data bank is currently in progress under the sponsorship of the NRC but may not be available for several years. In the meantime, the Handbook is a start.

In developing the estimated HEPs and the models, we tried to consider both typical and atypical modes of operations. We have the most confidence in applications of the models and HEPs to typical rule-based behavior in which the analyst estimates HEPs for routine operations, e.g., following written procedures for maintenance, calibration, tests, and subsequent restoration of components to normal status. We have less confidence in applications of the models and HEPs to skill-based behavior, e.g., performance of skill-of- the-craft maintenance tasks, and to continuous tasks such as reactivity control or the operation of a crane. For purposes of PRA, assessment of skill-based behavior is not generally important, as explained in Chapter 1. We have the least confidence in applications of the models and HEPs to knowledge-based behavior, e.g., interpretation and decision-making (the cognitive aspects of tasks), and to responses to unusual events in which high levels of stress constitute an important PSF. Applications in these latter areas are speculative.

At this early state in HRA technology it is not possible to model all the effects of PSFs either in general or in their applications to a specific NPP. This limitation must be considered in applying the models and numerical results to any NPP or group of NPPs. Assessment errors of the analyst will be reduced to the extent that he correctly evaluates the influence of PSFs. The best approach to such evaluation is the task analy- sis described in Chapter 4.

Our approach in model development was to use the above-mentioned sources in formulating psychologically sound human performance statements based on the operational environment, the tasks of the operators, and the relevant PSFs. In consultation with statisticians knowledgeable about NPP opera- tions,-we developed simple, standard mathematical expressions that reason- ably describe human performance. Advantages of using standard statistical distributions for these descriptions include the ready availability of tabled values and the added calculational convenience. For example, it was

iII-2

Models and HEPs

apparent in developing the curve of recovery of effectiveness for basic

walk-around inspections in Chapter 19 that the cumulative normal distribu- tion reasonably approximated this recovery function. Therefore, this dis- tribution was specified as the recovery function for basic walk-around inspections (Figure 19-2).

As another example, actual NPP experience as well as the vigilance litera- ture (cited in Chapters 3 and 17) indicate that the highest probability of detection of an unannunciated display of a deviant condition, in the absence of alerting cues, occurs during the initial scan of the control room panels. After this initial scan, a combination of the effects of fatigue and expectancy causes the probability of detection to decline rapidly and then level off at some low level. It was apparent that an exponential function was a reasonable fit for this performance curve (Figure 11-1). We therefore specified the exponential curve as descriptive of this performance rather than specifying some other mathematical function that might fit closer to our judgment but would be less convenient to use.

Our approach to the mathematical modeling of human performance is more

inductive than that used by classical modelers. We first induce the state- ments of human performance from our experience and the available data; then we select mathematical expressions that fit the statements. Thus, our basic premise is that the psychology must be sound; the mathematical

expressions are secondary. We confess to a strong bias against elaborate mathematical models of human performance. All too often, the elaborate mathematics seems to become the goal, obfuscating the purpose of the model.

The first model in Part III is our dependence model (Chapter 10). It is

presented first because most of the human performance models in the Hand- book reflect the estimated performance of one person, and the HEPs must be modified for the interaction of other persons or for the dependence of tasks performed by one person. Our dependence model has certain restric- tions; for some HRA problems, the model will not be relevant, and direct estimates of the effects of dependence must be substituted.

Chapter 11 includes several models dealing with response to displayed information. Most of the models and HEPs apply to control room displays, and most of the errors described are errors of commission.

In Chapter 12 we offer some very tentative models to deal with certain

cognitive aspects of responding to abnormal events. For PRA, it is impor- tant to estimate how long it will take control room personnel to diagnose correctly the plethora of signals coincident with an abnormal event. In this area, the lack of data is especially severe. Application of these models requires considerable judgment.

Chapters 13 and 14 provide estimated HEPs for errors of commission related to the operation of manual controls (including switches for motor-operated valves) and for locally operated valves. These two chapters are particu-

larly important in estimating errors in the restoration of safety-related equipment after maintenance, calibration, or testing.

111-3

Models and HEPs

Chapter 15 provides estimates of HEPs for errors of omission and commission related to the following of oral instructions or to the use of written materials. Most NPP tasks assessed in a full-scale PRA are rule-based tasks that are to be performed using written materials or in response to oral instructions. Certain rule-based tasks are critical in coping with an abnormal event after it has been diagnosed. For this application, Chapter 15 provides the nominal HEPs that must be modified for the influence of special PSFs related to the abnormal event, e.g., high levels of stress.

Chapter 16 deals with administrative control, the extent to which plant policies are carried out or monitored. For example, in a PRA, in addition to the HEPs for errors in use of written procedures, one also needs esti- mates of the probabilities that these procedures will be used at all, and if used, used correctly. These considerations fall under the meaning of administrative control. Effective use of the information in this chapter will help in arriving at realistic assessments of how well people will do what they are supposed to do.

Chapter 17 treats the difficult subject of the levels of stress under which tasks must be performed. Most of the HEPs in the Handbook are premised on a facilitative (optimal) level of stress--neither too little nor too much stress for reliable performance. This chapter provides the modifying factors for adjusting the HEPs for cases in which the presumed stress levels for NPP personnel are not optimal.

Chapter 18 presents assumptions about staffing in the control room for estimating HEPs for team tasks carried out after the onset of an abnormal event. The NRC rules for the minimum number of persons present in the control room are used as a starting point, and conservative assumptions regarding the appearance of other persons later in the event, and their dependence levels, are given as guidelines for PRA. As with all other models, the staffing model is a nominal model and should be modified for plant specifics. The chapter also presents a crude model for the effects of experience level.

Chapter 19 discusses the recovery factors that must be assessed to estimate the probability that a particular error will fail to be corrected before it causes undesirable consequences. Several models are presented, including a model for the basic walk-around inspection.

In summary, Part III presents the assumptions and rationale for the esti- mated HEPs that are useful for the HRA part of a PRA. Most of these HEPs are presented in tables. Although we have tried to make each table "stand alone," the reader should study the related text so that he may fully understand the limitations and assumptions behind each estimated HEP. Most of the tables in Part III are repeated in Chapter 20 (Part IV), in which a procedure for using the tables for HRA is presented. A search scheme in Chapter 20 directs the analyst to the appropriate tables for a full-scale HRA.

111-4

Models and HEPs

In using the HEPs in Part III, the analyst should remember that these HEPs are nominal HEPs, that is, they are estimates that have not considered plant-specific PSFs, and few of them consider .such common PSFs as stress, dependence, or recovery factors. The analyst will need to modify the nominal HEPs according to the results of his task analysis when performing a plant-specific HRA. Since there may be considerable uncertainty in estimating HEPs for a specific application, the analyst may want to perform sensitivity analyses (Chapter 5) in which, say, the upper or lower UCBs for an HEP are used to see what effects on the overall PRA such modification makes. He will often find that the PRA will tolerate considerable latitude in estimates of HEPs.

This Handbook is a first attempt to develop and apply human performance models to the real-world problems of NPPs. Although some readers may find many of the models and assumptions in Part III to be oversimplified, the practical user will find that the consideration and application of all the elements in Part III constitute an involved task, requiring considerable judgment as well as expertise with NPPs. We do not apologize for our models and assumptions being oversimplified--we regret that we were not able to simplify them further.

111-5

Ch. 10. Dependence Overview; Methods

CHAPTER 10. DEPENDENCE

Overview

A major problem in human reliability analysis (HRA) is the determination of how the probability of failure or success on one task may be related to failure or success on some other task. Two events are independent if the conditional probability of one event is the same whether or not the other event has occurred. That is, independence is the case in which the proba- bility of success or failure on Task "B" is the same regardless of success or failure on Task "A." This relationship may be expressed as follows in terms of our HRA symbology given in Table 5-1: b = bla = blA, or B = BIA = Bla. If events are not independent, they are dependent, and the nature of the dependence must be considered in the HRA. Some degree of dependence between performances on Tasks "A" and "B" exists if b y b6a 91 blA, or B 5 B[A 91 Bla.

Failure to make a realistic assessment of dependence may lead to an over-

optimistic assessment of the joint human error probabilities (HEPs) for nuclear power plant (NPP) tasks. A general guideline is to assume depen- dence between human activities unless a diligent search reveals no sig- nificant interaction between them.

Dependence can occur between and within people. Dependence between people is fairly common, e.g., one person restores some manual valves and a dif- ferent person checks the accuracy of the restoration. Generally, some dependence will occur in this case. Dependence also occurs between differ- ent tasks performed by the same person. For example, dependence is likely to occur in the case in which a person restores two adjacent valves or activates two adjacent switches.

This chapter describes different methods of assessing the effects of depen- dence. Most of the chapter is devoted to our model for assessing depen- dence, which is addressed to positive dependence among tasks or people performing tasks. This model has been used in several probabilistic risk assessments (PRAs) sponsored by the Nuclear Regulatory Commission (NRC) or by utilities. Because of the subjectivity involved in assessing depen- dence, the use of sensitivity analysis is suggested to assess the effects of different assumptions of levels of dependence.

Different Methods for Assessing Dependence

There are several methods for assessing dependence. In this chapter, we emphasize methods that can be used for today's PRAs rather than methods that require data or information that is not available. The major problem, of course, is the lack of data on conditional probabilities of success or failure in the performance of NPP tasks. Because such data are scarce, indirect approaches must usually be taken in assessing levels of dependence.

10-1

Two Types of Dependence

Four methods for estimating the effects of dependence in an HRA are dis- cussed:

(1) The use of actual data on the operations in question.

(2) Direct estimation made by qualified personnel, based on information from similar operations and the relevant performance shaping factors (PSFs).

(3) Use of our positive dependence model.

(4) Use of other dependence models.

The best method for assessing dependence is to determine the conditional probabilities from actual data. For example, if Tasks "A" and "B" have been performed under the applicable conditions a large number of times, the probability of failure on Task "B" can be calculated separately for the situations in which there is failure on Task "A" and in which there is success on Task "A," and the conditional probabilities can be determined.

A second method, used when objective data are not available, is to make judgmental assessments of the conditional probabilities on the basis of the nature of the tasks and their interrelationships. We often use this approach.

A third approach, developed for this Handbook, uses a positive dependence model that serves as an aid in the estimation of conditional probabilities. This model is based primarily on human factors considerations. Its purpose is to permit gross estimates of the influence of interaction between human tasks. The model represents the continuum of positive dependence by five points: zero, low, moderate, high, and complete.

Another approach to modeling dependence, based primarily on probabilistic considerations, has been developed by R. G. Easterling and is presented in Appendix B. Other dependence models have been developed, e.g., the geomet- ric mean model used in WASH-1400 (Swain and Guttmann, 1978) and a model developed at Brookhaven National Laboratories (Samanta and Mitra, 1981), but in our opinion are not as useful today as the dependence model de- scribed in this chapter since they may yield unrealistically low estimates of joint HEPs or may require data that are not currently available.

Two Types of Dependence

Dependence can be due to two different types of causes: direct and in- direct. Direct dependence exists when the outcome of one task directly affects the outcome of a second task. This is the case when the failure or success of one task changes the environment or other factors so as to change the probability of failure on another task. Some examples of direct dependence between Tasks "A" and "B" are:

(1) Failure on Task "A" causes an auditory signal that results in more careful performance of Task "B."

10-2

Two Types of Dependence

(2) Failure on Task "A" causes extreme anxiety with a resultant increase in probability of failure on Task "B."

(3) Failure on Task "A" causes Task "B" to be more difficult with an associated increase in probability of failure.

Indirect dependence occurs when some performance shaping factor (PSF) or set of PSFs influences the relationship between tasks such that the depen- dence between them changes. If the PSF merely raises or lowers the HEPs for tasks without changing the relationship between them, this is not an example of indirect dependence.

To illustrate these points, consider the PSF of stress. A high level of stress tends to increase the probability of error for all of the tasks to be performed, but unless it also changes the dependence between these tasks, we do not have a case of indirect dependence. This type of depen- dence would occur if the high level of stress also changes the interaction between operators in a control room so that, for example, under a high level of stress, the control room operators would have a much stronger tendency to defer to the suggestions of the shift supervisor. In effect, they would be manifesting a higher level of dependence on him than would occur under normal operating conditions. If all the control room personnel were senior reactor operators, it might be the case that a high stress level would not change the usual interaction between them. Thus, the combination of the PSFs of stress level and skill level may determine whether there is indirect dependence involved.

As mentioned in Chapter 1, this Handbook takes a scenario-oriented approach to the prediction of human performance. Each scenario includes the PSFs judged to be the most important in estimating the HEPs for any given task. The influence of indirect dependence is represented in these PSFs. The distinction between indirect dependence and direct dependence is sometimes arbitrary and not very important for PRA. In one sense, when we deal with human performance, almost all dependence stems from the underlying influ- ence of a person's mediating processes. If I make an error on Task "A,". this performance does not always directly influence my performance on Task "B." Except for essentially automatic human reactions, or some physical dependence of the task objects, my performance on Task "B" is affected by my mental/emotional reaction to my perception of my failure on Task "A." It is not the same as the direct cause and effect relationship that exists when a break in one pipe carrying caustic material can cause a nearby pipe to fail. In practical human reliability analysis, we don't ordinarily differentiate between the two types of dependence; we determine the appro- priate PSFs and their overall effects, including the appropriate weightings for direct dependence between human task performances. The cases in which a common influence may affect performance are allowed for in the assess- ments of direct dependence.

In the draft issue of the Handbook, we incorrectly employed the term "common-cause dependence" by defining it as the dependence that exists when the performance of two or more tasks is related to some common influence.

10-3

Characteristics of Dependence

We failed to restrict this term to changes in the dependence between two (or more) tasks as a function of some influence outside of the tasks them- selves. In this reissue of the Handbook, we use the term indirect depen- dence rather than common-cause dependence because the term common cause is often used in PRAs in a different manner. Common-cause events are single events having the potential to fail more than one safety function and to possibly cause an initiating event simultaneously (Zion Probabilistic Safety Study, 1981). We will restrict our use of this term to the above PRA meaning and use the term indirect dependence when speaking of a change in dependence due to some factor outside of the tasks themselves, e.g., stress. Thus, if a person fails to restore two blocking valves in the auxiliary feedwater system, that person's performance may constitute a common-cause failure mechanism for the system. The fact that the person forgot to restore both valves rather than just one valve may be the result of dependence, i.e., the two valves are so close together that if he for- gets one, he is almost certain to forget the other. Whether this type of dependence is labeled direct or indirect is not important for estimating the conditional probability of failure to restore the second valve, given that the first valve was overlooked.

Characteristics of Dependence

The degree of dependence among human actions, or between system events and the human reaction to them, ranges along a continuum from complete negative dependence through complete independence (zero dependence) to complete positive dependence. While our dependence model deals only with the zero to complete positive dependence part of this continuum, in HRA it is neces- sary to deal with both negative and positive dependence.

Negative dependence implies a negative relationship between events, e.g., failure on the first task reduces the probability of failure on the second task, or success on the first task increases the probability of failure on the second task. Most of us can think of everyday experiences in which, having erred in some task, we take extra care and have a higher probability of success on the second task. Conversely, success on the first task might reduce our probability of success on a second task through overconfidence. There are also instances of negative dependence on task performance between people in industrial situations, as when a rivalry exists. In terms of our HRA symbology, facilitative negative dependence exists when BIA < B or bIA > b. The opposite effects hold when Bla > B or bja < b.

Positive dependence implies a positive relationship between events, i.e., success on the first task increases the probability of success on the second task, and failure on the first task increases the probability of failure on the second task. In terms of our HRA symbology, facilitative positive dependence exists when bla > b. The opposite effect occurs when BIA > B. In PRA work, the latter effect is usually the one that is of most interest.

10-4

Data-Based CHEPs

Whether dealing with negative or positive dependence, then, we must judge the degree of dependence among tasks or events when performing an HRA. If one has the data, the best approach is to calculate directly the condi- tional probabilities of the actions under consideration and to incorporate the appropriate degree of dependence in the error probabilities.

Conditional Probabilities of Error Based on Data

Assume that we are studying two related tasks, "A" and "B," in which Task "A" is the first task performed, and Task "B" is the second. Data col- lected on 10,000 occurrences of the 2 tasks show that the ratio of suc- cesses to failures on both tasks is 9,900 to 100, that is, both tasks have an HEP of .01. The data also show that half of the failures on Task "B" occurred after a success on Task "A", and the other half of the failures on Task "B" occurred after a failure on Task "A." If the two tasks were completely independent, the joint occurrence of failures on both tasks should occur with a probability of .01 x .01 = 104. Thus, in the 10,000 occurrences of the 2 tasks, one would expect to see only 1 joint failure of the 2 tasks. However, the data show that there were actually 50 such joint failures, indicating a conditional probability of error of .5 for Task "B," i.e., BIA = .5. This conditional error probability for Task "B" can be multiplied by the error probability of .01 for Task "A" to obtain the joint error probability for the two tasks: .01 x .5 = .005 (a factor of 50 higher than would occur given independence).

Such a complete set of data is rarely available for NPP tasks, but some- times data on one of the tasks may be available. For example, assume that Person A will perform a task and Person B will check A's work. One may find data on how accurately Person A performs his task but none on how accurately Person B checks the task. However, it may be known that Person B's accuracy is about the same as that of Person A when performing the same work. In such a case, one approach is to use the estimated probability of failure for Person A and apply it to Person B, with some modifying factor to account for the judged level of dependence. If there is no dependence and if their error probabilities are equal, the joint probability that Person A will make an error and that Person B will fail to catch the error is the square of the estimated error probability for Person A. Such com- plete independence is most unusual, and the assumption of zero dependence could result in overly optimistic estimates of the joint HEPs.

At the opposite extreme (complete dependence), one might judge that when- ever Person A makes an error, so will Person B. That is, Person B will never detect an error made by Person A. In this case, the estimated proba- bility that Person A and Person B will make errors jointly is the estimated probability for Person A alone, since the conditional error probability for Person B is 1.0 under the assumption of complete dependence. Complete dependence is not as unusual as zero dependence; however, for the e: Ie stated, it would not be likely. The more usual state of affairs is a level of dependence somewhere between zero and complete dependence.

10-5

Direct Estimation of CHEPs

Direct Estimation of Conditional Probabilities of Error

Direct estimation of levels of dependence may be based on the analyst's own judgment or on judgments obtained from others who are knowledgeable in the field. Ordinarily, the analyst will not ask the "judges" to estimate directly the level of dependence between certain actions, since they may not be familiar with our concept of dependence levels. Instead, he may ask the judges to estimate the percentage of times an operator who has failed to perform Task "A" in a given situation will also fail to perform the subsequent Task "B." The most qualified judges would be theoperating personnel who perform the tasks. If several operators independently show agreement in their estimates, the analyst may be confident that the collec- tive judgment is approximately correct. He might then double their esti- mates to allow for the usual overconfidence of people when estimating their own abilities (as discussed in Chapter 6).

The following example illustrates a case in which we estimated conditional probabilities of error directly, based on our collective judgment. The example illustrates both negative dependence and complete dependence.

In this situation, a technician is checking the calibration of a series of setpoints consisting of three comparators. To do this, he must set up some test equipment, and he could make an error in this initial setup. For example, he could select the wrong decade resistance, set up the wrong scale on the decade, or make any other error in the test setup. Unless corrected, such an error would result in miscalibration of all three comparators.

The problem was evaluated in WASH-1400, p II-101. Figure 10-1 presents the HRA event tree for this task. In this evaluation, a probability of 10 was estimated for the common-cause event of a faulty setup. This estimate was modified by recovery factors as follows: it was reasoned that when any technician discovered that the calibration of the first setpoint had to be changed, he would change it without suspecting anything amiss. It was further reasoned that 9 out of 10 technicians who found that the second setpoint also had to be changed would be suspicious, and would recheck the test setup and discover the original error. One out of 10 technicians would not be suspicious, and, given that he has this unsuspicious nature, it was judged that the conditional probability of the third error (i.e., failing to become suspicious when he has to recalibrate the third setpoint) was 1.0. That is, complete dependence (CD) was assumed between the last two tasks. Thus, the joint probability of error in calibrating the three setpoints was .01 x 1.0 x .1 x 1.0 = .001.

Note that the above example assumes CD between the setup task and cali- bration of the first setpoint. CD is also assumed between the tasks of calibrating the second and third setpoints. However, a different level of dependence is assessed between the tasks of calibrating the first and second setpoints. The .9 probability that the technician will be alerted by the misalignment of the second setpoint is an instance of negative dependence, since failure on the second task is expected to enhance the

10-6

Figure 10-1

a =99

S 1 =.99

A= .01

b B=I.0

C C=.1

S 2 = .009 d D = 1.0

F 1 = .001

A = FAILURE TO

B = FAILURE TO

C = FAILURE TO

D = FAILURE TO

•0 = NULL PATH

SET UP TEST EQUIPMENT PROPERLY

DETECT MISCALIBRATION FOR FIRST SETPOINT

DETECT MISCALIBRATION FOR SECOND SETPOINT

DETECT MISCALIBRATION FOR THIRD SETPOINT

Figure 10-1 HRA event tree of hypothetical calibration tasks.

10-7

Direct Estimation of CHEPs

probability of success on the following task. Instances such as this one are decided by the judgment of the analyst rather than by any formal approach.

For illustrative purposes, we have evaluated this situation in more detail by considering both small and large miscalibrations. We define a large change as one that is not normally expected, while a small change is one that can be expected to occur occasionally because of variations in equip- ment and other conditions.

In Figure 10-2, Task "A" represents the test equipment setup. Alpha and beta refer to the probability of small or large miscalibrations that will invariably follow from A, the incorrect setup of the test equipment. The other letters refer to the calibration of the three setpoints, with plain letters used on the alpha side of the tree, and prime letters on the beta side. As noted in Chapter 5, we usually do not employ the complete nota- tion for the conditional probabilities of events. That is, we do not write aJA or PIA; we write a or P, with the "given A" understood.

In the analysis, the only error of interest was that of the miscalibration of all three setpoints; that is, the failure paths terminating with the end-failure terms, F1 and F2 . Path a was designated as a success path (SI) because, given that the technician set up his test equipment cor- rectly, there were no other failure probabilities of interest. Given a, even if we assume 'that the probability of miscalibrating any given setpoint is as high as 10 2 , the probability of missetting all three is 106, a negligibly small number for most PRAs. This example illustrates the usual case that an HRA need not depict all possible system outcomes.

Given A, we estimated that half the consequent miscalibrations would re- quire a small change, and half a large change. In a real analysis, system analysts would provide these estimates. It is likely that a >> in a real-world situation.'

Considering the alpha side'of the tree, B = 1.0 means that all of the technicians will make the small calibration change and then proceed to Task'"C," calibration of the second setpoint. C = 10-1 means that 1 per- son in 10 will fail to become suspicious when the second set point, also

:requires adjustment. c, then, is .9, which means that 9 out of 10 tech- nicians will become suspicious and recheck the test setup. This is why the c path ends at S2 , a success endpoint in the HRA event tree. How- ever, given C, D = 1.0 means that if a technician does not become suspi- cious at the second setpoint, he will make the calibration change to the third setpoint without becoming suspicious. In routine tasks that are done frequently, this level of inattentiveness is not unusual. This failure path ends at F1 , one of the two possibilities for miscalibrating all three setpoints, the only failure of interest in the analysis.

When we move to the beta side of the tree, the recovery factors are better because the required changes of the three setpoints will be substantial and should cue a competent technician. The reasoning for the beta side of the 0

10-8

Figure 10-2

)'= 1.0

F2 = .000005F1 =.0005 '0'

A = FAILURE TO SET UP TEST EQUIPMENT CORRECTLY

a - SMALL MISCALIBRATION OF TEST EQUIPMENT

B - FOR A SMALL MISCALIBRATION, FAILURE TO DETECT MISCALIBRATION FOR FIRST SETPOINT

C =FOR SMALL MISCALIBRATION, FAILURE TO DETECT MISCALIBRATION FOR SECOND SETPOINT

D = FOR A SMALL MISCALIBRATION, FAILURE TO DETECT MISCALIBRATION FOR THIRD SETPOINT

P = LARGE MISCALIBRATION OF TEST EQUIPMENT B'= FOR A LARGE MISCALIBRATION, FAILURE TO

DETECT MISCALIBRATION FOR FIRST SETPOINT

C' = FOR A LARGE MISCALIBRATION, FAILURE TO DETECT MISCALIBRATION FOR SECOND SETPOINT

D' = FOR A LARGE MISCALIBRATION, FAILURE TO DETECT MISCALIBRATION FOR THIRD SETPOINT

Figure 10-2 HRA event tree of hypothetical calibation tasks (small and large miscalibrations).

10-9

Positive Dependence Model

tree is the same as for the alpha side, except that the large calibration changes modify the estimates of conditional probabilities. Note that B' = .1, which means b' = .9; i.e., 9 persons in 10 will be suspicious after calibrating the first setpoint. Of the technicians that accept this need for change, c' = .99 means that 99 in 100 will suspect a second setpoint requiring a large change. D' = 1.0 implies an unquestioning technician; if he accepts the first two changes, he will not question the third one. The values assigned to this pathway mean that, given A x B, such unquestioning behavior would occur only one time in a thousand, B' x C' X D' = 10-1 x 10-2 X 1.0 = 10-3.

Obviously, the above assumptions are not valid in all cases. There is a possibility, for example, that the technician might recheck the test setup after becoming suspicious and accept the setup as correct, but we estimate this probability to be insignificant.

The Positive Dependence Model

Actual data describing conditional probabilities of error are not widely available, and the direct estimation of conditional probabilities often requires a degree of expertise which may not be available to the analyst. Our dependence model was developed to cover the many cases for which data are not available, as an aid to arriving at estimates of conditional proba- bilities. The model is restricted to positive dependence; when the analyst judges that negative dependence is involved, he will have to make his esti- mates of conditional probabilities on the basis of available :data or use judgment. Alternatively, he might elect to assume that no dependence exists; for most PRA applications this assumption, rather than the assump- tion of negative dependence, would lead to conservative estimates of HEPs.

One could take the reverse of our positive dependence model as a model of negative dependence. We did not do this because, for most PRA applica- tions, the assumption of negative dependence will often yield unrealis- tically low estimates of joint HEPs. For those cases in which negative dependence should be assumed, we believe it is better that the analyst make his analysis on the case specifics as done in an earlier section.

Levels of Dependence

Dependence is a continuum, and it is necessary to judge the appropriate

level existing between any pair of task performances. This may be diffi- cult, and some simplification has to to be made. The approach taken here is to reduce the positive continuum of conditional probability to a small number of discrete points. We use just five points: the two end points of zero dependence (ZD) and complete dependence (CD) plus three points in between. We call these intermediate points low dependence (LD), moderate dependence (MD), and high dependence (HD). The equations for the five points are presented later.

Figure 10-3 shows the relative positions of the five discrete points in this model. The rationale for the assignment of their relative positions is given later. Use of values between HD and CD would result in relatively

1 0-1c

Figure 10-3

ZD LD MD

..I I HD

I , CD

IF DEPENDENCE

ZERCqhI-. COMPLETE

Figure 10-3: Continuum of positive dependence represented by five discrete points.

10-11

Positive Dependence Model

little change in the estimated joint HEPs and very little impact in PRAs. Therefore, no discrete points are assigned in this area.

Complete-Failure Path and Complete-Success Path Dependence

For the three intermediate levels of dependence, (LD, MD, and HD), methods are provided for assessing (1) the complete-failure path dependence (i.e., dependence among successive tasks that are all performed incorrectly), (2) the complete-success path dependence (i.e., dependence among successive tasks that are all performed correctly), and (3) all other paths. In practical human reliability analyses, the first two paths are usually the ones of interest. The analyst is generally interested either in the proba- bility that a person performs at least one of the related tasks correctly (i.e., he avoids the complete-failure path) or that he performs all related tasks correctly (i.e., he follows the complete-success path).

Figure 10-4 shows a three-rta.sk HRA eventtiee consisting of the success and failure limbs for Tasksq,"I"A," "B," and "C.1" (The symbology from Table 5-1 is used.) On the, complete-success path in the tree, a nonzero level of positive dependence means that the probability of succe'ss on a task, given success on the preceding task, is higher than the basichuman success probability (-BHSP) (what the success probability would be in the absence of dependence). On the complete-failure path in the:;tree, a nonzero level of positive dependence-means that the probability of failure on'a task, given failure on the preceding task, is higher than the basic human error proba- bility (BHEP) (what the error probability would be in the absence of dependence). •

Considering only Tasks "A!' and "B" in Figure 10-4, 'Lthe, conditional proba- bility of success or failure of Task 'B." varies according to the influence of Task "A.'i Consider the complete-failure path and assume a nonzero level of dependence between these two tasks. If; Task "A" is failed, the proba- bility of also failing Task "B" will be higher than its BHEP (i.e., BIA will be higher than B). Conversely, the probability of successful perfor- mance of Task "B," given failure on Task "A" (i.e., bIA), will be lower than b, the BHSP.

For the complete-success path, still assuming a nonzero level of depen- dence, the probability of successful performance of Task "B," given suc- cessful performance of Task "A" (i.e., bja), will be higher that the BHSP, b. Conversely, the probability of failure of Task "B," given that Task "A" is performed correctly (i.e., Bla), will be lower than the BHEP, B.

When estimating the conditional probability of success or failure of Task "C" in Figure 10-4, one must decide what level of dependence is appropriate between this task and the preceding task, Task "B." The level may or may not be the same as the level between Tasks "A" and "B." (As noted in the following section, the conditioning influence of Task "A" on Task "C" is ignored.) Also, the level of dependence between ,a and Task "B" may not be the same as the level of dependence between A and Task "B." Thus, bla and Bla may be based on ZD while bIA and BIA may be based on, say, MD.

10-12

Figure 10-4

CIBIA

Figure 10-4 HRA event tree showing complete-success path (double line) and complete-failure path (heavy solid line).

10-13

Psychological Considerations

Dependence in Other Paths

For a human reliability problem consisting of only two tasks, the condi- tional probabilities of the two limbs not in the complete-success or the complete-failure paths are predetermined because at each branching the conditional probabilities for the limbs must sum to 1.0. Thus, considering only Tasks "A" and "B" in Figure 10-4, bla + Bla = 1.0, and bIA + BIA = 1.0. For any two-task tree, if we have estimated bja and BIA, the proba- bilities for the other two limbs can be obtained by subtraction.

In the case of an analysis involving more than two tasks, the conditional probabilities of subsequent limbs are derived by attributing all the effects of dependence to the immediately preceding limb.* For example, in Figure 10-4, the conditional probability CIBIa is assumed to be the same as that of CIBIA, and the conditional probability of cJbJA is assumed to be the same as that of cjbla. In both of these cases, the influence of Task "A" on Task "C" is ignored. Although we recognize that this assumption is not entirely valid, our rationale for simplification is that the immedi- ately preceding task is generally the prime factor influencing the success or failure of the task in question, given some nonzero level of dependence between the two tasks. It is possible to derive a model that assigns different weights to the conditioning factors of, say, Cibla and CIbIA, but considering the typical uncertainty in estimates of the basic success or failure probabilities, such exactitude is unwarranted. Such refinement is best postponed until the basic human performance data warrant such improve- ment. If we do not believe that Clbla = CJbJA, we must estimate condition- ing effects directly, as in the examples of dependence described earlier.

Psychological Considerations for Levels of Dependence

This section discusses the psychological meaning of each of the five levels of dependence and presents some examples. Guidelines are provided to aid in the judgment of which level of dependence is most appropriate for typi- cal situations. There are no hard and fast rules for deciding what level of dependence is appropriate for any situation; considerable judgment is required. In assessing the level of dependence between people, it is important to decide whether and when there is any interaction between them. One error in applying the dependence model that has been made in some PRAs is to assume that every person in the control room following a transient will be involved in each detailed activity. Actually, many of the actions, for example, switching actions, would be carried out by only one or two people. Additional examples of the assessment of dependence levels may be found in NUREG/CR-2254. A later section provides rules for assigning numerical values to the conditional probabilities associated with the various levels of dependence.

As described in Chapter 2, there are five categories of incorrect human

outputs for HRA: errors of omission and four categories of errors of commission: selection error, error of sequence, time error, and quali- tative error. In performing an HRA, different levels of dependence may be

This assumption is equivalent to the assumption of a Markov process.

10-14

Psychol. Considerations - ZD

used in estimating probabilities for these incorrect human outputs or for the underlying errors that caused them (e.g., cognitive errors, extraneous acts, incorrect procedures, etc.). For example, in the model for locally operated valves in Chapter 14, ZD is assumed for several errors of commis- sion related to the manipulation of a sequence of valves. In general, higher levels of dependence are assumed for errors of omission than for errors of commission within the same task.

Qualitative Difference Between Zero and Nonzero Levels of Dependence

The assumption of any level of dependence between two tasks implies that the behavioral phenomena related to those tasks are substantially different from those related to tasks that are independent. For this reason, we expect to find a correspondingly large difference between the conditional HEPs estimated for tasks that are independent of other tasks and the condi- tional HEPs for tasks that are dependent. The dependence model provides a method for estimating the quantitative effects of this qualitative difference.

Dependent tasks are related to each other in that, consciously or not, the operator performing a series of dependent tasks approaches them as parts of a task sequence. In assessing a nonzero level of dependence, we assume that making an error on any task in the sequence implies that the oprator is in an error-likely state at the time of the error and that this state carries over to the performance of the next task in the sequence. If another error is made, the error-likely state extends through the subse- quent task in the sequence, and so on. Our model assumes that the error- likely state terminates when a task within a sequence is performed cor- rectly, or upon completion of the dependent sequence, regardless of the number of tasks within the sequence. This is because the dependence model considers the effects of dependence of any task as extending from the immediately preceding task only. The end result is that the conditional HEPs in a dependent task sequence are considerably higher than the basic HEPs, the assessed level of dependence accounting for the magnitude of the increase.

Because our dependence model places considerable emphasis on the propensity for error following an initial error, the estimated conditional probability of error for the second task is influenced very little by the magnitude of the basic HEP of that task, until the BHEP is .1 or higher. With such high BHEPs, the tasks are obviously affected by unusual PSFs such as inadequate training, poor ergonomic design, poor procedures,. stress, and so on. Our dependence model also allows for this qualitative influence on conditional HEPs.

Zero Dependence

Zero dependence (ZD) applies to the case in which the performance or non- performance of one task has no effect on the performance or nonperformance of subsequent tasks. Although 100% independence is uncommon between human tasks, occasionally the dependence is judged to be so slight that we assume ZD for purposes of analysis.

10-15

Psychol. Considerations - ZD

An example of a situation in which ZD would be assumed between tasks for errors of commission is that of an operator who has to check-read several instruments under normal operating conditions as part of his periodic scanning of displays during a shift. If the basic probability of a check-

reading error on some instrument is estimated as .003, and there is a similar instrument displaying different information elsewhere on the con- trol panel that must also be check-read, the same error probability would be used for each.

ZD would not be a valid assumption for errors of commission if the charac-

teristics of the work situation include all of the following: (1) the meters are located side by side, (2) under normal operating conditions, the pointers on meters are parallel, and (3) the operating policy is to check- read both meters sequentially. In such a case, we would assume high or complete dependence between the check-readings.

The level of dependence for errors of omission should be assessed sepa- rately. If the above two check-readings are separated in time so that they represent unrelated tasks to the operator, ZD would be assumed. However, if the two check-readings must be made at the same time and are called for in the same step in the procedure, complete dependence would be assumed. If the operator fails to check one, he will fail to check the other.

We usually assume ZD when estimating the error probabilities for carrying

out individual steps in a written procedure. This applies to errors of

commission and omission, given that the operator has initiated the task and is using the written instructions properly. Sometimes operating personnel do not use written procedures properly. In such cases the assessment of ZD may be inappropriate. Consider the case in which there are several valves on a manifold and the operator has a set of written procedures that direct him to change the positions of all of the valves. It would be an unusual operator who would look at the written designation of a valve, change its position, and then read the written designation of the next valve, change it, and so on. Typically, a skilled operator will note from his procedures or his memory that "all these valves on the manifold are to be closed," and he will close them. In this case, even with written procedures, errors of omission are completely dependent for this operator. Some operators, especially if there is a written checklist, will use the checklist proper- ly, checking off each valve after they have closed it. ZD for errors of

omission would be an appropriate assessment for these operators. The

analyst, then, obviously must use judgment as to the percentages of opera- tors who will respond in different ways and must base his assessments of conditional error probabilities on these estimates.

ZD is rarely assessed for errors made by persons working as a team or for

one person checking another's performance. A nonzero level of dependence is nearly always the case. The interaction among control room personnel is discussed below and in Chapters 12 and 18.

10-16

Psychol. Considerations - LD, MD

Low Dependence

Low dependence (LD) represents the smallest level of dependence that is considered in our dependence model. Because of the qualitative difference between zero and nonzero levels of dependence, the assumption of even the lowest level of dependence generally results in a substantial change in the conditional probability of error if an error has been made on the preceding task. The change is much smaller for the case in which the preceding task has been performed successfully.

As an example of low dependence between people, consider the interaction between two control room operators who are newly acquainted with each other. They have not as yet developed confidence in each other and will tend not to rely on the other to carry out certain tasks. Therefore, each will try to consider most task requirements independently. However, the mere knowledge that the other is a licensed operator is cause for a certain amount of confidence, even though it may be minimal. The situation is similar to that of a new pilot assigned to a squadron--the other pilots accept him as a pilot, but they do not develop confidence in him until they have "tested" him over a period of time in a variety of situations. Simi- larly, an NPP shift supervisor will often check the performance of a new operator very carefully; his level of dependence with the new operator would be assessed as low. The shift supervisor may also take special care in assessing the status of safety systems after a transient has occurred, and an estimate of LD might be appropriate in this case even with experi- enced control room operators. In cases in which the shift supervisor assumes primary responsibility for a task, i.e., he takes over the opera- tion from one of the control room operators, the level of dependence of the operator on his supervisor will often be assessed as higher than LD-- perhaps HD or CD. This is because the less qualified, less experienced operator may well defer to the shift supervisor. Thus, in assessing levels of dependence between control room personnel, the analyst must specify the direction of interaction involved--is the shift supervisor checking the performance of the regular control room operator or is the latter checking the performance of the former?

In most cases of human actions, if there is any doubt as to an assessment of ZD, we ordinarily assume LD to allow for the various unforeseen sources of dependence. This applies to actions between individuals as well as to actions by the same individual, except for those cases where ZD is clearly indicated. In calculating the complete-failure path (Figure 10-4), the assumption of LD rather than ZD will yield a more conservative estimate of reliability. The effect on the complete-success path will be negligible if the BHSPs are .99 or higher.

Moderate Dependence

Moderate dependence (MD) is assumed when there is an obvious relationship between performance on one task and on a subsequent task, and the relation- ship is greater than LD would imply. In the case of human interactions, a moderate level of dependence is usually assessed between the shift super- visor and the operators for tasks in which the supervisor is expected to

10-17

Psychol. Considerations - HD, CD

interact with them. There is some latitude in this assumption since the dependence among personnel may exceed the moderate level under unusual conditions, such as stress.

High Dependence

High dependence (HD) is a level midway between ZD and CD on the positive dependence continuum. HD implies that performance on one task very sub- stantially affects performance on a subsequent task, but the influence is not total. HD exists among people when one person's authority or prestige is significantly greater than that of the others, and his decisions would have a very significant influence on the others. Although this does not imply blind obedience, it does imply that the others would not be likely to question his approach. This tendency is greatest when coping with some unusual or unexpected event.

High dependence is usually assessed between the two operators in a control room for those tasks in which they are supposed to interact. In this case, the assessment is made on the basis that neither of the operators expects the other to make mistakes, and the checking of one's actions by the other is often casual.

In the case of the complete-failure path, the assumption of HD will yield a less pessimistic estimate than would the assumption of CD. The assumption of CD implies that an error on task 1 inevitably results in an error on

task 2, whereas the assumption of HD reduces the conditional probability of error on task 2 to some value less than 1.0.

Complete Dependence

In normal circumstances, complete dependence (CD) between the actions of

two people working on a job is unusual, although not as unusual as ZD. CD between two actions performed by the same person is more common. For example, in certain NPP control rooms, switches are often paired, and the written procedures treat the pair as a unit. Thus, the procedures for a given situation in one pressurized water reactor (PWR) call for the oper- ator to "Open MOV-860A and B, suction to the low head SI pumps from the

containment sump." In the training simulator, the operator is taught to reach for both switches and operate them as a unit. If he fails to operate either, he is almost certain to fail to operate the other. Conversely, if he does operate either, he is almost certain to operate the other. The primary error here is one of omission. If he forgets the first one, he will certainly forget the other. Therefore, CD is the appropriate level of dependence. On the other hand, in cases where two switches are located such that simultaneous manipulation cannot be achieved with one motion, a lower level of dependence will often be assessed.

Errors of omission provide a common example of CD. If an operator fails to initiate a task (an error of omission), all the steps in that task will also be omitted. The converse does not necessarily hold; the fact that an operator initiates a task does not guarantee that all the steps will be performed.

10-16

Perceptual Units

In the case of tasks performed by two or more people, CD may develop under a variety of conditions. For example, in performing a calibration, one person might do the adjusting while the other reads and records the display indication. If the first person does not also read the display indication, the accuracy of the calibration is completely dependent upon the second person. Similarly, if one person reads a set of procedures to another person and does not check the work of the second person, the success of the task is completely dependent upon the second person, except for reading errors, of course. Although such instances of CD between people are uncom- mon under normal operating circumstances, they do occur frequently enough to warrant consideration by the human reliability analyst.

Functional Relationships among Tasks

Although we think of dependence as a phenomenon primarily between people (or within an individual), several parameters of the system interface affect the functional relationships among tasks, and influence the level of dependence between human actions. The highest level of relationship exists when two or more events or items are, in effect, regarded as a single unit by an operator. For example, in the above-mentioned case of certain pairs of switches that are operated simultaneously, the operator characteristi- cally uses both hands at once, one for each switch. For these situations, the switches are considered to be completely dependent with regard to errors of omission. If the operator remembers to manipulate one of the two switches, he will remember the other. Whether or not this highest level of relationship also applies to errors of commission will depend on the design of the switches. For example, if both switches are rotated clockwise to open the motor-operated valves (MOVs), the assumption of CD for errors of commission would normally be appropriate. If the switches operate as mirror images (one is rotated clockwise and the other is rotated counter- clockwise), the assumption of CD may not be appropriate, and the analyst will have to estimate the HEPs for commission on some other basis.

In the above example, the assumption of CD for errors of omission might be appropriate even if the two switches are separated such that simultaneous operation by an operator is not possible. The key here is the time rela- tionship and the typical mode of operation. If the operator typically manipulates one switch and then immediately manipulates the other, this association might be judged strong enough to warrant the assumption of CD for errors of omission. For example, in the Arkansas Nuclear One Unit #1 PRA (Kolb et al, 1982a), CD was assessed for errors of omission for pairs of identical-function switches in two different channels even though the switches were separated by as much as 3 feet. In this case, lines on the panels indicating different functional groups of controls made this assess- ment a reasonable one. Observation of operators verified this assessment.

Other items of equipment may not have the same function in an NPP, but because of their location or other association, may be regarded as one unit, at least for errors of omission. For example, if several locally operated valves are located in a small group and if the written procedures treat them as a group (in a single written step), the assumption of CD will

10-19

Perceptual Units

normally be appropriate. An exception is the case in which some of them are supposed to be open and some closed, breaking the perceptual pattern of a "single unit."

It would be helpful if we could quantify relationships on the basis of differences in time or location of operation. As yet we do not have such data, but Figure 10-5 indicates the direction of the relationships along the positive dependence continuum. The values indicated in the figure are suggestive only and should not be substituted for the judgment of the analyst who has plant-specific information.

Other parameters of displays and controls, such as similarity in appear- ance, mode of operation, mode of reading, etc., can be thought of in the same manner in regard to their relationship to the dependence continuum. We do not have any rules for combining the effects of such parameters. For example, we cannot say that HD on any two parameters equals CD, although it would increase one's confidence that the relationship is indeed HD (or higher). It is the operator's perception of what is functionally' related that must be evaluated. For example, if operators in a plant state that "if the temperature indication rises, I always check the pressure: indica- tion," one may consider that the two indications constitute a single func- tional unit.

Displays may have different levels of functional relatedness. Although temperature and pressure gauges for the reactor vessel are obviously related, this does not necessarily mean that an operator who checks one display will invariably check the other. In Chapter 11, special rules are formulated for the assessment of the level of dependence among displays.

In the Handbook, the term perceptual unit refers to an individual item of equipment (e.g., a display, control, valve, etc) or some group of items that are completely dependent with regard t0 errors of omission.' .The level of functional relatedness for errors of commission must always beý evaluated separately.

Awareness of One's Own Errors

It will be noted that the dependence model does not directly address the effects of a person's awareness that he has committed an error. In the case of a nonstressful situation, awareness that one has made an error will normally serve as a recovery factor for that task and may alert one to be more careful on the subsequent task, an example of negative dependence. Generally, however, we assess zero dependence as a conservative estimate for such cases. In the case of a highly stressful situation, the awareness of an error will still serve as a recovery factor, but the awareness may also increase the stressfulness of the situation, resulting in a higher error probability on the next task because of the increased stress, an example of positive dependence. We cannot assess the relative strength of these competing influences, so we assume they are balanced.

Thus, the awareness of error does not modify the assumptions of dependence in either the low-stress or high-stress situations and may be disregarded

10-20

Figure 10-5

DEPENDENCE LEVELS

ZD LD MD HD CD

- a - - a

I TIME BETWEEN OPERATOR ACTIONS (seconds)'

>+ -1.0 DISTANCE BETWEEN OPERATOR ACTIONS

OPERATOR DOES NOT HAVE TO CHANGE

HIS POSITION

.4 SIMULTANEOUS

SIMULTANEOUS OPERATION (ONE-HAND OPERATION OR

TWO HANDS TOGETHER)

OPERATOR MUST WALK TO A .

DIFFERENT PANEL

Figure 10-5 Direction of operator action relationships along the positive dependence continuum.

Geometric Mean Model Equations for ZD and CD

in low-stress situations. In the case of high-stress situations, the usual increases in estimated basic HEPs due to stress make adequate allowance for the difference in stress that arise from awareness of error, and no further changes in the estimated BHEPs could be made with any confidence.

General Guidelines in Assessing Level of Dependence

While there are no set rules to apply in assessing which level of depen- dence to use, there are a few general guidelines, as presented in Table 10-1.

Relationship of Dependence Model to an Earlier Model

The dependence model in this chapter replaces an earlier model developed in "Human Reliability Analysis," Section 6.1, in Appendix III of WASH-1400, and in Swain and Guttmann (1978). The earlier model used the geometric mean of the joint probabilities of error under ZD and CD to estimate the joint probability of failure of two or more human activities when a moder- ate level of dependence was assumed. Under the assumption of a high level of dependence, the joint probability of failure of two or more human activ- ities was calculated by using the geometric mean of the joint probabilities of error under MD and CD. The present model will yield somewhat more con- servative (i.e., higher) failure estimates than will the geometric mean model. The present model is more easily understood in a practical sense, and it has greater flexibility since there are three intermediate levels of dependence. In the earlier model, we had only Zb, MD, HD, and CD; in the present model, we have added LD.

One of the major problems with the geometric mean model is that with three or more tasks, each having BHEPs of 10-3 or lower, the joint HEP for the complete-failure path becomes unbelievably low. The present model is psychologically more valid even though it is not as mathematically attrac- tive as the geometric mean model.

Equations for Zero and Complete Dependence

The mathematical expressions of joint probabilities for ZD and CD are fixed by definition (see any standard statistical text). With ZD among n events, the probability of successful completion of all tasks, i.e., the joint human success probability (HSP) is

Pr[SIZD] = ab *.. n (10-1)

where n is the probability of successful performance of activity "N," and

the probability of incorrect performance of all tasks is

Pr[FIZD] = AB ... N (10-2)

where N is the probability of failure on activity "N."

10-21

Table 10-1

Table 10-1 General guidelines in assessing level of dependence

Item Guidelines

(1) Evaluate the influence of the failure or success of the immediately preceding task on the task of interest. The level of dependence may not remain constant throughout a series of activities.

(2) Use the higher of two levels of dependence when there is doubt about which of them is the more appropriate. This will result in a more conservative (i.e., higher) assessment of the failure proba- bility for the complete-failure path and only a slightly more optimistic (i.e., higher) assessment of the success probability for

the complete-success path.

(3) Evaluate the spatial and time relationships among all events.

Dependence between any two events increases as the *events occur

closer in space and time. For example, displays or controls that are physically close to each other or that must be manipulated at

about the same time have a higher level of dependence than items that are widely separated either spatially or as to the time of

their manipulations.

(4) Evaluate the functional relationships among events. Dependence be- tween any two events increases with their functional relatedness.

For example, events within a subsystem have a higher level of dependence with respect to each other than to events in some other

subsystem.

(5) Evaluate the effects of stress on dependence among personnel. Dependence increases with stress, and this tendency is more pro-

nounced for the dependence of lesser on more experienced personnel.

(Further discussion is presented in Chapters 12 and 18.)

(6) Evaluate the similarities among personnel with respect to all relevant factors. Dependence among personnel increases with simi-

larity in status, training, responsibility, and many social and psychological factors.

(7) Reevaluate the level of dependence assumed if the joint probability

of failure for two tasks is as low as 10 6 in a case in which one person is checking the other. Likewise, if the joint probability of failure of two related tasks performed by one person is as low

as 10-5, reevaluate the level of dependence assumed. Such low end- failure probabilities are possible but not usual.

10-23

Equations for ZD and CD

For the case in which the estimated failure (or success) probabilities for all n activities are equal, the joint HSP is

Pr(SjZD) = a- (10-3)

where a is the probability of successful performance for the first of n events with equal estimated success rates. The joint HEP becomes

n ] Pr[FIZD] f A ,. (10-4)

where A is the probability of error for the first of n events with equal estimated failure rates.

With CD among n events, the joint HSP is.

Pr[SICD] = a- (10-5)

where a is the estimated success probability for the first action in a sequence of n completely dependent events, whether or not the basic success probabilities of each event are equal. The joint HEP becomes

Pr[FICD] = A (10-6)

where A is the estimated failure probability for the first action in a sequence of n completely dependent events, whether or not the basic failure probabilities of each event are equal.

For the case in which it is not known which event will be performed first but in which all events have an equal opportunity of being first, if the estimated success (or error) probabilities for the n events are not equal, the arithmetic mean of the n probabilities is used:

Pr[SICD] = a + b + ... + n (10-7) *n

where n is the estimated probability of successful performance of the nth activity, and

Pr[FICD] = A + B + . +.N (10-8) n

where N is the estimated probability of incorrect performance of the nth activity.

In the HRA event tree (Figure 10-4), the above equations express the proba- bility of arriving at the end of either the complete-success path or the complete-failure path. In the case of ZD, the basic success or failure probabilities of each task are entered into the tree without modification. In. the case of CD, only the outermost limbs of the tree are used, since the conditional probabilities of success and failure would both be 1.0, and the first branching inner limbs would have values of zero. For the intermedi- ate levels of dependence, the conditional probabilities of success and

30-24

Intermediate Dependence Levels

failure for the inner branching limbs are obtained as described in the section "Dependence in' Other Paths" in this chapter.'

The definition of the term basic human error probability (BHEP) is repeated

here since this term is fundamental to calculating the conditional proba- bilities of all the limbs in an HRA event tree. The BHEP is the probabil- ity of a human' error on a' task that -i's considered as an isolated entity, unaffected by 'any other task. Except for cases in which the first branch- ing in a tree represents a carryover from another tree, the first failure limb in the tree is the BHEP of the first task considered. If ZD is

assumed, the HEPs assigned to all the failure limbs in the tree represent the BHEPs-for'the tasks.- If the assumption of ZD does not apply, the BHEPs

for succeeding failure limbs have to be modified to arrive at the appropri- ate conditional probabilities in accordance with the estimated level of

dependence. In Figure 10-4, for example, *the BHEPs for the limbs marked A and BIA might be equal, but, if CD is assumed, BIA becomes 1.0 rather than

B. For the intermediate levels of dependence, the BHEPs are modified to obtain conditional probabilities, as explained later.

The complement of the BHEP 'is" the basic human success probability (BHSP),

defined as 1.0 minus the BHEP. In'cases in which dependence exis'ts, the' BHSPs also must be modified in accordance with the appropriate level of dependence. Since our model deals only with positive dependence, these

modifications should be made on the complete-failure path when dealing with HEPs and on the' complete-success path when dealing with HSPs. In other

words, if there is dependence between events "A" and "B," the.effects' of dependence on event "B" should be estimated as BIA 'for HEPs or :bla for

HSPs. If Bla is the failure path of interest, it can be estimated by subtracting bla'.from""I.'0'. ' ..

Different Methods for Determining Intermediate Levels 'of D'ependence

The conditional HEP of one ta'sk, given failure or success on some other

task, is a function of the level of dependence between the two tasks. Our derivation of the intermediate levels of dependence results in the three intermediate points on-the continuum of dependence as shown in Figure 10-3.

These points were selected to divide the continuum of dependence among human activities that will lead to believable answers for the usual range

of conditional HEPs in NPP operations. Several different approaches were considered, including the geometric mean approach used in WASH-1400. These approaches' can be divided into two" types--ohe using nonlinear scaling and the other using linear scaling.

In the former method, the- Conditional HEP is defined on a nonlinear scale that has the probability corresponding to ZD as a lower bound and the value

1.0 as an upper bound. One' version of th~is method is a logarithmic scale, in'which the BHEP is expressed as BHEP = 1 0 x,l where x is the exponent of 10 that yields the 'BHEP". With a BHEP of I0-2, x = -2. For example, assume we

10-25

Intermediate Dependence Levels

have some basis to define the conditional HEPs as: Iox, 1 0 3/4x, I 0 1/2x, 101/4x, and 100 (or 1.0) for ZD, LD, MD, HD, and CD, respectively. Then, for a BHEP = .003 = 10-2.52, the conditional HEP for low dependence = 10 (3/4)(2.52) = .013.

There is a serious objection to this approach. When the BHEP is low, the method will generally yield unrealistically low joint HEPs. For example,

if the BHEPs for two tasks to be done in succession are both .003, the logarithmic method yields a joint HEP for the tasks of .000039 = .00004 for the low level of dependence. This probability is so small that one would often underestimate the influence of human errors.

The other approach to dividing the dependence continuum, in which the conditional HEPs are almost independent of the BHEP, is to select values of dependence on a linear scale between the BHEP (ZD) and 1.0 (CD). In an earlier application the continuum of dependence was divided into four roughly equal parts. LD was about one-fourth the distance from ZD to CD, MD about halfway between ZD and CD, and HD three-fourths of this distance. This approach yields estimates of .25 for LD, .50 for MD, and .75 for HD for the conditional HEPs of all tasks with BHEPs of .01 or less, given failure on the immediately preceding task. This method was discarded for two reasons. First, the difference in joint HEPs, given HD and CD condi- tions, was relatively small and made little difference in the results of a PRA. That is, a PRA was insensitive to whether HD or CD were assessed as the level of dependence. For two tasks with BHEPs of 10-3 each, the joint HEP would be .003 x .75 = .0025 if RD were used for a second task and .003 x 1.0 = .003 if CD were used for the second task. The factor of 1.3 dif- ference is seldom important in a PRA. The second reason for discarding the equal interval approach was because the difference in joint HEPs, given ZD or LD conditions, was much too large. For two tasks each with a BHEP of 10-3, given ZD, the conditional HEP for a second task given failure on the first task would be the BHEP for that task; i.e., 10 -, yielding a joint HEP of 106. Given LD, the conditional HEP would be approximately .25, and the joint HEP would be 2.5 x 10-4, a result very distant from the joint HEP

using ZD. Because of these two problems, it was decided to consider a different set of values.

The division of the dependence continuum shown in Figure 10-3 and the resultant values listed in Table 10-2 provide more realistic reliability assessments and have been chosen as interim values until a data-based dependence model can be developed. The problem remains of developing a dependence model that meets certain statistical requirements and that provides realistic estimates of joint probabilities of failure based on human factors considerations.

The "success equations" in Table 10-2 are used to calculate the conditional probabilities of success on Task "N," given success on the immediately preceding task, "N-I." The "failure equations" represent the conditional probabilities of failure on Task "N," given failure on the immediately preceding task, "N-i ." The failure equations were selected so as to pro- vide conditional probabilities of failure of about 5%, 15%, and 50% of the

I C 21C

Table 10-2 Equations for conditional probabilities of success and failure on Task "N," given

success or failure on previous Task "N-i," for different levels of dependence

Level of Dependence Success Equations Equation No. Failure Equations Equation No.

0 N -J

Pr [S,,N. IS.,N-11, I LD]

Pr S,,N,, Is,,N-1 11 IMD]

Pr[S,,Not Is.N-1'' I HD]

Pr [S,,N,, Is N_ ,CD]

1 + 19n 20

1 + 6n 7

1 + n 2

= 1.0

Pr [S,,N,, Is,,N-1, I ZD] (10-9)

(10-10)

(10-11)

(10-12)

(10-13)

Pr[F,, NIF,,N_ IZD] = N

Pr [FN I Few N I LD]

Pr [F IF,,N _ 1" I MD]

Pr[F,, I FN N-I HD]

1 + 19N 20

1 + 6N 7

1 +N 2

(10-14)

(10-15)

(10-16)

(10-17)

(10-18)Pr[F, N,,IF, NI,,ICD] = 1.0

Use of Dependence Equations

distance between the BHEP (ZD) and 1.0 (CD), for the low, moderate, and high levels of dependence when the BHEP is .01 or smaller. (The values for zero and complete dependence are, of course, fixed as the BHEP and 1.0, respectively.) With BHEPs > .01, the failure equations will yield higher conditional HEPs than .05, .15, and .5, as should be the case. The success equations were selected to provide an equivalent division of the dependence continuum between the BHSP and CD. When the BHSP is > .99, the conditional HSPs for any nonzero level of dependence must fall within a relatively small range of success probabilities (.99 to 1.0). Thus, for most prac- tical purposes, the conditional HSPs in such a situation are essentially unchanged by the effects of dependence on the BHSPs since the range available for the demonstration of such effects is quite small. As the BHSP increases above .99, the effects of dependence diminish further for conditional HSPs, due to the restriction in range. In such cases, the outcomes of risk assessments will not be affected significantly, except in some cases of first-order human failure terms.

One objection to this model, voiced by some reviewers, is that when the BHEPs are very low, the joint HEP for any nonzero level of dependence is much larger than the joint HEP for a zero level of dependence. Although our new model of dependence has reduced the magnitude of this difference as compared with the earlier equal interval approach, the disparity between the BHEP and conditional HEP of the second task still increases sharply as the BHEP decreases. For example, using LD and assuming a BHEP of .001, the conditional HEP is .05 with low dependence but .001 with zero dependence, a factor of 50 difference. With a BHEP of .1, the conditional HEP is .15 with LD but .1 with ZD, a factor of only 1.5 difference. As stated earlier, there is no psychological reason for this factor to remain con- stant as the BHEP decreases. The fact that the estimates of conditional HEPs are essentially unaffected by BHEPs < .01 reflects our judgment that the behavioral phenomena related to dependent tasks are substantially different from those related to tasks that are independent, that is, the dependent behavior induced by the contributing factors is a class of be- havior in itself.

Application of Dependence Equations

To illustrate an application of the equations, refer to Figure 10-4 and assume three tasks with BHEPs of A = .001, B = .003, and C = .005, with LD between the first two tasks and HD between the second and the third tasks.

To calculate the conditional probability BJA, Equation 10-15 from Table 10-2 is used:

Pr[BIAjLD] = 1 + 19(.003) = .05285 = .05

Since BjA + bJA = 1.0, bjA = .95.

10-28

Use of Dependence Equations

To calculate the conditional probability, CIBIA, Equation 10-17 is used*:

Pr[CIBIAIHD] + .005 .5025 .5

Since CIBIA + cIBIA = 1.0, cIBIA = .4975 = .5.

To calculate the conditional probability bla, Equation 10-10 is used:

PrbaILD] =1 + 19(.997) Pr=la 1 LD] 20 99715 = .997

Since bja + Bla = 1.0, Bla = .00285 = .003.

To calculate the conditional probability cibla, Equation 10-12 is used:

1 + .995=- = .9975 Pr~c~b~aJHD] 2 97

Since cjbla + CIbla = 1.0, CIbla = .0025.

The remainder of the Task "C" limbs are calculated as follows:

for the expression use the conditional probability equals

cjbIA cjb a .9975 cIBla cJB A .4975 CIB a C BIA .5025 CIbIA CIbla .0025

To calculate the joint HEP for the tasks that make up the complete-failure path in Figure 10-4, take the product of the probabilities assigned to the three failure limbs:

Pr[F] = (A x BIA x CIBIA) = .001 x .05285 x .5025 = 2.656 x 10-5 _ 3 x 10 -5

This value for the joint failure probability of these three actions, based on the levels of dependence assumed, can be compared with the joint failure probability of 1.5 x 10-8 that would be the product of the three BHEPs, assuming ZD. This would be an unbelievably low error probability for most human activities. Unless some reasonable estimate of dependence among human actions is used in calculating the effect of human errors on system reliability, one may grossly underestimate the effects of these errors.

To calculate the probability of the complete-success path in Figure 10-4, take the product of the probabilities assigned to the three success limbs in that path:

The calculation of the conditional HEP for a task is based on the influ- ence of the immediately preceding task only.

1 0-21

Use of Dependence Equations

Pr[S] = (a x bla x cibla) = .999 x .99715 x .9975 = .99366 = .994.

After the conditional probabilities have been assigned to all secondary limbs in the tree, the probability of any path through the tree is obtained by taking the product of the probabilities assigned to each limb in that path. Note that Pr[S] + Pr[F] < 1.0 because not all paths have been considered.

As indicated by the above examples, the level of dependence assigned be- tween successive limbs in the tree can vary. The equations permit complete flexibility in this regard. Also, the BHEPs or BHSPs for the various tasks can differ. Although a factor of 5 difference was used in the above hypo-

thetical example, quite often BHEPs are equal in a human reliability prob- lem in which dependence effects must be estimated, or they differ by a factor of less than 3. However, the equations are valid even when there are gross differences in BHEPs. For example, assume HD between Tasks "A" and "B" and assume that the BHEP for the first task is .01 and for the second it is .001. The conditional probability BIA becomes

1 + .001 A 2 - .5005 = .5

Should the BHEPs for the two tasks be reversed, the equation for BIA results in an estimate of .505 = .5. In both cases, the conditional proba- bilities of failure for Task "B" are essentially the same, but the joint probabilities of failure for the tasks differ by a factor of 10, depending on which task is performed first. That is, A x BIA = .01 x .5 = .005 for A = .01, but A x BIA = .001 x .5 = .0005 for A = .001.

From the failure equations in Table 10-2, it can be seen that as the BHEP gets smaller, the conditional HEPs for the succeeding tasks, given failure on the immediately preceding task, approach .05 for LD, .15 for MD, and .50 for HD. This was our purpose in developing those particular equations. Therefore, it is suggested that these rounded conditional HEPs be used as

point estimates for BHEPs of .01 or lower. For BHEPs higher than .01, refer to Table 10-3 for the conditional HEPs or apply the equations from Table 10-2.

Table 10-3 also shows a desirable characteristic of the dependence model. Note that with lower BHEPs the ratio of conditional HEPs given low de- pendence to conditional HEPs given zero dependence grows larger (see the F columns). Psychologically, this has some advantages. As noted in Chapter 6, our lack of confidence in estimates of very low HEPs is reflected in the large uncertainty bounds assigned to such estimates. When one assumes ZD between two events with low BHEPs, this uncertainty is increased. For example, if the BHEP for each of two tasks is estimated as .003, the joint probability of failure, assuming ZO, becomes 10-5 (rounded), an unbeliev- ably low figure for most human actions. If we substitute LD for ZD (usually a reasonable assumption), the resultant joint HEP of 10-4 (rounded) is relatively more realistic. In practice, when the joint HEPs are suspiciously low, we will question the estimates of the BHEPs, the

10-30

0 Table 10-3 Conditional probabilities of success or failure for Task "N" for the five levels

of dependence, given FAILURE on preceding Task "N-i"

Task "N" Conditional Probabilities*

I tem

(1)

(2)

(3) 0 wu

ZD**

S F

(a) (b)

.75 .25

.9 .1

.95 .05

.99 .01 t

.995 .005

.999 .001

.9995 .0005

.9999 .0001

.99999 .00001

S F

.71 .29

.85 .15

.9 .1

.94 .06

.95 .05

(4)

(5)

(6)

(7)

(8)

(9)

(e)

.64

.77

.81

.85

.86

F S

(f)

.36

.23

.19

.15

.14

.37

.45

.47

.49

.50

(h)

.63

.55

.53

.51

.50

F S

(i)

HD CD

(_j)

1.0

All conditional probabilities are rounded. to calculate the values in the F columns.

subtraction.

Equations The values

10-14 through 10-18 (Table 10-2) were used in the S columns were obtained by

(D The conditional probabilities given ZD are the basic probabilities for Task "N."

For PRA purposes, it is adequate to use CHEPs of .05 (for LD), .15 (for MD), and .5 (for HD)

when BHEP < .01.

Parallel-Series Application

level of dependence assessed, or both. Very low error probabilities indi- cate that the data may be suspect. Although very low error probabilities may indeed reflect an excellent design, further investigation is usually warranted.

In contrast to the above wide range of conditional HEPs, the range of values for conditional HSPs (given success on the immediately preceding task) will be restricted because most tasks in NPPs have a BHSP of at least .99. As shown in Table 10-4, for BHSPs of .99 and higher, there is not much difference in the conditional probabilities for Task "N" regardless of the level of dependence assumed.

The effects of dependence on system reliability are not unidirectional. In some cases, dependence between human activities will cause an increase in the overall probability of failure, while in others it will cause a de- crease. The following observations on the effects of dependence apply to HEPs < .5 (almost always the case).

For the situations in which all tasks have to be performed without error (the complete-success path in an event tree), the joint HSP increases as the level of dependence increases. The magnitude of the increase depends upon the size and relationship of the BHSPs. For example, for three tasks with BHSPs of a = .95, b = .99, and c = .999, assuming LD between b and a and between c and b, the joint HSP (using Table 10-4) is .95 x .99 x .999 .94. The total failure probability is 1 - .94 = .06. If we assume HD in the above situation, the joint HSP is .95 x .995 x .9995 = .945. The total failure probability is 1 - .945 = .055.

For BHSPs of a = .999, b = .99, and c = .95, the joint HSP, assuming LD, is .999 x .99 x .95 = .94. The total failure probability is 1 - .94 = .06. Assuming HD, the joint HSP is .999 x .995 x .97 = .964. The total failure probability is 1 - .964 = .036.

For the situation in which success is achieved if at least one task is per- formed without error (i.e., any path other than the complete-failure path), the probability of failure increases as the level of dependence increases. For example (using Table 10-3), for A = .05, B = .01, and C = .001, the probability of failure, ABC, increases from .05 x .06 x .05 2 x 10-4 for LD to .05 x .51 x .50 = 10-2 for HD. For A = .001, B = .01, and C = .05, the probability of failure increases from .001 x .06 x .1 = 6 x I0-6 for LD to .001 x .51 x .53 = 3 x 10-4 for HD. For more complex situations, the probability of failure may either increase or decrease with increasing dependence.

Application of the Dependence Model to a Parallel-Series System

To illustrate the application of the dependence model in a system context, two examples are offered, both of which are based on the system shown in Figure 10-6. The three equipment components are locally operated valves, and the three human tasks are to restore these valves to their normal

10-32

Table 10-4 Conditional probabilities of success or failure for Task "N" for the five levels of dependence, given SUCCESS on preceding Task "N-i"

Task "N" Conditional Probabilities*

ZD** LD MD

S F

HD CD

Item

(1)

(2)

(3)

(4)

(5)

(6)

(7)

(8)

(9)

(a)

.75

.95

.99

.995

.999

.9995

.9999

.99999

(b)

.25

.05

.01

.005

.001

.0005

.0001

.00001

F S

(c)

.76

.95

.99

.995

.999

.9995

.9999

.99999

(d)

.24

.05

.01

.005

.001

.0005

.0001

.00001

(e)

.79

.91

.94

.991

.996

.999

.9996

.99991

.999991

(f)

.21

.09

.06

.009

.004

.001

.0004

.00009

.000009

(g)

.87

.95

.97

.995

.997

.9995

.9997

.99995

.999995

(h)

.13

.05

.03

.005

.003

.0005

.0003

.00005

.000005

(i)

1.0

*All conditional probabilities are rounded. Equations 10-9 through 10-13 (Table 10-2) were used to calculate the values in the S columns. The values in the F columns were obtained by subtraction.

**The conditional probabilities, given ZD, are also the basic probabilities for Task "N." H

Parallel-Series Application

operating positions after maintenance. Any valve might fail to function on demand because of equipment defects or human failures. It is assumed that there is ZD between equipment and human failures, that there is ZD among equipment failures of the three valves, and that there is HD among the three human tasks, "A," "B," and "C," which are performed in that order.

For system success, the series leg (Valve #3) in Figure 10-6 and at least one of the parallel legs (Valve #1 or #2) must function properly. There- fore, there are three possible combinations of valve states leading to system success:

(1) All three valves succeed.

(2) Valve #1 succeeds, Valve #2 fails, and Valve #3 succeeds.

(3) Valve #1 fails, Valve #2 succeeds, and Valve #3 succeeds.

For any leg of the system, either a human failure, an equipment failure, or both, will cause that leg to fail. Thus, from the logic of the system, seven possible success paths can be constructed as follows:

Success Paths

1 E A x e b x e c 1 2 3

2 = eA x e2b x e3 c

3 =E a x eb x e c1 2 3

4 = e axE B x e c 1 2 3

5 = e a x e B x e c 1 2 3

6 =e axE b x e c1 2 3

7 =e a x e2b x e3c

For computational convenience, these seven paths can be reduced to three expressions that define the probability of system success as follows:

Pr[ST] = Pr[S 1 + Pr[S 2 + Pr[S 3

where Pr[S 1 is the sum of paths 3, 6, and 7 (equipment failures only or no failures); Pr[S 1 is the sum of paths 1 and 2 (human failures on the first2 parallel leg); and Pr[S ] is the sum of paths 4 and 5 (human failures on the second parallel leg). •herefore,

Pr[S ] = abc(E1e2e3 + eIE2 Ee + e1e2 ee ) + Ae 2be 3c(E1 + e ) + Be ae 3c(E2 + e2

Since (E + e ) and (E + e2) = 1.0, these terms can be dropped, leaving the following three expressions, with the terms rearranged to separate the human and equipment failure terms:

10- 34

Figure 10-6

VALVE #1

VALVE #3

INPUT "E3" "C" OUTPUT

VALVE #2

Figure 10-6 A parallel-series system with potential failures of

equipment (El, E 2 , and E 3 ) or of human tasks (A, B, or C).

10-35

Parallel-Series Application

S = abc(E1 e 2e3 + eE2e3 + e e 2e3

S2 = Abc x e2e3

S = aBc x e e

3 1 3

In the examples below, we have calculated the probability of system failure under the assumption of ZD among human actions as well as under the as- sumption of HD among human actions, the original premise. Note that in one case, the results are substantially the same whether or not dependence is considered but that in the other case the difference is appreciable.

Example No. 1

For this example, we assume that the BHEPs for all three tasks are .01 and that the equipment failure probabilities are all .001. If ZD is assumed, the approximate failure equation is Pr[F] = (E + A) (E2 + B) + (E3 + C) =

.011 x .011 + .011 = .011.

If HD is assumed for the human tasks, each success path must be calculated using values from Tables 10-3 and 10-4 for the human tasks:

Pr[S 1 = (a x bla x cjb) x [(E x e e ) + (E x e e 3) + e e 2e 3 1 1 23 2 13

= (.99 x .995 x .995) [(.001 x .9992) + (.001 x .999 ) + .999.]

.97914

Pr[S2] = A x bIA x cib x e 2 e 3 22

.01 x .49 x .995 x .999 - .00487

Pr[S3] = a x Bla x clB x ele 3

2 .99 x .005 x .49 x .999 = .00242

(It is conventional to postpone rounding of answers until the final answer

is reached.)

The total failure probability for the system, including both human and equipment contributions, is

Pr[FT] = 1 - Pr[S I + Pr[S I + Pr[S3] = 1 - .98643 = .014

T 1 21 3

In this particular example, the practical effects of dependence were negli-

gible (.014 vs .011).

Example No. 2

For this example, we will continue the assumption of .001 for the indivi- dual component failure probabilities and an HEP of .01 for Tasks "A" and

10-36

Parallel-Series Application

"B," but will assume an HEP of .05 for Task "C." If ZD is assumed, the approximate failure equation is

Pr[F T] = (E + A) (E2 + B) + (E3 + C) = .011 x .011 + .051 = .051

If HD is assumed for the human tasks, the success paths are calculated as before, using the appropriate HD values for the BHEP = .05 from Tables 10-3 and 10-4 as follows:

Pr[S 1 (a x bla x cjb) x [(E x e e ) + (E x e e ) + e e e 21 23 2 13 123

- (.99 x .995 x .97) [(.001 x .999 2) + (.001 x .999 ) + .999 3

- .95454

Pr[S2] = A x bJA x cjb x e2 e3

2 - .01 x .49 x .97 x .999 = .00474

Pr[S3] = a x BSa x cB x e e3

2 = .99 x .005 x .47 x .999 = .00232

The total failure probability for the system, including both human and equipment contributions, is

PrF T] = 1 - (.95454 + .00474 + .00232) = 1 - .96161 = .038

In this example, failure to consider the effects of dependence yields a

system failure estimate that is pessimistic by a factor of .051/.038 = 1.3. In other cases, failure to use the appropriate level of dependence could produce overly optimistic results. Since there is no convenient method to predetermine whether dependence will have a substantial effect on system reliability, the effects of dependence should be calculated for each case.

An Example of Sensitivity Analysis

Examples 1 and 2 illustrate sensitivity analyses in that several human per- formance measures were varied to determine their effects on overall system reliability. Example 1 showed that the difference in system outcome be- tween the assumptions of ZD or HD was inconsequential. One can say, then, that the total system failure probability is insensitive to the level of dependence in the range of ZD to HD for the postulated HEPs. If the factor of 1.3 difference is considered significant, Example 2 illustrates that the system failure probability is sensitive to the levels of dependence assumed when the hypothesized HEP for Task "C" is .05 instead of .01.

Finally, the two examples show how sensitive the system outcome was to an increase by a factor of 5 in the BHEP for Task "C." This increase in BHEP increased the total system failure probability from .014 to .038, roughly a factor of 3. In some analyses, this increase would be considered negligi- ble; in others, it could be important.

10-37

Uncertainty Bounding

Because of the large element of subjectivity in assessing levels of depen- dence in a PRA, it is important that sensitivity analysis be used to pro- vide a check on this judgment. The basic method for sensitivity analysis discussed in Chapter 5 can be applied also to estimates of dependence. If the analyst believes that an appropriate level of dependence for some HRA problem is "probably LD but may be MD," sensitivity analysis can be used in the PRA to see whether one level or the other would make any material difference to the outcome of the PRA.

Uncertainty Bounding in Assessing Dependence

All of the discussion and examples in this chapter have dealt with single- point estimates of HEPs. In many PRAs, it is necessary not only to provide these single-point HEPs but to consider them as measures of some central tendency and then to assign uncertainty bounds (UCBs) about them. Chapter 7 presents a procedure for assigning uncertainty bounds for conditional HEPs based on the dependence model. Table 7-3 presents approximate condi- tional HEPs and their UCBs for the different dependence levels. Appendix A presents the procedure in the context of propagating uncertainty bounds of all HEPs through an HRA event tree.

10-38

Ch. 11. Displays Overview; Assumptions & Terms

CHAPTER 11. DISPLAYS

Overview

This chapter presents some of the background and assumptions for estimating errors of omission and commission in reading various types of displays in nuclear power plant (NPP) control rooms. Most of the text is addressed to conventional displays, such as meters and graphic recorders, which are used in most U.S. plants currently in operation. The latter part of the chapter describes some contemporary displays, based largely on cathode-ray tubes (CRTs), that are in use in other countries and are being designed into newer plants and retrofitted into older ones in the U.S.

Specific human performance models for annunciated and unannunciated conven- tional displays are presented, and human error probabilities (HEPs) derived from the models are listed. Although this chapter is primarily addressed to displays in control rooms, many of the statements and estimates apply equally well to displays in other areas. The analyst must assess the relevant performance shaping factors (PSFs) and make extrapolations accord- ingly.

No attempt has been made to differentiate among the different design ver- sions of any given type of display with regard to their influence on human error. For example, the typical fixed-scale, moving-pointer analog meter may be vertical, horizontal, curved, or other form. While these meters may differ in their susceptibility to reading errors, the data are so sparse that it is not possible to substantiate such differentiation.

In general, NPP displays are not optimal for reducing human error, and in some cases the design of the display is conducive to error. Our perfor- mance models are based on current, typical designs in light water reactors (LWRs) and may be conservative by a factor of about 2 to 10 when applied to displays that conform to accepted human factors design practices.

Following some assumptions and definitions of terms, this chapter is divided into three major parts. Part 1 pertains to the use of unannun- ciated displays, including estimated HEPs and models for reading, check- reading, and informal scanning. Part 2 deals with annunciated displays, including an annunciator response model relating the probability of ini- tiating intended corrective action to the number of annunciators competing for the attention of an operator. Part 3 states some observations on the future role of new display techniques in NPPs.

Some Basic Assumptions and Terms

There are hundreds of displays in a typical NPP control room. Most of the displays are unannunciated and indicate the moment-by-moment status of selected plant parameters. They are the displays the operator uses to run

11-1

Assumptions & Terms

the plant. They include meters, counters, chart recorders, computer print- outs, CRTs, digital indicators, status lamps, and legend lights. Status lamps are simple YES-NO type indicators, displaying the state of some component or subsystem, such as ON for power, OPEN or CLOSED for a valve, and so on. Status lamps are usually positioned above, under, or next to the printed status word on a panel. Legend lights are transilluminated displays that illuminate a message or label, indicating that some component or subsystem is in some particular mode. In a few plants, the lights are color coded to indicate whether or not the mode is desirable.

A very important class of legend light is the annunciated legend light--an indicator that directs the operator's attention to some specific component or subsystem. Often the annunciated indicator conveys a message requiring the immediate attention of the operator. The message may be, for example, "Radiation Level High." This class of legend light is annunciated by some type of compelling sound, such as a horn, and also by a flasher circuit, which causes the legend light to flash on and off. In most plants, two rates of flashing are used: one rate for the initiation of the annunciator legend light and a slower rate to indicate that the problem has been cor- rected. By strict definition, the horn or buzzer that sounds the onset of the legend light is the annunciator, but it has become common usage to refer to the light itself as the annunciator. Since it is less cumbersome to use the short form annunciator instead of annunciated legend light, we will follow that usage. Another commonly used term is annunciator tile.

In most control rooms, the annunciators are grouped in arrays above the other control/display panels and are considerably above eye level. The status lamps, digital readouts, meters, and other displays are mounted on a variety of panels--some vertical, some horizontal, some slanted. The layout of the control/display panels in control rooms often increases the complexity of the operator's job and reduces the reliability of his per- formance.

Another term used in describing displays has to do with the frequency with which they are likely to change their indicators. Dynamic displays are those that an operator refers to in the course of normal operations and which may change frequently in accord with plant functions. Stable dis- plays are those that normally do not change their indications in the course of a shift. Some stable displays are not checked routinely during a shift. Others may be scanned during the initial audit and disregarded thereafter.

When a new shift begins, it is assumed that the oncoming operator will conduct an initial survey or initial audit of all the control panels and will then monitor them throughout the shift to note any deviant displays or deviant manual controls. Part of the initial audit may involve the use of a written checklist; in this case, the operator is to verify that certain status lamps are lit or that certain analog readings are within limits, or he might write down some quantitative value. We restrict the equivalent terms initial survey and initial audit to inspections performed without the use of written materials. If written materials are used, e.g., a written checklist for the operator to use, the initial audit is designated as a

11-2

Assumptions & Terms

written control room audit. For the latter type of inspection, the HEPs for errors of omission are estimated by use of the written instruction models in Chapter 15. For the initial audit without use of written materials, we make the assumptions that any deviant display will be in the operator's visual field; therefore, if he fails to note a deviant display in the initial audit, it is classified as an error of commission. He "saw" the deviant display, but he failed to discriminate that the indication on the display was deviant.

In the initial audit without written checklists, the operator acquaints himself with the status of the plant via the control room instruments. Since his primary interest is in maintaining power to the grid, he will pay most attention to displays directly related to this function. Thus, although he will not deliberately ignore displays related to other func- tions, such as safety- related displays, he is likely to ailot less scan- ning and monitoring time to them. The low incidence of failure indications in NPPs naturally lowers his expectancy of finding any deviant safety- related displays.

At some plants, the control room operator must manually log about 15 to 20 specific parameters every 1 hour, 2 hours, or some other period. Some of these parameters relate directly to system safety, e.g., pressurizer level. At other plants, manual logging is not required and reliance is placed on computer printouts. Obviously, the second practice reduces the likelihood that the operator will detect some deviant parameter before it causes an annunciator to alarm.

In this Handbook, the primary interest is in safety-related equipment and systems. The estimates of scanning efficiency pertain to safety-related displays that are not used in the moment-by-moment running of the plant. We assume that scanning and monitoring efficiency is less for solely safety-related displays than for displays related to keeping the plant on line. Even though most safety-related functions are annunciated, it is important for control room personnel to scan the related unannunciated displays to anticipate potential trouble and take preventive action as early as possible. For example, a safety-related function is moving toward some value at which automatic equipment normally takes over. In case of automatic equipment failure, the alert operator will be readier to initiate manual action than if his first warning of trouble is an annunciator.

For routine detection of unannunciated deviant displays, reliance must be placed on the frequency and reliability of the scanning patterns used by control room personnel. The models for unannunciated dynamic displays presented later are based on the assumption of hourly scanning. For some stable displays, we assumed one check per shift during the initial audit. For other stable displays, those which change rarely, the assumption of even one check per shift might be optimistic. These assumptions are based on interviews with control room personnel.

11-3

PART 1. UNANNUNCIATED DISPLAYS Major PSFs

PART 1. UNANNUNCIATED DISPLAYS

This part describes some PSFs influencing the use of unannunciated displays in NPPs and presents models and HEP estimates related to the three major uses of these displays: reading for quantitative information, check-read- ing for qualitative information, and the periodic scanning of displays for deviant indications or trends. We restrict the term scanning to the case in which no written materials are used to direct the operator's attention to specific displays.

Major PSFs for Unannunciated Displays

In deriving estimates of HEPs for unannunciated displays in the control room, the most relevant PSFs are the following:

(1) Stress level of the operator

(2) Rate at which the operator must process signals

(3) Frequency with which a particular display is scanned

(4) Whether a written checklist is used to direct the operator-to specific displays

(5) Relationship of the displays to annunciators or other attention- getting devices

(6) Extent to which the information needed for operator decisions and actions is displayed directly

(7) Human factors engineering related to the design and arrangement of the displays

For the first PSF, the power-generating mode of the NPP is assumed, with the operators functioning at a low-to-optimal level of stress (Figure 3-4). This range of stress is assumed for routine tasks of maintenance and cali- bration also. All HEPs in this part are premised on the optimal level of stress and must be adjusted if different levels are assessed in an analysis.

The second PSF relates to the first in that a requirement for rapid pro- cessing of information is often associated with a higher-than-usual level of stress. Usually best performance is obtained when the task is self- paced; i.e., the person proceeds at a comfortable pace. If the person has to function at a much higher rate, he is more error-prone. A requirement for processing signals at a high rate is a form of stress--time stress. Time stress does not affect performance in the same way as emotional stress; time stress usually leads to errors of omission, whereas emotional stress may result in much more severe incapacitation. In the power- generating mode of a plant, the stress level of the operator varies between

11-4

Selection Errors

low and optimal, and his required signal-processing rate is comfortable since he has ample time to read and interpret the displays.

The third PSF, scanning frequency, is a function of the relative importance of the display as perceived by the operator, the context in which it is being read, and specified procedures.

The fourth PSF, use of a checklist, refers to the availability and use of written procedures to ensure that certain displays are checked, and that the correct values of the displays are clearly indicated.

The fifth PSF, relationship to attention-getting devices, refers to the phenomenon that even though a deviant display is not annunciated, attention will probably be drawn to it if it is part of an annunciated subsystem. In many cases, an unannunciated display and a related annunciator may be considered to constitute a perceptual unit. If the annunciator is noticed, the operator will also look at the related unannunciated display.

The sixth PSF relates to the content of the information displayed. If the operator has no direct indication of certain functions, he will have to deduce the information from other displays; a problem in the Three Mile Island (TMI) incident was the absence of a direct indication of emergency coolant flow from the Auxiliary Feedwater System (AFWS). Interpretation errors are more likely with this type of design, especially under stressful conditions.

The seventh PSF, human factors engineering related to the design and place- ment of displays, is highly variable in NPPs. Generally, there has been no systematic application of this technology to the designs of currently operating U.S. NPPs. We estimate that the HEPs listed in the Handbook will be reduced by factors of 2 to 10 if the displays and controls are improved by the incorporation of standard human engineering concepts such as those described in Seminara, Eckert, et al (1979). Table 11-1 lists some of the human engineering deficiencies in operating U.S. plants, based on the review by Seminara et al (1976) and our own observations.

For a complete reliability analysis, the influence of dependence must be considered, as described in Chapter 10, and errors of both omission and commission must be treated. Our discussion of unannunciated displays deals only with errors of commission; errors of omission for these displays are described in Chapters 15 and 16.

Selecting an Incorrect Unannunciated Display

Because of the many similar-appearing displays in the control room of an NPP, errors of selection must be assessed in a human reliability analysis (HRA). Some displays are distinctive in appearance, or physically distant from similar-appearing displays, and selection errors would usually be negligible. This supposes that when an operator begins his search for a display, he is aware of the physical characteristics of that display. In other cases in which selection errors would be expected to occur, the

11-5

Table 11-1

Table 11-1 Human factors engineering deficiencies of displays in nuclear power plants*

(1) Displayed information that is primarily of actual status only; normal status usually is not indicated

(2) Poorly designed scales, and scale numeral progressions that are difficult to interpret; e.g., one scale division equals 0.5 units

(3) Parallax problems in relating pointers to scale markings and numerals on meters

(4) Placement of meters and recorders above or below eye level, making the upper or lower segment of the scale difficult to read

(5) Meters or recorders that can fail with the pointer reading in the normal operating band of the scale

(6) Glare and reflections

(7) Too many channels of information on chart recorders

(8) Illegible pen tracings or symbols on chart recorders

(9) No warning before a chart recorder pen runs out of ink

(10) Use of chart recorders where meters or digital readouts would be more appropriate, e.g., where lags in data can result in wrong decisions

(11) Functionally related displays that are widely separated physically

(12) Inconsistent coding and labeling among displays

(13) Mirror-imaging of control rooms

(14) Lack of limit marks on meters used for check-reading

(15) Meters not arranged with "normal" segments in the same relative positions (to facilitate check-reading)

(16) Displays and arrangements do not present the operator with a mental image of what is going on

Source: Seminara et al, 1976. Most of these deficiencies apply only to unannunciated displays; some also apply to annunciated displays.

0 11--C

Reading/Recording Quantitative Information

content of the displayed information would provide an immediate cue that an error had been made, and the probability of such a selection error remain- ing uncorrected would be assessed as negligible.

The most likely candidates for unrecovered errors of selection are the pairs of green and red status lamps. On some panels, there may be more than 100 such pairs of lamps, identified by labels only, and the probabil-

iyof a selection error is significant. Similarly, selection errors are likely to occur when attempting to get information from a chart recorder in a bank of similar-appearing chart recorders or from an analog meter in a row of similar-appearing analog meters. Use of functional groupings of displays or displays that are parts of mimic lines on a panel will reduce selection errors.

Table 11-2 lists estimated HEPs and error factors (EFs) for selecting unan-

nunciated displays during the course of noting quantitative or qualitative indications from these displays. Selection errors for annunciated displays are discussed in Part 2 of this chapter.

Reading and Recording Quantitative Information from Displays

In the normal power-generating mode, there will not be many deviant in- dications on displays other than those already tagged. Although we are primarily concerned with the probability of failure to detect deviant indications on displays in the control room (a scanning or a check-reading function), an operator sometimes has to read an exact value from a display. The following sections present HEPs and EFs related to the reading and recording of quantitative information. The estimated HEPs and EFs for these cases are listed in Table 11-3.

A given error of commission may or may not have any system-significance. For example, in reading a value from a digital display, a calibration technician may erroneously read 1-2-3-5 instead of 1-2-3-4. In such a case, the error usually will not be important. On the other hand, if he erroneously reads 1-4-2-3, *the error might have serious consequences. In performing a human reliability analysis, one must identify the important errors and estimate probabilities for them. As an approximation, we may assume that all possible reading errors are equally likely. Thus, in the above example, the technician has an equal likelihood of reading any digit incorrectly. If one is not interested in errors in the last digit of the four, the basic reading error for 4-digit digital readouts of .001, de- scribed below, can be multiplied by .75 to yield .00075 for the three digits of interest. (In practice, we would round this .00075 back to .001, since our HEPs are too inexact to permit such fine distinctions.) The

assumption of equal probability for all possible reading errors was made for convenience; the error probabilities are affected by familiarity, expectancy, arid other factors, which vary with the situation.

1-7

Table 11-2

Table 11-2 Estimated probabilities of errors in selecting unannunciated displays for quantitative or qualitative readings

Item Selection of Wrong Display: HEP* EF

(1) when it is dissimilar to adjacent displays** Negligible

(2) from similar-appearing displays when they are .0005 10 on a panel with clearly drawn mimic lines that include the displays

(3) from similar-appearing displays that are part .001 3 of well-delineated functional groups on a panel

(4) from an array of similar-appearing displays .003 3 identified by labels only

The listed HEPs are independent of recovery factors. In some cases, the content of the quantitative or qualitative indication from an in- correct display may provide immediate feedback of the selection error-, and the total error can be assessed as negligible.

This assumes the operator knows the characteristics of the display for which he is searching.

Table 11-3

Table 11-3 Estimated HEPs for errors of commission in reading and recording quantitative information from unannunciated displays

Item Display or Task HEP* EF

(1) Analog meter .003 3

(2) Digital readout (4 4 digits) .001 3

(3) Chart recorder .006 3

(4) Printing recorder with large .05 5 number of parameters

(5) Graphs .01 3

(6) Values from indicator lamps .001 3 that are used as quanti- tative displays

(7) Recognize that an instrument .1 5 being read is jammed, if there are no indicators to alert the user

Recording task: Number of digits or letters** to be recorded

(8) ( 3 Negligible -

(9) > 3 .001 (per 3 symbol)

(10) Simple arithmetic calcula- .01 3 tions with or without calculators

(11) Detect out-of-range .05 5 arithmetic calculations

Multiply HEPs by 10 for reading quantitative values under a high level of stress if the design violates a strong popula- tional stereotype; e.g., a horizontal analog meter in which values increase from right to left.

In this case, "letters" refer to those that convey no mean-

ing. Groups of letters such as MOV do convey meaning, and the recording HEP is considered to be negligible.

11-9

Reading/Recording Quantitative Information

Meters and Digital Readouts

Based on data reported in Chapter 6, the estimated probability of a reading error is .003 for analog meters and .001 for four-digit digital readouts.

Chart and Printing Recorders

Most chart recorders are analog displays with the added feature of pro- viding a record of the monitored parameters. Thus, they indicate the recent history of each parameter and enable the user to note any trends. The accuracy of chart recorders is somewhat less than that of well-designed panel meters of the same size (because of scale differences, pen lag, line width, etc.), but this is of minor consequence. Ordinarily, we would as- sumne an HEP for reading chart recorders that would be only slightly greater than the HEP for reading panel meters. However, in many NPPs, chart re- corders are considerably more difficult to read than are comparable panel meters because of positioning, scaling, chart characteristics, pen charac- teristics, or multiplexing of data. Also, the pens of chart recorders are more apt to stick than the pointers of panel meters. The extent to which these disadvantages combine naturally varies; as a working estimate for reading chart recorders we suggest doubling the estimated HEP of .003 for reading meters to .006.

In addition to pen-writing chart recorders, there are printing recorders that periodically stamp a number on the chart. This number corresponds to the monitored parameter, and the position where it is printed corresponds to the value of the parameter. A large number of parameters can be re- corded on a chart. Recorders of this type are particularly susceptible to reading errors because of faulty registration, faulty printing, double printing, and differences in scale associated with the different param- eters. Based on these difficulties, the HEP for reading recorders of this type is estimated to be .05. Most of these errors are due to confusing the identifying numbers (or letters) of the parameters.

Graphs

Graphs are not used very much in NPPs. When they are, the operators freely use any aids they desire, such as rulers, pencils, etc. It is difficult to read a graph with precision, but it is unusual for graphs to be used where precise interpolation is required. The estimated HEP for reading graphs is .01.

Indicator Lamps

Occasionally, a set of status lamps is used as a quantitative display. For example, in one plant the containment sump level is indicated by five lamps labeled 4, 18, 48, 64, and 205 inches (Figure 3-15). As water in the sump rises, the lamps light in sequence to indicate the water level. Under normal operating conditions, this type of indication should be relatively free of human error even though the design violates a populational stereo-

1 1-10

Arithmetic Calculations

type, in that the lamps are arranged in reverse order, with the 4 on top and the 205 on bottom. We assess the reading error for normal plant condi- tions to be the same as that for reading a digital readout, .001. Because it does violate a populational stereotype, we multiply the HEP of .001 by 10 for reading under stressful conditions.

"Funneling" of Attention in Use of Displays

In responding to a display, an operator may focus his attention on a par- ticular display to the exclusion of all others. Often, the operator will initiate the action indicated and then concentrate his attention on that display, to the exclusion of other displays, waiting for a change in the readout. This "funneling" of attention, sometimes called perceptual tun- neling, is more likely to occur when the operator is under stress.

An occasional display malfunction is known as "sticking", i.e., a pointer on a meter or a pen on a chart recorder jams for some reason and no longer yields useful information. Usually, there are several redundant displays for any significant parameter, and the operator can refer to any of them until the primary display is repaired. However, there is a strong tendency to focus on the first display without cross-checking. Because of the operator's involvement in the corrective action to be taken, this is most likely to occur when the display sticks in a position indicating the need for immediate corrective action. It is less likely to occur when the sticking display does not indicate a need for immediate action because the operator will be scanning the associated displays as well. When an oper- ator uses an instrument that has jammed without any indication to that effect, we estimate a probability of .1 that he will fail to cross-check until some other indication, such as an alarm, alerts him that something is amiss.

Recording Errors

If readings are to be recorded manually, there is an estimated error proba- bility of .001 per symbol in recording the reading if more than three digits or letters are to be written down. In this case, "letters" refer to those that convey no meaning and are used simply as designators. Groups of letters constituting an abbreviation, such as "MOV" (motor-operated valve), do convey meaning, and the recording HEP for such groups is considered to be negligible. For less than four letters or digits, the recording error probability is judged to be negligible.

Arithmetic Calculations

Simple arithmetic calculations are often required for calibration and maintenance tasks in NPP operations. These calculations may be done using paper and pencil or may be done using a calculator. In the Handbook, an HEP of .01 (EF = 3) is suggested as an overall estimate of error in arriv- ing at answers from simple arithmetic calculations, without considering recovery factors and whether the calculations are done with paper and pencil or with a calculator. The rationale for this estimate follows.

1 1-11

Arithmetic Calculations

Calculations Without Use of a Calculator

Planners of work operations often overestimate the accuracy with which technical people may perform routine arithmetic calculations. The rela- tively large HEPs for such calculations is illustrated by a study in which experienced inspectors measured the locations of holes in a calibrated test plate (Rigby and Edelman, 1968b). Using micrometer-type instruments, 12 inspectors measured X and Y coordinates for each of 9 holes, for a total of 216 6-digit data points. Typically, each inspector made more than one measurement for each value, so that there were different numbers of oppor- tunities for each inspector to make an error. When taking measurements, the inspector had to read values from a gauge and a meter and assign a plus or minus sign as appropriate. As he read each value, he wrote it on a work sheet and then performed additions, subtractions, and divisions to arrive at a six-digit value for an X or Y coordinate. In performing this simple arithmetic, the inspectors made a total of 9 errors in 698 opportunities, an HEP of about .01.

In earlier studies, HEPs per digit of about .002 for addition, subtraction,

and division, and about .003 for multiplication were obtained for univer- sity students acting as experimental subjects (Weldon, 1956; Weldon et al, 1955). In a study of naval cadets, HEPs per digit of about .007 for addi- tion and .016 for subtraction were obtained (Trumbull, 1952a and b). Obviously, the experimental conditions and subjects are different for the three studies. In the studies using students and naval cadets, speed was emphasized along with accuracy, and their only task was the set of required computations. In the inspector study, accuracy was emphasized, and the calculations were only one part of the task performed.

Calculations Using a Calculator

A search of the literature failed to reveal any data on errors in simple arithmetic when a calculator is used. We contacted educators (and others) who might have had some data on HEPs, but no records had been kept. We had thought that the use of calculators would result in lower HEPs for simple arithmetic problems, but the consensus of those we interviewed is that such an improvement would apply only in the case of those who use calculators frequently in their everyday occupation. Several educators noted that students who used hand calculators only occasionally tend to believe what the device displays, without performing a mental check to see if the result displayed is reasonable. Thus, simple errors in entering digits into the keyboard (Table 13-3) are not usually caught unless the calculation is

repeated (a recovery factor).

Based on the above information, we suggest that the basic HEP of .01 (EF = 3) for arithmetic calculations should be used for calculations with cal-

culators also, exclusive of recovery factors.

1 1-12

Recovery Factors for Quantitative Reading Errors

Recovery Factors for Arithmetic Calculations

In the performance of calibration and maintenance tasks in NPPs, we assume that the person carrying out the arithmetic calculations is aware of the approximate value that he should obtain. Therefore, if a calculation re- sults in some value which is grossly deviant from the value he would expect to use, a skilled person would have a good chance of realizing his error. For example, if the actual value for a setpoint is 8352, and his calcula- tion results in 4352, the calibration technician should have a high proba- bility of detecting this error. On the other hand, if the error results in the value 8355, he would very likely accept this value as reasonable, and presume that it is correct.

In the first case above, in which the values differ by 4000 units, the probability of an attentive technician failing to detect the error would be negligibly small. We have made the assumption that the person knows the approximate value that he should obtain, and any value that falls outside the expected range would be suspect. However, there will be times when the technician may be inattentive, or distracted, and might fail to notice the gross error. For lack of data, we suggest an HEP of .05 (EF = 5) for fail- ure to recover from calculational errors that would result in values out- side of the expected range. Even if the technician inattentively entered the incorrect value, the immediate consequences might often serve as recov- ery factors: the incorrect value might be out of range for the equipment or might require an inordinate amount of compensatory activity, alerting the technician to his error.

Some Recovery Factors for Errors in Quantitative Readings

Errors in reading displays can be recovered in several ways. Two of the most common are discussed in the following two sections. The first relates to the operator's expectancy of what the reading should be. The second is the recovery obtained when two people perform a reading task, i.e., the use of human redundancy.

Recovery Factors Based on Operator Expectancy

Note that the HEPs presented above apply to the probability that the error will occur at all--they do not consider the operation of recovery factors that can alert the user to his error. For example, the estimated HEP for recording four digits is .001; this includes the most common error, that of transposition--a person reads 3821 and writes 8321. If the entry is to be used immediately, as when calibrating an item, the resulting figure is so obviously deviant that usually it would be questioned and corrected. Simi- larly, transposing the two middle digits results in a grossly deviant figure. When we arrive at the last two digits, the error may or may not be obvious, depending upon the significance attached to them. If they are as significant as the others, the error usually will be noticed. If not, the error itself may not matter, as when a data form requires more accuracy than the situation warrants. The important concept about errors of the

11-13

Check-Reading Displays

type discussed in this section is that gross errors in taking data from displays usually will be noticed. However, in a preliminary probabilistic risk assessment (PRA), we often take the conservative position that recovery factors will not operate when needed. In the final PRA, recovery factors must be included.

The Use of Human Redundancy in Quantitative Readings

If two people act as a team in reading and recording display indications, the error probability will be a function of the procedures followed by the team. For example, the benefits of human redundancy are maximal if one operator reads the indication aloud, the other operator records it, and then they both check each other.

Consider the opposite case in which the team members do not check each other--for example, if one operator reads the indication aloud and the other records it without any checking. In thi's situation, we would not allow for any benefits of human redundancy. It is possible, of course, that if the reader makes a gross error, the person doing the recording might notice it. However, this type of team interaction is so passive that we assess the probability of a wrong entry as the sum of the reading and recording errors. There also is a possibility of an error of communication between the team members, but this type of error is considered to be negli- gible (Chapter 15).

The highest reliability would be attained if the two people read and recorded individually on separate sheets. In such a case, complete inde- pendence of the two team members might be assumed, and the error probabil- ity for an individual would be squared if there were some error-free com- parison of the two records.

Team members tend to have an alerting effect on each other in actual prac-

tice, and usually there is some informal checking on each other even if not required by written procedures, so that the performance of teams will usually be more reliable than that of individuals. Often, the reliability increases because a team is more likely to follow plant policies than is an individual working alone. We are unable to estimate the quantitative value of this informal recovery factor because it is highly variable. To be con- servative, we assign no recovery factor for this effect.

Although the interaction of team members will vary widely, we suggest an assumption of high dependence between team members as a conservative esti- mate of the benefits of redundancy in reading and recording display indi- cations, applying Equation 10-17 in Table 10-2 or the tabled values in Table 10-3.

Check-Reading Displays

In many cases, displays are merely "checked" rather than read quantita- tively. That is, the operator refers to the display to determine that it

il-14

Check-Reading Displays

is within certain limits, rather than to determine the exact value of the reading. The check-reading may be merely to note go/no-go indications such as which indicator light is on (e.g., is it a red light, or is it a green light) or whether the pointer is still within the acceptable range on a meter. At other times, the check-reading may require more detailed quali- tative discrimination, e.g., has the pointer moved upwards since the last time it was checked? The following sections describe check-reading of meters, digital readouts, chart recorders, and indicator lights. For some situations, display changes may occur too rapidly on a digital indicator for it to be used effectively to note whether the reading is acceptable. The estimated HEPs associated with these tasks are listed in Table 11-4. The HEP estimates for check-reading apply to displays that are checked in- dividually for some specific purpose, such as a scheduled requirement, or in response to some developing situation involving that display. For this reason, these HEPs are much smaller than the ones related to the passive periodic scanning of the control boards, discussed later.ý The check- reading may be done from memory, or a written checklist may be used. Use or nonuse of a written checklist has implications for errors of omission, as described in Chapter 15. If check-reading is done from memory, it is assumed that the operator knows the correct indications, and the HEPs for errors of commission in Table 11-4 reflect the total of all errors of this type.

Check-Reading Meters and Chart Recorders

Check-reading meters and chart recorders is facilitated by the use of limit marks to indicate the limits of acceptable readings. The "red lines" on tachometers are a familiar example.

The estimated HEP for check-reading meters is .003. This applies to meters without limit marks. If there are easily visible limit marks, we estimate that the error probability is reduced by a factor of 3, to .001. For analog-type chart recordexs, the above HEPs are doubled to .006 for charts without limit marks and to .002 for charts with limit marks.

In most NPPs, the meters used for check-reading are purchased and installed without limit marks. However, the operators usually add informal limit marks in the form of tape, grease-pencil lines, etc., that are almost as effective as factory-printed limit marks. At some plants, management policies do not allow the use of limit marks on meters, and the operators resort to the use of "invisible" limit marks, such as fine scribe lines. Since these are not easily seen, we assign an HEP between the two values above, .002.

Digital Readouts

With analog displays, check-reading does not involve a quantitative reading -- the indication is either "in" or "out." The relative position of the pointer provides all the required information. With digital displays, there are no such positional cues--the display must be read, so the esti- mated check-reading HEP for digital displays is the same as for their quantitative readings--.001.

11-IS

Table 11-4

Table 11-4 Estimated HEPs for errors of commission in check-reading displays*

Item Display or Task HEP EF

(1) Digital indicators (these .001 3 must be read - there is no

true check-reading function for digital displays)

Analog meters:

(2) with easily seen limit marks .001 3

(3) with difficult-to-see limit .002 3 marks, such as scribe lines

(4) without limit marks .003 3

Analog-type chart recorders:

(5) with limit marks .002 3

(6) without limit marks .006 3

(7) Confirming a status change Negligible** on a status lamp

(8) Misinterpreting the indi- Negligible

cation on the indicator lamps

"Check-reading" means reference to a display merely to see if the indication is within allowable limits; no quantitative reading is taken. The check-reading may be done from memory or a written checklist may be used. The HEPs apply to dis- plays that are checked individually for some specific pur- pose, such as a scheduled requirement, or in response to some developing situation involving that display.

If operator must hold a switch in a spring-loaded position until a status lamp lights, use HEP = .003 (EF = 3), from Table 13-3, item 10.

For levels of stress higher than optimal, use .001 (EF = 3).

11-16

Check-Reading Displays

Check-Reading Indicator Lights

Indicator lights are used to indicate the state of some component or sub- system. The discussion that follows assumes either a transilluminated legend light or a plain lamp with a colored cover and a label above or below the lamp.

In certain applications in NPPs, color conventions are observed. For example, valve states are usually indicated by red for open and green for closed. In a few cases entire subsystems are arranged to comply with a "green-board" philosophy; that is, if all components in a subsystem are in the normal operating mode, all the indicator lights will be green.

Aside from those described above, few conventions are followed in the color-coding of indicator lights in NPPs, and the lamps do not indicate the normal operating states of the items they are monitoring. (In this context "normal" refers to the power-generating mode of the plant.)

The three usual cases in which an operator'will refer to a status lamp are when:

(1) Confirming a status change after a manual operation such as changing a valve state

(2) Determining the status of a specific item (or group of items) for some immediate purpose

(3) Conducting a survey, as in the course of an initial audit or at pre- scribed periods thereafter

The HEPs associated with the first two cases are as follows:

(1) Confirming status change after an operation, such as changing the status of an MOV, is an active task. The operator has initiated the change and ordinarily will watch the indicator lights for confirmation of the response. In most situations, the probability of his failing to note the status change is negligibly small and will be disregarded. For an exception, see Table 13-3, item 10, "Failure to complete change of state of a component if switch must be held until change is com- pleted," in which an HEP of .003 (EF = 3) is assigned.

(2) Checking the status of a specific indicator light (or group of lights) for some specific purpose is also an active task. Assuming that the correct indicator is addressed, the probability that the indication will be misinterpreted (e.g., a reversal error will be made when looking at a pair of lamps consisting of a red and a green lamp) is assessed as negligibly small and will be disregarded. For selection errors (selecting the wrong indicator lamp), see Table 11-2. Use of this table means that we estimate the same selection HEPs for indi- cator lamps as for controls (Table 13-3, items 2, 3, 4):

11-17

Periodic Scanning

.003 probability of selection error when the lamps are identified by labels only

.001 probability of selection error when the lamps are part of well-delineated functional groups on a panel

.0005 probability of selection error when the lamps are on a panel with clearly drawn mimic lines that include the lamps

The HEPs associated with the third case are discussed in a later section, "Status Lamps and Legend Lights."

Detection of Unannunciated Deviant Displays During Periodic Scanning

Previous sections describe the cases in which some specific cue directs an operator to look at a particular unannunciated display. For example, an annunciator alarms, and the operator checks meters functionally related to that annunciator, or the operator follows some schedule that requires him to log some reading or to check-read some display. In such cases, there is a high probability of detecting a deviant indication on the display he observes. The following sections on periodic scanning of unannunciated displays deal with the probability that an operator will detect some devi- ant indication on a display when no such cues are present and when a writ- ten procedure listing the displays to be checked is not used. This refers to stable displays and to dynamic displays that are in a stable condition.* In such cases, detection will depend largely on the operator's scanning pattern and frequency.' Although the probability of detecting a deviant unannunciated display during periodic scanning is relatively low, scanning does provide a useful recovery factor for previous errors and also provides a possibility of detecting an incipient, potentially serious condition.

Hourly scans are assumed as the average practice of control room operators for dynamic displays in a stable condition and for those stable displays which may change ih the course of a shift. For certain other types of stable displays,.only one scan per shift is assumed, at the initial audit. For example, stable displays such as a meter showing the level of refueling water in a tank are not likely to be scanned more than once per shift. Finally, there are many displays that operators report they never check unless alerted by some signal or an alarm, e.g., status lamps for blocking valves.

The modeling of scanning accuracy is based on the observation that,, although he may "look at" a given display, there is a probability that a deviant indication will not register on the consciousness of the operator

A stable condition of a display means that it is not rapidly changing status, lamps are not blinking, and the auditory signals for annunciators are canceled. These displays present no special alerting cue to the operator.

11-18

General Display Scanning Model

unless some alerting cue is present. This error is classified as an error of commission since the operator "looks at" the display but does not per- ceive the deviation. His expectancy to see the normal state influences his perception.

Deviant indications may appear on any of the displays described earlier in this chapter. The modeling that follows and the HEPs presented later differentiate between go/no-go displays, such as status lamps and legend lights, and displays that present additional information, such as meters, chart recorders, and digital readouts.

The next section presents a general scanning model illustrating hypothe- sized relative effectiveness of scanning at different points in a shift. Following sections present changes to the model according to the number of deviant displays and/or operators participating in the scanning. This part of the chapter concludes with several sections that present estimated HEPs based on different types of displays and scanning assumptions.

A General Display Scanning Model

If we consider a single deviant safety-related display in a stable condi- tion that was not detected in the previous shift, the oncoming shift opera- tor will have some probability of detecting that deviant display during his initial scan of the control boards. This probability varies according to the type of display. For any initial probability of detection, we hypothe- size that the probability of detection over the remainder of the shift follows an exponential curve. This relationship also holds for the case in which an operator regards a functional group of displays as a single unit.

Since we are assuming skilled operators, we expect them to know what is associated with what. For example, if an operator notes an increase in containment temperature, he will usually check the containment pressure. These two indicators are jbdged to be completely dependent. There are many other cases in which more than one display may be treated as a single perceptual unit when estimating the probability of detection. We do not offer specific rules for what constitutes a perceptual unit, since plants differ. The principle is that closely associated indications in a system may be regarded as perceptual units. The models for displays pertain to individual indicators and to any group of indicators that constitutes a perceptual unit.

Under high-stress conditions, when immediate action is required, some operators may not respond to what would normally be considered a perceptual unit (e.g., both temperature and pressure displays) but may fixate on one element of the unit, as described in the previous section, "Funneling Attention in Use of Displays." Several incidents have occurred in which an operator did not cross-check directly related instruments but concentrated on one instrument. In some cases, the one instrument displayed erroneous information or insufficient information to enable the operator to interpret the situation correctly.

11-19

General Display Scanning Model

It is general policy in NPPs that outgoing control room personnel brief the incoming shifts. Interviews and observation reveal much variability in the thoroughness of this intershift consultation. This is in part a function of what problems the outgoing shift has experienced, and in part it is a function of personalities and the administrative control exercised in the plant. Ideally, the incoming operator would perform his initial audit in the company of the outgoing operator, as is done at some plants. During this period, the probability of detection of any deviant condition is highest. For calculational convenience, we assume that the initial audit takes place at the very beginning of the shift and that its duration is zero time.

For scanning purposes, we identify three kinds of displays:

(1) Those stable displays which ordinarily are not checked in the initial audit or during a shift (e.g., a pair of status lamps that indicates whether some blocking valve is open or closed),

(2) Those stable displays which usually would be checked in the initial audit (e.g., the level in the Refueling Water Storage Tank), and

(3) Those dynamic displays on which important information may change during the shift, requiring that they be observed several times per shift (e.g., containment temperature).

For the first kind of display, we assume no periodic scans. For the second kind, we assume one scan during the initial audit. For the third kind, we hypothesize an initial scan, followed by hourly periods of scanning if nothing unusual occurs. We assume that scanning effectiveness for such displays decreases for the rest of the shift because of the cumulative effects of fatigue, boredom, and expectancy, with an end spurt in effec- tiveness coincident with shift turnover to the oncoming operator. We are disregarding the effects of "breaks," such as lunch breaks, coffee-breaks, etc.

The shape of the distribution of scanning effectiveness is conjectural because of lack of data, variation among operator scanning habits, and the variety of possible situations in a plant. As shown in Figure 11-1, we have selected the exponential curve to represent the decline of scanning effectiveness after the initial scan. The improvement on the eighth scan is suggestive only, representing the typical end spurt effect in detection performance effectiveness characteristic of shift work. For calculational convenience, we have incorporated the detection improvement shown in the dotted line into the detection effectiveness of the initial audit on the next shift.

During the initial audit, the probability of detection is highest because

the operator is fresh and unaffected by the recent history of the displays. If he misses a deviant indication in the initial audit, he will be less likely to notice it on the next scan because he has already accepted it as

I 1-20

Figure 11-1

DISPLAY SCANNING MODEL VERY HIGH

z LU

LI.

LII

VERY LOW L 1 2 3 4 5 6 7 8

HOURLY SCANS U"It IS THE INITIAL AUDIT AT BEGINNING OF SHIFT

Figure 11-1 Hypothetical curve representing detection effectiveness

at initial audit and at hourly intervals thereafter, ending with last scan at beginning of the eighth hour.

11-21

One Unannunciated Deviant Display - Two Operators

normal. With each successive failure to detect, the probability of detec- tion on the subsequent scan decreases as the operator grows more accustomed

to the deviant indication.

In cases involving an indication that becomes deviant after the initial audit, there are two opposing psychological influences. Because a change has occurred, we expect that the highest probability of detection will occur on the first scan after that change, with this probability of detec- tion declining exponentially on subsequent scans. However, there is also an expectancy effect that will reduce the probability of detection of a

deviation that occurs after the initial audit. Consider, for example, a deviation that occurs midway in the shift, i.e., just before the fifth

scan. Having seen "good" indications in the previous four scans, the operator expects to see a good indication on subsequent scans as well. This is analogous to the situation of an inspector on a production line where quality is very good; he seldom experiences a bad unit, and, expect-

ing them all to be good, frequently misses the occasional bad item.

We do not know how to quantify the effects of these opposing influences.

We will assume they cancel each other and will consider the exponential curve in Figure 11-1 to represent the relative probability of detection of any deviant display regardless of when it becomes deviant.

When the initial probability of detection of a particular type of deviant

display is very low, a simplification can be made. For displays with a success probability of about .01, we ignore the exponential curve and assume a constant probability of successful detection per scan. Our rea- soning is that when the initial probabilities are so low, the use of the exponential curve constitutes questionable exactitude.

One Unannunciated Deviant Display - Two Operators

If two operators are assigned to the control panels, and if both operators scan for deviant indications, the probability of detection should be greater than with only one operator. This section presents our estimates of the added effectiveness that can be expected from the use of more than

one operator.

Several factors must be considered:

(1) Number of operators assigned to the control panels of a single

reactor,

(2) Division of the task; for example, one operator may scan the reactor

control board and the other scan the rest of the panels,

(3) Level of dependence among the operators assigned to a single reactor,

and

(4) Percentage of time the operators are available to scan the panels.

11 -22

One Unannunicated Deviant Display - Two Operators

Number of operators

It is possible that more than two operators might be assigned to monitor the control panels of a reactor, but ordinarily this would not add mate- rially to the probability of detecting deviant items. Usually, the third operator would be assigned to other duties, either formally or informally, and only two operators would be effectively scanning the panels. Even if more than two operators are assigned to one reactor per shift, only two should be assumed in a human reliability analysis (HRA).

Assignment of Panel Responsibilities

Seminara et al (1976) state in their review that when two operators were assigned to a control room for a given reactor, they were not assigned to specific panels in the normal power generating mode, that is, they did not divide the work. We assume the same practice in this Handbook. (However, in emergencies, the supervisor will often assign the operators, "You take the turbine, you take the reactor," etc.)

For the case in which a second operator in a control room is assigned a particular function or functions, we assume that his reliability for those functions equals that of the regular control room operator for the rest of the control room tasks. For example, following the incident at TMI, some plants assigned a dedicated operator to maintain sufficient water in the steam generators. In general, the error probabilities for the relatively, few tasks performed by a dedicated operator will be materially reduced because of the elimination of the interpretations and decisions charac- teristically required of the regular operator.

Level of Dependence Between Two Operators Assigned to Same Reactor Panels

For the case of two operators assigned to a control room, if both operators are active monitors in the sense that both are equally responsible for the panels, we judge that moderate dependence (MD) best reflects the interac- tion between them when both are performing their duties in the control room. If this condition is not met, the use of MD will result in an over- estimate of their joint probability of detecting a deviant display. The estimate of MD is for the power generating mode; under abnormal conditions, a high level of dependence is assumed between the two operators.

It is possible there would be one active operator and one relatively pas- sive operator assigned to the control room. In such a case, a high level of dependence would best reflect their interaction. If the second operator functions as a "rover" (that is, checking on things outside the control room), obviously he is not available for control room activities during his absences.

Percentage of Time Operators Are Available To Scan the Panels

In deriving the probabilities of detection of deviant displays, we did not assume that an operator would be constantly scanning the control boards,

1).23

Multiple Unannunciated Deviant

Displays - One Operator

since this would be unrealistic. We have assumed no scans, one scan, or hourly scans per shift for different condition or types of displays.

We judge that when two operators are present and both have been instructed

to scan the same control boards, each will show less scanning effectiveness than if he were the sole operator. This was the rationale for the assump- tion of MD. Also, it is likely that because of the imposition of extra tasks when two operators are assigned, each will have less time for scan- ning than if he were the sole operator. The question is, For what per- centage of the shift will two operators be available for scanning and for what percentage will only one operator be available? Instances during which only one operator is available for scanning include any time during which only one operator is present in the control room as well as those times when one operator is engaged in paperwork or other activity that precludes effective scanning. If we were to assume a 50-50 split, half of the time MD would be assumed, and half of the time only one operator would be counted as present.

Multiple Unannunciated Deviant Displays - One Operator

If more than one unannunciated display becomes deviant during a shift,

there is obviously a greater probability that at least one of them will be detected, since there are more chances to see a deviant display.* In practice, it is unlikely that there will be more than five unannunciated deviant displays in an NPP, as with six or more deviant displays there ordinarily will be annunciation.

The extent to which each additional deviant display facilitates detection

is a function of the detectability of the individual displays. For exam- ple, a deviant meter usually attracts more attention than a deviant status lamp (when displays are not functionally grouped or related via mimic lines). Thus, there is a higher probability of detecting at least one of several deviant meters than of detecting at least one of several deviant

status lamps. Because of the differences in the cumulative facilitative effects of signals with different basic probabilities of detection, it was necessary to develop a detection model that allows for these differences. Application of the concept of dependence satisfied this requirement and is in accord with what is known of human performance in NPPs. The application of dependence to detection is premised on the statistical relationship

between 2 events or sets of events. The most important PSF affecting detection of deviant indications is a facet of human performance over which

we have very little control--the scanning habits of the operator. For a variety of reasons, operators will pay more attention to some displays than

to others and will be more responsive to some displays than to others. The practical effect of differences in the PSFs of displays is identical to the effects of direct dependence among events and may be analyzed similarly.

It is assumed that the deviant indication will remain deviant until something is done about it.

SI -2q

Multiple Unannunciated Deviant Displays - Two Operators

For the detection model, the application of high dependence (HD) among events yields realistic estimates of HEPs as a function of the number of deviant displays. Table 11-5 lists probabilities of detecting at least one deviant display when there are from one to five such deviant displays and Table 11-6 lists the HEPs (the complements of the entries in Table 11-5). The entries are obtained by use of the following equation, which is derived from Equation 10-17 in Table 10-2. Equal HEPs for the deviant displays are assumed.

Pr[ Sone or more deviant displays Iequal HEPs]

1 + Pr[F n.-i

I Pr[F ] oneone 2-

where n is the number of deviant displays.

This equation can be modified for the case in which the HEPs are not equal. Such application should be made only within the same class of displays, e.g., to meters or to chart recorders or to some combination of status lamps and legend lights. Thus, for the case of four deviant meters, two with limit marks and two without limit marks, two equations would have to be used because the HEPs for these two types of meters are different. Since one would not know the order in which the meters will be scanned, the arithmetic mean of the probabilities of success obtained from the two equations would be used as the estimate of the probability of detecting at least one of the four meters.

Under normal conditions, when an operator detects a deviant display, he also checks functionally related displays. The probability of detecting deviant related displays is a function of the operator's perception of what is related. The TMI incident teaches us that operators do not always perceive all the functional relationships that would enable them to cope with unusual events. Differences in training and experience will mate- rially affect this perception. No detailed guidelines can be given for the probability that related displays will be checked under unusual conditions -- this will vary with the situation.

In addition to checking functionally related displays, there will be an arousal effect that will heighten sensitivity to other, unrelated deviant indications. For these displays, we assume that the arousal will raise the detection effectiveness to the level at the time of the initial audit. The arousal effect will last until the operator is satisfied that everything is back to normal. For simplicity, we assume instantaneous arousal and dissipation.

Multiple Unannunciated Deviant Displays - Two Operators

Equation 11-1, for calculating the probability that an operator will detect at least one of up to five unannunciated deviant displays of the same

I 1-25

Table 11-5

Table 11-5 Estimated probabilities of detecting at least one of one

to five unannunciated deviant displays, as a function of the BHEP for detection of a single deviant display during periodic scanning*

Number of Deviant Displays

BHEP BHSP** Pr[S) of detecting at least one deviant display

Item 1 1 2 3 4 5

(1) .99 .01 .015 .02 .025 .03

(2) .95 .05 .07 .10 .12 .14

(3) .9 .1 .15 .19 .23 .27

(4) .8 .2 .28 .35 .42 .48

(5) .7 .3 .41 .49 .57 .63

(6) .6 .4 .52 .61 .69 .75

(7) .5 .5 .63 .72 .79 .84

(8) .4 .6 .72 .80 .86 .90

(9) .3 .7 .81 .87 .92 .95

(10) .2 .8 .88 .93 .96 .97

(11) .1 .9 .95 .97 .98 .99

(12) .05 .95 .97 .99 .993 .996

(13) .01. .99 .995 .997 .999 .999

Based on Equation 11-1. The -values apply when no written materials are used.

BHSP = basic human success probability, 1 - BHEP.

11-26

Table 11-6

Table 11-6 Estimated probabilities of failing to detect at least one* of one to five unannunciated deviant displays as a function of the BHEP for detection of a single deviant display during periodic scanning**

Number of Deviant Indications

1 2 3 4 5

BHEP Pr[F] to detect at least one deviant displayt

I tem. (a) (b) (c) (d) (e)

(1)

(2)

(3)

(4)

(5)

(6)

(7)

(8)

.99

.95

.90

.80

.70

.60

.50

.40

.30

.20

.10

.05

.01

.985

.93

.85

.72

.59

.48

.37

.28

.19

.12

.05

.03

.005

.98

.90

.81

.65

.51

.39

.28

.20

.13

.07

.03

.01

.003

.975

.88

.77

.58

.43

.31

.21

.14

.08

.04

.02

.007

.001

.97

.86

.73

.52

.37

.25

.16

.10

.05

.03

.01

.004

.001

(9)

(10)

(11)

(12)

(13)

To estimate the HEP for failure to detect other concurrent unannunciated deviant displays when one has been detected, use the HEP for the initial audit for those displays that are not functionally related to the display detected (from Table 11-7) and use the annunciator response model for those displays that are functionally related to the dis- play detected (from Table 11-13). The HEPs apply when no written materials are used.

Except for column (a), the entries above are the com- plements of the entries in Table 11-5.

t For EFs, refer to Table 7-2.

11-21

Estimated Scanning HEPs

class, should be modified for the case in which there are two operators as- signed to the panels of a reactor. If we assume that the two operators are both scanning for half of the shift, their joint probability of failure to detect at least one of up to five deviant displays would be as follows:

Pr[FFboth operators] = .5 x Pr[Fboth operatorslMD]

+ .5 x Pr[Fone operator1, n 4 5 (11-2)

where n is the number of deviant displays of the same class, both operators are available 50% of the time, and only one operator is available the other 50% of the time. As an example, assume four deviant displays with equal HEPs of .5. From Table 11-5, the HSP for one operator would be .79, yield- ing an HEP of .21 (EF = 5) for one operator. For two operators, given MD, we use Equation 10-16 (Table 10-2) to obtain the conditional failure proba- bility for the second operator:

Pr[F,,,FI M] 1 + 6N 1 + 6 x .21 " I MD= 7 = .32

From Table 7-3, item 3e, the UCBs for .32 are .1 and 1.0. The joint HEP = .21 x .32 = .07 for two operators both full time. If one operator is available only half the time, applying Equation 11-2, the combined failure probabilities equal (.5 x .21) + (.5 x .07) = .14. The UCBs for the joint HEPs of the two operators can be calculated using the method in Appendix A.

Estimated Scanning HEPs

Scanning HEPs are provided for status lamps and indicator lights, digital readouts, and analog displays. Table 11-7 presents the estimated HEPs and EFs for the various kinds of displays, and the sections that follow present the related discussion of the HEPs.

Status Lamps and Legend Lights

In the following discussion, the term "one indicator" refers to an individ-

ual indicator or to any completely dependent pair or group of indicators that may be regarded as a perceptual unit, e.g., the two status lamps monitoring the AFWS blocking valves at TMI.

Without any alerting cues, the probability of detecting a deviant status lamp or legend light during a scan, when there is only one such deviant indicator, is very low. For scanning of a status lamp (e.g., noting that the green lamp is on when the red lamp should be on), our best order-of- magnitude estimate of this detection probability is .01 per scan; i.e., an error estimate of .99 (Table 11-7, item 7). For a legend lamp, the

Table 11-7

Table 11-7 Estimated probabilities of failure to detect one (of one) unannunciated deviant display* at each scan, when scanned hourly**

(Initial t Audit) Hourly Scans

Display Type 1 2 3 4 5 6 7 8

Itenm (a) (b) (c) (d) (e) (f) (g) (h)

Analog meters:

(1) with limit marks .05 .31 .50 .64 .74 .81 .86 .90

(2) without limit marks .15 .47 .67 .80 .87 .92 .95 .97

Analog-type chart recorders:

(3) with limit marks .10 .40 .61 .74 .83 .89 .92 .95

(4) without limit marks .30 .58 .75 .85 .91 .94 .97 .98

(5) Annunciator light no .9 .95 .95 .95 .95 .95 .95 .95 longer annunciating

(6) Legend light other .98 .98 .98 .98 .98 .98 .98 .98 than annunciator light

(7) Indicator lamp .99 .99 .99 .99 .99 .99 .99 .99

"One display" refers to a single display or a group of completely dependent displays, i.e., a perceptual unit.

For error factors, refer to Table 7-2.

SWritten materials not used.

These displays are rarely scanned more than once per shift, if at all. Hourly HEPs for each are listed for completeness only.

11-29

Estimated Scanning HEPs

estimate of detection probability is doubled since the latter has more information to aid the operator. This still yields an HEP of .98 (Table 11-7, item 6).

In most applications, either one' scan per shift or no scan is assumed for status lamps and legend lights. If more than one scan is made per shift, a constant detection rate for each scan is assumed throughout the shift, beginning with the initial audit. Thus, the estimated failure probabili- ties per scan to detect a given deviant status lamp or legend light, when there is only one deviant indicator, are .99 and .98. (The exponential curve in Figure 11-1 is not applied since the probabilities of detection are so low relative to meters and other dynamic displays for which the curve is assumed.)

Digital Readouts

The scanning model for analog displays does not apply to digital readouts because the latter must be read rather than scanned. If a particular digital readout is read periodically, we assume that its criterion value is commonly known (e.g., reactor temperature) or that it is posted beside the display. In such a case, the only significant error is the error of read- ing, as discussed later.

Analog Displays

For analog displays, we assume the exponential decrease in detection effi- ciency described earlier (Figure 11-1). For multiple deviant analog dis- plays, and for more than one operator, special performance models are used. The ne xt several center headings present numerical values to illustrate the application of the models to meters and chart recorders. As is the case with other displays, the estimates apply to single displays or to groups of displays considered as perceptual units.

Multiple Unannunciated Deviant Displays Present During Scanning

,If more than one unannunciated display becomes deviant during a shift, the probability of detecting at least one of them is calculated using Equation 11-1 or Table 11-5. For two operators, use Equation 11-2 with appropriate modification for the percentage of time both operators are available for scanning. If the operator has detected a deviant display, the probability of detecting functionally related deviant indicators follows the model for annunciators described in Part 3 of this chapter. Under normal plant con- ditions, detection of any deviant display should cause arousal such that related deviant indications take on the same attention-getting values as annunciated displays.

If there are any deviant displays not functionally related to the detected deviant display, the arousal effect is assumed to raise detection effec- tiveness to the initial audit level of effectiveness.

11-30

Scanning of Unannunciated Deviant Analog Displays

Detection of Unannunciated Deviant Analog Displays During Periodic Scanning

The scanning model in this section is based on the general scanning model from Figure 11-1, as applied to hourly scanning of meters with limit marks and with no cue or special instruction that would draw attention to a particular meter. If there is some cue or special instruction, the HEPs for reading or check-reading should be used instead of the scanning model. The model is based on the assumption of one deviant display and one oper- ator. For more than one deviant display, and for more than one operator, use Equations 11-1 and 11-2.

Since the model in this section is based on meters with limit marks, ad- justments are made for application to meters without limit marks or to analog-type chart recorders with and without limit marks. For this purpose, scanning may be considered as a special kind of check-reading, and the modifying factors for check-reading errors may be applied to scanning errors when comparing different analog displays.

Scanning of Meters with Limit Marks

As indicated in Figure 11-1, the probability of an operator detecting one of only one unannunciated deviant display is assumed to follow an expo- nential decrease in detection efficiency over the shift if there are no alerting cues. We assume that the operator scans displays at hourly inter- vals, beginning with the initial audit and ending with the eighth scan an hour before the shift ends. Zero duration time for each scan is assumed for mathematical convenience (in typical PRAs the system failure estimates are not changed materially by this assumption). Thus, the initial audit, T , takes place at time zero, the second scan, T , at the start of the s4 cond hour on the shift, and so on to the last •can, T8 , which takes place at the start of the eighth, hour on the shift. The initial audit on the subsequent shift corresponds to the end of the eighth hour.

While the exponential shape of the curve is in general agreement with what is known about detection, efficiency over time, a search of the literature revealed no data with which to assign probability estimates to the eight points on the curve. For example, Murrell (1969, pp 62-63) notes the lack of data on control room errors in large process plants. On the basis of our experience, we hypothesize a .95 probability of successful detection of one of only one deviant analog display at the initial audit, T , and a .1 probabirity of successful detection for the last hourly scan, T , given nondetection on the previous trials. With values assigned to t~ese two points, the other points were determined from the exponential function and are listed in column b in Table 11-8. These figures represent the hypothe- sized hourly decline in detection probability through the 8-hour shift.

Equipment malfunctions are assumed to occur midway through the shift. This assumption means that a meter indication related to some deviant condition

1)-3,

Table 11-8

Table 11-8 Estimated per scan and cumulative probabilities of detection for each hourly scan for one of one

unannunciated deviant meter with limit marks*

Item

(1)

(2)

(3)

(4)

(5)

(6)

(7)

(8)

Trial (Scan) Number

(a)

T **

T 2

T 3

T 4

T 5

T 6

T 7

T 8

Pr[S.] 1

Pr[F.1

(b)

.95

.69

.50

.36

.26

.19

.14

.10

(c)

.05

.31

.50

.64

.74

.81

.86

.90

Pr[S ]

(d)

.95

.03

.008

.003

.001

.0007

.0004

.0003

Pr[S<i]

(e)

.95

.98

.988

.991

.992

.9927

.9931

.9934

"One meter" refers pendent meters.

to a single meter or a group of completely de-

T is the initial audit.

NOTES:

(1) To determine EFs for Pr[F] terms, refer to Table 7-2.

(2) These estimates are rounded values. Four places are used merely

for completeness; in practice, these should be rounded further.

th (3) Pr[S.] (column b) is the probability of detection on the i trial,

1 given there is one deviant display to be detected.

(4) (Column c); Pr[F.] = 1 - Pr[S.] 1 1

(5) Pr[Si) I (column d) is the probability of first detection on the

ith trial only, given that the deviant condition occurred before T

of the present shift but after T of the preceding shift. For8

example, Pr[S ] = Pr[F 1 x Pr[F ] x Pr[F ] x Pr[S ] = .05 x .31 x(4) 1 2 3 4 .50 (all from the Pr[F i column) x .36 (from the Pr[S.I column) =

.00279 = .003.

Table 11-8 (p 2 of 2)

(6) Pr[S4i] (column e) is the probability that detection occurs on or th

before the i trial, given that the deviant condition occurred

before T of the present shift but after T of the preceding shift.1 8 It is the cumulative sum of the Pr[S I values.

(1)

(7) To calculate Pr[SiW I and Pr[S I for deviant indications that

occurred just before hourly scans other that T1 , start the cal-

culations from the T of interest. (See Appendix C for somei

calculations.)

(8) If fewer than eight scans per shift are assumed for a specific

application, use the T. values from the table according to the 1

times for which the scans are assumed. Thus, if scans at the

beginning and midway through the shift are the two scans assumed,

use the Pr[S. and Pr[F.] values for T and T .3l 1 1 5

(9) It is assumed that if the deviant meter has not been detected

within 30 days, it will not be detected unless some other stimulus

calls its deviation to the operator's attention.

11-31,

Scanning of Unannunciated Deviant Analog Displays

in the plant will occur just before T . As explained earlier, the simpli- fying assumption is made that the tabled detection probabilities hold regardless of when the deviation occurred. The estimates in Table 11-8 refer to either a single display or a set of displays for which the proba- bilities of detection are completely dependent, i.e, a perceptual unit. The tabled values apply to the power-generating mode only and should not be applied to conditions in which disabling levels of stress can occur.

The estimates in Table 11-8 are predicated on the passive nature of scan- ning in comparison to check-reading or quantitative reading activities; the operator is looking around the control room to see if "everything is OK." Under normal conditions, he expects to find everything within proper lim- its, and usually they will be. He is not actively "probing" each indica- tor. He is inclined to accept readings as being within proper limits unless his attention is caught by a grossly deviant indication.

Unavailability Calculations

In unavailability calculations, the mean number of trials to detection, t, or the median number, M, are often of interest. Table 11-9 lists the mean and median numbers of trials to detection for various starting points. For example, at T the mean value of 3.33 and the median value of 4 represent the number of trials to detection for some unannunciated display that becomes deviant after T and before T_, i.e., midway through the shift. The calculations for defiving these Alues are illustrated in Appendix C.

Scanning of Other Types of Analog Displays

Table 11-10 presents the estimated probabilities of success and failure for each hourly scan for meters without limit marks and for chart recorders with and without limit marks. The Pr[F.] values for T were determined by multiplying the equivalent Pr[F ] of ..O for meters with limit marks (Table 11-8) by a factor of 3 for meters without limit marks, a factor of 2 for chart recorders with limit marks, and a factor of 6 for chart recorders without limit marks. These factors are taken from Table 11-4 on the assumption that the factors that differentiate between check-reading HEPs for meters with and without limit marks and chart records with and without limit marks also apply to scanning HEPs for these four different types of displays. The Pr[F.] values for T were based on the Pr[F.] of .9 for meters with limits marks, as follows:

1 - Pr[F 8 for meters with limit marks 1- 8

the appropriate factor of 2, 3, or 6

The values for T through T are based on the exponential relationship used in Table 11-8.

Appendix C shows how mean and median numbers of trials to detection of these types of unannunciated displays could be calculated for any given scan T. from Table 11-9.

11 -14

Table 11-9

Table 11-9 Mean and median numbers of trials to detection for unannunciated meters, with limit marks, that become deviant prior to any given scan, T.1.

Scan T. Mean* Median1

T ** 1.06 1

T 1.78 12

T 2.55 1 3

T 3.13 2 4

T 3.33 45

T 3.16 4

T 2.68 3 7

T8 1.96 2

The mean values are based on a total of 8 trials from any T. of interest because, with the values given in Table 11-8, 1

use of more than 8 trials does not change the mean materially.

T is the initial audit.

The derivations-of the mean and median values for T are shown in Appendix C. 5

Table 11-10

Table 11-10 Estimated per scan probabilities of detection for each hourly scan for one (or one completely dependent set) of one unannunciated deviant display

Type of Display

Meters Without

Limit Marks

Chart Recorders

Trial (Scan) Number

T 1

T 2

T 3

With Limit Marks

Without Limit Marks

T 4

T 5

T 6

T 7

T 8

Pr[S.]1

.85

.53

.33

.20

.13

.08

.05

.03

Pr[F]

.15

.47

.67

.80

.87

.92

.95

.97

Pr[S.]1

.90

.60

.39

.26

.17

.11

.08

.05

Pr[F. 13

.10

.40

.61

.74

.83

.89

.92

.95

Pr[S.

.70

.42

.25

.15

.09

.06

.03

.02

Pr[F] 3.

.30

.58

.75

.85

.91

.94

.97

.98

T is the initial audit.

NOTES:

(1) To determine EFs for Pr[F] terms, refer to Table 7-2. th

(2) Pr[S.] is the probability of detection on the i trial, given there is oAe deviant display to be detected.

(3) Pr[F i = I - Pr[S).

(4) If fewer than eight scans per shift are assumed for a specific application, use the T values from the table according to the times for whi-ch the scans ari assumed. Thus, if scans at the beginning and midway through the shift are the two scans assumed, use the Pr[S ] and Pr[F.I values for TI and T5.

(5) A 30-day cutoff is assumed as per note 9 in Table 11-8.

.1-14

PART 2. ANNUNCIATED DISPLAYS Types of Annunciated Displays

PART 2. ANNUNCIATED DISPLAYS

The following several sections present a model describing the probabilities of errors of omission for scanning and reading of annunciators (ANNs). The model applies to four situations in the control room: (1) the power- generating mode, (2) maintenance or calibration operations involving con- trol room personnel, (3) transients, and (4) loss-of-coolant accidents (LOCAs).

In performing an HRA, the analyst may elect either of two approaches in assessing the probability of timely response to an abnormal event. One approach is to use the Annunciator Response Model in this chapter to esti- mate the probability of noticing a particular annunciator or combination of annunciators and initiating action in response to them. The other approach is to use the Nominal Diagnosis Model in Chapter 12, which includes per- ception and interpretation of the annunciators and the necessary diagnosis and decision-making activities.

The Annunciator Response Model is suggested for those parts of an HRA in which the primary interest is in responding to the annunciators, without emphasis on interpretation, as when plant rules stipulate the response. For example, a plant may have a rule that whenever safety injection occurs, the operator is to initiate the immediate actions for a loss of coolant accident. No diagnosis is involved. The Annunciator Response Model can also be applied to the analysis of operator response to a slowly developing transient or LOCA, or for recovery actions after the Nominal Diagnosis Model has been applied. For example, if the initial diagnosis was incor- rect, it is possible that some later annunciator(s) may enable control room personnel to recover from their incorrect diagnosis if sufficient time remains for appropriate corrective action.

The Nominal Diagnosis Model is suggested for those parts of an HRA in which interpretation, diagnosis, or decision-making are required. The model provides the cumulative HEP for recognition and diagnosis of a transient, as a function of time after the problem is annunciated.

Whichever model is used, the reliability of the actions that are carried out subsequent to the alarm(s) can be assessed through the use of this chapter and others in Part III.

Types of Annunciated Displays

Any display that is accompanied by an attention-getting signal when it changes state is said to be annunciated and is popularly called an annunci- ator. The annunciators of interest in NPP evaluations are legend lights and automatic printout equipment. Usually, the annunciated legend lights are mounted in panels above the vertical control boards, above eye level. There are from 15 to 66 legend lights (or tiles) per panel, and there may be 400 to 750 tiles per reactor. When any annunciated function deviates from a specified condition, an automatic signal initiates an auditory alarm

11-37

Types of Annunciated Displays

(horn, bell, or buzzer) and causes one or more of the tiles to blink. Separate buttons cancel the auditory and blinking signals. When the blink- ing signal is canceled, the tile remains illuminated in a steady-on condi- tion until the trouble is cleared. Generally, 20 or more tiles are in the steady-on condition at any one time due to various conditions that do not require immediate action. These can be regarded as "visual noise" for pur- poses of signal detection by the operator.

At most plants, when the trouble is cleared, auditory and visual "clear" signals occur that differ from those signaling the onset of the problem. The clear signals may then be canceled with the appropriate button.

There also are auditory alarms for automatic printout equipment. Finally, .there are auditory high-radiation alarms, fire alarms, security alarms, and so on. This discussion of annunciators deals only with annunciated legend lights and printout equipment.

Following are the major types of possible error in connection with annun- ciators:

(.1) Annunciatedlegend lights (tiles)

a. omission Errors: failure to initiate some kind of intended cor- rective action as required

b. Errors in scanning of unannunciated conditions of tiles that are lit steadily: during any subsequent scan, failure to recover the initial error of omission

c. Reading Errors: either an error is made in reading the tile or the wrong tile is read.

d. Diagnosis Errors: the tile is read correctly but the operator makes the wrong decision as to the required action or decides incorrectly that action is required

(2) Annunciated printout equipment

a. Omission Errors: same as la above, but this error may include a failure to read the printout

b. Errors in Scanning: same as lb above

c. Reading Errors: the message is incorrectly read

d. Diagnosis Errors: same as id above

The performance model and HEPs for responding to annunciated indicators are speculative and may be modified when objective data so indicate. In the interim, we have taken a conservative position in assessing the reliability of operators in responding to annunciated indicators. We use the term

11l-36

Major PSFs Reading Errors - ANN Tiles

"responding" to mean initiating what the operator believes is the appro- priate corrective action, as indicated by the annunciator. The initial acts of canceling the auditory signal or the flasher circuitry are not regarded as part of this response; it is assumed these actions are always taken. The annunciator response model does not address the correctness of the action the operator takes. Correctness of his action is based on the correctness of interpretation and diagnosis, as discussed in Chapter 12.

A basic problem is that one can expect a wide variety of responses because of the very large number of annunciated indicators and the fact that the indicators do not always provide the precise information the operator needs for making timely and correct decisions. In future plant designs, if sound ergonomic practices are followed (cf Seminara, Eckert, et al, 1979), large improvements in operator reliability can be expected because many of the factors contributing to operator error will be eliminated, including unnecessary alarms. The model and HEPs in this section reflect the design of present plants.

Major PSFs for Detecting Annunciated Signals

The most important PSFs are (1) the number of signals per unit time that the operator must process, (2) the number of relatively unimportant indica- tors, (3) the number of false alarms and nuisance alarms,* (4) the place- ment and design of the tiles or other indicators, and (5) the stress levels (ranging from boredom to the emotions associated with a major accident).

Table 11-11 lists some human engineering deficiencies related to annun- ciator warning systems that affect the above PSFs. The table is based on the human reliability analysis in WASH-1400, on subsequent observations by the authors in NPPs in the U.S. and Europe, on the Electric Power Research Institute (EPRI) control room review by Seminara et al, (1976), and on NUREG/CR-2147 (Banks and Boone, 1981).

Errors in Reading Annunciated Legend Lights

When an annunciator light comes on, the operator normally cancels the sound, looks to see which tile is blinking, reads the message on the tile, and then cancels the blinking. Two reading errors are possible. First, when the operator looks away from the blinking light to find the cancel button for the blinking, he may look at the wrong steady-on tile when he looks back. This error is possible because of the plethora of "normally

False alarms refer to annunciations that occur because of faulty cir- cuitry, misset limits (or deliberately making them tight), and so on. The result of a false alarm is that the operator must respond to a signal that does not require the kind of action that would be required if the alarm were not false. Nuisance alarms are those that provide unnecessary and distracting signals to an operator responding to an abnormal event, e.g., a LOW OIL PRESSURE alarm on an idle pump.

I 1-39

Table 11-11

Table 11-11 Human engineering deficiencies in annunciator warning systems

(1) The large number of annunciator lights per reactor (400 to 750)

coupled with the large number of alarms even under normal operating conditions (estimated by EPRI interviewees as from 2 to 30 times per hour, depending on the situation) call for complex response patterns. The discrimination requirements can be excessive.

(2) The number of false alarms per shift (estimated by EPRI interviewees as ranging from 15% to 50%, with as many as 75 false alarms in some cases) leads to the expected reaction to a "cry wolf" situation.

(3) The normal background (20 or more) of. annunciator lights in the steady-on state reduces the signal-to-noise ratio for an operator searching for meaningful displays. These steady-on lights constitute visual noise.

(4) Critical warnings are interspersed wit'h noncritical warnings. This reduces the arousal effects of the former and makes them more diffi- cult to identify.

(5) The lack of location aids as well as the large number of illuminated annunciator lights increase the difficulty of finding some safety- related annunciators.

(6) In some control rooms, large viewing distances combined with small lettering on indicators force the operators to move around quite a bit to read all the indicators. Some operators try to identify an alarm by its position on the board rather than by approaching it to read its label.

(7) In some cases, the intensity of auditory alarms is loud enough to evoke a startle response. This motivates operators to silence the alarm immediately, sometimes without even looking up at the annun- ciator boards. The intensity is so compelling as to interfere with the task at hand and cause forgetting. In some cases, coins are used to lock HORN SILENCE buttons in the cancel position. In some plants, the operator may have to leave a panel where he is performing some critical task to silence an auditory alarm. Operators have been observed to lose track momentarily of what was going on.

(8) Some annunciated indicators tell the operator that either a high or a low setpoint has been exceeded, but not which one. In some cases, an annunciated indicator means that any one of four possible conditions exists.

(9) Acknowledgment of an annunciator causes its legend light to go to a steady-on state and to blend in with all the others in the steady-on state. This can result in a loss of information.

11I-V

Table 11-11 (p 2 of 2)

(10) The difference between the Alert and Clear blink rates is not always clearly distinguishable to the operators.

(11) The alarm audio frequency may be too high for some operators to hear.

(12) Simulator experience shows that in a major accident, annunciators come on in such bewildering numbers that it is not possible to read, much less absorb, the meaning of all the annunciators. In some cases, they are deliberately ignored (Kemeny, 1979).

(13) According to some simulator instructors, the primary motivation of even skilled operators during situations in which large numbers of annunciators alarm is to "Turn off that . . noise!" This often causes delays in responding to critical indicators.

(14) During transient conditions, shifts to different power levels, shut- downs, start-ups, and other out-of-the ordinary conditions, the large number of annunciators that come on can easily mask safety-related annunciators because the operator is intent on coping with the un- usual condition.

(15) Labeling on tiles is often cryptic, with nonstandard jargon and abbreviations.

(16) Some "nuisance alarms" come on to indicate that a particular subsys-

tem is working normally. This serves only to distract the operators.

(17) Functional color codes for annunciators are often inconsistent within a plant.

(18) In dual control rooms, the different alarm frequencies often mask

each other when activated simultaneously, causing discrimination problems for the operators.

Scanning Errors - Unannunciated ANNs

on" tiles that do not require any operator action. However, because most operators will read the message while the tile is blinking, we regard his error as negligible. The second reading error can occur because the tile may be some distance away, and the operator may not walk over close enough

to see the legend clearly. He sees the correct legend but misreads it because of the distance. This error is infrequent but has been reported in operator interviews (Seminara et al, 1976).

There are no data on either type of error. We make the assumption that they are included in the estimated HEPs in the Annunciator Response Model presented later.

In the case of automatic printout equipment, we estimate a zero reading error for the general sense of a message, e.g., "oil coolant temperature is high." If he reads the printout at all, he will read such messages cor- rectly. For coded messages, or for series of numbers, the reading errors described in Table 11-3 for digital readouts apply.

Scanning Errors for Unannunciated Conditions of Annunciated Displays

If an operator has failed to initiate action after canceling the auditory and blinking indications of a legend light because of distractions or other activities, there is still some probability that he will recover from the error. Similarly, if the previous shift operator has turned off these indicators without initiating action, there is some chance that the oncoming shift operator will see the signal and respond to it. If the abnormal situation persists, it is very likely that additional annuncia- tions will alert the operator to his oversight. If not, there is still a possibility that the steady-on condition of the tile (or tiles) will be noticed at a later time. The remainder of this section presents some guidance in assessing this possibility.

For any application, it has to be judged whether the control room operator

scans the annunciator panels periodically or only when an alarm sounds or some other cue signals him. If there is no basis for such a judgment, we assume only one scan in the absence of alerting cues, during the initial audit. If periodic scanning is assumed, the exponential decline in the probability of detection (Figure 11-1) is not applied to the detection of unannunciating tiles, regardless of the frequency of scanning. Two oppos- ing factors stabilize the probability of detection. First, in most NPPs, all annunciated legend lights indicate abnormal plant conditions, so a certain minimal level of attention-getting is always associated with these indicators. Second, because of the large number of these legend lights that are usually on, the operator has to distinguish the "new" tile from the background of "old" tiles. This requires an awareness of all the "acceptable" abnormal conditions reported by the old tiles and is a very error-likely situation.

17 -4,Z

Responses to ANNs - Power-Generating Mode

We estimate that the probability that an operator will detect a steady-on tile that requires action is .05 per scan, except at the initial audit, for which the extra care doubles this probability to .1. Thus, the estimated HEP per scan is 1 - .05 = .95 for all but the initial audit, when it is .90. These are very high HEPs because the steady-on tile of interest fades into the normal background of 20 or more steady-on tiles. These extra steady-on tiles constitute visual noise that increases the difficulty of noticing a single steady-on tile that provides a useful signal. If the visual noise were eliminated, the estimated HEPs of .95 and .90 should be greatly reduced. Table 11-12 shows these estimated HEPs and those for more active perception, as discussed in the next section.

Recovery factors for the failure to detect a steady-on tile that requires action include functionally related signals and the onset of other annun- ciators. In the case of automatic printout equipment, we estimate a zero probability of recovery from later scanning in a shift. If the operator ignored the message when it alarmed, he would not be likely to check it later. Our rationale for this estimate is based on observation and on statements by operators that much of the information printed out is of little consequence and does not require immediate action.

Responses to Annunciating Indicators -- Power-Generating Mode

ordinarily, responses to annunciating indicators in the power-generating mode will be made by the operator assigned to the control room. With additional operators available as "backups," human reliability should be increased. Human reliability will be decreased as additional annunciators compete for an operator's attention and as additional false alarms occur. At some point, of course, the number of annunciating indicators means that normal operating conditions no longer exist. As more and more annunciators compete for the operator's attention, his ability to process each annunci- ated indicator decreases as a function of increased signal load. This is one type of stressor. In deriving the Annunciator Response Model, the effects of stress are already considered. Therefore, unlike other human performance models in the Handbook, the Annunciator Response Model does not have to be adjusted for different stress levels.

One Annunciating Indicator, One Operator

The Annunciator Response Model applies to any number of annunciators. The simplest application is for the case of one operator responding to one of only one annunciating indicator when the plant is in the power-generating mode. The term one annunciator. also applies to any functionally related group of annunciators that will be regarded as a perceptual unit by the operator, i.e., the annunciated indicators are completely dependent percep- tually. Such a functional group might consist of two, three, four, or even five annunciators. No specific rules can be stated for defining such groups; the guiding principle is that the operator responds to them as if they were a unit. An example of such a perceptual unit is given in Chapter 21 in the section, "Failure to Initiate Steam Generator Feed-and-Bleed Procedures."

io-'13

Table 11-12

Table 11-12 Estimated HEPs for annunciated legend lights*

Item Task HEP EF

(1) Respond** to one or more annunciated See Table 11-13 legend lights

(2) Resume attention to a legend light .001 3 within 1 minute after an inter- ruption (sound and blinking cancelled before interruption)

(3) Respond to a legend light if more .95 5 than 1 minute elapses after an interruption (sound and blinking cancelled before interruption)

(4) Respond to a steady-on legend .90 5 light during initial audit

(5) Respond to a steady-on legend light during other hourly scans .95 5

No written materials are used.

"Respond" means to initiate some action in response to the indicator whether or not the action is correct. It does not include the initial acts of canceling the sound and the blinking; these are assumed to always occur.

1'.'H

Responses to ANNs - Power-Generating Mode

Because of the compelling nature of the auditory alarm, failure to make a timely response to an annunciator is infrequent when there are no competing signals. However, in our interviews with operators and researchers associ- ated with both U.S. and foreign plants, we established that occasionally an operator forgot an annunciated indication after the sound and blinking of the light were canceled. No data exist on which to base an estimate of the probability of this failure to respond in a timely manner. We assign a 10- probability to this error as an order-of-magnitude estimate. (For human responses, a 10- HEP is an extremely small number; it means that 9,999 out of 10,000 times the operator will respond to a single annunciator within the allowed time.) A history of frequent false alarms could in- crease the 10- error estimate by one or more orders of magnitude. Al- though we have no data on the effects of false alarms from annunciator panels, data on the effect of false alarms in a different context (guard duty) indicate that they have a major influence.

The total response is defined as perception of the alarm, acknowledgment of the alarm, decision as to what action is appropriate, and initiation of that action (which maý include a decision to take no action). Note that the error term of 10 applies to the act of responding, not to the ac- curacy of the action taken, which could consist of operating switches in the control room, communicating to some other location for action to be taken at that site, and so on. The correctness of the action taken must be evaluated separately.

Ordinarily, the operator acknowledges the alarm by turning off the audio signal almost immediately and then looks for the blinking light. When he finds which tile is blinking, he reads the legend and then cancels the blinking function, usually in that order. The lamp will remain on until the problem has been corrected. For our purposes, the acknowledgment of the alarm is not significant; we regard the initiation of intended correct action as the actual response.

Depending on the nature of the alarm, the operator is expected to initiate corrective action within some specific time. For safety-related alarms, this action would normally take place within a minute or so, although several minutes may be allowed in some cases. If corrective action is not initiated within, say, 1 minute after acknowledging the alarm, the proba- bility that the action will be overlooked increases to 1.0 for some inde- terminate period and then declines. The rationale is that ordinarily the operator will take action immediately after acknowledging an alarm. If he does not, it is because something more pressing requires his attention. While attending to the other event, he cannot attend to the initial event and so the HEP for the initial event becomes 1.0. Upon completing the required action on the second event, he is free to return to the initial event. However, he may have forgotten about the initial event while work- ing on the second event, and the only indication remaining is the tile in the steady-on state, which is much less compelling than the blinking

1 1 -4c,

Responses to ANNs - Power-Generating Mode

state.* Furthermore, this indication will normally be only one of several since there will be several steady-on indications (estimated at 20 or more) on the annunciator panels at any time. These constitute noise and must be filtered out by the operator.**

A probability of 10-3 is estimated for failure to respond to a steady-on annunciator within 1 minute after the interrupting task has been taken care of. The rationale is that the interruption increases the operator's error probability by a factor of 10 (from the 10 HEP), due to disruption of his short-term memory. If he does not initiate action within some very brief period (say, 1 minute), we assume that he has forgotten the alarm, and the steady-on tile blends into the background of other steady-on tiles on the annunciator panels. The probability of his responding to it later will be much lower--a .,05 probability of detection per scan of the annunciator panels, assuming hourly scans or fewer. The estimated HEP is .95, as discussed under the earlier heading, "Scanning Errors for Unannunciated Conditions of Annunciated Displays."

Figure 11-2 outlines the error probabilities for initiation of corrective action as a function of time after the initial annunciator comes on. We have arbitrarily assumed that, in the case of an interruption, the first scan takes place at the end of the minute following the interruption.

The figure shows some of the considerations involved in accounting for time spent by operators as it relates to the probability of oversight in re- sponding to an annunciator. The times involved vary. For example, some safety-related annunciators require a quicker response than do others. The

10- HEP is the estimate to use in answer to the question, What is the probability that an operator will fail to initiate a response, within the required time, to a single annunciator (or perceptual unit), assuming no interruptions, with the plant in the power-generating mode?

This problem suggests an obvious human engineering design that we suggest be evaluated. This design would incorporate timing mechanisms that would cause steady-on annunciators to resume blinking, perhaps with a unique auditory signal, if not corrected within some time period. Different annunciators might have different time periods for resumption of the blinking. This concept is the basis of the snooze alarm found on some alarm clocks.

This problem, too, is a candidate for an ergonomics solution, since several of these continuing indicators may be related to repairs or other long-term conditions. One solution used in some plants is to paste repair stickers over such annunciator lights. A more effective solution would be to reduce the illumination level of the tile, either electri- cally (e.g., using half-brilliance circuits) or optically, by placing translucent covers over the tiles. Reduction of the illumination would still enable an operator to see a blinking signal if the system were put back into service without restoration of the full illumination level.

11-46

Figure 11-2

REACTION TIME (.25-1 sec)

TIME SPENT ATTENDING TO INTERRUPTION, SUCH AS SECOND ALARM

Z-

W z uz -a :az ILL

ILO 01I-

-z

a z I 0

Ilu

1.0

10-1 -

10-2 -

x x

RESUME ACTION ON FIRST ALARM (ONE MINUTE)

x x X IX. X x x EACH X REPRESENTS A .95 PROBABILITY THAT ON EACH HOURLY SCAN ANY STEADY-ON TILE REQUIRING ACTION WILL NOT-BE NOTICED

TIME SPENT ON FIRST ALARM

I t ACKNOWLEDGE ALARM ALARM SOUNDS

TIME (VARIABLE) 10

Figure 11-2. Initiation of action in response to annunciators in control room, given one operator, only one annunci- ating indicator, and power-generating mode.

I I1I,41

Responses to ANNs - Power-Generating Mode

If there is some distraction or interruption before the operator can ini-

tiate action, the estimated probability of failure to take the action after the interruption is 10 for the first minute. Thereafter, if hourly scans are assumed, the HEP is estimated as .95 per scan. If hourly scans are not assumed, the next chance to see the steady-on tile will occur during the initial audit of the next shift. Our model indicates that unless the operator responds when the annunciator first comes on, or within a minute after an interruption, he has a relatively low probability of recovering from the error on a timely basis because the steady-on indication of the tile fades into the usual background of other steady-on tiles.

One Annunciating Indicator, Two Operators

If two operators are assigned to the control room as active operators, we

assume MD between them, per Equation 11-2. Thus, the HEP of 10-4 to acknowledge an annunciator and take corrective action would be modified per the equation for MD.* If A is the HEP for person "A," and BIA is the conditional HEP for person "B," given that person "A" has failed, the joint HEP is as follows:

Pr[FIMD] = A x BjA

-4 -5= 10 x .15 = 10

The above estimate is based on the assumptions that both operators are in- deed present and that they have been instructed to monitor all the panels. If either assumption is not met, the above estimate will be too low.

Seminara et al (1976) noted that when two operators were assigned to a reactor control room they both looked at all the panels. However, often only one operator is actively operating the controls. If we assume that both operators will monitor the control boards simultaneously about half the time, the above equation would be modified as follows:

Pr[F] = [(failure probability for one person) x

(percent of time first person only is available)] +

[(failure probability for one person) x (percent

of time second person is available) x (conditional

probability of failure of second person assuming MD)] =

[I-4 x 5 I-4 -5 [104 x.5+[10 x .5 x .15] • 6 x

Under abnormal plant conditions, we assume a high level of dependence between two operators assigned to the control room panels.

Responses to ANNs - Power-Generating Mode

Multiple Annunciating Indicators, One Operator

(The term "one annunciator" also refers to a set of annunciators that trained operators regard as a single unit.)

If an operator has to attend to two or more annunciators, there is an increased load on him, and some decrease in his reliability is expected. As the number of annunciating indicators increases, the operator load is estimated to increase exponentially. This is the basis for the Annunciator Response Model presented below.

Operating personnel have two primary responsibilities: keeping the plant operating and ensuring its safe operation. Because serious safety problems occur very rarely, most of the operator's attention is directed to the instrumentation and controls related to the first responsibility. Inter- views with operators indicate that they do not expect serious safety prob- lems, even after TMI. Furthermore, they have confidence in the ability of their plant's safety systems to cope with safety problems automatically. These attitudes, coupled with the usual lack of practice in dealing with the unexpected, may result in reluctance on the part of an operator to take action that would interfere with keeping the plant operating. To what extent the TMI incident and the subsequent changes in operator training and onsite practice have affected these attitudes is not known.

It follows that if several annunciators sound simultaneously, an operator will probably give priority to the ones related to the maintenance of power output. This is not meant to imply that the operator will purposely ignore safety-related annunciators in favor of annunciators related to keeping the plant operating, but the tendency might be there. This tendency is strengthened as the number of false alarms in safety-related annunciators increases. Seminara et al (1976) note that some operators complained that false alarms were frequent. Estimates included "occasional," 15%, 30%, up to 50% false alarms, 50 to 100 per shift, and even 100 per hour in one unusual case.

For many transients, the distinction between safety-related systems and power-generating systems is academic. For example, in the case of loss of main feedwater, if auxiliary feedwater is not supplied on a timely basis, both safety and power generation are affected. Since loss of main feed- water will automatically trip the turbine and reactor, the above "conflict" between power and safety considerations is not very important.

There is still another consideration in attempting to predict the opera- tor's response to some annunciator or group of annunciators. Often the operator will have standing orders in the event of a turbine/reactor trip. For example, prior to an NRC requirement for automatic switchover to auxi- liary feedwater of main feedwater trips, at some plants the transfer had to be done manually. The operators were instructed to shift to auxiliary feedwater immediately whenever there was a turbine/reactor trip. A case study involving this situation is presented in Problem 4 in Chapter 21.

11-49

Responses to ANNs - Power-Generating Mode

Our model for multiple annunciators does not address unique situations such as the above. It is based on the simplifying assumption that all annun- ciators are equal in importance and attention-getting capacity. The user must modify the model for circumstances under which this assumption is invalid. The auxiliary feedwater case study in Chapter 21 shows how we handled this problem in one application.

The "Annunciator Response Model" is expressed in the following two equa- tions and the resultant HEPs in Table 11-13. (Note that in all discussion of annunciators, "one" annunciator (ANN) also refers to any group of annunciators that can be regarded as a perceptual unit.)

10-4,

Pr[F.] 2 i-2 x 10-13 < i 4 10 (11-3)

.25, i > 10

th where Pr[F.] is the failure to initiate action in response to the i annunciatoi in a group of n annunciators.

n - Pr[F.i

Pr[F-iT (11-4)

where Pr[F.] is the failure to initiate action in response to a randomly1 selected annunciator in a group of n annunciators, and is calculated as the arithmetic mean of all Pr[F.] terms. An EF of 10 is assigned to each Pr[F ] or Pr[Fi]. 1

The HEPs for annunciators after the second one are doubled instead of increased tenfold to allow for the arousal effect of any situation that causes a number of alarms to sound almost simultaneously. (A rationale for the doubling of HEPs under stress is given in Chapter 17.) There is still a substantial increase in the probability that an operator will overlook some annunciators after he has canceled their sound and blinking because of the increase in task load as the number of alarms increases. There may even be a point at which some operators may deliberately ignore the annunciators (except for canceling the disruptive sound), as evidenced by the TMI incident.

The cutoff HEP of .25 for the 10th annunciator (or 10 sets of completely dependent annunciators) was selected because more than 10 implies a stress- ful situation, such as a transient, and an error probability of .25 is assumed under extremely high stress (Chapter 17). Thus, for all annunci- ators beyond the 10th, an HEP of .25 is assigned.

I1-50

Table 11-13 The Annunciator Response Model: estimated HEPs* for multiple annunciators alarming closely in time*

Number Pr[F.] for each annunciator (ANN) (or completely dependent set of of AANs) successively addressed by the operator

ANNs 1 2 3 4 5 6 7 8 9 10 Pr[F.]I

Item (a) (b) (c) (d) (e) (f) (g) (h) Ci) Li) (k)

(1) 1 .0001 --------------------------------------------------------------------------. 0001

(2) 2 .0001 .001 ------------------------------------------------------------------. 0006

(3) 3 .0001 .001 .002 ----------------------------------------------------------. 001

(4) 4 .0001 .001 .002 .004 --------------------------------------------------. 002

(5) 5 .0001 .001 .002 .004 .008 ------------------------------------------. 003

(6) 6 .0001 .001 .002 .004 .008 .016 ----------------------------------. 005

(7) 7 .0001 .001 .002 .004 .008 .016 .032 --------------------------. 009

(8) 8 .0001 .001 .002 .004 .008 .016 .032 .064 ------------------. 02

(9) 9 .0001 .001 .002 .004 .008 .016 .032 .064 .13 -------. 03

(10) 10 .0001 .001 .002 .004 .008 .016 .032 .064 .13 .25 - .05

(11) 11-15 .10

Pr[F.] for each additional ANN beyond 10 = .25

(13) 21-40 .20

(14) >40 .25

The HEPs are for the failure to initiate some kind of intended corrective action as required. The

action carried out may be correct or incorrect and is analyzed using other tables. The HEPs include

the effects of stress and should not be increased in consideration of stress effects.

EF of 10 is assigned to each PrCF.I or PrCFi]. Based on computer simulation, use of an EF of 10 for

Pr[F . yields approximately correct upper bounds for the 95th percentile. The corresponding lower 1

bounds are too high; they are roughly equivalent to 20th-percentile rather than the usual 5th-percen-

tile bounds. Thus, use of an EF of 10 for the mean Pr[F.I values provides a conservative estimate1

since the lower bounds are biased high.

"Closely in time" refers to cases in which two or more annunciators alarm within several seconds or within a time period such that the operator perceives them as a group of signals to which he must

selectively respond.

Pr[F.] is the expected PrIF] to initiate action in response to a randomly selected ANN (or completely

dependent set of ANNs) in a group of ANNs competing for the operator's attention. It is the arithmetic

mean of the Pr[F Is in a row, with an upper limit of .25.1

Responses to ANNs - Maintenance or Calibration Operations

The Pr[F.] column in Table 11-13 is for use in reliability analyses in which the order of attending to annunciators cannot be predicted and the probability of failing to initiate action in response to a specific an- nunciator is of interest. This probability is taken as the arithmetic mean of the Pr[F.]s of the sounding annunciators, under the assumption that any annunciator has an equal chance of being first, second, or nth to be selected. Since the values in Table 11-13 are speculative, the estimates for Pr[F.] for more than 10 annunciators are grouped as shown in the table; i.e., 11 1to 15, 16 to 20, 21 to 40, and over 40 annunciators. Although Pr[F.] will never equal .25, this value is approached when n exceeds 40.

If the order of attending to some particular annunciator is predictable, the annunciator model would not be used. For example, at some plants the operators are instructed to look for certain annunciators when a turbine/ reactor trip occurs, e.g., safety injection. In such a case, if it is known that the operators are well versed in the rule, the HEP for noticing that safety injection has occurred would be estimated as 10 4, the same as the HEP for responding to one of one annunciators. This is because the operator is actively seeking that particular information, rather than sur- veying the entire array of tiles.

Multiple Annunciating Indicators, Two Operators

For the case of two operators, the error terms above should be modified as described in the section "One Annunciating Indicator, Two Operators."

One problem that was mentioned in both the control room review by Seminara et al (1976) and the Kemeny report (1979) is that as the number of annunci- ators sounding off increases, more personnel want "to get into the act," and a very confusing situation can arise, with errors in communication between people and, on occasion, one operator taking actions that negate the actions of another operator. There are no means of predicting such ef- fects--these are functions of plant discipline and administrative controls.

Responses to Annunciating Indicators - Maintenance or Calibration Operations

During certain calibration or maintenance procedures, the technicians may have to communicate with the control room operator by phone or intercom. For example, in calibrating setpoints that are alarmed in the control room, the technician will want to know whether the annunciator alarms at a cer- tain level. In this type of activity, the control room operator is an active partner and is fully alert to annunciated indicators. For this reason, we judge that his probability of responding to other annunciated indicators will not be degraded.

In other procedures, however, the control room operator may be passive in that alarms sound frequently, and he merely notes that these alarms are part of the calibration procedure and turns off the audio and the blinking light indications. We have observed that some operators become annoyed by

Response to ANNs - Transients, Startup, Shutdown

these interruptions and eventually may turn off the audio and blinking light without checking that the annunciating, indicator does indeed relate to the ongoing calibration or maintenance. Under these conditions, we assign an increase of an order of magnitude to the HEP for normal operating conditions when a single annunciator comes on. Thus, for the one-operator situation, the oversight probability would be .001.

If two or more annunciators sound more or less simultaneously the operator should recognize an indication that is not associated with the calibration procedure. If the procedure involves just one annunciator, the onset of two annunciators will be perceived by the operator as "something differ- ent." This arousal effect should offset the adaptation to the calibration procedure, and the model for normal operating conditions applies.

The above statements apply to annunciated legend lights. Ordinarily, an- nunciated printout equipment is not affected by calibration and mainte- nance, as described above.

Responses to Annunciating Indicators - Transients, Startup, and Shutdown

A transient is a perturbation in the power-generating mode of an NPP that requires reactor shutdown.* Tables I 4-9 and I 4-12 in WASH-1400 list the pressurized water reactor (PWR) and boiling water reactor (BWR) transients that were considered in that study. For most of those transients, the reactor will automatically trip, but for others (sometimes called Antici- pated Transients Without Scram [ATWS]), the operator will have to take action to control the situation. This may involve a rapid manual shutdown of the reactor. In any case, transient events are very demanding of an operator's attention. He will tend to concentrate all his attention on coping with the transient event and will be less likely to attend to annun- ciated signals not directly related to the transient. Also, as noted in simulator studies (Woods et al, 1982), once operating teams have diagnosed one transient, it may take them some time to recognize that there is another simultaneous transient.

Certain start-up and shutdown procedures are also very demanding of an operator's attention, reducing the probability of his responding to a safety-related annunciator in time. We estimate the same probabilities of failure to respond to annunciators related to transient events and to

Transients may be anticipated or unanticipated. In approximately 1800 reactor-years of large commercial and production reactors in the noncom- munist world, no unanticipated transients have occurred, while there are about 10 anticipated transients per reactor year. For purposes of HRA, we regard unanticipated transients as psychologically similar to a small LOCA.

1 I-5

Responses to ANNs - LOCAs

start-up or shutdown procedures as were estimated for annunciators in general. However, when the plant is in a transient mode or in the shutdown or start-up mode, the estimated probability of the operator failing to respond to an annunciated indicator not directly related to these condi- tions is increased by an order of magnitude. This increase applies to the values in Table 11-13 for annunciated legend lights and to the probability of failure to respond to an annunciated printout.

Responses to Annunciating Indicators - Loss-of-Coolant Accidents

When a large LOCA is simulated in a dynamic simulator, the noise and con- fusion are often overwhelming, even for experienced operators, and frequent errors of oversight and commission would be expected. When a small LOCA is simulated, there is more time to take reasoned action, but the diagnosis problem is more difficult, and the number of errors may be the same as with large LOCAs.

As yet we do not have a data bank of operator responses to LOCAs (simulated or actual), so estimates of human reliability in coping with LOCAs are highly conjectural. The HEPs listed in Table 11-13 allow for considerable degradation of operator performance under the stress of a large number of simultaneously sounding annunciators--we do not believe that these figures will degrade further under the stresses of a LOCA.

Given the high probabilities of effective automatic responses of engineered safeguard features to a LOCA, the most important potential post-LOCA errors are those made by operators during manual switching from the injection mode to the recirculation mode or at later times during the recirculation mode. In Problem 2 in Chapter 21, we present the WASH-1400 analysis of changing from the injection to the recirculation mode and include changes to the original HEP estimates based on this Handbook.

1-54

PART 3. RECENT DEVELOPMENTS

Operational Support Systems

PART 3. RECENT DEVELOPMENTS IN DISPLAYS

The displays discussed thus far are the conventional displays that are in use in most of the NPPs currently in operation. The development and avail- ability of new display concepts have not been ignored in the nuclear power industry, but the extremely long time involved in putting a plant into operation after the initial decision to build introduces a lag in the application of new developments. Cathode-ray tubes (CRTs) have been used in many process plants for a number of years and are now coming into use in NPPs. CRTs provide great flexibility in the manner in which information can be displayed and in the type of information that can be displayed. They have the advantage that the operator can call up any information of interest without leaving his position.

Problems in the use of CRT displays, also called visual display terminals (VDTs), are described in a number of reports compiled by the Visual Perfor- mance Technical Group of the Human Factors Society (Bhise and Rinalducci, 1981a and b). The problems related to use of CRT displays are readily correctable with the application of good ergonomics techniques. The dis- play problems are minor ones, such as occur with conventional displays: poor choice of print size, inadequate brightness, glare, and so on. Another set of correctable problems is related to work station dimensions: keyboard and screen height, screen angle, knee room, chair dimensions, and so on. Based on the reports in the two volumes edited by Bhise and Rinalducci, it is clear that the design of VDTs should incorporate good ergonomics techniques to avoid health problems (such as eye strain) and to maximize worker acceptance. The human factors considerations of most concern in the use of CRTs in NPPs are reported in Banks et al (1982). A final concern is that the display of information on CRTs be organized so as not to overload the operator. Felkel and Roggenbauer (1983) note that in operating plants, computer-generated listings displayed on CRTs and on printers may overwhelm the operator. They ask, "How should the operator be able to scan a listing containing almost two thousand messages (lines) to find the one or two that are missing?"

Operational Support Systems

Conventional displays present "raw" data, in the sense that the operator has to interrelate their readings and determine their implications. The new generation of displays is being designed to utilize computer logic systems to reduce a large amount of raw data to a small number in a format that is easier for the operator to handle. Such systems, which process the data in some manner to aid the decision-making process of the operator, are called operational support systems. Three such systems will be described.

Handling of Alarms with Logic (HALO)

The HALO program has been under development with the Halden Reactor Project in Norway (Visuri et al, 1981). The rationale of the program is to use computer logic to select the alarms that are most relevant to a situation

1/_55

Operational Support Systems

and to suppress all nonessential alarms. A similar concept, Nuclear Power Plant Alarm Prioritization (NPPAP), has been described by Roscoe (1982). In a study of one PWR, Roscoe observed that 83 alarms would sound within 2 minutes of a steam line leak but only 13 were essential to a diagnosis of the problem and maintenance of safety functions. The presence of the other 70 alarms tends to mask the 13 essential ones, making it difficult for the operator to select the ones that are relevant to his immediate decision- making requirements. The HALO and NPPAP concepts are intended to reduce the number of nonessential alarms that distract the operator from the es- sential ones. The intent is to advise the operator that a deviant condi- tion exists, allowing the relevant alarms to sound and suppressing the others. There is no "enrichment" of the displayed information; only status is presented, as with the annunciators currently in use. In the case of the situation described by Roscoe, the number of alarms could be reduced by over 80%, which would greatly reduce the load on the operator.

Safety Parameter Display System (SPDS)

The Safety Parameter Display System (SPDS) i's a display of selected safety- related parameters. The primary function of the SPDS is to help operators assess plant safety status quickly. It also aids in verification of auto- matic actions. The regulatory requirements for the SPDS are presented in NUREG-0696 (1981), which stipulates that ". . . the important plant func- tions related to the primary display while the plant is generating power shall include, but not be limited to:

Reactivity control Reactor core cooling and heat removal from primary system Reactor coolant system integrity Radioactivity control Containment integrity"

NUREG-0835 (1981) lists the human factors design review acceptance criteria to use in evaluating designs for the SPDS. Several designs have been evaluated by the nuclear power industry (Woods et al, 1982; Mullee et al, 1983; Buckley et al, 1982). The designs all used CRTs to present the information in a variety of formats, which included bar graphs, polaro- graphic displays, mimic displays, and time histories. The bar-graph dis- plays used an array of horizontal bars to depict the values of the param- eters. The bars were either green or red, to indicate normal or abnormal states, a color code that agrees with the population stereotype. The polarographic display (sometimes called "iconic polar display") resembled a wheel with concentric circles, in color, indicating normal and abnormal limits, with each parameter represented by a spoke of the wheel, the length of the spoke corresponding to its value. If it extended to the innermost (green) circle, the parameter was at its normal value. Beyond the green circle an intermediate yellow circle and an outer red circle indicated increasingly abnormal values.

The mimic displays of the PWR presented schematic diagrams of the reactor core and other critical plant system (Woods et al, 1982). In the BWR study

Comment on Recent Developments

(Mullee et al, 1983), the mimic display depicted the reactor vessel, with vertical bars for water level, pressure, etc. The time history displays presented the data in the form of a graph for each parameter, plotting parameter value against time for a fixed interval, such as 6 minutes or 30 minutes, prior to the most recent update. This allowed the operator to observe the recent trend of the parameter.

The evaluations were carried out in training simulators. No data comparing performance with and without the SPDS were gathered in these preliminary evaluations, but much was learned about the relative use of the different data available from the displays. A number of user comments indicated a need for improvement in such areas as scaling of displays, labeling, read- ability, etc., which can be implemented in the operational versions of the SPDS. Probably the most important finding was that user acceptance of the SPDS was high--the operators seemed to appreciate the efficient data pres- entation of which the CRT was capable.

Disturbance Analysis and Surveillance Systems

Disturbance Analysis and Surveillance Systems (DASS) are computer-based systems that process data from the plant and display the results in their order of importance, with the objective of improving plant safety and availability. The concept of DASS is that of a system that "looks at" all the important plant parameters, those concerned with operational availa- bility as well as safety. Whereas SPDS is a safety-oriented system, addressing about 20 parameters, DASS will integrate data from about 2,000 parameters or more.

Early work on the DASS concept has been described by Bastl and Felkel (1981) and Felkel and Roggenbauer (1983). They also describe the current development work in Europe. Recent work on DASS in the United States has been conducted by the Electric Power Research Institute (EPRI) and Sandia National Laboratories (SNL) in parallel programs to assess the scope and feasibility of developing a plantwide DASS (Long et al, 1980). All the work on the DASS concept is premised on the use of CRTs to display the information. A variety of display formats has been studied that are appli- cable to various functions that would be performed by the system, such as identifying a disturbance, predicting equipment failure, suggesting opera- tor response, etc. The flexibility of the CRT allows any of these outputs to be displayed.

Comment on Recent Display Developments

Although the display concepts described above have not as yet been imple- mented in operating NPPs in the United States, they are indicative of the improvements we may expect in NPP displays within the next few years. As yet, we do not have any experience to draw on for quantitative estimates of the extent to which operator errors will be reduced through their use, but the concepts are in accord with sound human factors engineering principles, and we anticipate a substantial reduction in error probabilities as the new designs become available. The Japanese experience with computer-coupled

11-57

Comment on Recent Developments

color CRT displays in the Fukushima Daini Boiling Water Reactor Unit One, completed in 1981, has been reported as favorable, although no quantitative data on operator performance were described, and plans are underway to increase the use of computer-processed information and CRT displays in plants now being designed in Japan (Itoh et al, 1982). In the United States, all existing NPPs will be retrofitted with SPDS, in compliance with NUREG-0696 (1981).

We can conclude that the use of new display techniques holds great promise for increased human reliability in the operation of NPPs. The potential exists for organizing and presenting information that will make it rela- tively easy for an operator to keep abreast of normal conditions in the plant'and to cope successfully with abnormal conditions. However, it will still be necessary for some time to come for the operators to maintain proficiency in using conventional displays and controls for those situa- tions in which the CRT systems may fail.

Because no one knows precisely how the new display systems will affect operator performance, this issue of the Handbook is restricted to conven- tional display technology such as that in presently operating NPPs. We anticipate that some of our estimated HEPs related to displays and controls will be reduced after operating experience is obtained with new visual display terminals in NPPs.

11-58

Ch. 12. Diagnosis of Abnormal Events Overview: Abnormal Events in HRA

CHAPTER 12. DIAGNOSIS OF ABNORMAL EVENTS*

Overview

In a probabilistic risk assessment (PRA), it is important to analyze and estimate human error probabilities (HEPs) of control room operator response to specified events that result in abnormal situations in a nuclear power plant (NPP). Cognition is involved in this response, e.g., deciding what course of action to take in coping with an abnormal event. We could find no quantitative data on cognitive behavior in a form suitable for the human reliability analysis (HRA).done in a PRA. Therefore, we are forced either to ignore this important aspect of human behavior or to develop an interim model that incorporates those cognitive aspects that are the most critical in an HRA. In choosing the second course, we have addressed that part of cognitive behavior that involves the interpretation by control room-person- nel of patterns of signals to diagnose the nature of the abnormal event or at least to decide which course of action to take.

This chapter describes the kinds of abnormal events for which human errors are usually assessed in a PRA, discusses some problems in quantitative modeling of cognitive processes, and provides a rationale for restricting our treatment of cognitive behavior to diagnosis. Two diagnosis models are presented, one for initial screening in a PRA and a nominal model for subsequent analysis. Guidelines are given for the subjective judgment required to apply these models. The section on initial screening includes screening values for the nondiagnostic aspects of responding to an abnormal event. Finally, we note the need for plant and simulator collection of error relative frequencies and subjective estimates of diagnosis error probabilities. Such information can be the basis for the development of better models of operator response to abnormal events in NPPs.

Abnormal Events Considered in HRA/PRA

Only certain specified abnormal events are generally considered in a PRA. These events are initiating events, i.e., events that require the plant to trip. Initiating events are classified either as external events, such as fire, flood, or earthquake, or internal events. The latter are divided into loss-of-(primary-)coolant accidents (LOCAs) and transients. A transi- ent is a condition that causes a requirement for reactor shutdown not caused by a LOCA. (For PRA purposes, a loss of secondary coolant is clas- sified as a transient, not a LOCA.) There are other internal abnormal events that are not usually considered in PRAs, such as a leak in the spent fuel pool.

Some of the material in this chapter was presented at the Eighth Congress of the International Ergonomics Association (Swain, 1982b).

12-1

Problems in Modeling Cognition

In addition to initiating events, PRAs also consider consequences. Conse-

quence analysis refers to the analysis of health and financial effects resulting from a release of radioactive material that resulted from some initiating event or events. Associated with consequence analysis is an

analysis of emergency response, e.g., evacuation of personnel who could be affected by the release of radioactive material.

To date, HRAs performed as parts of PRAs have been applied only to the

activities related to internal initiating events (LOCAs and transients). In this chapter, the emphasis is on the same events, but the models can be applied to other abnormal events displayed in the control room. We do not

address operator response to external initiating events except insofar as they produce the same plant reactions as are produced by internal initi-

ating events. Neither do we address consequence analysis or emergency response.

Some Problems in Modeling Cognitive Behavior for PRA

It is suggested that the reader review the first parts of Chapter 3, which

introduced a "simplified model of the human component in a man-machine system for the purpose of probabilistic risk assessment," illustrated in

Figure 3-1. For convenience, this figure is reproduced as Figure 12-1.

Treatment of Cognitive Behavior for PRA

Cognitive behavior is complex. Cognition is defined as "the act or process

of knowing including both awareness and judgment" (Webster, 1975). For purposes of PRA, we must be more restrictive. Our models of cognitive behavior are limited to the diagnosis of abnormal events displayed in the

control room. For those interested in other aspects of cognitive behavior and in theories of cognition in general, the references footnoted below are

recommended.* Readers in this area will note an absence of models that estimate the reliability of cognitive processes in applied situations. This is not a criticism of researchers in this most difficult and important

area of human behavior since it is still largely theoretical. Our study of

the area indicated that we could not use the same approach to the develop- ment of a model for the cognitive aspects of NPP operator tasks that we

have used in developing other human performance models in the Handbook.

Problems and points of view in the area of cognitive modeling are addressed in volumes edited by Kaplan and Schwartz, 1975; Sheridan and

Johannsen, 1976; Lucaccini, 1978; Rasmussen and Rouse, 1981; and Sheridan et al, 1982. Other books dealing with the topic are those by Neisser, 1976; Janis and Mann, 1977; and Norman, 1981b. Reports by Wohl (1981 and 1982a and b) describe cognition applied to military decisions and to main-

tenance of military equipment; several of Wohl's concepts could be applied to NPP operations.

12-1.

Figure 12-1

A EXTERNAL INPUTS TO HUMAN VIA SIGNALS, SIGNS, & CONDITIONS

DISPLAYS: VISUAL AUDITORY VIBRATORY OLFACTORY ETC

COMMUNICATION: ORAL WRITTEN

ENVIRONMENTAL CONDITIONS

CORRECT/INCORRECT HUMAN COMPONENTS

APPARENT PATH FOR WELL-PRACTICED ASSOCIATIONS A-INTERNAL FEEDBACK LOOP--I

B I D o INTERNAL INPUTS MEDIATING RESPONSES

TO HUMAN VIA (COGNITIVE) (HUMAN SENSING, ACTIVITIES & OUTPUTS TO DISCRIMINATION, PROCESSES SYSTEM) AND PERCEIVING1

FILTERING, ORGANIZING, & RECOGNIZING VIA: SENSE ORGANS (RECEPTORS) & MEDIATING ACTIVITIES OR PROCESSES

PROCESSING OF PERCEIVED INFORMATION INCLUDING EFFECTS OF MEMORY, GOALS, INTENTIONS, PERSONALITY (ATTITUDES, MOTIVATION, EMOTIONS, ETC), INTERPRETATION & DECISION- MAKING

NO ACTION; USE OF HAND, FOOT, OR OTHER BODY MEMBER TO CONTROL SOME NPP FUNCTION, OR ISSUANCE OF ORAL OR WRITTEN COMMANDS OR INFORMATION

E EXTERNAL RESULTS ON MAN-MACHINE SYSTEM

CORRECT OR INCORRECT HUMAN INPUTS TO SYSTEM VIA:

NO ACTION SWITCHES VALVES COMPUTER KEYBOARD CRANE HANDLING EQUIPMENT & TOOLS CONNECTORS INSTRUCTIONS

I I II

L -EXTERNAL FEEDBACK LOOP----

Figure 12-1 A simplified model of the human component in a man-machine system for the purpose of probabilistic risk assessment. (Note: This is a repeat of Figure 3-1.)

12-3

Problems in Modeling Cognition

Instead, we have developed tentative models that are based entirely on speculation--ours and others'. The user of these models will have to employ considerable judgment.

In analyzing operator response to abnormal events, the basic needs for PRA are to estimate how long it will take control room personnel to diagnose the nature of the unusual event correctly and to estimate the HEPs that may occur in carrying out the rule-based or skill-based activities (Rasmussen, 1981) that are appropriate to coping with the event. In making these estimates, it is useful to separate the three primary components of behavior involved so that available data or models may be applied to each. These are as follows:

(1) Internal Inputs (Perception and Discrimination) (Box B, Figure 12-1) (2) Cognitive Activities (Interpretation and Decision-Making) (Box C) (3) Response (Action) (Box D)

It is first necessary for the operators to recognize that something unusual has happened and to distinguish the relevant signals; these are functions of perception and discrimination. Although this is an involved perceptual process, we treat this as primarily a display and communication problem. To estimate HEPs related to these aspects of the operator response, other chapters, especially Chapter 11, "Displays," provide derived data.

Having discerned that something unusual is happening, the operating per-

sonnel must diagnose* the problem and decide what action to take: this involves interpretation and decision-making, the primary subject matter of this chapter. Finally, actions must be carried out (the response). Again, other chapters provide derived data relevant to this aspect of the HRA, especially Chapter 15, "Oral Instructions and Written Procedures."

A Comment on Errors of Judgment and Other Cognition-Related Terms

Webster (1975) defines judgment as "the process of forming an opinion or evaluation by discerning and comparing." This definition is very broad and can, be construed to include perceptual as well as cognitive aspects of behavior. The phrase "errors of judgment" has been used broadly in the sense that if someone performs an action that subsequently proves to have been inappropriate, the performer is deemed guilty of an error of judgment. Often no attempt is made to determine the role of the performer's training and practice, work material, or environment in incorrect performance. In aircraft accident investigations, for example, pilot errors have been classified as errors of judgment even though they were obviously associated with difficulty in reading or interpreting a display (Hurst, 1976).

It is understood that diagnoses may be incorrect as well as correct; in this text we use the word to mean correct unless we specify otherwise.

12-4

Problems in Modeling Cognition

As in the aviation field, errors in NPP operations are sometimes classified as errors of judgment although the operators were responding in accordance with procedures that had been overemphasized in training but which were inappropriate to a particular abnormal event. This type of erroneous classification occurred in some evaluations of operator actions in the Three Mile Island (TMI) incident. In addition, some evaluations overlooked the fact'that some so-called errors of judgment at TMI were related to serious display inadequacies.

Our position is that the term "judgment" is inadequate for HRA because it

is too imprecise. We therefore prefer to use more definitive terms such as perception, discrimination, interpretation, diagnosis, and decision-making (two levels). Table 12-1 presents dictionary definitions of these terms as well as of cognition, judgment, and action and briefly describes how these terms are used in the Handbook.

The general, but not invariable, sequence of activities in a control room after it is realized that some problem exists is indicated in Table 12-1, beginning with the word "perceive." The sequence is not fixed; there are various loops in this interactive and complex process of responding to an unusual event, as indicated in Figure 12-2. A more detailed breakdown of the cognitive aspect of human behavior is found in Rasmussen (1980), which describes the iterative nature of the human response to a system perturbation.

To illustrate the use of the terms in Table 12-1 and to show the limita- tions of our diagnosis models, following is a representative sequence of activities after an abnormal event (in this discussion, the term "operator" refers to the team of personnel present following the abnormal event--see Chapter 18, "Staffing and Skill Levels"):

(1) Abnormal Event Begins. If it is a slowly developing problem, some time may pass with the only indications being changes in information displayed in meters, recorders, etc., in the control room. As noted in Chapter 11, "Displays," the operator is not likely to perceive such indications quickly. Generally, no credit is given in a PRA for human intervention in the abnormal event until more obvious indications occur.

(2) Perception of Abnormal Condition. Some compelling indication of trouble occurs; this usually consists of one or more annunciators alarming. At this point, the operator will notice that annunciator tiles are blinking and will turn off the auditory signals (as dis- cussed in Chapter 11).

(3) Discrimination of Indications of Abnormal Event. The operator moves close enough to read the blinking displays; i.e., he discriminates the message conveyed by the annunciator tiles. If he makes an incorrect discrimination, he will very likely make an incorrect interpretation.

17-5

Table 12-1

Table 12-1 Definitions of cognition-related terms and usage in the Handbook

Term Dictionary Definition* Handbook Usage

Cognition

Judgment

Perceive

Discriminate

Interpret

Diagnosis

the act or process of knowing, including both awareness and judgment

the process of forming an opinion or evaluation by discerning and comparing

to attain awareness or under- standing; to become aware through the senses

to mark or perceive the dis- tinguishing or peculiar fea- tures of; to distinguish one like object from another

to conceive in the light of

individual belief, judgment, or circumstance

a statement or conclusion concerning the nature or cause of some phenomenon

restricted to those aspects of behavior involved in

diagnosis of abnormal events

not used in our models--too imprecise; used only in the context of expert estimation

used in the very narrow sense of "awareness" with- out the further meaning of "understanding," e.g., "some annunciator tiles over there are blinking"

distinguishing one signal (or a set of signals) from another, e.g., "the coolant level in Tank A is 37 feet," or if there are limit marks on the meter, "the coolant level is out of limits" (in the latter case, some interpretation is done for the operator by the design of the display)I

the assignment of a meaning to the pattern of

signals (or stimuli) that was discriminated, e.g., "the coolant level in Tank A is low, which means that the make-up pump is not running, or there is a leak somewhere, or the indicator is out of order"; if there is only one possible cause for the observed signal, the interpretation is equivalent to diagnosis

the attributing of the most likely cause(s) of the abnormal event to the level required to identify those systems or components whose status can be changed to reduce or eliminate the problem; diagnosis includes interpretation and (when necessary) decision-making

"decision-making" used instead of "deciding"

(1) decision-making as part of diagnosis: the

act of choosing between alternative diagnoses, e.g., to settle on the most probable cause of the pattern of stimuli associated with an abnormal event

(2) postdiagnosis decision-making: the act of

choosing which actions to carry out after a diagnosis has been made; in most cases, these actions are prescribed by rules or procedures, and decision-making is not required

carrying out one or more activities (e.g., steps or

tasks) indicated by diagnosis, operating rules, or written procedures

0 Decide to make a choice or judgment

Decision- Making

Action a thing accomplished usually

over a period of time, in stages, or with the possibility of repetition

Webster (1975)

1 2-6

Figure 12-2

.PERCEIVE

DISCRIMINATE

'DIAGNOSIS

Figure 12-2 Cognition wheel.

12-7

Problems in Modeling Cognition

(4) Interpretation of Discriminated Signals. The operator assigns meaning to the indications he has discriminated. If he makes an incorrect interpretation, he will likely make an incorrect diagnosis. In some cases, interpretation is equivalent to diagnosis because there is only one possible cause of the pattern of signals displayed, or there are operating rules that require the operator to perform certain actions without delay or further diagnosis (e.g., "if safety injectionhas occurred, initiate the procedures for response to a loss-of-coolant accident"). In many cases, the onset of certain annunciator tiles will direct the operator to information displayed on related instru- ments, and these indications will become inputs to the task of interpretation.

(5) Diagnosis of the Discriminated Signals. On the basis of his interpre- tation of the meaning of the signals he has discriminated, the opera- tor selects the most likely cause or causes of the abnormal events. In thinking of "cause," the operator's primary concern is a cause that will lead him to action (or deliberate inaction) that will control the abnormal event or at least will mitigate its likely undesirable effects. If more than one possible cause is involved, the first level of decision-making, as defined below, is required.

(6) Deciding Between Alternative Diagnoses. When there is more than one possible cause for an abnormal event, the operator must select one to act upon, at least as an hypothesis. Skilled operators may keep more than one diagnosis in mind while trying certain actions to see which is most likely to be the correct one. This type of decision-making is part of the diagnosis itself and may be highly iterative and time- consuming (as exemplified by the TMI incident). Our diagnosis models end at this point.

(7) Deciding What Actions To Take after Diagnosis. After the operator has made a diagnosis, he must decide what actions to. carry out. In most cases, this postdiagnosis decision-making is of a simple nature; most of the responses are prescribed by rules or written procedures, and the operators usually follow these established practices, although they may use different strategies in carrying them out. With the advent of improved emergency operator procedures and improved training and practice requirements since the TMI incident, it seems likely that after an abnormal event has been diagnosed, correct decisions as to what action to take will follow. The cases that are not covered by procedures or by special training are the exceptional instances that we are unable to anticipate. Therefore, the models in this chapter do not specifically include postdiagnosis decision-making. We do not offer an HEP for this activity, but we believe that the uncertainty bounds (UCBs) associated with the HEPs in the sections that follow are broad enough to include the occasional error of decision-making that may take place following a diagnosis. When the analyst believes that a particular case of postdiagnosis decision-making warrants analysis, he must use sources other than the Handbook. Due to the lack of data

12-8

Problems in Modeling Cognition

on this aspect of human performance, the use of psychological scaling techniques may be required (Chapter 8).

(8) Carrying Out the Actions Indicated by the Diagnosis. Generally the actions to be carried out will be rule-based activities, and the relevant HEPs and human performance models for such noncognitive activities are found in other chapters. However, these actions may provide display feedback to the operator that indicates that the previous diagnosis was incorrect or incomplete. Our diagnosis models do not treat this iterative aspect of cognitive behavior. We believe that the UCBs associated with our models are sufficiently broad to include such contingencies.

There is, however, one problem involving postdiagnosis decision-making that has been addressed in some PRAs. This is the case in which the diagnosis of the problem calls for actions that have a potential for serious economic impact on the plant or that require the operator to perform actions that he normally should not perform. An example of the latter situation is a LOCA in which subsequent events indicate that the operator should turn off (or throttle way down) the high- pressure injection system after its automatic initiation; whereas, after TMI, there may be a natural reluctance to disrupt the emergency coolant flow. Cases like this one, or ones involving economic aspects, can result in a conflict of operator motives. Such cases must be treated as plant- and situation-specific, and no prescriptions can be offered. One general approach has been to consider such cases as highly stressful to the operators (Caplan et al, 1975, Appendix D) and to estimate how long a delay in implementing the correct proce- dures would result from this conflict. Estimates of such postdiag- nosis decision-making activities are speculative.

(9) Repetition of Any or All of the Above Activities. We emphasize that the human response to abnormal events is highly interactive and itera- tive. Our modeling representation of the role of the human does not include this complexity, but the model can be reentered at any point the analyst chooses.

Some Problems in Estimating Diagnostic Errors

There are several problems facing the system analyst and human reliability analyst in assessing how long it will take control room personnel to diag- nose any unusual event in an NPP. Some of the more difficult problems for PRA are as follows:

(1) The event may be slow in developing so that considerable time elapses before it is recognized that something is wrong. Therefore, models that assume that the onset of an abnormal event will be noticed quickly can be in error. A model for diagnosis of abnormal occur- rences should begin with the onset of compelling signals such as annunciators. However, detection time based on less compelling indi- cations (e.g., periodic scanning of control boards) can be estimated

12-9

Problems in Modeling Cognition

using the display models in Chapter 11 or the, estimated HEPs in using written procedures presented in Chapter 15.

(2) After an operator has recognized that there is a problem, there may be some delay in calling for assistance from other operators, the shift supervisor, or other technical personnel such as the shift technical advisor. Care must be taken not to give credit for people who may not be present in the control room or who are involved in other duties.

(3) Following an abnormal event, the roles of different people in the control room vary considerably from plant to plant. In some plants, supervisors take over complete direction, while in others, they oper- ate in a more interactive manner. Even when there are four people in a control room, not all of them would be involved in every control action taken; thus, estimates of dependence among control room person- nel are subject to considerable uncertainty.

(4) The training and onsite experience of control room personnel and tech- nical advisors in coping with abnormal events vary greatly. For some plants, the simulators on which training and practice are received provide reasonably accurate representations of what would occur in the operator's plant, at least for the initial diagnosis phase. For other plants, the simulator is much less representative of plant responses to abnormal events. Some personnel are not given sufficient training in pattern recognition for various unusual events; instead, training is largely devoted to following the appropriate set of written proce- dures, given that the event has been recognized and diagnosed.

(5) For some transients and LOCAs (e.g., a repeat of the TMI incident), the control room personnel at a plant may be so well trained in what they must do that errors in the cognitive aspects of behavior may be very unlikely (their responses are almost reflexes). For other inci- dents, in which training is less emphasized, or in which the time to cope with the incident is short, or when there are multiple problems, errors in the cognitive aspects are likely.

(6) With the advent and probable acceptance of symptom-oriented proce- dures, (also called function-oriented procedures) it is possible that the need to diagnose an unusual event, at least the need to do this early in the event, may diminish in importance for PRA. That is, if the intent of these new procedures can really be fulfilled so that the safety of the plant can be assured, whether the operators can quickly catalog the abnormal event correctly may cease to be important. For the time being, however, we must base our cognitive models on current written procedures that are not symptom-oriented in most cases. However, in Table 12-1, we have defined diagnosis as the attributing of the most likely cause(s) of the abnormal event, where "cause" is defined in terms of systems or components whose status can be changed to reduce or eliminate the problem. Correct diagnosis, then, does not require that the operators correctly catalog (name) the initiating event or have full understanding of the nature of the event. All the

12-10

Problems in Modeling Cognition

correct diagnosis entails is that the operators correctly decide on what actions to take to cope with the initiating event.

(7) With the probable growing emphasis on Operational Support Systems, such as software-driven cathode-ray tube (CRT) displays of safety parameters described in Chapter 11, and as additional guidance for diagnosis is included in the displayed information, two positive results can be anticipated: the probabilities of diagnosis errors should decrease, and the requirement for diagnosis itself should be, lessened as rule-based behavior is substituted for cognitive behavior. One important negative result can be anticipated: unless the opera- tors have sufficient practice in coping with unusual events without the help of the new equipment, the operators' diagnosis skills will decline. If events occur that are associated with either a breakdown

.of the CRT-displayed information or with situations not foreseen in the computer software design, the probability of correct diagnosis by the operators is likely to be very low. In other systems (e.g., commercial aircraft systems), maintenance of the necessary diagnosis skills is provided by the use of partial panel operation, usually in dynamic simulators, in which operators must respond to problems with- out the use of a full panel of displayed information.

Despite these and other problems, it is necessary, for PRA purposes, to develop interim models for analysis of diagnosis errors even though these models will be subject to large uncertainty. As more data are obtained, better estimates of diagnosis errors can be formed, and the uncertainties can probably be reduced. Also, as we learn more about symptom-based pro- cedures and the new Operational Support Systems, we will be able to assess their effects on correct operator response to transients and other unusual events.

The models described in the remainder of this chapter constitute a depar- ture from our usual human performance models. Other models in the Handbook pertain to the performance of a single person, and the dependence model (or its equivalent) is used to estimate the joint performance of the team of persons who would be involved in carrying out various procedures. The diagnosis models, on the other hand, represent the performance of a typical team and require no adjustment for the effects of dependence among person- nel. (However, as will be seen, our dependence model was used in deriving the estimates of team performance in the models presented in this chapter.) The team of people assumed for purposes of PRA is defined in Table 18-2.

Another major difference between the models in this chapter and those in other chapters is that diagnosis models are based in part on a consensus of PRA practitioners and have less basis in behavioral data and theory than the other models. For these reasons, the diagnosis models must be con- sidered as highly speculative.

12-11

Initial-Screening Models

Initial-Screening Models for Operator Diagnosis and Postdiagnosis Performance

In PRA, initial screening involves the assignment of conservative (high) failure probabilities to each system event or human task as an initial type of sensitivity analysis. If a screening failure probability does not have a material effect in the system fault tree or system event tree, it may be dropped from further consideration.

Two initial-screening models are presented, one for the diagnosis of an abnormal event by the control room team and the other for carrying out the appropriate activities after the diagnosis has been made. In both cases, estimated HEPs and UCBs are listed. The UCBs are presented as error factors (EFs). Since initial screening must be conservative, downward adjustments to the HEPs should not be made, although upward adjustments may sometimes be appropriate. Optimistic screening values can result in fail- ure to consider human errors properly in analyses subsequent to the initial screening.

Both initial-screening models are based in large part on a group consensus reached at the National Reliability Evaluation Program (NREP) Reliability Data Workshop on April 14-15, 1982 (Oswald et al, 1982). For the diagnosis model, the participants used the approach described by Wreathall (1982a, p 104, and 1982b), but as suggested by J. R. Fragola (Hall et al, 1982, p 20), assumed lognormality for time to diagnosis rather than assuming that the probability of failure is a logarithmic function of time. UCBs for the estimates in both models were also derived by consensus of the group. As described later, we suggested slight modifications to the diagnostic model- to bring it into closer correspondence with the large LOCA curve (Figure 17-2) and our dependence model. The resultant model is the one presented in Oswald et al (1982) and in the NREP Procedures Guide (Bari et al, 1982, Appendix 5A). In this Handbook, we have expanded the model to account for cases in which more than one abnormal event may occur at the same time, and we have modified the postdiagnosis procedures model to make it more con- servative than the model described in the above references.

Initial-Screening Model for Diagnosis of Abnormal Events

Figure 12-3 shows the initial-screening model for the probability of opera- tions team diagnosis error in the case of a single abnormal event. The solid line plots the median joint HEP for control room personnel against time after T0 , where To represents the time at which they notice that some abnormal condition exists; that is, the curve shows the probability of a team not diagnosing an abnormal event by a given elapsed time, T. The dotted lines represent the lower and upper UCBs. T is defined as the onset of a compelling signal of an abnormal event, usually consisting of one or more annunciators. Therefore, when the initial screening model for diagnosis is used, it is not appropriate to derive a separate estimate of the probability that the compelling signal will be observed--a 1.0 proba- bility is assumed.

12-12

Figure 12-3

I-

0 z 0

.01

.001

.0001

.000001

.000001 1 10 100 1000

To TIME T IN MINUTES AFTER A COMPELLING SIGNAL OF AN ABNORMAL SITUATION

Initial-screening model of estimated HEPs and UCBs for diagnosis within time T of one abnormal event by control room personnel (based on Oswald et al, 1982).

Figure 12-3

12-13

Initial-Screening Models

The left half of Table 12-2 lists the point values (and related EFs) from which the plots in Figure 12-3 were drawn. These represent the modified consensus estimates described above for the case of a single abnormal event. These estimates are probably too optimistic as screening HEPs for multiple faults, e.g., a loss of feedwater combined with a failure of the reactor to scram automatically. For the case of two or more abnormal

events annunciated closely in time, we treat the first fault diagnosed as a single fault, applying the above screening values. "Closely in time" refers to cases in which the annunciation of the second abnormal event occurs while the control room personnel are still actively engaged in diagnosing and/or planning the responses to cope with the first event. This is situation-specific, but for the initial analysis, use "within 10 minutes" as a working definition of "closely in time." Based on simulator research (Woods et al, 1982), we assume that the plot of diagnosis HEPs by time is not changed for the first fault to be diagnosed but that the second fault to be diagnosed will take noticeably longer. In a PRA, the first fault would be designated as the most obvious fault and could be determined in interviews with skilled operating personnel. Also based on the above simulator study, for the second fault, we add a constant 10 minutes to the values for the first fault to determine the estimated HEP-time plot for the diagnosis of a second fault. The right half of Table 12-2 shows HEP-time values for the control room personnel's diagnosis of the second fault. As one would expect, the constant of 10 minutes has greater effect early in the development of an abnormal event, when it can be assumed that the stress and confusion levels of the operating personnel are higher. For points between the times shown in the table, the medians and EFs may be chosen from Figure 12-3.

As Table 12-2 indicates, an HEP of 1.0 is assigned as the screening esti- mate for the diagnosis of the third and subsequent abnormal events occur- ring closely in time. This deliberate conservatism should ensure that the postscreening analysis would include the human contribution to the unusual combination of more than two closely occurring abnormal events, should the combination itself survive the initial screening.

Initial-Screening Model for Postdiagnosis Performance

Our screening values for postdiagnosis performance apply to the activities to be carried out by control room personnel following diagnosis of the problem (whether or not the diagnosis was correct). It is understood that certain actions will be taken by the operators almost automatically in response to the onset of an abnormal event, but only after the condition has been diagnosed will the operators refer to the appropriate written procedures to cope with the event. Chapter 15 presents HEPs for following written procedures under a variety of realistic conditions, but for screen- ing purposes, we deliberately use very high values.

For screening, we recommend an HEP of .05 for carrying out activities based on written procedures when recovery factors are not considered and an HEP of .025 when recovery factors are included. An EF of 3 is assigned in each case for estimating UCBs. Our rationale is that .05 is the upper bound for

~2-i4

Table 12-2

Table 12-2 Initial-screening model of estimated HEPs and EFs for

diagnosis within time T by control room personnel of abnormal events annunciated closely in time*

Median Median joint HEP for joint HEP

T diagnosis of T for diagnosis (Minutes** a single or (Minutes** of the

Item after T ) the first event EF Item after T ) second event EF 0 0

(1) 1 1.0 -- (7) 1 1.0 --

(2) 10 .5 5 (8) 10 1.0 --

(3) 20 .1 10 (9) 20 .5 5

(4) 30 .01 10 (10) 30 .1 10

(11) 40 .01 10

(5) 60 .001 10

(12) 70 .001 10

(6) 1500 (= 1 day) .0001 30

(13) 1510 .0001 30

"Closely in time" refers to cases in which the annunciation of the second abnormal event occurs while CR personnel are still actively engaged in diagnosing and/or planning responses to cope with the first event. This is situation-specific, but for the initial analysis, use "within 10 minutes" as a working definition of "closely in time."

Note that this model pertains to the CR crew rather than to one individual

For points between the times shown, the medians and EFs may be chosen from Figure 12-3.

t To is a compelling signal of an abnormal situation and is usually taken as a pattern of annunciators. A probability of 1 .0 is assumed for observing that there is some abnormal situation.

.tt Assign HEP = 1.0 for the diagnosis of the third and subsequent abnormal events annunciated closely in time.

12-15

Nominal Diagnosis Model

some HEPs of .01 for rule-based behavior, and this seems to be appropriate for screening purposes. The .025 value, when recovery is included, is based on allowing a modifier of .5, which is the upper bound on the nominal HEP of .1 for typical monitoring activities in which one person checks the performance of another (Chapter 19, "Recovery Factors"). These screening values should be modified to account for any dependence among critical tasks done by the same or different operators.

The above screening values of .05 and .025 apply to the cases in which written procedures are available to the operators. If an abnormal event is being analyzed for which written procedures are not available, further evaluation is mandatory, and a screening HEP of 1.0 should be assigned. This HEP can be reassessed in the subsequent analysis, when more realistic failure terms are developed.

At the NREP workshop, several participants advocated lower screening values (.01 and .001 were suggested instead of the .05 and .025 recommended here). Objections to the recommended values were based on the assumption that the higher values would result in failure to screen out any rule-based activi- ties, since the application of these HEPs to every step in a multistep procedure would almost always yield a joint failure probability requiring further analysis. Actually, in the typical HRA done as part of a PRA, there has already been considerable screening of human error terms by the usual method, in which only those potential human errors considered to be important by system analysts are included in the system event and fault trees. That is, from the very beginning of the system analysis, some elimination of possible human errors has already taken place.

In reviewing the written procedures, the analysts ascertain which particu- lar steps are the critical ones and apply the HEPs only to those, not to every step in the procedures. The HEP of .01 suggested at the workshop is no greater than the estimated HEPs in this Handbook for routine tasks in which a long list of procedures is used without a formal checkoff. If this HEP were used as an initial-screening value, it might result in the screen- ing out of system-critical rule-based activities from further analysis. We believe that because of the large uncertainty in estimating the effects of human errors in a PRA (Vesely, 1983), every precaution should be taken to avoid inadvertently excluding such effects from further consideration in a PRA.

Table 12-3 summarizes our recommendations for screening values for the system-critical rule-based behavior following diagnosis of an abnormal event. The EFs are large, reflecting the greater uncertainty in the assignment of screening HEPs than nominal HEPs.

Nominal Model for Diagnosis of Abnormal Events

The nominal model is used subsequent to the initial screening in the HRA. This model is restricted to diagnosis of single and multiple abnormal events; it does not include postdiagnosis activities. In analyzing the performance of the postdiagnosis activities, the models and derived data in

12-16

Table 12-3

Table 12-3 Initial-screening model of estimated HEPs and EFS for rule-based actions by control room personnel after diagnosis of an abnormal event*

Item Potential Errors HEP EF

Failure to perform rule-based actions correctly when written procedures are available and used:

(1) Errors per critical step without .05 10 recovery factors

(2) Errors per critical step with .025 10 recovery factors

Failure to perform rule-based actions correctly when written procedures are not available or used:

(3) Errors per critical step with or 1.0 without recovery factors

Note that this model pertains to the CR crew rather than to one individual.

12-17

Nominal Diagnosis Model

the other chapters in this Handbook can be used. In all cases, appropriate consideration must be given to the higher levels of stress and dependence that may be associated with the response to abnormal events (Chapters 17 and 18). The nominal model for diagnosis includes the activities listed in Table 12-1 as "perceive," "disciminate," "interpret," "diagnosis," and the first level of "decision-making." The model includes those aspects of behavior included in the Annunciator Response Model; therefore, when the nominal model for diagnosis is used, the annunciator model should not be used for the initial diagnosis. The annunciator model may be used for estimating recovery factors for an incorrect diagnosis.

The nominal model for diagnosis of abnormal events represents an attempt to derive some response times and HEPs that are more realistic than the overly conservative estimates in the initial-screening model. The nominal model is just as speculative as the screening model; at present, there is no sufficient body of data that would enable us to derive a data-based model. Hence, our nominal model is merely a gross approximation that is adequate for most PRA purposes. As usual, when our knowledge about some aspect of human behavior is slight, our estimated HEPs will be conservative and the associated UCBs will be wide.

The left third of Table 12-4 lists the HEPs and EFs for the diagnosis of a single abnormal event or the first of more than one abnormal event when the events are annunciated closely in time. As for the initial-screening diag- nosis model, "closely in time" refers to cases in which the annunciation of the second (or more) abnormal event occurs while the control room personnel are still actively engaged in diagnosing and/or planning the responses to cope with another event. This is situation-specific, but for the initial analysis, use "within 10 minutes" as a working definition of "closely in time." The HEPs were obtained by shifting the initial-screening model HEP estimates from Table 12-2 for a single abnormal event one time step upwards after the first minute. The diagnosis HEPs and EFs for one abnormal event are plotted in Figure 12-4 to facilitate interpolation.

The nominal diagnosis model provides probabilities of failure to correctly diagnose an abnormal event within time T. The model does not indicate how many incorrect diagnoses may have been made. If one or more incorrect diagnoses of an abnormal event are plausible, the PRA team must estimate their probabilities and identify the likely consequences of each plausible incorrect diagnosis. This is obviously an area of considerable spe'cula- tion. We can offer no specific guidance beyond suggesting that alternative (incorrect) diagnoses be determined through interviews and talk-throughs with reactor operators and consulation with training personnel at simulator facilities.

For the case of multiple abnormal events, we suggest using the tabled values for the first event to be diagnosed and adding 10-minute constants to derive the HEPs for the additional events. Table 12-4 lists estimated HEPs for one, two, and three abnormal events. We remind the reader that the sequence of diagnosis for multiple abnormal events should be determined

12-I1

Table 12-4 Nominal model of estimated HEPs and EFs for diagnosis within time room personnel of abnormal events annunciated closely in time*

T by control

,,.,

Median joint HEPtt Median joint Median

for diagnosis HEPtt for joint HEPtt T of a single T diagnosis of T for diagnosis

(Minutes** or the first (Minutes** the second (Minutes** of the Item after T ) event EF Item after T 0) event EF Item after T ) third event EF

(1) 1 1.0 -- (7) 1 1.0 -- (14) 1 1.0 --

(2) 10 .1 10 (8) 10 1.0 -- (15) 10 1.0 --

(3) 20 .01 10 (9) 20 .1 10 (16) 20 1.0 --

(4) 30 .001 10 (10) 30 .01 10 (17) 30 .1 10

(11) 40 .001 10 (18) 40 .01 10

(19) 50 .001 10

(5) 60 .0001 30

(12) 70 .0001 30

(20) 80 .0001 30

(6) 1500 .00001 30

(13) 1510 .00001 30

(21) 1520 .00001 30

"Closely in time" refers to cases in which the annunciation of the second abnormal event occurs while the control room personnel are still actively engaged in diagnosing and/or planning the responses to cope with the first event. This is situation-specific, but for the initial analysis, use "within 10 minutes" as a working definition of "closely in time."

Note that this model pertains to the CR crew rather than to one individual.

The nominal model for diagnosis includes the activities listed in Table 12-1 as "perceive," "discriminate," "interpret," "diagnosis," and the first level of "decision-making." The modeling includes those aspects of behavior included in the Annunciator Response Model in Table 11-13; therefore, when the nominal model for diagnosis is used, the annunciator model should not be used for the initial diagnosis. The annunciator model may be used for estimating recovery factors for an incorrect diagnosis.

For points between the times shown, the medians and EFs may be chosen from Figure 12-4.

t To is a compelling signal of an abnormal situation and is usually taken as a pattern of annunciators. A probability of 1.0 is assumed for observing that there is some abnormal situation.

tt Table 12-5 presents some guidelines to use in adjusting or retaining the nominal HEPs presented above.

Figure 12-4

I-

0 z

L-

.1 .UPPER BOUND, JOINT HEP

.01

.001 N •I- MEDIAN

.000 1

LOWER BOUND, -- \ .00001 JOINT HEP

.000001

.0000001 I I 1 ,11111 I I t t t 1 10 100

To TIME T IN MINUTES AFTER A COMPELLING SIGNAL OF AN ABNORMAL SITUATION

Figure 12-4 Nominal model of estimated HEPs and UCBs for diagnosis within time T of one abnormal

event by control room personnel.

1000

12-20

Nominal Diagnosis Model

through operator interviews. Such interviews will also enable the analyst to discover when certain pairings of abnormal events might be considered, for HRA purposes, to constitute one event because of operator training and practice.

As discussed in Chapter 18, the nominal HEPs are intended to apply to the presumed manning of the control room after the occurrence of an abnormal event. Thus, it is assumed that the primary control board operator (OP 1) is present from T0 , the onset of the compelling signal that an abnormal situation is present, a backup operator (OP 2) is present at T2 , the shift supervisor (SS) is present at T5 , and the shift technical adviser (STA) is present at T 1 5 , where the subscripts represent minutes after T0 . Chapter 18 also provides a rationale for not giving credit for more than the above personnel.

Our reason for believing that the nominal joint HEPs in Table 12-4 are conservative can be illustrated by the following calculations. Assume that the basic HEP (BHEP) for OP 1 is .1 for 10 minutes, that is, 9 times in 10, he will diagnose the problem without any assistance by time 10.* For pur- poses of calculation, we will assume the .1 BHEP for OP 1 applies also to 20 or 30 minutes. The calculations below, using the dependence model to calculate the conditional HEPs (CHEPs) for the other control room person- nel, show some of the other assumptions that would be necessary to.approxi- mate the joint HEPs (JHEPs) in the table for these three times for single abnormal events:

(1) For 10 minutes, tabled JHEP = .1 .1 BHEP for OP I 1.0 CHEP for OP 2, assuming complete dependence (CD) .55 CHEP for SS, assuming high dependence (HD) No credit for STA Calculated JHEP = .055

(2) For 20 minutes, tabled JHEP = .01 .1 BHEP for OP 1 .55 CHEP for OP 2, assuming HD .23 CHEP for SS, assuming moderate dependence (MD) .55 CHEP for STA, assuming HD Calculated JHEP = .007

(3) For 30 minutes, tabled JHEP = .001 .1 BHEP for OP 1 .55 CHEP for OP 2, assuming HD .15 CHEP for SS, assuming low dependence (LD) .15 CHEP for STA, assuming LD Calculated JHEP = .0012

The HEP of .1 is based on an interim model we employed in the Arkansas Nuclear One Unit #1 PRA (Kolb et al, 1982a, Appendix B15, "Human Interface System").

12-21

Concluding Comments

The tabled JHEPs are nominal HEPs and must be adjusted for plant-specific conditions. For example, it must be determined whether the abnormal event being analyzed is one in which diagnosis errors are credible. It may be judged that for a particular transient, the operating personnel are so well versed in recognizing the pattern of stimuli associated with the transient that the cognitive aspect of behavior may be negligibly small. The deci- sion of the analyst should be based on the frequency with which each member of each control room operations team practices diagnosing the transient in question. The type of practice can consist of simulator requalification exercises and less formal talk-throughs in the plant. Since optimism in a PRA should be avoided, one should rarely assess an HEP of zero for the cognitive aspect; instead, if it can be determined that all of the opera- tions personnel fully understand the cues for a given abnormal event, a more conservative approach would be to assign the lower UCBs of the nominal JHEPs.

If it is judged that a reasonable possibility for diagnosis error exists, it is suggested that the analyst use the tabled joint HEPs as the initial cut. He must then decide, on the basis Qf the information he has from observation, interviews, and task analysis, whether he should modify these estimates. The use of sensitivity analysis will indicate how sensitive the PRA is to different assessments of the probability of diagnosis error. (Sensitivity analysis is described in Chapter 5.)

Table 12-5 presents some following general rules to use as a guide in adjusting or retaining the nominal HEPs in Table 12-4. If other than the nominal HEP is used as the median joint HEP, the UCBs should be adjusted in accordance with Figure 12-4.

Concluding Comments

Estimating HEPs for the cognitive aspects of human tasks in complex systems is still highly speculative. Data from simulators and other sources such as plant records are badly needed. When such data are obtained, the degree of judgment required in HRA should be lessened, but there will always be some judgment necessary when data from one situation are extrapolated to another, even highly similar, situation. The judgment should become more reliable as data-based models of diagnosis and other aspects of cognitive behavior replace the interim models in this chapter. In the meantime, the models in this chapter can be used as a starting point in a PRA, and expert judgment can be used to modify the nominal values, as outlined in Chapter 8.

I- 22

Table 12-5

Table 12-5 Guidelines for adjusting nominal diagnosis HEPs from Table 12-4

I tem General Rules

(1)

(2)

Use upper bound if:

(a) the event is not covered in training,

(b) the event is covered but not practiced except in initial training of operators for becoming licensed,

(c) the talk-through and interviews show that not all the opera- tors know the pattern of stimuli associated with the event.

Use lower bound if:

(a) the event is a well-recognized classic (e.g., TMI-2 inci- dent), and the operators have practiced the event in the simulator requalification exercises,

and

(b) the talk-through and interviews indicate that all the opera- tors have a good verbal recognition of the relevant stimulus patterns and know what to do or which written procedures to follow.

Use nominal HEP if:

(a) the only practice of the event is in simulator requalifica- tion exercises and all operators have had this experience,

(b) none of the rules for use of upper or lower bound apply.

(3)

12-23

Ch. 13. Manual Controls Overview; Types of Manual Controls

CHAPTER 13. MANUAL CONTROLS

Overview

Operation and maintenance of a nuclear power plant (NPP) involves thousands of controls of many different types. Although most of the operating con- trols are located in the control room, the material in this chapter applies to controls in general, regardless of location. (The special case of controls for locally operated valves is discussed in Chapter 14.)

Within the control room, the controls handled by the operator are manual controls: they are the means by which the human enters his inputs to the system. Most of the controls are electrical switches of various types. Many of the switches control components, such as motor-operated valves (MOVs), that are also under automatic control, in that the components will respond to signals from sensors or computer commands as well as from the operator. In such cases, if a function exceeds some limit, a sensor may automatically signal some component, such as a pump, to turn on or off, as required. Usually, operators try to maintain the functions of the compo- nents within limits by use of their manual controls.

This chapter describes the most common types of controls in NPPs, some of the important performance shaping factors (PSFs) affecting their operation, types of errors in their operation, and the nominal human error probabili- ties (HEPs) associated with the errors. Errors of omission are not dis- cussed here; they are covered in Chapters 11, 15, and 16. The errors addressed here are those errors of commission associated with the operation of the controls--the unintentional errors. The errors of decision-making regarding which control should be operated are described in Chapter 12. In this chapter, except for the case of inadvertent operation of a control, we assume that the operator had the intent to operate the control, and we restrict ourselves to possible errors in carrying out the intent, whether or not the intent was the result of a correct decision.

Types of Manual Controls

Almost all the manual controls in an NPP control room are electrical. These controls may be either continuous or discrete. Continuous controls, e.g., a potentiometer, may be adjusted to any point within their range. Discrete controls, e.g., a switch, have a fixed number of positions and are used to select one of a limited number of states. Most of the electrical controls in an NPP are discrete. The most common controls in NPPs are the following:

Multiposition selector switches Transilluminated switches J-handle (pistol-grip) switches Thumbwheels Pushbuttons Toggle switches

13- I

Errors in Manual Control Activation Performance Shaping Factors

Rotary knobs (continuous or discrete) Circuit breakers Keyboards Levers Cranks Handwheels Connectors (cables, jumpers, and interlocks)

Errors in Manual Control Activation

When an operator reaches for a control, the decision to manipulate that control has already been made. We are not addressing the correctness of the decision, only the probability of error in carrying out a decision, such as "Turn on Switch #3." Whether or not the decision was correct, there are three possible errors in the manipulation of the control: (1) selection of a wrong control, (2) incorrect operation of the control, and (3) inadvertent operation of a control by unintentional contact. Errors of the first type are usually due to inadequate distinction among controls, as when the labeling is poor. Errors of the second type result from poor design features such as a nonstandard relationship of control movement to expected result, inadequate or ambiguous indications of control position, inadequate feedback from displays, or from a requirement for special han- dling, such as that the control be held in place by the operator. Errors of the third type result from a variety of causes and in almost all cases can be eliminated by appropriate shields or guards, either electrical or mechanical, for those controls that are most likely to be inadvertently activated.

Performance Shaping Factors

The probability of each of the above types of errors is largely a function of the PSFs related to the placement and identification of controls, as listed in Table 13-1. The manner in which the above PSFs can affect per- formance may be inferred from the list of deficiencies in the layouts of manual controls that were noted at NPPs by Seminara et al (1976). These deficiencies are listed in Table 13-2.

The tabled deficiencies are such that, in most cases, the operator has difficulty locating the specific control he requires. In only a few cases does the layout of the control panels help the operator locate and identify the controls. In most cases, the operator has to rely on his knowledge of the control room layout. In a few plants, the operators are rotated among control boards for different reactors with different layouts, and in times of stress, the operators could easily make errors. This is a particularly difficult situation when operators are rotated to a control room that is a mirror image of the one they were operating immediately previously. Many operators report problems in adapting to the reversal of "left-right" relationship of the panels even during normal operations.

In addition to the above deficiencies, an often-noted problem is that of the possible inadvertent activation of controls when operators have to

13-2

Table 13-1

Table 13-1 PSFs related to controls

(1) Relationship of control to its display (includes physical distance and

direction of movement)

(2) Identification of control with its function (includes labeling, func- tional grouping of controls, and use of mimic panels)

(3) Specific identification of control (includes control labeling-- position, wording, and legibility of label; and control coding--color,

shape, size, and position)

(4) Anthropometrics (includes spacing, ease of reach, and ease of visual access)

(5) Indicators on controls (includes types of indicators such as pointers

and position marks, and visibility and distinctiveness of indicators)

(6) Direction of motion (compliance with populational stereotypes)

(7) Operator expectancies regarding layout of controls

(8) Immediacy of feedback after control operation

(9) Control room layout (includes distance to controls and placement)

13-ý

Table 13-2

Table 13-2 Deficiencies in layouts of manual controls observed at NPPs

(1) There are inadequate means of distinguishing controls: large numbers of identical controls are arranged on panels without any identifica- tion other than their labels; controls are not grouped by function, and no mimic lines are used to help identify the functions of the controls; there is a lack of control coding, such as color or shape, to assist in identifying the controls.

(2) Generally, the controls are not arranged in a logical relationship to

their displays.

(3) In some cases, the distance between a control and its display is such that the operator cannot easily see the display while manipulating the control.

(4) In many cases, the controls are difficult to reach and the operator may fail to use them if he is busy.

(5) In many cases, the indicators on the controls do not clearly indicate control position.

(6) In many cases, there are no provisions for placement of items such as procedures manuals, and the operator will be forced to cover some controls with a manual for want of some place to rest it.

(7) Mirror-imaging of control panels presents the same problems in the use of controls as in the use of displays (Chapter 11).

(8) Labeling of controls is often very terse, and the operator can easily err in control selection.

(9) Illumination levels at some control surfaces are barely adequate to read the labels.

Source: Seminara et al, 1976.

13-4

Estimated Probabilities of

Errors of Commission

stretch across a panel to read a display or reach a control or when they

lean against or sit on part of a panel. In such cases, the operator's body

is likely to brush against the controls closest to him and may activate any

that are not adequately guarded. At one plant, it was observed that a row of J-handle (pistol-grip) switches was mounted very close to the near edge

of a horizontal panel. The panel was at approximately table-top height,

and it was natural for the operators to lean their backs against the panel

while observing a display across the room and to brace themselves by plac-

ihg their hands on the panel. Since the switches were unshielded and had

no requirement for a deliberate release act prior to activation,. there was a definite likelihood of inadvertent activation. The Handbook does not

list an HEP for this situation; it is plant-specific. The analyst on the

job made an estimate of a .5 probability that an inadvertent activation

would occur once in 2 weeks when the NPP was in the power generation mode.

This estimate disregarded recovery factors. We do not have any basis for

disagreement with'such an estimate, since there-are no data available. The

analyst formed his estimate on the basis of observation of and interviews

with the operators. This is an example of the many situations in which the

analyst will have to formulate estimates ,of HEPs without any guidelines.

from the Handbook. (At one plant, a similar problem-was ameliorated by repositioning the handles on their shafts, rotating them 1800 from their

original positions. The direction of operation was unaffected. Although

not an ideal solution, it was a reasonable, inexpensive quick fix.)

Estimated Probabilities of Errors of Commission

The nominal HEPs in Table 13-3 apply to the power generation mode of the

plant. For conditions associated with higher levels of stress, the HEPs

should be modified as described in Chapter 17. Note that some of the

errors of commission listed in the table relate to simple acts, such as

switch manipulations, which usually are affected only slightly by stress.

However, we apply the modifiers from Chapter 17 for the sake of- conserva-

tism and for convenience in carrying gut an HRA, since it is easier to

apply the modifiers to all actions than to concern oneself with exceptions.

The error introduced by following this uniform modification of HEPs is

negligible.

The HEPs and error factors (EFs) in the table apply to the operation of a

single control. In actual operation, only one control is manipulated at a

time. In those instances in which controls are handled as pairs, complete dependence is assumed between the two controls; thus, the HEP for a single

control will apply.

Complete dependence is assumed whenever a number of controls can be re-

garded as a perceptual unit, whether it be the case of two controls always

handled as a pair, or a group of controls that can be regarded as a dis-

tinctive subset because they are handled in succession with no pauses

between activations.

1-3-5

Table 13-3

Table 13-3 Estimated probabilities of errors of commission

in operating manual controls*

Item Potential Errors HEP EF

(1) Inadvertent activation of a control see text

Select wrong control on a panel from an array of similar-appearing controls**:

(2) identified by labels only .003 3

(3) arranged in well-delineated functional groups .001 3

(4) which are part of a well-defined mimic layout .0005 10

Turn rotary control in wrong direction (for two-position switches, see item 8):

(5) when there is no violation of populational .0005 10 stereotypes

(6) when design violates a strong populational .05 5 stereotype and operating conditions are normal

(7) when design violates a strong populational .5 5 stereotype and operation is under high stress

(8) Turn a two-position switch in wrong direction

or leave it in the wrong setting

(9) Set a rotary control in an incorrect setting .001 10 %

(for two-position switches, see item 8)

(10) Failure to complete change of state of a .003 3

component if switch must be held until change is completed

Select wrong circuit breaker in a group of circuit breakers**:

(11) densely grouped and identified by labels only .005 3

(12) in which the PSFs are more favorable .003 3 (see text)

(13) Improperly mate a connector (this includes .003 3

failures to seat connectors completely and failure to test locking features of connectors for engagement)

The HEPs are for errors of commission only and do not include any errors of decision as to which controls to activate.

If controls or circuit breakers are to be restored and are tagged, adjust the tabled HEPs according to Table 16-2.

tDivide HEPs for rotary controls (items 5-7) by 5 (use same EFs). ti.

This error is a function of the clarity with which indicator position can be determined: designs of switch knobs and their position indications vary greatly. For plant-specific analyses, an EF of 3 may be used.

¶?-

Estimated Probabilities of Errors of Commission

The HEPs in the table are estimates of the occurrence of an error, without consideration of recovery factors. When the effects of recovery factors are combined with the tabled HEP, the joint HEPs are smaller than the original HEPs.

The error "Turn control in wrong direction" is also called a "reversal error." Normally, reversal errors for two-position switches are so infre- quent that they can be assigned a probability of zero. However, for proba- bilistic risk assessment (PRA) purposes, we recommend use of the tabled value of .0001 (EF = 10) because of the occasional exception that occurs when a switch was left in the wrong position previously. For example, assume that a switch was left ON that should have been left OFF. When an operator addresses that switch to turn it ON, there is a possibility for a reversal error. If the operator has the intent to "change the position of the switch," he may not notice that the switch is already ON and might make a reversal error, turning the switch to OFF. If the operator has the intent to "turn the switch to ON," he is not nearly as likely to make the reversal error.

Regardless of the intent, the reversal error for a two-position switch is conditional on a previous error. Let us assume a probability of .003 for the previous error of leaving the switch in the wrong position, with a recovery factor of .9, so that the joint probability of the switch being in the wrong position is .1 x .003 = .0003. Assume that half of the operators have the "change position" intent and half have the "turn the switch ON" intent. We estimate that the conditional HEP for the first group is .9, and that for the second group it is .01. The HEP for the joint probability of leaving a two-position switch in the wrong position and later changing it to an incorrect position becomes

Pr[F] = .0003 [(.5 x .9) + (.5,x .01)] = .0001

Obviously, the above HEP is subjective, since we have no data on the per- centage of operators who would have either intent when addressing the switch. However, the exercise illustrates the rationale for assigning an HEP to the event of committing a reversal error with a two-position switch, and we suggest using the value of .0001 (EF = 10) for PRA purposes. For other purposes, the HEP for this error would be regarded as negligible.

Keyboards are a special class of control in that they are usually employed to enter or to request information rather than to exercise direct control over a component or system. In present generation NPPs, the most common use of a keyboard is to address the computer for information about some system, subsystem, or component. Although it is possible to make an error in using the keyboard, the consequences of such an error would not be serious. If the operator intended to request the status of system "X" and erroneously entered system "Y," the error would be obvious as soon as system "Y" appeared on the display (the display could be either a cathode- ray tube [CRT] or a printout). At some installations, the message entered into the keyboard is displayed as the keyboard is operated, so that the operator can verify it before entering it.

13-'?

Estimated Probabilities of Errors of Commission

Considerable research has been done on errors in keyboard usage, such as the operation of typewriters, push-button telephones, calculators, and other specialized keyboards. In the following studies, the reported error rates range from .0002 to .025. Klemmer and Lockhead (1962) reported error rates ranging from .0002 to .0006 per keypunch stroke for experienced bank clerks. Deininger (1960) reported an error rate of .025 per stroke for subjects entering seven-digit numbers into experimental push-button tele- phones. Butterbaugh and Rockwell (1982) reported error rates of .003 to .004 for entering data into an alphanumeric keyboard. Conrad and Hull (1968) observed rates of .009 to .012 per key stroke for a group of inex- perienced housewives entering eight-digit numerals into push-button tele- phones and calculators.

In all of the studies, the subjects were required to work as rapidly as they could, since speed was one of the performance measures. In an NPP, there is no requirement for speed, since keyboard entries are made at a pace convenient to the operator, and the almost immediate feedback from the display provides an excellent recovery factor if an error should occur. Therefore, the HEP for keyboard entry when requesting information is con- sidered to be negligible.

Future control rooms will probably change the role of the operator from that of direct controller of individual components and subsystems to that of supervisory controller, in which he will enter instructions to the computer system to initiate and carry out. certain functions. We do not know what opportunities there may be for human errors in such systems, since the intent is to relieve the operator of many small tasks, releasing him to act as a decision-maker. The number of opportunities for error in control room operation should be reduced considerably.

Circuit breakers are a specialized form of switch providing protection from overloads. Most circuit breakers are positioned elsewhere than on the working panels of control rooms. Insofar as operation is concerned, they are the equivalent of two-position switches, and the probability of acti- vating one in the wrong direction is no greater than that of turning any two-position switch in the wrong direction (HEP = .0001, EF = 10). How- ever, the probability of the error of selection for small circuit breakers is likely to be higher than for control room switches, because small circuit breakers are often grouped in dense arrays, without mimic lines or other such aids. In most cases, only labels are available to identify the circuit breakers, and often the illumination levels are inadequate for easy reading of the labels. For these reasons, we estimate an HEP of .005 (EF = 3) for the selection of a circuit breaker in a group of identical circuit breakers identified by labels only. For those situations in which the PSFs of illumination, density of array, and legibility of labels are more favorable, the HEPs may be reduced to the level used for controls in general (HEP = .003, EF = 3).

13-8

Recovery Factors

Among the PSFs in Table 13-1, Item #8, "Immediacy of feedback after control operation," refers to the importance of providing the operator with prompt knowledge of the results of his actions. Feedback is a very important recovery factor in case of error in the operation of a control. The avail- ability of prompt feedback does not affect the probability of error in operating a control, but it provides information about the results of the operation, which will usually cue the operator that an error was committed. Thus, the HEP for selection of a control is the same whether or not feed- back is available, but the probability of recovery from the error is greatly increased by prompt feedback, so that the joint probability of the occurrence of the error and failure to correct the error is greatly re- duced. The recovery value of the feedback is a function of the nature of the feedback. For example, if the error results in activation of an annun- ciator, the HEPs from the annunciator model would apply for the recovery factor. Similarly, the HEPs associated with other displays that provide feedback are presented in Chapter 11. Feedback may also take the form of an admonition from another person who notices the error, but we do not allow credit for that possibility in PRAs unless a second person is also assigned to the task. In this case, the dependence model should be applied to estimate the effects of human redundancy.

Feedback may be either immediate or delayed. When a person turns on a lamp, the feedback is immediate: the light comes on instantly. In other cases, there may be a short delay, as when operating a switch for an MOV. Depending upon the size of the valve, as long as 2 minutes may pass before the indicator lamps in the control room show the change of state to be complete. Although the intervals are relatively brief, a lag of even 30 seconds can seem to be a very long time when coping with a transient, and we have observed operators jiggle a switch repeatedly, apparently trying to assure themselves that they have activated it (in much the same manner that a person presses a call button repeatedly while waiting for an elevator). Of course, the additional activations of the control do not cause problems and are not regarded as errors in themselves. The errors caused by the delay would result if the wrong switch had been activated and the operator had not received any indication of his error of selection during the wait- ing period. Most of the controls in an NPP provide reasonably prompt feedback that the control has been activated or that the controlled com- ponent has reached the desired state. The occasional instance of an indi- cator that is not "telling the whole truth," such as the pressure-operated relief valve status indicator at TMI-2, is a case of design deficiency rather than delayed feedback--the feedback was prompt but misleading. Fortunately, such instances are infrequent. The analyst normally assumes that the indicators are telling the truth.

Delayed feedback in an NPP refers to the change in plant parameters that is

expected to follow a control action but that takes some time to develop. For instance, when an operator observes low water level in a pressurizer and turns on another charging pump, several minutes will pass before he

13-9

Recent Improvements in Control Rooms

will be able to see a change in the pressurizer water level. Such delayed feedback may be the only indication of the adequacy of an action taken in response to some condition.

Ordinarily, if a person operates an incorrect control or operates a control incorrectly, the feedback from the act provides the first recovery factor. For human reliability analysis (HRA), we assume that no other recovery factors are available prior to the feedback from the display related to the control. Therefore, in cases where time of control activation is impor- tant, the nominal HEP for the control activation is assumed to prevail until the feedback is due to occur. There will be instances in which the analyst will use a longer time than the known time for the feedback to occur, on the assumption that the operator may not be attending to the expected feedback exactly when it is due. This depends upon the situation.

There are some controls that have to be held by the operator until the change of state has been completed. Such controls are susceptible to the error of early release, since most controls require only a momentary action. The HEP listed for this type of control operation applies to the error of releasing the control before the change of state is completed.

Recent Improvements in Control Rooms

Since the incident at TMI-2, action has been taken to enhance the usability of control panels via inexpensive modifications that could be implemented readily. These modifications, sometimes.referred to as the "paint, label, and tape approach," were suggested in NUREG76-6503 (Swain, 1975) and in NUREG-0700. The approach involves the use of paint, labels, and adhesive tape on the panels to set off functionally related groups of controls and displays and to create mimic lines that establish "what leads to what" among the controls and displays. This easily implemented bit of human factors engineering simplifies the task of the operator in locating any particular control or disyplay, reducing his search time and his probability of selecting the wrong control or of reading the wrong display. The HEPs for errors of selection in Table 13-3 indicate the extent to which errors may be reduced by such simple modifications.

13-10

Ch. 14. Locally Operated Valves Overview: Assumptions

CHAPTER 14. LOCALLY OPERATED VALVES

Overview

The status of valves is crucial in plant operation and to the availability of engineered safety features (ESFs). We identify two classes of valves: locally operated valves and motor-operated valves (MOVs). Although both are manually operated, we will use the term manual valve in referring to locally operated valves and the term "MOV" in referring to valves that are operated by an electrical switch (generally located in the control room). This chapter deals mainly with errors in the manipulation of manual valves. MOV controls are included in the category of switches that is discussed in Chapter 13, "Manual Controls."* Errors of omission are discussed in later chapters.

This chapter deals with the activities and estimated HEPs associated with (1) manipulating manual valves during the course of normal plant operations or in response to abnormal events and (2) manipulating the valves to permit testing, maintenance, calibration, or other work. As is usual, the nominal HEPs in this chapter must be adjusted for the effects of other performance shaping factors (PSFs) such as high stress.

For convenience, we will use the term change to mean "change the state of a valve from the normal operating position to a nonnormal position" and the term restore to mean "restore the valve to its normal operating position." The normal operating position is the valve's position during the plant's power generating mode.

This chapter presents general assumptions about valve manipulations along with some of the relevant PSFs. These PSFs include the work situations in which such manipulations are attempted, the quality of valve labeling, the position of a valve relative to other valves, valve location, tagging activities, position indicators, and physical accessibility. HEPs for valve manipulations taking place under various combinations of these PSFs are then discussed.

General Assumptions

The locally operated valves of most concern for probabilistic risk assess- ment (PRA) are turning-wheel valves with or without rising stems and with or without position indicators.** A rising-stem valve provides a fairly

Some MOVs can be operated at the valve site (locally) by electrical or mechanical fheans. These manipulations should be considered as special cases. The human error probabilities (HEPs) for manual valves should be modified to account for the effects of any special performance shaping factors (PSFs) acting in these cases.

** A position indicator incorporates a scale that indicates the position of the valve relative to a fully opened or fully closed position.

14-1

General Assumptions

good indication of whether the valve is open (stem all the way out) or closed (stem all, or nearly all, the way in). Confusion sometimes arises because of nonstandardization in design of these valves; there are dif- ferences in the length of the rising stem and in whether the stem is all the way in when fully closed or whether there is still some stem showing (and how much). Finally, some valves have no indication of their position; the operator must try to turn the valve to determine whether it is open or closed. Valves are manipulated by nuclear power plant (NPP) personnel responding to directives such as oral instructions, written procedures, and maintenance work requests for test, calibration, or maintenance procedures. Operations personnel (including unlicensed reactor operators called "auxil- iary operators") generally are responsible for changing and restoring valves. They are usually following a set of written procedures or a list of valves based on a work request. With respect to the work activities, operations personnel usually perform all valve activities associated with achieving system isolation as required. Maintenance personnel (main- tainers) may perform valve manipulations during the course of maintenance activity. Restoration of the system following maintenance is accomplished by operations personnel. (In this chapter, we use the term operators to refer to operations personnel.)

The manipulation of valves to permit work operations is coordinated between operators and those who will perform the work. The latter will not begin work until the work is cleared by the operators, and operators will not restore equipment until the restoration is cleared by the workers. Commu- nication between the operators and the workers may be formal (through a "chain of command") or less formal, depending on individual plant policies.

To estimate probabilities of errors in manipulating valves, the type of error must first be identified. For errors of omission, the directive being followed defines the HEP. For example, valves themselves are not omitted--rather, steps that pertain to valve manipulation are omitted from a sequence in a directive. Therefore, the probability of omitting a step from a set of written procedures or of overlooking the restoration of an item of equipment while following the plant's tagging procedure must be estimated. These errors are discussed in Chapter 15, "Oral Instructions and Written Procedures," and Chapter 16, "Management and Administrative Control."

For errors of commission, several PSFs must be defined. In all cases of manual valve manipulation, a basic work situation is defined as one in which a single valve is to be manipulated. The valve is in a separate location from the control room, and it is not adjacent to any similar valves. Initially, the HEPs are presented for this simplified situation; then alternative situations are described with their applicable HEPs.

Other PSFs affecting valve manipulations include the quality of the label- ing of the valves; the identifiers of the valves, as called out in proce- dures or other documents; and the similarity of the labels and the identi- fiers. Typical NPP valve labeling is assumed (as described in Chapter 3 in the section "Inadequate Labeling and Status Indications of Manual Valves")

14-2

HEPs in Valve Manipulations

unless otherwisespecified. The location of the valve with respect to valves of similar systems or functions is also important. If the valve is among a group of valves from another system, it may be difficult to locate and one of the others may be manipulated by mistake. The configuration of the valve with respect to other valves in the area will also affect the probability of error in manipulating it. If a valve is the only one in an area or is the only one of its size or type in an area (and the operators know this), it is less likely to be selected incorrectly than are valves

:that are difficult to distinguish. In dealing with changing or restoring valves for work operations, the presence or absence of a tag pertaining to the activity is a PSF. To change a valve, operators usually would not expect to find a valve already tagged; therefore, in most cases, they will search for an untagged valve.

Other PSFs affecting the probability of correct change or restoration of a valve are the means for determining the normal status of the valve and for determining its position on inspection. For example, if an operator is to restore a certain valve and finds on it only the tag he is to remove, he will in some cases simply change its position. If there is an indicator on the valve showing that it is already in its normal position, the operator is more likely to detect that a restoration act is not called for. Like- wise, if he has specific instructions as to how far to open or close a valve, he is more likely to do' this correctly if there is a position indi- cator on the valve that provides unambiguous feedback as to its position. (In many cases, whether a valve is 60% or 75% open will have little effect on plant performance. The effect of the difference must be estimated by experts in plant performance.)

HEPs in Valve Manipulations

Valve manipulations are called for in various types of NPP directives: oral instructions, written procedures, informal lists, valve alignment sheets or checklists, and administrative control procedures (tagging lists). Errors of omission for valve manipulations must be estimated in conjunction with estimates of misuse of the directive. HEPs specific to valves will be discussed here--errors of commission affected by PSFs peculiar to valves.

Selection Errors

The first error discussed is one of selection. In this case, the operator or maintainer intends to change or restore a particular valve. For some reason, though, he changes/restores a different valve from the one called for. There are a number of causes for this type of error. He may have misread the directive and chosen a valve incorrect for the purpose but correct according to his intention. The instruction in the directive may have been ambiguous, causing him to mistake the valve called for. 'He may' have misread the label on the valve itself, thought he had selected the correct valve, and manipulated the wrong one. He may have selected a valve from his "memory" of its function or location but may have been mistaken in his recollection of it.

14-3

HEPs in Valve Manipulations

The result of such an error on the system is the same as that of an error of omission (the called-for activity did not take place) unless the manipu- lation of the valve chosen by mistake results itself in some system effect. In most cases, the probability of a selection error can be estimated, but it is not possible to provide Handbook HEPs for which valve will be manipu- lated erroneously. Expert judgment based on detailed task analyses or plant history data may reveal which items of equipment are most likely to be chosen incorrectly, and estimates of HEPs can be made on these bases for those errors of consequence in a probabilistic risk assessment (PRA).

The primary PSFs affecting the probability of selection errors include the quality of labeling on the valve, the correlation of those labels to the identifiers of the valve in the directive, the presence of tags on valves the operator expects to find tagged, and the location of the valve in the plant and with respect to other, similar-appearing valves. The labels on the valves should be clearly printed and legible. They should be unam- biguous with respect to how they are identified in plant directives. The more difficult the label is to read or the more difficult it is to identify the valve called for in the directive, the higher will be the HEP for selection errors. The more isolated a valve is, the less likely it is that the operator will fail to select it when searching for it. The valve could be physically separated from other valves in that area. It could have distinctive features that set it apart from other valves in the immediate area, such as being the only valve with a tag on it, being the only valve of its size or shape in an area, or (in a group of valves whose states can be ascertained at a glance) being the only valve in a certain state in an area. The more similar a valve appears to others in the vicinity, the more likely errors of selection are to occur.

The presence or absence of tags on a valve influences the probability of selection errors, especially for restoration tasks. If the operator knows he is looking for a valve that should have a tag on it and there is only one tagged valve on a panel, he may be likely to choose that valve, paying little or no attention to other identifiers. He is certainly less likely to choose the wrong valve if it has no tag on it at all. The derived HEPs for selection errors for locally operated valves are given in Table 14-1. The statements in the table are guidelines; the analyst may find cases that don't match exactly any of the statements. In such cases, he should make appropriate modifications to the tabled HEPs.

Reversal Errors

Another type of commission error involving valves is that of reversal. In this case, the operator intends to perform the task and has chosen the cor- rect valve but manipulates it incorrectly--he opens it instead of closing it or vice versa. For example, this error could occur if he is to manipu- late 10 valves, closing 9 and opening 1. He may simply close all the valves. In another case, he may have the correct valves in mind and intend to change their states, regardless of the states in which he finds them. In this case, he has been "locked in" to changing the state and may fail to notice that a valve is already in the correct state.

14-4

Table 14-1

Table 14-1 Estimated HEPs for selection errors for locally operated valves

Item Potential Errors HEP EF

Making an error of selection in changing or restoring a locally operated valve when the valve to be manipulated is

(1) Clearly and unambiguously labeled, set apart .001 3 from valves that are similar in all of the following: size and shape, state, and pres- ence of tags*

(2) Clearly and unambiguously labeled, part of .003 3

a group of two or more valves that are simi- lar in one of the following: size and shape,

state, or presence of tags*

(3) Unclearly or ambiguously labeled, set apart .005 3 from valves that are similar in all of the following: size and shape, state, and presence of tags*

(4) Unclearly or ambiguously labeled, part of a .008 3 group of two or more valves that are simi- lar in one of the following: size and shape, state, or presence of tags*

(5) Unclearly or ambiguously labeled, part of a .01 3 group of two or more valves that are simi-

lar in all of the following: size and shape, state, and presence of tags*

Unless otherwise specified, Level 2 tagging is presumed. If other levels of tagging are assessed, adjust the tabled HEPs according to Table 16-2.

14 - ý

HEPs in Valve Manipulations

As another example, suppose a valve's normal position is open. If the valve is changed for maintenance, an operator closes it. To restore it, he opens it. There should be a tag on the valve whenever it is closed since that is not the normal position for the valve. If the operator intends to close the valve and finds it already closed, there are two possible causes for this: an error was made previously in restoring the valve, or there is maintenance ongoing that calls for the valve to be closed. For the opera- tor to make a reversal error (to open the valve instead of closing it), the valve must be in an "incorrect" state--one that he does not expect to find. Therefore, the possibility for a reversal error is dependent upon the valve being in the unexpected state. The joint probability of the incorrect ini- tial condition and the resulting reversal error is so small that it can be regarded as negligible for all cases.

The PSF having the most effect on this type of error is the presence of a tag on the valve. If the valve has already been changed or restored, chances are that the tag will have been attached or removed. If the tag is in place, there will usually be an indication on the tag of the valve's normal operating position. If the valve has a position indicator directly on it, this will reduce the probability of a reversal error since the operator can determine whether he is changing or restoring a valve. The above PSFs all have an effect on the probability of a reversal error. The probability of such an error has been estimated to be negligible; however, if the combination of the PSFs is particularly poor, a nominal estimate of .0001 may be applied to this error.

Stuck Valves

In changing or restoring locally operated valves, there is another type of commission error possible. This involves an attempt to open or close a valve and failing to do so fully. For example, an operator may try to close a valve and succeed in closing it only 80%. The usual cause of this is that the valve sticks as it is being manipulated. Given that this happens,* there is some probability that the operator will fail to detect that the valve has stuck instead of "bottoming out." This situation is easier to detect if the valve is a rising-stem valve than if it is only a turning wheel. It is also much easier to detect if the valve has a posi- tion indicator on it. In fact, if the valve has a position indicator without a rising stem, it is more likely that he will check the position indicator and receive a clear indication that the valve is not fully opened or closed.

The HEPs for the operator's failure to recognize a stuck valve are pre- sented in Table 14-2. Naturally, persons other than the human reliability analyst must determine the extent of travel at which the valve is likely to stick and the effects on system performance of this occurrence if it is undetected.

Equipment reliability specialists have estimated that the probability of a valve's sticking in this manner is approximately .001 per manipulation, with an error factor of 10.

14-6

Table 14-2

Table 14-2 Estimated HEPs in detecting stuck locally operated valves

I tem Potential Errors HEP EF

Given that a locally operated valve sticks as it is being changed or restored,* the operator fails to notice the sticking valve, when it has

(1) A position indicator** only

(2) A position indicator** and a rising stem

(3) A rising stem but no position indicator**

(4) Neither rising stem nor position indicator**

.001 3

.002 3

.005 3

.01 3

Equipment reliability specialists have estimated that the probability of a valve's sticking in this manner is approxi- mately .001 per manipulation, with an error factor of 10.

A position indicator incorporates a scale that indicates the position of the valve relative to a fully opened or fully closed position. A rising stem qualifies as a position indicator if there is a scale associated with it.

14-7

HEPs in Valve Manipulations

Other Potential Errors in Valve Manipulation

There are many other potential errors in the operation of locally operated valves. These include general and special inspections of valve status, opportunities to detect previous errors, and orders given relating to valve operation, maintenance, and surveillance. These activities will be covered in Chapter 16 and in Chapter 19, "Recovery Factors."

14-8

Chapter 15. Oral Instructions and Written Procedures; Overview

CHAPTER 15. ORAL INSTRUCTIONS AND WRITTEN PROCEDURES

Overview

This chapter lists the estimated human error probabilities (HEPs) and uncertainty bounds (UCBs) or error factors (EFs) for some errors of omission and commission associated with oral instructions and written mate- rials. Errors related to oral instructions to perform one or more activi- ties may occur in giving the instruction (i.e., giving incorrect instruc- tions), in the recipient's understanding or recalling of what was given, or in carrying out the activities. Errors related to written materials may occur in their preparation, in the failure to use them properly (or at all), or in the performance of any of the written steps in a procedure. The failure to use a procedure or to use it properly overlaps the subject of administrative control (Chapter 16).

We define oral instructions as short directives given in verbal communica- tion between personnel. Written materials include written notes and writ- ten procedures. The latter can be ad hoc, i.e., one-of-a-kind procedures developed for a special purpose, or formal printed procedures subject to various levels of review and use. This chapter addresses primarily routine procedures (those for test, maintenance, calibration, and operations) and emergency operating procedures (EOPs) (those that deal with the recognition and mitigation of abnormal operating conditions). In the latter area, the estimated HEPs are for the most common type of emergency procedures, known as system-based (or event-based) procedures. Not enough is known about a new kind of procedure, called the symptom-oriented (or function-oriented) EOPs, to derive estimated HEPs for their use. For a comparison and exam- ples of the event- and symptom-based EOPs, see INPO 82-107 (INPO, 1982), vonHerrmann (1983), and vonHerrmann et al (1983).

The estimated HEPs for written procedures and oral instructions are based on the assumption that the technical content of the material used by the originators to generate the procedure or instructions is correct. No attempt is made to ascertain the probability of errors in the source mate- rial. Our estimates do include reading and/or writing errors of those who prepare the material as well as those who receive the material. For veri- fication of the written correctness of EOPs, and to ensure that the generic and/or plant-specific technical aspects have been properly incorporated in the EOPs, see INPO 83-004 (INPO, 1983).

Most of the errors identified in this chapter are errors of omission since most of the errors peculiar to the use of oral instructions or written procedures are drrors of omission. The probabilities of errors of commis- sion are based more on performance shaping factors (PSFs) peculiar to an item of equipment than on the characteristics of the entire task series. For this reason, errors of commission in carrying out instruction items from oral instructions or from written procedures are discussed in the chapters relevant to the type of equipment involved.

15-1

Oral Instructions

For oral instructions, we assume clear channels of communication, usually face-to-face interaction, but also include communication by telephone or equivalent. We do not include any degrading factors for noise communica- tion channels or for misunderstanding of what is spoken. This problem simplification is based on studies of speech intelligibility reported in Kryter (1972, p 1975) who states, "If the speech peaks are 30 dB or more above the noise throughout the frequency band from 200 to 6100 hertz, the listener will make essentially no errors." This is characteristic of modern control rooms. We also assume that if the recipient of an oral instruction is not certain he has heard the message clearly, he will ask for a repeat of the message. The only errors in oral communication con- sidered are those made by the originator of the message in reading displays or written information and errors by the recipient in writing down informa- tion. This chapter and others in the Handbook provide the estimated HEPs for these kinds of errors.

Types of Oral Instructions

Oral instructions are short, spoken directions or information given by one person to another. They may be general or detailed. General oral instruc- tions consist of instructions for the recipient to perform one or more tasks or activities but do not state each task or activity by name, e.g., "Open the blocking valves for the Auxiliary Feedwater System." Detailed oral instructions do include mention of each task or activity by name, e.g., "Open the blocking valves for the Auxiliary Feedwater System--those are locally operated valves AF7757, AF7758, and AF7759."

In the case of general oral instructions, the person speaking the instruc- tions assumes that the recipient of his instructions knows which valves are involved in the restoration task. That is, it is presumed that the knowl- edge required is skill-of-the-craft. If the analyst agrees with this pre- sumption, he may use the lower HEPs associated with following detailed oral instructions.

Whether the oral instructions are general or detailed, the analyst will have to assess levels of dependence between the instructions involved. In either the general or the detailed oral instruction examples listed above, the three separate blocking valves might be considered by an analyst to constitute a perceptual unit for a skilled person, as defined in Chapter 10. That is, if an operator remembers to restore one of the valves, he will always remember the other two. In this case, opening the valves is completely dependent for errors of ommission.

In other cases, oral instructions may involve several things to remember that are not grouped as a perceptual unit, e.g., the several isolation valves for a sodium hydroxide tank may not all be located in a central place. If one valve is, say, behind a partition and separated from the other valves, the arrangement might not be consistent with an assumption of complete dependence for errors of omission. In this example, the analyst

15-2

Oral Instructions

might judge that all but one valve constitute a perceptual unit but that the other valve (behind the partition) constitutes a different thing *to be remembered, and he would assign less than a complete level of dependence between that valve and the set of other valves for errors of omission.

Oral instructions may well include several unrelated things for the recipi- ent to do. For example, he may be told, "Go down to the second level and check that oil leak in the AFWS pump, you know, the one that is next to the north wall. Make sure it is still a slow leak--a few drops in a minute or SO. Oh, while you're down there, see if they cleaned up the spillage and left a can to catch the leaking oil. On the way back up, would you check the main isolation valve for the ABC System and also check the pressure meter to see it is well below the red line." As more and more unrelated things to remember are given, the probability of remembering all of the individual items of oral instruction rapidly diminishes. Each independent thing to be remembered is designated as one oral instruction item. If several completely dependent things are involved (i.e., a perceptual unit), this set is also considered to be one oral instruction item.

Oral Instruction Model

If oral instructions are given, it is very unlikely that the recipient will fail to initiate at least the first task or activity given. The only factor that might interfere with initiation of the first oral instruction item the operator intends to perform would be some compelling distraction, such as a major-leak of some liquid or gas, a crack in a coolant pipe, a disturbing phone call from home, or some other unusual event that could "erase" the oral instructions. As a conservative estimate, we assign an HEP of .001 (EF =3) for this possibility, exclusive of recovery factors.

*For analytical convenience, we assign the .001 estimated probability of failure to initiate an oral instruction task as the estimated probability of an error of omission for the first (or only) oral instruction that the operator performs. The above .001 HEP is not to be used twice for the same set of instructions. It is the estimated HEP for failure to initiate the task that is also taken as the HEP for omitting the first thing the oper- ator intends to do. In most cases, which item of more than one that the operator does first is not important. The probabilities of errors of commission for oral instruction items must be estimated separately, using the appropriate tables in Part III of the Handbook.

Experimental studies of retention indicate that errors in forgetting increase rapidly as more items must be remembered. Typical university laboratory studies test the ability of a person to remember relatively meaningless items such as numbers, syllables, or unrelated words (cf. Chapter X, "Retention and Forgetting," in McGeoch and Irion, 1952). The studies all yield substantially similar data: people can recall up to five items of the above kind for short intervals, usually of no more than a few minutes.

15-3

Oral Instructions

However, meaningful material is retained for much longer intervals with relatively little decline over time. If only one meaningful oral instruc- tion item must be remembered, this requirement is no more difficult than that of requiring a person to carry out an order. Our estimate of .001, then, is for some distraction that causes the recipient of an oral instruc- tion to fail to initiate the first (or only) activity he is planning to do. Given normally responsible personnel, deliberate failure to carry out orders is so unusual that it may be disregarded as a major source of error.

In considering the retention of more than one oral instruction item, can one generalize from the laboratory data that suggest that people can retain five unrelated things in their memory stores? The differences in required retention times (usually very short in laboratory studies) and the number of competing stimuli (usually much higher in a plant) make such a generali- zation questionable. Our experience indicates that an individual cannot remember more than two to three unrelated items or perceptual units. (The reader may recall his own errors of omission when sent to the grocery store to pick up certain items, but no list had been provided him.)

Based in part on experimental data and in part on our own judgment, we have derived the oral instruction model presented in Table 15-1. The table differentiates between detailed and general oral instructions and between the case in which order of recall is not important (the usual case) and in which order of recall is important.

With reference to the first column of HEPs in the table, lines 1 through 5, it can be noted that the values estimated for Pr[F] (probability of failure) increase by a factor of 3 (rounded) with each additional oral instruction item to be remembered. This factor was selected because we judged it provided a reasonable modification of laboratory results for the added retention value of meaningful instructions. Laboratory data show a somewhat faster increase in loss of retention of orally presented items, but in these studies, the items are meaningless digits or nonsense words (Woodworth, 1938, p 18). The HEPs in this column are for the case in which order of recall is not important. Thus, the .003 (line 2) is for the probability of forgetting the second of two things that the operator should remember, given that he has remembered the other one. The .0.1 estimate (line 3) is for failure to remember the third of three things he should remember, given that he remembered two of them in no particular order, and so on.

The values in the second column of HEPs also are for the case in which order of recall is not important. The .001 in line 1 refers to the failure to remember one item (or to initiate the task), and the .004 in line 2 is the estimated probability of failure to remember both of two items the person should remember. Thus, Pr[F] occurs if he forgets either or both of the two items he should remember. The .004 is obtained by converting the HEPs in the first column of HEPs to human success probabilities (HSPs). To obtain HSPs, we subtract the HEPs from unity. We then apply the following equation:

15-4

Table 15-1

Table 15-1 Estimated probabilities of errors in recalling oral instruction items not written down*

HEPs as a function of number of items to be remembered**

Number of Oral Instruction Items

or Perceptual Units

Item%

Pr[F] to recall item "N," order of recall not important

Pr[F] to recall all items, order of recall not important

Pr[F] to recall all items, order of recall is important

(a) HEP EF

(b) HEP EF

Oral instructions are detailed:

(1) (2) (3) (4) (5)

1tt 2 3 4 5

2 3 4 5

.001

.003 .01 .03 .1

3 3 5 5

.001 .004 .02 .04 .2

3 3 5 5 5

.001 .006 .03 .1 .4

3 3 5 5 5

Oral instructions are general:

(6) (7) (8) (9)

(10)

.001

.006

.02

.06

3 .3

5 5 5

.001 .007 .03 .09 .3

3 3 5 5 5

.001 .01 .06 .2 .7

3 3 5 5 5

*It is assumed that if more than five oral instruction items or perceptual units are to be remembered, the recipient will write them down. If oral instructions are written down, use Table 15-2 for errors in preparation of written procedures and Table 15-3 for errors in their use.

**The first column of HEPs (a) is for individual oral instruction items,

e.g., the second entry, .003 (item 2a), is the Pr[F] to recall the second of two items, given that one item was recalled, and order is not im- portant. The HEPs in the other columns for two or more oral instruction items are joint HEPs, e.g., the .004 in the second column of HEPs is the Pr[F] to recall both of two items to be remembered, when order is not important. The .006 in the third column of HEPs is the Pr[F] to recall both of two items to be remembered in the order of performance specified. For all columns, the EFs are taken from Table 7-2 as explained in Chapter 15.

t The term "item" for this column is the usual designator for tabled

entries and does not refer to an oral instruction item. The Pr[F]s in rows 1 and 6 are the same as the Pr[F] to initiate the task.

15-5

Oral Instructions

Pr[Ftoremember n items] 1 - HSP x HSP2 .. x HSPn (15-1)

For the case of two items:

Pr[Fto remember two items= 1 - .999 x .997 .004

If the operator must remember five unrelated oral instructions in no par- ticular order,

Pr[F remember = 1 - .999 x .997 x .99 x .97 x .9 .14

which we set to .2 for conservatism and to indicate an increase from the .1 value for Pr[F] to recall the fifth item in the first column of HEPs.

For the unusual case in which order of recall of the oral instruction items is important, the last column in the table indicates that the HEPs increase more rapidly as more items must be remembered. These values are obtained in the following arbitrary manner, using our notation in Table 5-1:

n Pr[Fto remember all items] = 1 - (15-2)

where n is the Pr[S] for item "N" from the first column of HEPs. Thus, if the operator must remember five oral instructions in the order given, n is the fifth item and n equals 1 - HEP = 1 - .1 = .9. Therefore,

Pr[Fto remember all 5 items 1- .9 4

Such calculations are speculative but do reflect a sizeable increase in probabilities of errors of omission since more items must be recalled in a fixed order.

The estimated HEPs in the first column for lines 6 through 10 (general oral instructions) in the table are obtained by doubling the first column of HEPs in the first five lines except for the first item, which remains at .001, the probability of forgetting the first item. The factor of 2 is selected as our best estimate of the "penalty to pay" for having to remem- ber general rather than detailed oral instructions.

The HEPs for general oral instructions apply to each item encompassed by the instructions, not to the number of general oral instructions. Thus, in the example of the general instruction, "Open the blocking valves for the Auxiliary Feedwater System," the three valves encompassed by the instruc- tion are each counted as an item.

15-6

Some Problems with Written Procedures

The factor of 2 increase in HEPs for items encompassed in general instruc- tions should not be used if the analyst judges that the operators have such familiarity with a task that the general oral instructions are equivalent to detailed oral instructions for them. The values in the second and third columns of HEPs in lines 6 through 10 are obtained in the same manner as those in lines 1 through 5 except that they are based on the HEPs in the first column of lines 6 through 10.

Note that the table stops with five items. We think it unlikely that an operator would be given more than five unrelated oral instruction items and not write them down or be handed a list by the person giving the instruc- tions. The estimated HEPs in the table indicate that a written list should be used if more than two or three items are to be remembered. The next sections include estimated HEPs for those cases in which oral instructions are written down, usually a more reliable way of performing.

The UCBs for the HEPs in Table 15-1 are taken from Table 7-2, and are expressed as EFs. For the last two columns of EFs in Table 15-1, the EFs are somewhat larger than the EFs that we obtained by applying the UCBs propagation method presented in Appendix A. For both simplicity and con- servatism, we have selected the EFs using the rules stated in Table 7-2.

Some Problems with Written Procedures

The HEP estimates related to written procedures are based in part on our reviews of written procedures in nuclear power plants (NPPs), on reviews of test, maintenance, and calibration procedures by Brune and Weinstein (1980) and of emergency operating procedures (Brune and Weinstein, 1981), and on the Essex study of operator performance at Three Mile Island (TMI) (Malone et al, 1980). The Essex study lists deficiencies typical of procedures in many plants:

- Serious deficiencies in content and format

- Little consistency between nomenclature in procedures and on panel components

- Instructions for control actions that seldom indicate the correct (or incorrect) system response

- Excessive burden placed on operator short-term memory

- Charts and graphs not integrated with the text

- Not clear which procedures apply to which situations

- No formal method for getting operator inputs into updates of pro- cedures

15-7

preparation of Written Material

- Grossly deficient instructions for assisting operators in diagnosing the problems related to the TMI accident.

From Table 3-8, we estimate that the HEPs related to written procedures

will be reduced by factors of 3 to 10 if the above types of deficiencies are corrected. The Essex study describes in detail the effects of pro- cedural deficiencies on human errors during the TMI incident.

Preparation of Written Material

Three major problems in the preparation of written material are the qualif- ications of the writers, the accuracy of the original material, and the provisions for changing existing procedures. These problems apply to the three types of written material usually evaluated in the performance of a probabilistic risk assessment (PRA):

(1) Formal written procedures that may be used many times (either in actual or simulated exercises), including test, calibration, main- tenance, normal operations, and abnormal operating conditions.

(2) Ad hoc written procedures that are one-of-a-kind, informally prepared procedures for some special purpose, e.g., isolation and restoration procedures to allow maintenance on one particular pump during a plant's power-generating mode.

(3) Written notes prepared in response to oral instructions to reduce the probability of forgetting any of the oral instruction items.

The first type is usually prepared by a select group of personnel, and these procedures receive extensive verification before being accepted as formal written procedures. Updating these procedures may be done on some periodic or as-needed basis.

The second type is usually prepared by the shift supervisor, his deputy, or

other qualified person in response to a specific need. Ad hoc procedures may or may not be kept on file.

The third type is the most informal and may be written on the proverbial back of an envelope. Such notes would usually be discarded after the oral instruction tasks had been completed and, if required, a record made in the shift log of what had been done.

Qualifications of the Writers

Formal written procedures used in NPPs typically are written by engineers, operators, or other qualified personnel. It is a truism in technical writing that engineers write for engineers and that their written communi- cations must be "translated" for others. It is not surprising, then, that typical NPP written procedures do not match the capabilities, limitations, and needs of those who use the procedures. Some of these problems are discussed in Chapter 3 under the heading, "Job and Task Instructions."

15-8

Preparation of Written Material

Although some utilities hire technical writing organizations to prepare their written procedures, samples of these materials indicate that this practice has not solved the problem of inadequately written materials. Until some standard set of specifications or guidelines for formal written procedures in NPPs can be developed and accepted, these problems will continue.

Accuracy of Original Written Instructions

Because operators are expected to follow carefully the mandated written procedures, it is important that the procedures be accurate. Inaccuracies will often lead to errors or at least to ineffective performance. For example, during a simulated loss-of-coolant accident (LOCA) in a dynamic simulator, a highly skilled operator became obviously flustered when he could not locate a critical switch. To cope with the simulated emergency, it is imperative that this switch be operated on time. After several minutes of running back and forth between sections of the engineered safety features (ESF) panel, he realized what the problem was. The four-digit identifying number for the switch was incorrect in the procedures. The procedures he was using came from the NPP, not from the simulator facility. As he noted later, people tend to trust the written word, and serious problems can arise when it is incorrect.

Malone et al (1980, pp 72-74) evaluated the TMI emergency procedure, "Loss of Reactor Coolant/Reactor Coolant System Pressure," in which several problems were found. Several steps critical to handling the incident were not included, e.g., the procedure did not tell the operator what to do if the high-pressure injection system had been initiated automatically. Critical symptoms for leak or rupture, such as a rapid, continuing decrease of pressurizer level, were not described. No tolerances were given for critical readings such as the pressurizer level. Of 15 written procedures considered relevant to the accident, only 7 were judged adequate.

There are no means to quantify the probabilities of the above types of inadequacies in written materials. Such errors reflect failure to test the procedures in a dynamic situation (the simulator exercise described above) as well as failure to anticipate the full scope of situations in which the procedures must be used (the TMI incident). The HEPs below apply to those formal or ad hoc written procedures that reach the working crew for use on the job and to errors within the sections that were actually printed. We have no way of determining what additional material should be included or whether the procedures that are presented are appropriate.

Within these limitations, we estimate an HEP of .003 for the probability that an item that is intended to be included in a formal or ad hoc written procedure will not be included, as well as for the probability that there will be some error of commission for each item that is included. This estimate applies to valve restoration lists and the preparation of tags to designate positions of valves and other safety-related equipment.

S5~-9

Preparation of Written Material

The HEPs of .003 for errors of omission and commission, per item, to be entered into a procedure, imply that if a written procedure consists of only 10 required activities, there is a .06 probability that at least one item will be omitted or entered incorrectly (1 - .99720)., This is the probability without recovery factors. As described below, the recovery factors for errors in written procedures are not very effective in many plants. Because of the uncertainty in how procedures are prepared and checked, we assign an EF of 5 rather than the usual EF of 3 to the HEPs for preparation of procedures.

The nominal HEP of .003 may be adjusted up or down as a function of the recovery factors observed at the plant. One important recovery factor is the type of validation used for newly prepared or revised procedures. Proper validation requires a trial (or walk-through) of the procedures before release to the users. Often the procedures are released without such validation, and many errors are discovered in use. Required correc- tions are frequently passed along among the users by word of mouth, and the written procedures themselves remain uncorrected. The error opportunities in such a situation are obvious. If this is found to be the case in a plant being studied, the .003 estimate will be too optimistic, and a higher HEP will be appropriate, e.g., the upper UCB.

In the case of written notes prepared in response to oral instruction items, the .003 estimate is too pessimistic for errors of both omission and commission. We presume that the recipient will write down the items to be remembered as they are given to him. In the face-to-face situation, there is ample opportunity for him to ask for repeats and clarification if needed. Therefore, we assess the probabilities of errors of both omission and commission as negligible for up to five items. If more than five items

are to be written down, the situation has more potential for error, and an arbitrary estimate of .001 per each item in the list is assessed for errors of both omission and commission.

Table 15-2 presents the estimates for errors in the preparation of written materials.

Updating of Written Procedures in a Plant

Changes to existing procedures constitute a significant source of error. In a study of test, calibration, and maintenance procedures in five plants, none of the plants had a systematic process for identifying procedures requiring revision (Brune and Weinstein, 1980). Reliance was placed upon human memory to identify procedures that might be affected by change (there was no index relating procedures to equipment). Consequently, out-of-date procedures would sometimes be overlooked. We are unable to estimate the probabilities of- error in making changes to existing written procedures.

The problem of obsolete procedures is complicated by two factors: the length of time required to update procedures and the tendency of users to retain "personal" copies of procedures. Some plants mandate revision and

15-10

Table 15-2

Table 15-2 Estimated HEP per item (or perceptual unit) in preparation of written material*

Item Potential Error HEP EF

(1) Omitting a step or important instruction from a formal .003 5 or ad hoc procedure** or a tag from a set of tags

(2) Omitting a step or important instruction from written Negligible notes taken in response to oral instructionst

(3) Writing an item incorrectly in a formal or ad hoc pro- .003 5 cedure or on a tag

(4) Writing an item incorrectly in written notes made in Negligible response to oral instructionst

Except for simple reading and writing errors, errors of providing incom- plete or misleading technical information are not addressed in the Handbook.

The estimates are exclusive of recovery factors, which may greatly reduce the nominal HEPs.

Formal written procedures are those intended for long-time use; ad hoc written procedures are one-of-a-kind, informally prepared procedures for some special purpose.

t A maximum of five items is assumed. If more than five items are to be written down, use .001 (EF = 5) for each item in the list.

15-11

HEPs in Use of Written Procedures

review of procedures within 10 days after a change that affects the pro- cedures; whereas, other plants do not specify any time requirements for revision and reissue. Obviously, the longer the revision period, the more likely it is that incorrect procedures will be in use. The other factor in the use of obsolete procedures is the tendency of some personnel to make personal copies of procedures they retain and continue to use even after revisions have been issued. As a result, the copy is not accessible for revision. This situation is less likely to happen in plants where per- sonnel must retain procedures after use.

One consequence of these problems is that procedures with handwritten changes will be in use for various periods of time. These handwritten changes are frequently difficult to read, and errors are likely. For example, a numerical value will be crossed out and the new value handwrit- ten between the lines, even in a single-spaced copy. If the analyst finds that such poor administrative control practices are tolerated in a plant, he should use the upper bound of the .003 estimate, .015, in Table 15-2.

HEPs in Use of Written Procedures

This section presents some estimated HEPs for errors when use of typical NPP written procedures is called for, and also gives some adjustments that can be made to these nominal HEPs, depending on the quality of the procedures.

Written Procedures Models

In Chapter 3, we indicated that typical NPP procedures do not conform to established principles of good writing. The typical format is a narrative style with an excessive number of words to convey essential information--in engineering parlance, the signal-to-noise ratio is low. Steps in these procedures often include several special-instruction items. The potential problem is that the user may perform one of these special-instruction items, such as, "Check that the pressure is 40 psig ±5%," look at a gauge to verify this setting, and then return to the written instructions--but to the wrong place and skip an instruction.

Another difficulty with the typical NPP written procedures is that the low signal-to-noise ratio discourages the highly skilled person from using them. It is boring and offensive to read so many words to pick out the few important items. We find that failure to use written instructions properly is often related to the fact that the procedures are poorly written.

Considering only the typical narrative-type NPP procedures, we have devel- oped the assumptions and estimates described below.

Zero dependence (ZD) between the steps is assumed unless an interaction is obvious. For example, if the procedure is written so that Step 5 is to adjust some rotary control and Step 6 tells what instrument reading should result from that adjustment, the assumption of ZD between the two steps is clearly inappropriate; complete dependence (CD) would usually be assessed.

15-12

HEPs in Use of Written Procedures

Apart from such obvious cases, ZD is usually assumed among the steps in a written procedure.

Table 15-3 presents estimated HEPs and EFs made on the assumption that typical narrative procedures are being used. For estimates of nonuse of written procedures, see Table 16-1. Adjustments of the nominal estimates in Table 15-3 are discussed under the next heading.

In developing the written procedures model presented in Table 15-3, we recognized the fact that as the list of items grows, there may be a higher probability of overlooking any given item. We have arbitrarily represented this variable, number of items in a list, by two points: a short list, defined as 10 or fewer items, and a long list, defined as more than 10 items. The table also indicates that a procedure without a checkoff pro- vision is more susceptible to errors of omission than a procedure with a checkoff provision that is used properly. The HEPs for errors of omission of procedural items when a checkoff provision is improperly used are the same as in the case of a procedure without checkoff provisions. Proper use of a checklist is defined as reading an item in the checklist, performing the operation called for in the item, and then checking off that item on the checklist to indicate that it has been done. Whether a check mark or one's initials are used makes no difference. (If it is desirable to keep track of who performs a certain job, a signature can appear somewhere to indicate the person responsible for carrying out the procedure.) Any other use of a checklist is defined as improper and is considered tantamount to ignoring the checking function of the checklist. In such cases, assessment of errors in use of the checklist is the same as for the case of use of a written list without any checkoff provisions.

Finally, the table lists a higher probability for errors of omission (item #5) for not using written procedures when they are available and should be used. in such cases, the operator is relying on his skill-of-the-craft to remember every item. Although this may be reasonable when performing very familiar tasks, most of us can think of instances in which we have over- looked a critical step in a frequently performed task (e.g., forgetting to reinstall the drain plug in the oil pan on our automobile before pouring in new oil). We make the assumption that if a person does not use available written procedures, it is because he knows well the tasks involved. While it is quite possible for him to make an error of omission, errors of com- mission (including selection and setting errors) will not be significantly different for the case in which available written procedures are not used. Therefore, for errors of commission when written procedures are not used, consult the relevant tables in Chapter 11 (for display readings), 13 (for manual controls), and 14 (for locally operated valves).

Adjustments of Nominal HEPs Using Written Procedures

Several possible formats can be employed for NPP procedures. We favor the columnar format shown in Figure 3-3, with the addition of a contingency column to tell the operator what to do if the anticipated results or feed- back for each step fails to materialize. The signal-to-noise ratio is

15-13

Table 15-3

Table 15-3 Estimated probabilities of errors of omission per item of instruction when use of written procedures is specified*

Item** Omission of item: HEP EF

When procedures with checkoff% - provisions are correctly used

(1) Short list, 410 items .001 3

(2) Long list, >10 items .003 3

When procedures without checkoff provisions are 1tt used, or when checkoff provisions are incorrectly used

13) Short list, 410 items .003 3

(4) Long list, >10 items .01 3

(5) When written procedures are avail- .05 5 able and should be used but are not usedtt

The estimates for each item (or perceptual unit) presume zero dependence among the items (or units) and must be modified by using the dependence model when a nonzero level of dependence is assumed.

The term "item" for this column is the usual designator for tabled entries and does not refer to an item of instruction in a procedure.

%tCorrect use of checkoff provisions is assumed for items in which written

entries such as numerical values are required of the user.

ttTable 16-1 lists the estimated probabilities of incorrect use of checkoff

provisions and of nonuse of available written procedures.

*If the task is judged to be "second nature," use the lower uncertainty

bound for .05, i.e., use .01 (EF = 5).

15-14

HEPs in Use of Written Procedures

high, each numbered item has only one special instruction item (or com- pletely dependent set of items--such as the four switches in Step 2 in the figure), and there is provision for checking off each step as it is per- formed. The "Check" column is placed next to the "Step" column so that the user (or later checker) can quickly run down the two columns and see which steps might have been omitted. Thus, while placement of the "Check" column at the right side of the sheet seems logical in view of our normal left-to- right reading, it is more convenient to place this column as shown in Figure 3-3.*

Extrapolating from the Haney study (1969) described in Chapter 3 under the heading, "Job and Task Instructions," we estimate a factor of 3 reduction in errors of omission and commission if the columnar format shown in Figure 3-3 is used instead of the typical narrative style of NPP procedures in which several activities (special instruction items) to be performed are listed in a paragraph. There should also be some increase in the number of people who use the procedures properly, but we cannot offer any estimate of the increase.

All the narrative-style procedures we have observed have ample room for improvement. Brune and Weinstein have developed checklists for use in evaluating written procedures for routine operations (1982) and for abnormal operating conditions (1983). These checklists show how the signal-to-noise ratio of narrative-style procedures can be improved so that they are nearly as good as columnar-style formats. If a narrative-style procedure conforms to the Brune and Weinstein criteria, the nominal HEPs in Table 15-3 can be reduced by a factor of 2. If a written procedure follows a columnar rather than a narrative style and also meets their criteria, the factor of 3 reduction in tabled HEPs can be used. Conceivably, the reduc- tion could be greater, but until some performance data can be obtained, we prefer not to exceed the factor of 3 reduction.

Two indices of the quality of written routine procedures from Brune and Weinstein (1982) are listed here because they provide a quick measure of the relative quality of a procedure. These are the Complexity Index (CI) and the Specificity Index (SI).

CI =No. of actions in sample of steps or paragraphs No. of steps or paragraphs sampled

As a training aid during the instructional process, we recommend combining the columnar format with any necessary background or explanatory detail in a narrative format so that a trainee can read the text, perform the re- quired task, and check it off--thus acquiring the habit of using the pro- cedures correctly as part of the training program. The procedures to be used after completion of training will not have the explanatory material.

15-15

HEPs in Use of Written Procedure

where

(1) the number of actions is usually the number of verbs (or compound verbs) in a step or paragraph,

(2) the CI should be 1.5 or less, and

(3) the sample is 20% of all steps but no less than 10 steps.

SI = No. of steps in sample meeting all 3 specificity criteria

No. of steps sampled

where

(1) the three criteria are

(a) the action to be taken is specifically identified (e.g., open, close, torque, etc.),

(b) the limits (if applicable) are expressed quantitatively (e.g., 2 turns, 100 inch-pounds, etc.), and

(2) The SI should be at least 0.9, and

(3) the sample is 20% of all steps but no less than 10 steps.

If the CI and SI for routine procedures meet all of the above criteria, they are better-than-average procedures.

For the case of emergency operating procedures, the Brune and Weinstein (1983) criteria are more stringent. For example, there must not be more than one action per numbered step, and all steps must meet the specificity criteria stated above.

The top half of Figure 15-1 shows an example of a poorer-than-average writ- ten step for which we would assign the upper UCBs from Table 15-3. The figure also shows a revision using the columnar format. An in-plant exper- iment by Haney (1969) showed that people accustomed to the narrative format made about one-third fewer errors with the columnar format and liked this format better. (We think the sample columnar format could be further improved.)

i -16

Figure 15-1

Narrative Format:

3.4 On the FTM panel, depress the DC push to measure button. The lamp will go out. Depress and hold the RMS push to measure button. The RMS VM will read between 0.55 and 0.56, inclusive, for "A" thru "C" units, and 0.049 thru 0.050, inclusive, for "D" thru "G" units. Record the readings for "A" through "C" units as 3.4A, and the "D" thru "G" units as 3.4B.

Same material as above in columnar format:

PARA.

3.4

ITEM

FTM

panel

ACTION

a. Depress DC

push to measure

b. Depress and hold RMS push to measure

Read RMS VM:

ACCEPTABLE

lamp goes

out

NOTE

for "A" thru "C" units

for "D" thru "G" units

0.055 thru 0.056

inclusive

0.049 thru

0.050 inclusive

record (3.4A)

record (3.4B)

Figure 15-1 Examples of narrative and columnar formats (from Haney, 1969).

15-11

HEPs in Use of Written Procedure

In a recent study by Kozinsky (1983a) of errors made by teams of skilled NPP operators in training simulators, it was found that when plant proce- dures similar to the top half of Figure 15-1 were employed, all of 13 teams undergoing requalification exercises overlooked critical actions when they were buried near the end of a narrative paragraph. Figure 15-2 shows an example of such a paragraph. None of the 13 teams carried out the hidden instruction, "Depress the 'governor valve position limit, raise to 100 percent and verify governors open fully." For this type of extremely poorly designed procedure, even the use of the upper UCB of .1 (Table 15-3, item 4) would be optimistic.

Finally, an analyst can obtain a useful impression of the quality of a written procedure by having a small sample of operators walk through the steps, as described in Chapter 4. This approach, coupled with the appli- cation of the Brune and Weinstein checklists described above, can provide the analyst with a sound basis on which to assess estimated HEPs for errors of omission in the use of written procedures. The walk-through is impera- tive when assessing HEPs related to the use of the new symptom-oriented emergency operating procedures since there is insufficient recorded oper- ating experience with these procedures to derive estimated HEPs for this issue of the Handbook.

15-18

Figure 15-2

"RELATCH THE UNIT BY DEPRESSING THE-TURBINE 'LATCH' PUSH-BUTTON AND TURBINE TRIP HS-47-24 TO RESET. HOLD FOR ABOUT TWO SECONDS. THE 'UNIT TRIP' LAMP WILL GO OUT AND THE 'LATCH' BUTTON WILL STAY ILLUMINATED. CHECK THAT INTERCEPTOR AND REHEAT STOP VALVE REOPEN FULLY AS INDICATED BY THE POSITION AND LAMPS. GOVERNOR AND THROTTLE VALVES SHOULD BE CLOSED POSITION. DEPRESS THE 'GOVERNOR VALVE POSITION LIMIT' RAISE TO 100 PERCENT AND VERIFY GOVERNORS OPEN FULLY. THE 'REFERENCE' AND 'SETTER' DISPLAYS WILL BE ENERGIZED WITH 0000 DISPLAYED AND THE 'SPEED CONTROL' LAMP WILL BE ILLUMINATED."

Figure 15-2 An example of a less-than-average-quality written

procedure (from Kozinsky, 1983a).

15-19

Chapter 16. Management and Administrative Control

CHAPTER 16. MANAGEMENT AND ADMINISTRATIVE CONTROL

Overview

Administrative control refers to the extent to which plant policies are carried out and monitored. It reflects the type of structure inherent in a plant and reinforces the lines of responsibility. Good administrative con- trol systems can significantly reduce the probabilities of certain types of errors, and the converse is also true.

This chapter discusses administrative control in general then focuses on areas in which changes in administrative control can cause changes in human error probabilities (HEPs). These areas include estimated HEPs for the following: the use of written procedures when such is called for, the cor- rect use of checkoff provisions, the implementation of tagging procedures, and the control of scheduled activities. The effect of plant management on the implementation of administrative control procedures and on general in- plant performance is discussed in conjunction with each of these areas, and examples are provided of different levels of administrative control sys- tems. These levels emphasize different types of tagging procedures ob- served in several of the nuclear power plants (NPPs) visited or analyzed by the authors.

Administrative Control

Types of Administrative Control

The term "administrative control" encompasses many areas of plant opera- tion. In a general sense, it refers to the degree to which the plant is run in conformance to the guidelines by which it was designed to run. If the plant is run "like a tfght ship," it can probably be assumed that the administrative control system is adequate. Every functioning NPP involves a complex interaction of human, hardware, and software systems. In most cases, safety-related equipment is controlled automatically by a set of programmed responses. The program determines how each element of the safety systems should support the others in situations calling for their operation and specifies the responses of the hardware. In the same way, the human elements are supposed to interact with other system elements in predetermined ways. The humans are responsible for the performance of certain tasks necessary to reliable and safe plant operation in both normal and abnormal situations. Each plant has a number of control features de- signed to ensure that the humans can perform these responses successfully.

These control features include formal written procedures that are to be followed when a certain situation arises. Another control feature is the policy established in the plant to ensure that these procedures will indeed be followed (as opposed to a policy that would allow for alternative courses of action).

16-1

Administrative Control

Another control feature consists of the provisions for keeping track of the status of individual equipment items. These provisions include indications of status in the control room or elsewhere in the plant and a surveillance system for verifying equipment status. The latter can involve a system of inspections that are scheduled at regular intervals, inspections performed when a certain set of criteria are met, or record-keeping systems to track equipment status on paper. These systems and the methods used to ensure their employment constitute part of the adminstrative control system. For example, in a human reliability analysis (HRA), to estimate the probability of a restoration activity, we must analyze correct performance of the restoration procedure, and we must estimate the probability that the pro- cedure will be followed as mandated. These probabilities are entered into HRA event trees as described in Chapter 5.

The quality of administrative control can be inferred from the extent to which written procedures and checklists are used properly, the type and use of tagging systems for critical items, the type of inventory system and its use, and the general attitudes and practices of operating personnel. These performance shaping factors (PSFs) range from poor to good at different plants. Good administrative control means that certain types of errors can be held to a minimum. For example, as noted in Chapter 15, lower HEPs are associated with proper use of written materials. Good administative con- trol increases the likelihood of proper use of these important job aids.

Examples of Poor Administrative Control

Following are some examples of poor administrative control that make errors more likely than if good control is used. In one plant, the use of step- by-step written procedures was mandated by plant management with severe penalties, such as loss of pay, for offenders. Despite this plant policy, we frequently observed that highly skilled technicians performing the work made only casual reference to the procedures. Much of the time, the page to which the written procedures were opened did not list the operations being performed. This is an example of poor administrative control because there was no enforcement of plant policy, and the policy was openly disre- garded. Very often, improved human factors engineering facilitates the implementation of administrative control. In the above case, this problem was reduced only when the procedures were rewritten so as to remove excess wordiness, yielding a higher signal-to-noise ratio (as described in Chapter 15).

At several plants., we have observed people working without mandated check- lists or using them incorrectly, e.g., performing several steps and then checking them off all at once on the checklist. This same casual attitude has been noted in the use of tags and keys. At one plant, keys for criti- cal valves were distributed to several operators. Obviously, the keys were not under administrative control but were used for a different purpose, as described later.

A final example shows the importance of evaluating the extent to which mandated administrative controls will be used properly. In two separate

16-4

Administrative Control

plants, elaborate tagging and record-keeping systems had been developed to monitor the status of equipment items when they were changed to permit test or maintenance. The record-keeping system at both plants required that all items changed have tags applied to them, that the information on the tags be entered into a logbook in the control room, that an operator be assigned the responsibility of maintaining these records, that a cross-reference file be kept, cataloguing each piece of equipment by function and by the expected date of return to service, etc. Each of the systems called for checks and cross-checks of paperwork and of the equipment items themselves. On paper, both systems looked like excellent systems for controlling equip- ment change and restoration.

On visiting the plants, however, a different picture of the effectiveness

of these systems emerged. At the first plant, we asked the operator in charge of the tagging system to talk us through the procedure he would follow in recording equipment status. He gave a clear presentation of the operation of the system as it was outlined in the plant procedure on tag- ging control. We also asked to see examples of equipment items, tags, and logbooks in various stages of the tracking process. Everything we saw agreed with the procedure. The administrative control system was excellent not only on paper, but in practice as well.

At the second plant, the implementation was a different story. The opera-

tor in charge of tagging was unable to give us an accurate account of the process followed in applying, recording, and removing tags. He apologized for his lack of knowledge, explaining that none of the operators cares to be in charge of tagging. Management's solution was to rotate operators to the position of tagging controller on a weekly basis. The operator on duty told us that each operator was on tagging duty for a week about every 6 months. Usually, some of the procedures were new each time an operator rotated to tagging duty. At the beginning of the week, all was confusion-- there was no formal turnover of duties from one operator to the next. The new man was required to familiarize himself with the new policies by read- ing the procedure. The operator said that by Friday of the week in which he served as tagging control officer, he usually felt he had a good picture of the way things should be run., By that time, of course, his duty was almost finished, and it would be another 6 months before he was assigned to these duties again.

From this example, it is easy to see that what a plant has on paper is not always what is carried out. Both these systems looked good on paper, but the second plant could not be counted on to maintain the program as speci- fied. In an HRA, we would assign to the second plant a fairly high proba- bility of failure to carry out the policy called for in the tagging control procedure. The subsequent probabilities of errors of omission and commis- sion would also be higher for this case. These examples also illustrate the necessity for a talk-through to ensure that the task analysis is com- plete and accurate. To rely solely on a published plant policy may lead the analyst to overly optimistic estimates of the probabilities of success-

ful task performances in areas defined by the plant policy.

1.6-3

Plant Policy

Purpose of Administrative Control

It must be assumed that administrative control policies are designed with

some purpose in mind. The model of plant operation calls for people to perform certain functions at some level of reliability. Failure to main- tain this level of reliability can degrade the probability of effective plant performance. Any policy that does not contribute to the safety, efficiency, or productivity of the NPP should be discontinued. Useless or cosmetic policies demanded by plant management will be treated as such by personnel. This attitude could carry over to other policies. If a policy is to be carried out consistently and effectively, it must have the respect of the personnel involved in terms of its contribution to plant safety, efficiency, or productivity. If operators cannot see the use of a required policy, they will be less likely to enforce it than if its utility is obvious.

Clearly, in performing an HRA, some estimate should be made of the percent- age of people who carry out the various procedures properly and the per- centage of those who do not. If one is performing an HRA of a plant with-

out knowledge of the types of administrative control at the plant, it is still necessary to form some estimate of the quality of administrative control. In Table 16-1, we list estimated HEPs for administrative control.

Each HEP is discussed in the following sections. We believe these esti- mates are conservative. Since there is so much uncertainty in an across- plants analysis, the conservative estimates are intended to ensure that HRA is not overly optimistic.

Plant Policy

The term plant policy refers to a set of operating requirements that plant management generally expects to be followed. Usually they are described

fully in a formal set of written instructions that are available to all personnel in relevant positions. For example, operations policies are available to all members of the operations staff. Although these policies

are written, they usually include information that the employee is expected to memorize, since he should not have to refer to the policy document each

time he performs duties affected by them.

These general information documents can consist of such subjects as em-

ployee conduct, security procedures, radiological safety practices, tagging procedures, and job-performance specifics. In most cases, the scope of these documents is such that they will affect the performance of several tasks of a generic nature; e.g., tagging procedures will affect all changes and restorations of equipment, conduct of operations procedures will man- date certain practices to be followed by all operations personnel while performing their duties, maintenance policies will describe the interaction between maintenance and other departments for all tasks performed, and so forth.

Each of these plant policies is designed for continued use. Just as per-

sonnel in most plants are expected to wear identification badges at work,

16-4

Table 16-1

Table 16-1 Estimated HEPs related to failure of

administrative control

Item Task HEP EF

(1) Carry out a plant policy or scheduled tasks .01 5 such as periodic tests or maintenance per- formed weekly, monthly, or at longer intervals

(2) Initiate a scheduled shiftly checking or .001 3 inspection function*

Use written operations procedures under

(3) normal operating conditions .01 3

(4) abnormal operating conditions .005 10

(5) Use a valve change or restoration list .01 3

(6) Use written test or calibration procedures .05 5

(7) Use written maintenance procedures .3 5

(8) Use a checklist properly** .5 5

Assumptions for the periodicity and type of control room scans are discussed in Chapter 11 in the section, "A General Display Scanning Model." Assumptions for the periodicity of the basic walk-around inspection are discussed in Chapter 19 in the section, "Basic Walk- Around Inspection."

** Read a single item, perform the task, check off the item on the list. For any item in which a display reading or other entry must be written, assume correct use of the checklist for that item.

16-5

Plant Policy

so are they expected to carry out the other plant policies relevant to their jobs. However, errors occur even in the best-controlled systems. In some circumstances, the plant policy requirements may be too cumbersome for the employee to deal with at the time he performs a set of tasks. In other cases, an employee may overlook part of a requirement--he may put off com- pleting necessary paperwork until a later date, omit some part of a re- quirement in an effort to be more ,efficient or because he considers it unnecessary, or his attention may be drawn away from the tasks at hand by another, more compelling demand for action. As a conservative estimate, we assign an HEP of .01 (EF = 5) to the probability that a plant policy will not be carried out when it should. This estimate is not overly pessimistic since it assumes that 99 times out of 100, the employees will act according to the policy. For example, if plant policy dictates that each shift the shift supervisor will order the performance of a certain task, the .01 estimate means that the task will fail to be ordered only about 11 times a year. (Recovery factors may catch most of these oversights.) Theerror factor (EF) of 5 is used rather than the usual EF of 3 for HEPs 4.01 (Table 7-2) because of the greater uncertainty when identifying adherence to plant policies.

This .01 estimate also applies to the probability of failure to perform scheduled tasks such as periodic test and maintenance that are performed weekly, monthly, or at longer intervals. In some plants, such periodic activities are controlled by the shift supervisors, who must keep track of the times by which they must be done. The supervisors must inform the maintenance department of the expected date of these periodic activities and arrange for their performance with other plant departments. In one plant we visited, a staff member had been assigned to maintain a list of all scheduled test, maintenance, and calibration activities. Her primary job responsibility was to inform the departments involved in these activi- ties in plenty of time to allow them adequate preparation, to interface with other involved departments, to provide documentation for the events, and to report the results. In this case, we assign a lower estimate of the failure of plant policy, perhaps using the lower bound of .002.

In other plants, an elaborate computer-based system with a daily printout is used to alert the staff to the various tests, maintenance, and other scheduled activities. The printout describes what is to be done and lists the documents to be provided to the shift supervisor by the personnel who operate the computer system. Such a system can lower significantly the probability of errors of omission as well as of commission (e.g., using the wrong procedures for a task).

There is a special subset of tasks that are considered part of plant policy. These are regularly scheduled checking activities such as shiftly inspections of.the engineered safety features (ESF) panels in the control room and the shiftly walk-around inspection of items outside the control room. Since these activities take place every shift and are usually per- formed by the same person (or one of a very small number of people) every shift, the probability of failing to perform them is lower than that of

16-6

Use of Available Written Procedures

failing to carry out a general, infrequently required task such as quar- terly maintenance. The HEP estimate for failure to initiate a shiftly checking function that is part of the regular plant policy is .001 (EF = 3).

Use of Available Written Procedures

A common error in an HRA is to assume that available written procedures will always be used properly. Even in work with severe penalties for nonuse or incorrect use of written procedures, nonuse and misuse have been observed.. In the absence of such penalties, it is reasonable to assume a greater frequency of such practices. In a study of test, calibration, and maintenance procedures (Brune and Weinstein, 1980), at least one manager or supervisor at every plant visited expressed concern that personnel might not use or follow written procedures as intended. Although the nonuse or misuse of written procedures is a direct function of the administrative control in a plant, these problems are also related to the quality of the written materials themselves, as discussed in Chapter 15.

For those tasks for which the use of written procedures is mandated by plant policy, we have not been able to determine the ratio of tasks per- formed without written procedures to tasks performed with them. Brune and Weinstein (1980) found no mention of a procedure in 24% of almost 1,000 Licensee Event Reports (LERs) in the maintenance, test, and calibration areas. In general, these LERs described relatively simple tasks, commonly regarded as falling within the skill-of-the-craft for which detailed writ- ten procedures are not provided. For example, a maintainer may be in- structed to repair Valve HC1234, and it is assumed that this is all the instruction he needs.

On the basis of their observations at five operating NPPs, Brune and Wein- stein concluded that many maintenance tasks are performed without written procedures. A detailed analysis of the LERs indicates that 4% of the tests and 16% of the calibrations were performed without procedures, whereas 37% of all maintenance was performed without procedures--which suggests that written procedures are less available or perhaps less useful for mainte- nance tasks.

In a substantial number of LERs, unapproved actions were taken even when a written procedure was available. Of 132 maintenance LERs classified as noncompliances, 57 were considered unauthorized. These 57 involved failure to use or to follow an available procedure, performing an action not in the procedures, or performing procedural steps out of sequence.

The above data do not tell us how often oral instruction are used in place of written procedures, nor how often written instructions are ignored. However, they do suggest that there is much. reliance on memory, especially during maintenance operations.

The problem of the use of written procedures is a difficult one to model. It is possible to estimate the percentage of time that procedures will not

16-7

Use of Available Written Procedures

be used, but an estimate of the necessity of the procedures called for cannot be provided. A procedure may not be used when its use is required for several reasons, including poor administrative control and poor human factors engineering. An example of the former is seen in cases in which

the supervisor of a group does not insist on the use of mandated proce- dures. An example of poor human factors engineering is a procedure's non-

use because it is difficult to locate or to use. If the procedures are not used because the tasks involved are ones with which the employee is so familiar that his use of them is clearly unnecessary, a different question

arises. If skill-of-the-craft techniques are all that are necessary, use

of the procedures should not be called for. On the other hand, in some cases the employee may feel familiar with a set of tasks but fail to recall

them correctly. When safety-related tasks are involved, allowing operators or others to work without reference to procedures greatly increases the potential for degrading plant safety. Therefore, the analyst should not assume that the task being analyzed falls within the skill-of-the-craft category without a thorough analysis of the situation. In probabilistic risk assessment (PRA), we take the conservative position that nonuse or misuse of procedures will lead to increased error probabilities.

As the results of the Brune and Weinstein study cited above indicate, the

probability of nonuse of written procedures changes according to the per-

sonnel involved. Reactor operators are more likely to use procedures than are calibration technicians, who, in turn, are more likely to use them than are maintenance personnel. This tendency is reflected in the estimated HEPs for nonuse of procedures in Table 16-1. It is estimated that opera- tions personnel will fail to use mandated written procedures 1 time out of 100; the HEP is .01 (EF = 3). This HEP applies to the use of normal oper- ating procedures such as a valve change or restoration list. They are more likely to use procedures under abnormal operating conditions in spite of the fact that the immediate actions required of the operators following an

abnormal event must be memorized. The lower probability estimated for nonuse of procedures under abnormal operating conditions reflects our opinion that in such a stressful situation, the operators will be looking for "reinforcement" to support their decisions. This behavior pattern is reflected in the'generally higher levels of dependence assumed following accidents and in the lower estimate of the probability of nonuse of written procedures, given as .005 (EF = 10).

Calibration technicians are less likely to use procedures than are opera-

tions personnel, possibly because of the repetitive nature of their tasks. However, they are more likely to use procedures than are maintenance per- sonnel. This may be due to the fact that calibration technicians must read

and record measurements that should fall within some predetermined toler- ance limits. The estimated HEP for the failure of calibration technicians to use written procedures is .05 (EF = 5). The failure of maintenance personnel to use procedures is estimated as .3 (EF = 5).

The question of misuse of procedures must be addressed for procedures

designed as checklists or procedures having a checkoff provision. Correct

use of a checklist entails reading an instruction, performing the task

16-8

Tagging Systems

called for, and checking it off on the checklist to indicate that it has

been completed. At times, the items are not read, performed, and checked off in order. The checklist user may decide that constant reference to the

checklist is unnecessary or annoying, or the checklist may not have been designed with the user in mind, so that following it exactly is awkward. There are several possibilities for misuse of a checklist. For example, if the items are not attended to in order, one at a time, the increased proba-

bility of completing each item afforded by the checklist provision is dis- counted. If the checkoff is misused, following the procedure is modeled as

though there were no checkoff provision.

Estimating the likelihood of correct use of procedures should be done on a plant-specific basis. Observation of personnel performing checklist tasks is the best method of determining what percentage of the personnel actually use the lists as they have been designed. In the absence of plant-specific information, use an estimated HEP of .5 (EF = 5) for the probability of

incorrect use of a procedure. This reflects our opinion that half of the personnel will use it as it is intended to be used and half will not, or that half of the time the procedure will be used correctly. However, for any checklist item in which the user is required to write down some reading or other value, that particular item is modeled assuming correct use of the checklist.

Tagging Systems

In most plants, a tagging system is used to indicate the status of equip- ment when such status has a special meaning or calls for special notice, e.g., a normally closed valve has been opened to permit testing or main- tenance. Tagging systems are a special form of administrative control in that the use of tags and their management is one of the most important methods for ensuring awareness of the changed status of equipment relative to test or maintenance tasks and for ensuring the later restoration of that equipment to its original status. For some plant activities, it will be necessary for items of eqtipment to be placed in nonnormal states. For example, it may be necessary to close a normally open valve to permit main- tenance. After the maintenance is completed, it is important that the

valve be returned to its normal state: open. In another case, it may be imperative that a valve remain in its normal state during maintenance, i.e., in the protected normal state.* If another activity is attempted that requires the valve to be closed, it is essential to have some means of

alerting personnel that the valve must remain open for maintenance. In both cases, tags can be applied to the equipment to indicate the state required for the activity.

As used here, the term "tagging system" includes all administrative con- trols that ensure (1) awareness of any valves or other items of equipment

A valve or other item that is tagged or tagged and locked in its normal

state to guard against someone changing its status is said to be in the protected normal state.

16-o

Tagging Systems

that are in a nonnormal state or a protected normal state and (2) prompt restoration of this equipment to the normal state or to the unprotected state after the completion of test or maintenance operations. Thus, a tagging system includes the use of (1) tags, (2) chains, locks, and keys, and (3) logs, suspense forms, and any other techniques that provide a record of the above items.

Obviously, the quality of tagging systems can vary widely. As a guide for HRA, we identify four levels of tagging systems, listed in Table 16-2. The

HEPS in the Handbook related to tasks involving the application and removal of tags are premised on a Level 2 tagging system, as defined in the table. Level 2 tagging is typical of plants we have visited. If a better or worse tagging system is observed at a plant, the analyst must adjust the nominal HEPs he uses upward or downward.

Because of the wide variety in tagging employed in NPPs, the table lists

only some qualitative differences. It is up to the analyst to judge which level or levels are used in a plant. In general, Level 1 tagging provides good recovery factors for errors of omission and selection in the restora- tion process. The matching of numbers on the removed tags to the numbers in the records in the tagging office provides a check on such errors. An operator is assigned full time as a tagging controller for each shift in which safety-related items may be changed from or restored to their normal operating states. If this duty is rotated among personnel, each tour of duty lasts at least a month, and there is provision for adequate training

and turnover procedures for the oncoming tagging controller.

For purposes of HRA, if the analyst assesses Level 1 tagging, the HEPs in

the Handbook related to valve or other safety-related equipment change and restoration should be reduced. As an operating rule, we suggest the use of

the lower uncertainty bound (UCB) for each relevant HEP. This includes the effects of the recovery factors afforded by record keeping and tight tag control. The analyst may be able to justify even lower HEPs, but this must be based on a detailed analysis of potential errors in change and restora- tion procedures plus the recovery factors afforded by record keeping and tag control. In judging that there is a Level 1 tagging system employed, the analyst must look for exceptions to this excellent level, as described later.

The Level 2 tagging system is similar to the Level 1 except that the qual-

ity and extent of control is materially less. The formal system (on paper) may be described as a Level 1 system, but there are several exceptions, so that the analyst will want to conservatively assign Level 2 tagging in his assessment and will use the nominal HEPs for change or restoration proce- dures. Although there may be a designated tagging controller in a Level 2 system, the job is rotated frequently, or it is but one part of an opera- tor's regular duties. In assessing Level 2 rather than Level 1, the ana- lyst is making the judgment that the record keeping and control of tags does not provide a thorough checking for errors of omission or selection--

either for change or restoration procedures.

16-10

Table 16-2

Table 16-2 The four levels of tagging or locking systems

Modifications

to Nominal Level Description HEPs*

A specific number of tags is issued for each job Each tag is numbered or otherwise uniquely identi- fied. A record is kept of each tag, and a record of each tag issued is entered in a suspense sheet that indicates the expected time of return of the tag; this suspense sheet is checked each shift by the shift supervisor. An operator is assigned the job of tagging controller as a primary duty. For restora- tion, the numbers on the removed tags are checked against the item numbers in the records, as a recov- ery factor for errors of omission or selection. OR The number of keys is carefully restricted and under direct control of the shift supervisor. A signout board is used for the keys. Keys in use are tagged out, and each incoming shift supervisor takes an inventory of the keys.

2 Tags are not accounted for individually--the operator may take an unspecified number and use them as re- quired. In such a case, the number of tags in his possession does not provide any cues as to the number of items remaining to be tagged. For restoration, the record keeping does not provide a thorough check- ing for errors of omission or selection. If an operator is assigned as tagging controller, it is a collateral duty, or the position is rotated among operators too frequently for them to maintain ade- quate control tags and records and to retain skill in detecting errors of omission or selection. OR The shift supervisor retains control of the keys and records their issuance but does not use visual aids such as signout boards or tags.

3 Tags are used, but record keeping is inadequate to provide the shift supervisor with positive knowledge of every item of equipment that should be tagged or restored. No tagging controller is assigned. OR Keys are generally available to users without logging requirements.

4 No tagging system exists. OR No locks and keys are used.

Use lower UCBs

Use nominal HEPs

Use upper UCBs

Perform separate analysis

The nominal HEPs are those in the Handbook that relate to tasks involving the application and removal of tags and, unless otherwise specified, are based on Level 2 tagging.

16-11

Tagging Systems

The Level 3 tagging system represents an unusual lack of record keeping and tag control. One characteristic would be the absence of the position of tagging controller. Another would be records that are so general as not to list each item to be changed or restored. A Level 3 tagging system is a

system out of control. We have not observed a Level 3 tagging at any of the plants we have visited, but bince our sample of plants is relatively small, the Level 3 tagging system is listed in Table 16-2 as a possibility. If the analyst assesses this level, he should raise the HEPs related to tasks involving the application and removal of tags. We suggest using the upper UCBs of these HEPs.

Level 4 tagging is assessed when there is no tagging at all. This level frequently occurs during a major shutdown when alignment lists are used instead of tags because of the large number of items involved. In this case, the analyst must carefully analyze the manner in which such lists are

used in conjunction with the change and restoration procedures to be em- ployed. We offer no HEPs or modification factors of nominal HEPs for this "system." We have never observed a Level 4 tagging system used in U.S. plants for valve change or restoration under normal operational conditions,

although we have noted one foreign plant in which no tags were used; re- liance was placed on written procedures only.

Since a system of locks and keys is considered as part of the tagging system, levels of lock and key control are included in the tagging table. In HRA, we do not differentiate between tags per se and locks and keys. If both are used, the analyst should identify the system with the highest level of control (according to Table 16-2) and base the analysis on that level. To be conservative, no extra credit is allowed where both tags and locks are used, even when both are the highest level.

The preceding discussion deals with lock and key systems as part of admini-

strative control. At some plants, key-lock switches are used on critical systems, such as control rod drive, to alert the operator that the manipu- lation of these switches should be considered carefully. However, since these switches must be operated immediately in the case of a stuck rod, the keys are left in the switches at all times. In this case, the keys for the switches and the locks serve no adhinistrative control function. The keys in the lock serve as a form of shape coding but have no other alerting value. At other plants, several designated personnel carry these types of keys on a large key ring.

When used as part of an administrative control system, locks and keys serve with tags to reduce the probability of inappropriate manipulation of equip- ment items. If keys are controlled, items that have been locked into some state for maintenance cannot be released without some administrative in- volvement and perhaps some documentation.

The rating of tagging systems must be based on a thorough evaluation of plant practices, since there is a great deal of inconsistency in the appli- cation of tagging practice even within individual plants. For example, at one plant, we observed a Level 1 tagging system. All tags were numerically

16-12

Tagging Systems

assigned, logged, and accounted for by a dedicated operator, the Tagging

Controller. Before maintenance, the Tagging Controller would tag the ap- propriate items of equipment and retain a tab stub in a special file. The

other stub would be issued to the maintenance organization. After main- tenance, the maintainer would return the stub(s) for the completed job to

the Tagging Controller, who would then go to the appropriate items of equipment and remove the tags for which he had been given stubs. If there

were no other tags on those items of equipment, he would restore them. On returning to his office, the Tagging Controller would make the necessary

entries in his log. He could also tell from the log if any item of equip- ment was still tagged for pending tests or maintenance.

This is clearly an excellent tagging system. However, on occasion, this

excellent system is circumvented. Sometimes the Tagging Controller is too busy to restore some valves, and he removes the tags from a set of valves but leaves the valves in the unrestored positions. He then reports to the

shift supervisor or to the senior control room operator, requesting that the latter assign another operator to complete restoration by using a valve

lineup sheet. The fact that all the tags are removed before valves are restored creates opportunities for serious oversights due to any number of

reasons. Thus, the reliability of the normally excellent tagging control system breaks down when the "dedicated" Tagging Controller is called away from his dedicated function. The tendency for such lapses to occur can be ascertained only by site visits and thorough interviews with the staff.

Another serious problem occurs at the same plant during shutdown, when a

large number of valves are placed in nonnormal positions. A Level 4 tag- ging system, i.e., no tagging, is used at shutdown because of the very

large number of tags that would be required. Realignment of valves is accomplished with valve alignment lists. One person is responsible for the restoration of any given valve--no human redundancy is utilized (Chapter 19). Such reliance on an individual in the case of valve restoration is

typical in NPPs and is not unique to this particular plant.

The analyst may discover that, as illustrated in the examples above, a

plant may use a Level 2 or Level 1 tagging system for most restoration tasks and employ a less effective level for some restoration tasks. In the HRA event tree, this dichotomy is modeled as a branching, showing the esti- mated proportion of tasks for which each level is employed. For example, a plant may employ a Level 1 tagging system for 90% of the restoration opera-

tions and a Level 3 system for the remaining 10%. In the HRA, the higher HEPs associated with the Level 3 tagging system may have a significant

effect on the overall estimated probability of restoration task failure despite the fact that the lower level is used only 10% of the time.

In evaluating a tagging system as a form of administrative control, the

analyst must estimate which level (or levels) of tagging is used, what percentage of the time each level is used, and the likelihood of errors of omission or commission in preparing tags (Chapter 15). These estimates

will affect the estimates of HEPs in carrying out the restoration activi- ties, as discussed in Chapter 19.

16-1 3

Management Systems

Finally, the analyst should remember that the tagging level itself is a good qualitative indicator of the overall goodness of a plant's admini- strative control system. In most cases, evaluating the tagging control (i.e., how it is actually implemented) will enable the analyst to estimate the probability that plant policies in general will be carried out. An inefficient tagging system can serve as a warning to the analyst that the plant may be weak in other areas of administrative control.

Management Systems

By "management system," we mean the utility or plant management hierarchy responsible for ensuring the safe, efficient, and productive running of the plant. While the plant's management may or may not have been involved in the design and construction of the NPP, it is usually involved in the selection and training of personnel for all plant jobs. It is responsible for the establishment and maintenance of the plant's administrative control system.

Some plant managements have an "absentee landlord" relationship with the employees. As long as job performance is acceptable in terms of return on investment, interaction between management and labor is very limited. At other plants, there may be a high level of interaction between management and the employees. Some management hierarchies may breed reactions by the employees that adversely affect their performance. In other plants, the relationship between management and labor may create a work environment that is conducive to good performance.

Although the above and other sociological considerations undoubtedly affect the performance of plant personnel, the quantitative aspects of these ef- fects are currently unknown. No human performance models dealing with these effects have been developed in forms useful for HRA. Therefore, we cannot offer detailed guidelines on how to modify nominal HEPs on the basis of qualitative impressions an analyst can get regarding the attitudes and interactions of management and working personnel in a plant. These socio- logical considerations are too complex .for considerations in a handbook on HRA. The best we can do is state that as analysts we tend to use higher- than-nominal HEPs for tasks related to the quality of administrative con- trol if we judge that the labor/management relationship seems unusually combative, distrustful, or otherwise negative. Because we are conserva- tive, we do not assess lower HEPs for tasks in a plant in which there seems to be an excellent labor/management relationship. In these cases, we pre- fer to judge the quality of administrative control on a more direct basis. It has been our observation that the usual labor/management relationship in NPPs is one of cooperation.

16-14

Ch. 17. Stress Overview; Stress Levels

CHAPTER 17. STRESS

Overview

Under the heading "Stress," Chapter 3 presents a brief discussion of psychological and physiological stress. This chapter extends that discus- sion and presents human error probability (HEP) estimates of human perfor- mance and uncertainty bounds (UCBs) for different levels of stress. The four levels of stress identified in Chapter 3 are redefined as follows: very low task load, optimum task load, heavy task load, and threat stress. A stress model provides guidelines for estimating HEPs for these four levels of stress. The chapter includes a doubling rule for HEPs under greater than optimum levels of stress and includes the large loss-of- coolant (LOCA) curve from WASH-1400. Some studies on which these models are partly based are briefly described.

Objective data on the effects of stress are spotty, and at this time, there is no comprehensive treatment of the effects of stress on performance, although the problem has received much serious attention (Caplan et al, 1975; Cox, 1980; Poulton, 1977; Welford, 1973). In particular, the data are sparse on the performance of technical personnel under stress in an applied setting. In this chapter, we attempt to apply what little is known about stress to the performance of nuclear power plant (NPP) personnel. Some of these extrapolations are based on factual information, whereas others are speculative.

In Chapter 3, we defined a stressor as "any external or internal force that causes bodily or mental tension," indicating that stress is the equivalent of tension. Although this everyday definition has been used widely, for human reliability analysis (HRA) purposes, we prefer the following defini- tion of stress: "a continuum, ranging from a minimal state of arousal to'a feeling of threat to one's well-being, requiring action." In our usage, the threat may be to one's psychological well-being as well as to one's physical well-being. As will be developed later, the threat to one's psychological well-being is of primary concern in HRA and NPP operations.

The Four Levels of Stress

The classical stress curve in Figure 17-1 indicates that performance fol- lows a curvilinear relationship with stress, from very low to extremely high. For HRA, it is adequate to represent the entire continuum of stress by only four levels. The levels we have used throughout the Handbook are as follows:

(1) Very low (insufficient arousal to keep alert) (2) Optimum (the facilitative level)

(3) Moderately high (slightly to moderately disruptive) (4) Extremely high (very disruptive)

17-1

Figure 17-1

TASK LOAD

HIGH

~UJ

Wm; 0L1

W A

LOW

VERY OPTIMUM MODERATELY EXTREMELY LOW HIGH HIGH

STRESS LEVEL

Figure 17-1 Hypothetical relationship between performance and stress (based on Figure III 6-1 from WASH-1400) with task stress and threat stress division.

17-2

Very Low Task Load

For HRA purposes, we consider the moderately high level of stress to be moderately (rather than slightly) disruptive. We use the term high stress to include both moderately high and extremely high levels of stress.

In this chapter, we still use four levels of stress, but we designate them differently for explanatory purposes. The first three levels are attri- buted to the task load, and the fourth level is attributed to feelings of threat. The four levels are as follows:

(1) Very low task load - insufficient arousal to keep alert

(2) Optimum task load - the facilitative level

(3) Heavy task load - approaches or exceeds the human's normal capacity, moderately disruptive

(4) Threat stress - implies emotional reactions, very disruptive

The effects of the first three levels of stress can be approximated by applying modifying factors to the HEPs in the Handbook. The fourth level of stress is qualitatively different from the other three levels--the effects of this level of stress will outweigh other performance shaping factors (PSFs). For this reason, a different set of HEPs is assigned to the threat stress situation.

Note that our discussion of stress is limited to the stress associated with task-loading and threat, such as may develop in an NPP. Physiological stress is not addressed since there is relatively little such stress in an NPP other than that resulting from the occasional requirement to wear protective clothing in radiation environments. Sociological factors, such as management-labor relationships, domestic problems, etc., ordinarily are not addressed in HRAs.

Unless otherwise stated, the HEPs in the Handbook presume an optimum level of task load, or stress. In some cases, the rationale for an estimate will be based on other than the optimum level, e.g., the high HEP estimate in Chapter 19 for the relatively passive task and low arousal of the basic walk-around inspection. In performing an HRA, one must decide whether the stress level for a task is other than optimum, and, if so, how to modify the HEPS.

The rest of this chapter presents dicussions of each level of stress and provides guidance in determining the levels associated with various tasks and conditions in NPPs. A summary set of guidelines for estimating HEPs for various types of tasks as a function of stress level is presented in Table 17-1. This table is expanded in Chapter 18 to include the effects of skill level (Table 18-1).

Very Low Task Load

The characteristic of a very low task load, or stress level, is that there is not enough stimulation to maintain the person at an alert level; his

17-3

Table 17-I

Table 17-1 Modifications of estimated HEPs for the effects

of stress on skilled personnel*

Item Stress Level Modifiers for Nominal HEPs**

(1) Very low x2 (Very low task load)

Optimum (Optimum task load):

(2) Step-by-step xl

(3) Dynamic Xl

Moderately high (Heavy task load):

(4) Step-by-step x2

(5) Dynamic t x5

Extremely high (Threat stress)

t (6) Step-by-step x5

(7 Dynamic tt .25 (EF = 5) This is the Diagnosis actual HEP to use with

dynamic tasks or diagnosis-- this is not a modifier.

A skilled person is one with 6 being assessed. Modifications presented in Table 18-1.

months or more experience in the tasks for lesser experienced personnel are

The nominal HEPs are those in the data tables in Part III and in Chapter 20. Error factors (EFs) are listed in Table 7-2.

Step-by-step tasks are routine, procedurally guided tasks, such as carry- ing out written calibration procedures. Dynamic tasks require a higher degree of man-machine interaction, such as decision-making, keeping track of several functions, controlling several functions, or any combination of these. These requirements are the basis of the distinction between step-by-step tasks and dynamic tasks, which are often involved in re- sponding to an abnormal event.

tt Diagnosis may be carried out under varying degrees of stress, ranging from optimum to extremely high (threat stress). For threat stress, the HEP of .25 is used to estimate performance of an individual. Ordinarily, more than one person will be involved. Tables 12-2 and 12-4 list joint HEPs based on the number of control room personnel presumed to be involved in the diagnosis of an abnormal event for various times after annunciation of the event, and their presumed dependence levels, as presented in the staffing model in Table 18-2.

17-4

Very Low Task Load

state of arousal is below normal. This situation is common to a class of activities known as vigilance tasks, such as the night-lookout on a ship or an inspector visually inspecting large numbers of uniform items, almost all of which are good. A person's effectiveness declines very rapidly under such conditions. In situations where failure of detection may have serious consequences, the person engaged in the vigilance task should be relieved at the end of 30 minutes of continuous watch. The decrease in effective- ness on vigilance tasks may be attributed to a decline in the person's level of arousal, caused by insufficient stimulation.* The lack of suffi- cient stimulation is the condition that we describe as the very low task load level. It is a familiar phenomenon that as people have less to do, they tend to become less alert. As this period of very low arousal per- sists, the level of alertness decreases even further.

For the very low task load level, most people will manifest a minimal level of alertness. Some control room tasks, such as the periodic scanning of unannunciated displays described in Chapter 11, are characterized by such a low level of arousal. For periodic scanning tasks, the HEP estimates have taken this very low level of stress into account. Arousal can occur very quickly, as when an auditory signal sounds. For this reason, a separate set of data was developed for the responses to annunciators (Chapter 11).

Lower error probabilities are estimated for tasks with a specific require- ment to look at a display at a particular time. This reflects the higher level of arousal generated by a requirement to do a specific task rather than merely to look around the control room to see if everything is as it should be, as is done in periodic scanning. In NPPs, tasks involving detection of infrequent signals are the most likely to suffer performance degradation due to low arousal levels.

As a working rule, the HEPs and UCBs pertaining to tasks performed under the optimum level of stress should be multiplied by a factor of 2 if the tasks are performed under conditions of very low arousal. To illustrate this modification, assume the .003 general HEP (Table G-1) for an error of omission or commission for a task performed under optimum stress. Also assume an error factor (EF) of 3, yielding lower and upper UCBs of .001 and .01 (rounded). Now assume that this task is to be performed under the very low stress level. Using the factor of 2, the new HEP is .006 with UCBs of .002 and .02. The EF of 3 (rounded) is unchanged.

Overqualified persons are more likely to experience a very low level of arousal than less qualified persons for whom the tasks would prove inter- esting or challenging. Traffic safety studies reveal that very bright

There is controversy in the psychological literature over the whole con- cept of vigilance and on the applicability of laboratory studies to indus- trial settings (Buckner and McGrath, 1963; Jerison and Pickett, 1963; O'Hanlon and McGrath, 1968; Smith and Lucaccini, 1969; Craig and Colquhoun, 1975; Mackie, 1977). However, Fox (1975) cites studies that show evidence of the vigilance effect in industry.

17-5

Optimum Task Load Heavy Task Load

people make poor cab drivers--the job is too boring and they tend to day- dream, incurring a disproportionate number of accidents. Similarly, tele- phone answering services have found that employee turnover was highest among those who had the highest levels of education--again a matter of inadequate challenge (or arousal) in the task. The industrial literature has many articles on the unfavorable effects of assigning a job to a person whose qualifications are far in excess of the job demands (Swain, 1973). If the analyst judges that the person assigned to some job in an NPP is so overqualified that the job would be dull and uninteresting for him, the very low level of stress, and its modifying factor of 2, should be assigned. Of course, the occasional assignment of a mundane task to a highly qualified person does not fall into this category. However, extended periods of understimulation, such as may occur on the night shifts, can be regarded as the very low stress level for most operators.

At the opposite end of the continuum is the person with limited capacity who finds challenge in a job that most would consider dull. The utiliza- tion of slightly retarded people in simple industrial jobs, such as eleva-,. tor operators, has been very successful.

Optimum Task Load

Unless otherwise stated, the HEPs in this Handbook are based on a level of stress that is optimal for most people. This is the optimum level in Figure 17-1, which is characterized by an active interaction between the person and his environment--talking with others, reading displays, adjust- ing controls, making decisions, etc., at a pace that the person can manage comfortably.

Examples of tasks for which we assumed an optimum stress level are test, maintenance, and calibration; the initial audit of the control room; the reading of an annunciated legend light; and the scheduled reading and recording of information from a display.

Heavy Task Load

A heavy task load is one that requires a person to perform at a pace close to or exceeding his capacity. Most people operating under a heavy task load experience some degradation in their performance. People cope with heavy task loads in different ways. We should note that a peak load situa- tion, such as an emergency, may impose a loading in excess of 100% of normal capacity on the operator. The competent operator copes with this situation by various techniques. First, he briefly increases his maximum capacity by means of physiological changes, such as are evoked by adrenal gland activity. Second, he allocates a sequence to the various subtasks that comprise the overload, thus flattening the peak. Third, he decides to ignore those subtasks that may not be essential. Other stratagems are also employed, depending upon the situation. Regardless of the stratagem employed, the operator is more likely to commit errors when performing under a heavy task load than when performing under an optimal level.

17-6

Heavy Task Load

For the HRA analyst, a question arises regarding the determination of what constitutes a heavy task load in the NPP situation. Clearly, the more skill and experience a person has, the more likely he will be able to handle an emergency without suffering serious degradation of performance. However, the analyst is not expected to determine the skill levels and experience levels of the individual operators. As a conservative solution to this question, we suggest that certain NPP situations shall be classi- fied as imposing a heavy task load on the operators. Examples of these situations are

(1) Single transients that involve shutdown of the reactor and turbine (other than large LOCAs)

(2) Certain tasks during startup and shutdown that must be performed within time constraints

(3) Work performed in a radiation environment, where protective clothing must be worn*

In general, situations that impose time pressure on the performers are

classified as heavy task load situations.

We do not have any data on the relative frequency of errors committed by people performing under a heavy task load, such as the NPP situations above. A study by Brown et al (1969) indicates that decision-making is degraded to a greater extent under heavy task loading than are routine, overlearned skills. Although the study concerned driver behavior, not plant operation, it seems reasonable that the relative degradation of performance between decision-making and routine, skill-based tasks would be similar. On the basis of judgment, we multiply the HEPs for step-by-step, rule-based tasks by a factor of 2 when performed under heavy task loading, and for dynamic tasks requiring considerable interaction between the operator and system indications, we use a multiplier of 5. To illustrate the changes to the HEPs and to the associated EFs, we take the .003 (EF = 3) general HEP for a task done under optimum task loading and modify it for the heavy task load level. For step-by-step tasks, the multiplier of the HEP is 2 (Table 17-1, item 4) and the EF is 5 (Table 7-2, item 5). The new HEP is .006 (.0012 to .03). For dynamic tasks, the multiplier of the HEP is 5 (Table 17-1, item 5) and the EF is 10 (Table 7-2, item 8). The new HEP for this case is .015 (.0015 to .15). (Normally, we round these latter UCBs to .002 and .2.)

Under heavy task loading, the level of arousal is so high that the effect is moderately disruptive. For transients and other unusual events re- quiring quick response, a wider distribution of operator performance is

If special masks, air filters, or self-contained breathing apparatus must be worn in addition to the usual protective coats, gloves, and booties, the nominal HEPs should be multiplied by 5 instead of 2 because of the increased difficulty of working under those conditions.

17-7

Heavy Task Load

expected than for normal plant operations. Some operators will respond to the unusual with a calm, cool approach to solving the problem. Other oper- ators may "freeze," mentally withdraw from the situation, or even panic. The diversity of reaction is a function of many PSFs. Probably the three most important of these are (1) the emotional stability of the operator, (2) his skill level associated with the unusual condition, and (3) the

extent to which displayed information directly supports the actions the operator should take to cope with the situation.

Regarding the first PSF, we cannot predict the emotional stability of the operator. Although there are tests of emotional stability (Matarazzo,

1972), none has been validated in the NPP environment. In general, psycho- logical tests for predicting an individual's performance in a stressful

situation have not proven to be very useful.

The second critical PSF is the operator's skill and experience in respond- ing to the unusual situation. A problem here is that after completion of the initial formal training, the operator seldom receives further practice

in dealing with emergencies except for the customary annual requalification and biannual recertification, which involve dealing with a relatively small sample of emergencies in a dynamic simulator. Thus, for those emergencies that are not exercised in the simulator, his skill level is expected to follow the skill decay curve in Figure 3-6. Our general estimates of HEPs under heavy task loading take into account this lack of practice.

The annual exercises in simulators will refresh skill levels in coping with

the exercised emergencies for a relatively brief time only, unless supple- mented by practice in the plant. If personnel at a plant indeed have such frequent practice that their responses to the stituations could be regarded as "second nature," the HEPs for heavy task loading will not apply since the stress level will be closer to optimum. In judging whether plant per- sonnel possess the necessary skills, one must determine whether they really do receive the frequent practice required. Some NPP personnel we have in- terviewed believe that an operator has a high state of skill in a task if he performed that task at the last training session in a dynamic simulator,

even though that session might have occurred many months ago without fur- ther practice. Obviously, skills must be exercised much more frequently than that to be maintained at a high level. The fourth example in Chapter 21 presents an HRA in which the use of in-plant practice justifies an HEP

estimate of essentially zero for carrying out a well-rehearsed sequence of actions, given that the correct decision has been made to initiate the sequence.

The third critical PSF relating to stress is the human factors engineering of equipment. As long ago as 1975, Dr. Stephen Hanauer, then of the Nuclear Regulatory Commission (NRC), said, "Present designs in NPPs do not make adequate provisions for the limitations of people" (Rogovin and

Frampton, 1980). This report also states, "During the period in which most large nuclear plants have been designed, the nuclear industry has paid

remarkably little attention to one of the best tools available for inte- grating the nuclear operator into the system: the relatively new discipline

17-L

Threat Stress

of 'human factors.'" It continues, "The NRC gives short shrift in the design safety review process to determining how well operators will be able to diagnose abnormal events, based on what they see on their instruments, and respond to them." In the Three Mile Island (TMI) incident, these limitations were manifested by the operators' incorrect and tardy diag- noses, which worsened an already serious situation. It seems that much of the stress experienced by the operator in an emergency may be due to his inability to diagnose the cause of the emergency--the displays do not present all the essential data in an immediately usable form, nor do they help the operator filter the essential from the nonessential data. The inability to "size up the situation" promptly is a stressful experience, as has been observed in simulator exercises. Our conservative estimates of HEPs take these instrumentation limitations into account. (Chapter 11 describes the improvements in control rooms that are .in progress or are planned. The Handbook addresses the current design of most NPPs in the U.S.)

The above three PSFs are interrelated. Military experience indicates that a person with a tendency to panic or freeze in some particular abnormal situation is much less likely to do so after he is thoroughly skilled in diagnosing and responding to that situation and if the information provided him is directly related to the situation. This observation may seem to be obvious, since the abnormal situation will seem commonplace by the time the person has become thoroughly skilled in diagnosing and responding to it. However, until the person really is thoroughly skilled in diagnosing and responding to the situation, the tendency toward inadequate behavior exists. The high HEPs and EFs that we list for performance under stress may seem pessimistic but are justified in view of the inadequate human factors engineering of existing plants and the less-than-optimum training that operators receive in dealing with emergencies.

Threat Stress

Threat stress corresponds to the level that we call "extremely high" in other chapters. It is qualitatively different from the three other levels, in that it involves an emotional component: the feeling of threat to one's well-being. In the case of NPP personnel, the threat to one's well-being is rarely a threat to one's physical well-being--the threat is usually a threat to one's self-esteem or professional status. The threat can engender fear of disciplinary action, of loss of one's job, or of "loss of face."

When any abnormal event occurs, the activities involved in restoring normal status to the plant impose a high level of task load on each operator. This corresponds to the third level of stress. If the system responds to the operator's actions as expected, the operator knows that he has the system under control and ordinarily does not feel any threat. However, if the system does not respond as expected, the operator is likely to feel that he has lost control and will begin to feel threatened. Such a situa- tion could arise in the case of misdiagnosis of an event, in the case of multiple transients, or in any case in which the system does not respond as

17-9

The Data Problem for Threat Stress

expected to attempts to restore control. Misdiagnosis may result from operator error or from incorrect or inadequate presentation of information about plant status, as occurred in the TMI incident. Multiple transients can be deceptive, since the operators do not expect more than one failure at a time and the mixture of indications complicates the diagnosis.

Regardless of the cause, the realization that one does not have the plant under full control is a source of threat to the operating team. The feel- ing of threat involves emotional reactions, with levels of intensity and effects on performance that differ with the individual. In most cases, the emotional reaction has an adverse effect on performance.

In the case of a small or slowly developing LOCA, we would not expect a stress level higher than that of a heavy task load. In some incidents involving small LOCAs, the initial stress level may not be very high, but subsequent events may raise the stress level. In the TMI incident, which involved a small LOCA, some of the operators were considered to have been under extremely high levels of stress at various times (Kemeny, 1979; Rogovin and Frampton, 1980). This is in accord with our premise that the sense of loss of control results in increased stress.

There is great variability in perceived threat--a situation that seems threatening to a novice may be perceived as routine by a more experienced person. For example, a novice driver feels considerable stress when enter- ing a busy freeway, whereas an experienced driver perceives it as a common- place occurrence. The stress curve in Figure 17-1 is intended to represent the relationship of performance to the perceived level of threat, which will vary with an individual's knowledge, experience, preparation, person- ality, and many other factors. The curve represents average performance. The UCBs assigned to the HEPs at different levels of stress are intended to include the middle 90% of the industrial population in response to stress, as well as the usual sources of uncertainty described in Chapter 7.

The Data Problem for Threat Stress

Most of the experimentation on threat stress deals with artificial tasks in situations in which the experimental subjects clearly realize that nothing catastrophic will result from any ineptitude on their parts, and their emotional reactions are minimal. See Harris et al (1956), Klier and Linskey (1960), and Robbins et al (1961) for literature reviews.

Another body of literature deals with the performance of military personnel under combat stress (Grinker and Spiegel, 1963; Marshall, 1961). The latter reference reports results of interviews with World War II combat soldiers in which it was found that only 15% of the men interviewed had actually fired at enemy positions or personnel during an entire engagement (p 54). This suggests that most of the men were ineffective, assuming that the remaining 85% had opportunity to fire their weapons. In the best com- panies, no more than 25% of the men used their weapons even though most of the action occurred under conditions in which it would have been possible for at least 80% of the men to fire and in which nearly all personnel were

17-10

The Data Problem for Threat Stress

at some time operating within firing distance of the enemy. This suggests about 30% net effectiveness for the best companies (i.e., .25 ` .80).

Comparisons of extremely stressful conditions in an NPP with the stress of combat are obviously open to error. First, in combat, the participant's life is literally and obviously at stake, and he perceives this as fact. Such is not the case in most NPP emergencies. Second, death in combat is seen and is not rare; close calls are numerous. In NPPs, life-threatening emergencies are rare. Third, much of the evidence on combat stress includes the effects of combat fatigue (i.e., the cumulative effects of unrelieved stress over a long period of time) and therefore does not apply to an industrial situation. Fourth, combat training emphasizes coping with emergencies; this is a major purpose of that training. In NPP training, responding to emergencies is only a small part of the training, since nearly all of the tasks to be performed are routine and are performed in an optimal stress situation.

Despite the potential inaccuracy in generalizing the results of military studies to the behavior of NPP personnel, we will consider two such studies that are classics in the applied area of stress. In one series of studies, performance of soldiers was measured under conditions in which they did not realize that the experimental stressors were artificial. They really thought that either their own lives were in danger or that they had caused others' lives to be endangered (Berkun et al, 1962; Berkun, 1964). In the other study, critical incidents were collected from U.S. Air Force aircrews who survived in-flight emergencies (Ronan, 1953).

Because our estimates of HEPs under extremely high levels of stress are based on these two studies, they are described briefly. The major problem in estimating the performance of NPP personnel under threat stress is that very few of them have been subjected to this level of stress. Moreover, the kind of accident that is generally considered to represent the highest level of stress in an NPP, a large LOCA, has never occurred in over 1,800 reactor-years of operation of commercial and production reactors in the noncommunist world and is unlikely to occur. The best we can do is genera- lize from other types of emergency situations that may be only marginally related to the performance of NPP personnel under very high stress.

The Ronan Study

In the Ronan study, aircrews surviving in-flight emergencies in the B-50 heavy bomber were interviewed, and critical incidents were noted. As defined by Flanagan (1954), The critical incident technique (CIT) ". consists of a set of procedures for collecting direct observations of human behavior in such a way as to facilitate their potential usefulness in solving practical problems and developing broad pscyhological principles." Fivars and Gosnell (1966) define an incident as "any observable bit of human behavior sufficiently complete in itself to permit inferences to be made about the person performing the act." For an incident to be critical, "an incident must make a significant difference in the outcome of the behaviors; it must contribute either positively [a positive incident] or

17-11

The Data Problem for Threat Stress

negatively [a negative incident] to the accomplishment of the aim of the activity." By convention, a positive incident is called a "blue" incident, and a negative incident is called a "red" incident. The critical incident technique was used by U.S. Army Air Corps investigators in World War II and has been used extensively in postwar military investigations and in safety analyses (O'Shell and Bird, 1969).

In the Ronan study, aircrews from several Air Force bases were interviewed, using a carefully structured interview with assurance that all reports would be strictly confidential, i.e., no word as to who said what would get back to the interviewee's superior officer. Dr. Ronan stated that there was no apparent reluctance on the part of the interviewees to describe in detail the positive and negative critical behaviors they observed or engaged in. The data we use are from 153 aircraft commanders (ACs). These ACs were highly trained, with an average of 2,971 flying hours. Twenty- nine categories of emergencies were described, ranging from very serious emergencies, such as engine loss on takeoff, to less serious problems, such as an engine with rough operation. We hav6 excluded one category, crew coordination problems, since we wish to base the derived HEPs entirely on AC performance. The ACs submitted a total of 2,450 critical incidents over the remaining 28 categories. Of these, 360, or 15% of the total, were red incidents. Thus, of the critical actions taken during in-flight emergen- cies, 15% were ineffective in that the situation was not improved or was made worse as a result of them. This percentage probably underestimates all the errors made by ACs, since no data could be collected from nonsurvivors.

In WASH-1400, we equated the percentage of red incidents with the error probability for tasks performed by aircraft commanders under the stress of inflight emergency conditions. Obviously, this is not the same kind of error probability defined' in Chapter 2 (the number of incorrect responses divided by the number of opportunites for response). In the Ronan study, the denominator is not the number of opportunities, but the total number of incidents reported, both red and blue. Therefore, there is some unknown error in using the 15% figure as an error probability, but it is the best estimate available from such real-life situations.

The Berkun Studies

In the Berkun studies, raw recruits and experienced soldiers were placed in several elaborately simulated emergency situations. Data from the few who perceived the deception were excluded from the results reported. Self- reports showed that those subjected to the experimental (stressful) condi- tions reported they felt "timid," "unsteady," "nervous," "worried," or "unsafe;" whereas, the control subjects, not subjected to stressful condi- tions, reported that the conditions "didn't bother me," or that they them- selves were "indifferent" to the situation. Interview results indicated that the experimental subjects did indeed believe that their safety was endangered or that their actions had imperiled the safety of others.

In one study, subjects were passengers on an apparently stricken aircraft that was going to "ditch" in the ocean. Through a plausible fabrication of

17-12

The Data Problem for Threat Stress

events, the subjects were led to fill in two forms shortly before "ditch- ing." Only 5 of the 20 experimental subjects saw through the deception, attesting to the realism of the study. The performance scores on filling in the forms for the remaining 15 were significantly lower than for a flying control group, showing a i0% decrement for the first form adminis- tered and a 36% decrement for the second form. In terms of error proba- bilities on the first form, the experimental subjects had an HEP of .30, compared with .22 for the control subjects, an increase of 36%. On the second form, the HEPs for the experimental and control subjects were .59 and .37, a 59% increase for the experimental groups.

In two other test situations ("artillery" and "demolitions"), the subject was led to believe that he was in immediate danger of losing his life or that he had caused serious injury to someone else. The performance task was that of repairing a "defective" radio to summon help. In both of these situations, the experimental subjects performed at a level statistically inferior to the performance of the control subjects. In the situation in which their own lives were threatened (known as the "artillery" study), the decrement was 33%, and in the situation in which someone else had been injured, the decrement was 18%. Interestingly, in the first group, the 33% decrement was due largely to the low scores of one-third of the subjects who panicked and fled. The two-thirds who did not panic performed less well than the control subjects, but only at a borderline level of signifi- cance. This finding points up the great range of individual differences in reactions to threatening situations. The men who panicked were affected by threat to the extent that they were incapable of functioning. Of those who remained on duty, we can only speculate as to whether they did not perceive the situation to be as threatening, whether they perceived the same degree of danger but believed the repair of the radio to be the best way to cope with the threat, or whether they had a stronger sense of commitment to performing their duties.

Derivation of the Estimated HEPs under Extremely High Stress on NPP Tasks

In WASH-1400, we used data from the Ronan study and the Berkun artillery study to establish boundaries for the error probabilities of NPP personnel in a high-stress situation such as a large LOCA. It is not likely that NPP personnel would react as calmly or perform as reliably as Air Force air- craft commanders, considering the extensive practice they undergo in simu- lated aircraft emergencies. Generally, there is little if any onsite practice in simulated emergencies for operators of nuclear power plants. Hence, the error probability of plant operators was estimated to be higher than the .15 error probabilities observed by Ronan.

On the other hand, NPP operators have been extensively trained in NPP operations. A newly assigned operator has learned to recognize the poten- tial for accidents and to be prepared to cope with them, although not at a level comparable with that of the Air Force pilots. Hence, we judged that the operators should be better able to cope with emergencies than would raw Army recruits, and their error probability should be lower than the .33 probability of completely inadequate behavior of the artillery subjects. Therefore, we assumed an estimate of .25 for the average error probability

17-13

The Stress PSF and the Large LOCA

for NPP personnel in a high-stress situation such as a large LOCA. This estimate is based on the further assumption that the perceived threat in a large LOCA situation is comparable to the perceived threat in the Ronan and Berkun studies; whereas, in fact, it might be lower.

It is regrettable that there are no better data available to develop HEPs

for the high-stress condition in NPPs. Until better data are collected, this kind of rationale is all we have.

Using the value of .25 as the estimated HEP of trained NPP personnel under threat stress, it is necessary to assign UCBs for use in some reliability calculations. Because of the greater variability in performance that may be expected under threat stress, a large error factor is appropriate. We assign an EF of 5 to the HEP of .25, which yields a lower UCB of .05 and an upper UCB of 1.0. The EF of 5 is selected rather than a higher EF because we want to select a conservative (i.e., higher) lower bound for tasks performed under threat stress.

The above HEP of .25 does not apply to simple skill-based tasks, such as manipulation of a control. Such tasks are so highly practiced that degra- dation will be considerably less, even under threat stress. However, all

tasks requiring some mental involvement will be degraded, even the rela- tively simple step-by-step tasks of following written procedures. For conservatism, we apply a multiplier of 5 to the nominal HEPs listed in the appropriate data tables in Part III for step-by-step tasks performed under threat stress. An EF of 5 is assigned to the adjusted HEPs. Tasks in- volving dynamic interaction with the system, or requiring cognitive func- tioning, such as diagnosis, are the ones most susceptible to degradation, and these are the ones to which the HEP of .25 applies. Fortunately, these

latter types of tasks are the ones that usually involve several people, so that the benefits of human redundancy are available.

The Stress PSF and the Large LOCA

In WASH-1400, we used a large LOCA as the example of a situation resulting

in an extremely high stress level for the operators, and we estimated the HEPs for an operator from the first moments of a large LOCA until the operating crew could establish control of the situation. Figure 17-2 shows our estimates as a function of time after the onset of the accident. Obviously, the curve is speculative since a large LOCA has never occurred. Our rationale for the curve has not changed since WASH-1400 and was then explained as follows:

"Following a LOCA, human reliability would be low, not only because of the stress involved, but also because of a probable incredulity response. Among the operating personnel the proba- bility of occurrence of a large LOCA is believed to be so low

that, for some moments, a potential response would likely be to disbelieve panel indications. Under such conditions~it is estimated that no action at all might be taken for at least one minute and that if any action is taken it would likely be

inappropriate.

17-14

Figure 17-2

ILJ

1.00

.90

.70

.50t

HEPs

1.0 -v- 1st min .9 AT 5 min .1 AT 30 min .01 AT 2 hr

IF THREAT STRESS PERSISTS, LEVEL OFF AT .25 AT ABOUT 25 min

IF THREAT STRESS PERSISTS

AUTOMATIC SYSTEMS WORKING OK

.25

.10

.01 1510 30 60 120

TIME (minutes after large LOCA)

Figure 17-2 Estimated human performance after a large LOCA.

17-15

The Doubling Rule

"With regard to the performance curve, in the study the general

error [probability] was assessed to be .9 five minutes after a large LOCA, to .1 after thirty minutes, and to .01 after several hours. It is estimated that by seven days after a large LOCA there would be a complete recovery to a normal, steady-state condition and that normal error [probabilities] for individual behavior would apply." (WASH-1400, p 111-61)

The solid line in Figure 17-2 indicates the estimated HEPs that apply if

the automatic recovery systems function normally to mitigate the effects of the accident. Otherwise, threat stress is assumed, as shown by the dashed line, and the error probability will not decrease below the value of .25 as long as the threat stress conditions persist. The wide UCBs around the .25 estimate (.05 to 1.0) allow for some individuals to perform well and for others "to be a part of the problem."

The human performance curve in Figure 17-2 is based on the assumptions used

in WASH-1400 and applies to a lone operator under extremely high stress (threat stress). In a realistic situation, more than one operator will be present, and the joint HEPs for the several operators would be calculated

in the manner described in Chapter 12 in the section "Nominal Model for Diagnosis of Abnormal Events." That section indicated how we modified the basic HEP for an individual reactor operator by considering the increased reliability to be expected from the presence of other personnel and the reduction in the theoretical maximum reliability because of the dependence among them. The same approach would be used in deriving a joint HEP from Figure 17-2. For example, at 30 minutes, the HEP of .1 for a lone operator would be modified by assuming a conditional HEP of .55 for a second oper- ator (with high dependence) and a conditional HEP of .15 for both the shift supervisor and the shift technical advisor (both with low dependence). (The values for the conditional HEPs can be calculated by using Equations 10-15 and 10-17 from Table 10-2.) Thus, the joint HEP for the four people on duty would be

.1 x .55 x .15 x .15 = .0012 = .001

Dependence among people varies with stress and with differences in author-

ity (actual or perceived). Dependence between a junior-level operator and a senior-level operator will normally be greater than the dependence be- tween a shift supervisor and a senior operator. In general, dependence among all personnel increases as stress levels increase.

The Doubling Rule

There is an important corollary to the performance curve in Figure 17-2 for

the condition of heavy task loading. This corollary applies when the time available to take corrective action is severely limited. In our early work on human reliability (Swain, 1963b), we developed a theory of behavior under time stress. This theory holds that given that an error has been made and recognized as such, or that corrective action has failed to have its intended effect, the error probability for the following attempted

17-,6

Concluding Comments

corrective action doubles. Thus, if one working under severe time stress attempts a task that has an HEP of .1 and fails on the first attempt, it takes only three more unsuccessful attempts to reach the limiting case of an error probability of 1 .0 on the following attempt. This limiting con- dition corresponds to the complete disorganization of the individual, as described in some of the references cited earlier (Berkun, 1964; Marshall, 1961; Grinker and Spiegel, 1963).

Although not developed in the Swain reference, it may be seen that the results of the initial error, under heavy task loading, induce a feeling of loss of control, which engenders the next level of stress, threat stress. When any action fails to restore control, the sense of threat increases. Experimental studies of Navy aircraft pilots landing on carrier decks indicate a stress curve with repeated attempts after failure that closely matches the doubling rule (Siegel and Wolf, 1969). The doubling rule is a convenient estimate for the performance of an individual working alone. In the case of an NPP, we are dealing with teams rather than individuals. However, our models for team performance take into account the dependence among team members, and the rationale of the doubling rule is not violated if applied to team performance.

A variation of the doubling rule is based on the hypothesis that if an error is made when working in optimum conditions, the HEP on the subsequent attempt would be reduced by a factor of 2 because the person would be more careful after an error if there is ample time to exercise all due caution. This is an interesting speculation, but with our present limited knowledge of all possible modes of human behavior, we cannot justify adoption of the hypothesis, and we do not use it in HRA.

Concluding Comments

The above discussion of stressful situations and their effects on perfor- mance indicates the difficulties of quantifying this PSF and the uncertain- ties associated with such attempts. The entire area of stress is a very broad one, and this chapter addressed a very narrow aspect of the area: performance of skilled personnel in the operation of an NPP. Our approach constitutes a deliberate simplification of the topic for the purpose of rendering the PSF of stress manageable to analysts conducting HRAs. The rationale we have presented is probably suitable for other process indus- tries as well as NPPs, and we feel that the model provides reasonable estimates of the effects of stress on performance.

17-17

Ch. 18. Staffing and Experience Levels Overview; Staffing

CHAPTER 18. STAFFING AND EXPERIENCE LEVELS

Overview

This chapter presents assumptions to be used in probabilistic risk assess- ment (PRA) about staffing in the control room (CR) for estimating the joint

human error probabilities (HEPs) for tasks carried out after the occurrence of an abnormal event. The postevent staffing model uses the minimum number of persons specified by the Nuclear Regulatory Commission (NRC) as a start- ing point and presents guidelines for assessing the number of CR personnel at various times into the event and for assessing the dependence levels among them. The chapter also presents a simple model for assessing the effects of two experience levels (novices and skilled personnel) on estimated HEPs as a function of type of task and stress level.

Staffing

All PRAs are based on certain assumptions regarding staffing of the plant. In our HRAs, we address staffing in the CR only. A distinction is made between the CR proper and the CR area. The CR area includes the actual control panels and auxiliary equipment, the shift supervisor's office, a rest room (usually), a relaxation area for lunch breaks, etc., and any other facilities that are frequently used by the CR personnel. The entire area is secured via controlled access. The CR proper is the delineated area in which the central panels with operating controls, displays, etc., are located, which are used by the operators to carry out the actual operation of the plant.

Guidance for the minimum number and types of CR personnel that must be on duty during routine operation of a nuclear power plant (NPP) was outlined in NUREG-0737, Clarification of TMI Action Plan Requirements, November 1980, and is summarized below. The requirements include a position that did not exist when the draft version of the Handbook was published in October 1980--the position of Shift Technical Advisor (STA). The STA position is to be held by a person with expertise in the area of opera- tional safety to reduce the probability of abnormal or emergency conditions and to mitigage their consequences if they should occur (NUREG-0737, p C-12). The STA is not required to be a licensed reactor operator, and most are not. An STA must be available on all shifts. NUREG-0737 also lists preliminary training and academic requirements for the STA, based on recommendations prepared by the Institute for Nuclear Power Operations (INPO).

The manning requirements for CR personnel, exclusive of the STA, are listed below for the case in which all reactors in a reactor complex are in the power-generating mode:

a. One unit, one CR

1 shift supervisor (SS); must be a senior reactor operator (SRO) 1 SRO 2 reactor operators (RO) 2 auxiliary operators (AO)

18-1

Staffing

b. Two units, one CR 1 Ss (SRO) 1 SRO 3 RO 3 AO

c. Two units, two CRs I SS (SRO) 2 SRO 4 RO 4 AO

d. Three units, two CRs 1 SS (SRO)- 2 SRO 5 RO 5 AO

Except for the AOs, all the, above personnel are licensed reactor operators. The AOs are usually trainees who are preparing to qualify for reactor operator licenses.

Although it is not necessary for all of the listed personnel to be in the CR area at all times, they must be on duty. In the CR proper, at all times, there must be at least one licensed operator for each operating unit to scan the control panels and operate the controls. There must also be at least one SRO within the CR area at all times. The required SRO may be the SS, and the required operator at the controls may be the nonsupervisory SRO. Thus, the various team members can relieve each other as required. The AOs will often be away from the CR area, performing different plant functions. NUREG-0737 does not stipulate that the STA be on site at all times. At some plants, a different STA is assigned to each shift. At other plants, an STA is on duty for 24 hours, during which time he may sleep at the site, somethifig like a military Duty Officer. At all U.S. plants we have visited, an STA is always on site; this seems to be the usual practice.

For PRA purposes, the assumptions regarding availability of personnel will obviously vary with the number of operating units and control rooms. As an example, assume a plant like the one in case b, above, with two units operating and the control panels for both units in one CR. Each unit's panels must be manned continuously--usually an RO would be operating each unit. In addition, an SRO must be in the CR area at all times. These three represent the minimum number of licensed operators that would have to be in the CR area at all times when both units are in the power-generating mode. Since the guidelines require five licensed operators to be on duty, two of them with SRO ratings, the minimum requirements for duty in the CR can be met, even with allowances for relief periods and any other require- ments for temporary departures from the CR.

0 18-2

Experience Level

Licensed Reactor Operators

The skill levels of qualified NPP personnel range from acceptable to supe- rior. (Anyone whose skills were less than acceptable would not be quali- fied.) For HRA purposes, we designate the skill continuum by two experi- ence levels: skilled and novice. The distinction is used in our models of dependence among licensed operators and in their performance in different types of tasks and in their performance under different levels of stress. It is understood that performance skills continue to improve with experi- ence, well beyond the novice level, but we do not have any data on the extent to which additional experience affects dependence, response to stress, etc. For this reason, we address only the two levels on the skill continuum.

We define the novice as a person with less than 6 months on the job for which he has been licensed (in the case of ROs), or otherwise qualified (in the case of AOs, maintainers, and technicians). In view of the require- ments for becoming a licensed RO,* it might seem that there should be no difference in performance between a skilled operator and a novice operator. Our distinction is based on the following rationale: the years of training prior to licensing are certainly adequate for acquisition of the knowledge required to operate the plant; however, about 6 months of unsupervised experience is required for a person to develop the confidence to exercise his decision-making authority fully when he is responsible for the resolu- tion of some unusual problem that might arise.

Over a 6-month interval, the new operator will experience one or more transients and will develop a "feel" for the plant that cannot be acquired in trainee status. In most industrial settings, 6 months is accepted as the time required for a person to achieve full performance capability after a promotion or reassignment. Thus, the 6-month interval for distinguishing between novice and skilled operators is based on observation in comparable industrial settings and on the opinions of SROs.

Under current staffing requirements for CRs, the only operators that might be novices would be the ROs operating the controls, since the SS and the SRO are highly skilled people. Since at least one SRO must be in the control room at all times, the concerns about novices are less pressing today than in 1976, when Seminara et al wrote '. . . some shifts are manned entirely with novices, with only several months of experience." However,

Minimum regulatory requirements stipulate 2 years of power plant experi- ence, with 6 months at the site where the operator is to be licensed. Actually, the operators will have much more than the minimum requirements by the time they apply for their licenses. Usually, they will have about 2 years on site, plus previous experience, such as in the Military or at fossil-fuel plants.

18-3

CR Personnel - Abnormal Situation

despite the additional competence available by virtue of the SRO on duty, if an analyst knows that a CR will be manned by novice-level operators, or that certain maintenance tasks will be performed by novices, he should take this into account when performing an HRA. We judge that the HEPs for novices will be higher than those of an experienced operator for nonroutine tasks or when the stress level is higher than optimal. Table 18-1 presents modifying factors to apply to the nominal HEPs in Part III for novices and skilled personnel under different levels of stress and for different types of tasks. For uncertainty bounds, see Table 7-2.

Under very low levels of stress, performance is degraded because of insuf- ficient stimulation from the task. It has been suggested that under such conditions, the skilled person may actually suffer more degradation in performance than the novice because a greater number of tasks have become a simple routine for the experienced person. Although this argument seems reasonable, we have no data to support it, so we assume equal loss of performance for both classes of personnel.

Under optimum stress, the novice is assumed to be twice as error-likely as the skilled person when performing tasks involving dynamic interaction (which includes decision-making). For step-by-step tasks, performance is assumed to be the same. For the two levels of high stress, it is assumed that the HEPs assigned to skilled persons should be doubled for novices. The modifying factors in Table 18-1 are based on our judgment and may be changed as data become available.

Other Personnel

For AOs, maintainers, and technicians, the same factor of 2 is used between the estimated HEPs of novices and skilled personnel. This factor of 2 may be an underestimate for maintenance novices. Based on observations and interviews by Seminara, Parsons et al (1979) and by Brune and Weinstein (1980), it seems that the training of maintenance personnel is not as complete as that of AOs and technicians. Most of the maintainer's training of NPP specifics is on the job, with considerable dependence on his pre- vious background. Therefore, it is possible that the difference between a novice and a skilled maintainer may be greater than that reflected by the factor of 2. However, with no objective data, the factor of 2 will be used as an interim figure.

CR Personnel Interactions in an Abnormal Situation

In WASH-1400, we used an HEP of .1 for the tasks involved in changing over from the injection to the recirculation mode after a large loss-of-coolant accident (LOCA) (see the second example in Chapter 21). The .1 was pre- mised on an estimate that these procedures would be performed 30 minutes after the LOCA, by which time the average HEP would be down to .1. We further assumed that when these tasks were attempted, there would be at least three qualified operators in the CR and that their joint HEP would be approximately 10 3. This 10-3 assumes zero dependence (ZD) among the three operators, i.e., (10-1)3. Our rationale at that time was that although the

0 18-4

Table 18-1

Table 18-1 Modifications of estimated HEPs for the effects of stress and experience levels

Modifiers for Nominal HEPs* Stress Level Skilled** Novice**

I teem (a) (b)

(1) Very low x2 x2 (Very low task load)

Optimum (Optimum task load):

(2) Step-by-stept xl xl

(3) Dynamic t xl x2

Moderately high (Heavy task load):

(4) Step-by-step t x2 x4

(5) Dynamic t x5 xl0

Extremely high (Threat stress)

(6) Step-by-stept x5 xl0

(7) Dynamic ttý .25 (EF = 5) .50 (EF = 5) Diagnosis These are the actual HEPs to use

with dynamic tasks or diagnosis-- they are NOT modifiers.

The nominal HEPs are those in the data tables in Part III and in Chapter 20. Error factors (EFs) are listed in Table 7-2.

A skilled person is one with 6 months or more experience in the tasks being assessed. A novice is one with less than 6 months or more experi- ence. Both levels have the required licensing or certificates.

tStep-by-step tasks are routine, procedurally guided tasks, such as carry-

ing out written calibration procedures. Dynamic tasks require a higher degree of man-machine interaction, such as decision-making, keeping track of several functions, controlling several functions, or any combination of these. These requirements are the basis of the distinction between step-by-step tasks and dynamic tasks, which are often involved in re- sponding to an abnormal event.

ttDiagnosis may be carried out under varying degrees of stress, ranging

from optimum to extremely high (threat stress). For threat stress, the HEP of .25 is used to estimate performance of an individual. Ordinarily, more than one person will be involved. Tables 12-2 and 12-4 list joipt HEPs based on the number of control room personnel presumed to be involved in the diagnosis of an abnormal event for various times after annunciation of the event, and their presumed dependence levels, as presented in the staffing model in Table 18-2.

1 P-5

CR Personnel - Abnormal Situation

assumption of ZD might be inappropriate, the 10-1 estimate was gross enough that cubing this HEP for three people would not materially affect the results of the analysis. We now believe that this rationale may be too optimistic for HRAs in general, and we present guidelines below for the inclusion of dependence in such analyses.

In writing about the Three Mile Island (TMI) incident, Sheridan (1980) states:

"Nuclear plant operators work in teams, based on the premise that two or more heads are better than one. But there is a great deal of interaction among team members, some of it subtle and unspoken. Such interpersonal communication is little under- stood but assuredly does affect the reliability of human perfor- mance. For example, operators unintentionally could reinforce one another's misimpressions, making the team less reliable than a single operator who would be more likely to think a matter through carefully. This means that human error rates for indi- viduals may differ from those for teams."

The Rogovin Report indicates that it took some time for the operating personnel at TMI Unit 2 to become organized to the extent that they were stationed at strategic places (Rogovin and Frampton, 1980).

Present-day staffing requirements, which involve an SRO in the CR at all times, should reduce the confusion described in the Rogovin report. There have been no episodes such as the TMI incident since that time, so the advantages of the new requirements have not been fully tested to date.

Until we gather a reasonable amount of observational data on the inter- actions of operators in abnormal situations, we offer the following guide- lines, based on the assumption of the minimum number of operators specified by NUREG-0737.

For the first minute after recognition of an abnormal event, assume that the RO on duty is responding to the situation alone. Although the event is announced by compelling signals that will attract the attention of the assigned SRO, such compelling signals are not uncommon, and most of them are handled routinely by the RO, so we do not expect the SRO to become involved immediately. Furthermore, he may be in a location within the CR area where he is not in a position to see the operating panels; also, for some period of time, the RO may not think it necessary to call for help.

At 1 minute into the event, we assume that the SRO joins the RO at the panels. A full minute is adequate time for the SRO to determine that the RO may need assistance, or for the RO to call upon the SRO for help. Note that although we refer to this person as "the SRO," he may be either the assigned SRO or the SS. We assume high dependence (HD) between the SRO and the RO. This means that if we were performing a detailed, individual analysis, we would estimate an HEP for the RO and then use the dependence equations or tables in Chapter 10 to determine a conditional HEP for the

18-6

CR Personnel - Abnormal Situation

SRO. An exception to this would be the case in which we know the RO to be a novice. Although Table 18-1 indicates the correction factor for novices in an abnormal event, we would disregard that factor for dynamic tasks or diagnosis on the assumption that the more experienced operator would assume the dominant role. For most HRAs, we assume skilled ROs, however, and assign the conditional HEPs from Chapter 10 to the supporting personnel, even though they are more experienced. We do this because the RO has the most immediate knowledge of the plant status preceding the abnormal event, and of course he knows exactly what remedial action he took during the

first minute after the initiation ofl the event.

At 5 minutes into the event, we assume that the SS has joined the RO and

the SRO. We assume low dependence (LD) to moderate dependence (MD) between the SS and the other operators.

At 15 minutes into the event, we assume that the STA has joined the others. We assume low to moderate dependence between the STA and the others for

diagnosis, and high to complete dependence (CD) between the STA and the

others for details of operations. The high levels of dependence for details of operations are assumed because the STA is not supposed to be involved in the actual mechanics of running the plant.

At any time in the course of plant operation, there may be one or more AOs in the CR. For some abnormal events, AOs will have to perform PRA-signifi-

cant duties during the first few minutes after the annunciation of these events. The table allows credit for one or more AOs in the CR 5 minutes into an annunciated abnormal event. The table does not list estimated

levels of dependence for AOs, since usually they would be acting on instructions from licensed operators. Estimates of dependence must be made on a plant- and situation-specific basis.

Table 18-2 takes us only 15 minutes into an abnormal event, because by that

time there should be four people available to cope with the situation. At least two of the four would be SROs (the SS and the assigned SRO), and the STA will be specially trained in mitigating the consequences of abnormal events. The lowest level of experience that would be found among the

licensed operators would be an RO with less than 6 months on the job since receiving his license.

The four people listed above are the minimum that would be available to

cope with an abnormal event, since there would be another RO available. However, he could be performing duties elsewhere in the plant, so we are assuming only the initial group of four personnel for PRA purposes. We recognize that additional qualified-people may become available with time, but we do not know how to assess their influence independent of plant specifics and the characteristics of the event. Their presence may or may not help cope with the event. The Rogovin report describes instances in the TMI incident in which incorrect diagnoses were still being made more than 2 hours into the event even though several additional qualified per- sonnel were present. For PRAs we have performed, we have given credit only for the above four persons.

16-7

Table 18-2

Table 18-2 Number of reactor operators and advisors available to cope with an abnormal event and their related levels of dependence: assumptions for PRA*

Time after recognition** of an abnormal

event

Operators or advisors handling reactr unit affected

Dependence levels with

otherst

Item (a) (b)

(1)

(2)

(3)

0 to 1 minute

at 1 minute

at 5 minutes

on-duty RO

on-duty RO, SRO (assigned SRO or shift supervisor, an SRO)

on-duty RO, assigned SRO, shift supervisor

1 or more AOs

on-duty RO, assigned SRO, shift supervisor

- - - high with RO

- - - - high with RO - - - - low to moderate

with other operators

- - - - high with RO - - - - low to moderate

with other operators

(4) at 15 minutes

shift technical advisor-

1 or more AOs*

- - low to moderate with

others for diagnosis & major events; high to complete for detailed operations

These assumptions are nominal and can be modified for plant- and situa- tion-specific conditions.

For PRA, "recognition" is usually defined as the response to a compelling signal, such as the alarming of one or more annunciators.

No credit is given for additional operators or advisors (see text). 1tThis column indicates the dependence between each additional person and

those already on station. The levels of dependence are assumed to remain constant with time and may be modified in a plant-specific analysis.

*Availability of other AOs after 5 minutes and related levels of depen-

dence should be estimated on a plant- and situation-specific basis.

18-8

CR Personnel - Abnormal Situation

The above guidelines present a more conservative approach than we used in WASH-1400, but their use will not always increase the results of an HRA. For example, the probability of correctly shifting from the injection to the recirculation mode was recalculated, using the above guidelines, and compared with the probability calculated in WASH-1400; the results were a factor of 33 lower (see the second example in Chapter 21).

Although the staffing guidelines and associated dependence effects are more conservative than the approach we used in WASH-1400, they are also more realistic than the nominal diagnosis model presented in Chapter 12. Chapter 12 presents a comparison of joint HEPs from the nominal model with joint HEPs calculated using the dependence considerations outlined in this chapter. The comparison indicates that the assumptions of dependence among the operator personnel would have to be increased substantially to arrive at a joint HEP comparable to the joint HEP of the nominal model for the initial 10 minutes following an abnormal event. This is in accord with the philosophy of the nominal diagnosis model described in Chapter 12, which is to use estimates that are expected to be conservative. For those cases in which more realistic estimates of HEPs are desired, the guidelines in this chapter can be used. If this approach is followed, the analyst will have to derive estimates of diagnostic HEPs and modify them for the dependence among the personnel involved.

18-9

Ch. 19. Recovery Factors Overview

CHAPTER 19. RECOVERY FACTORS

Overview

A recovery factor is any element of a nuclear power plant (NPP) system that acts to prevent deviant conditions from producing unwanted effects. It can be anything that prevents a deviant condition, mitigates its effects, or provides the opportunity for detecting it so that its effects can be avoided or limited. Deviant conditions can be caused by human error, e.g., after maintenance, a safety-related valve is left in the wrong position. Deviant conditions can also arise from some equipment failure, e.g., a primary coolant pipe bursts due to some inherent weakness. In many cases, deviant conditions result from some combination of human error and equip- ment failure. If a human error is made and is not detected and corrected, it is designated as an unrecovered error. An error may remain unrecovered either because there were no recovery factors or because the recovery factors failed. If recovery factors resulted in detection and correction of the error in time to prevent undesirable effects, the error is desig- nated as a recovered error.

Most recovery factors in NPPs are based on information provided the opera- tions personne! via displays or other visible indications of equipment conditions, via direct observation of another's work, or via subsequent checking of that person's work. The term human redundancy is used to denote the recovery factor resulting from the use of one person to check another's work. In the Handbook, we designate the former as a checker. The term inspection is used for the recovery factor when someone canvasses items of equipment to ascertain their status. In the Handbook, we desig- nate this person, often an auxiliary reactor operator, as an inspector. He is looking for deviant equipment states, not directly to verify that some- one else's work is satisfactory.

In the control room, there are many types of recovery opportunities with greatly varying levels of human reliability. Annunciations of deviant conditions are the most attention-getting but may occur too late to prevent serious problems or may occur in such numbers that they may become over- whelming, as discussed in Chapter 11. Deviant conditions, or trends towards deviant conditions, are often displayed on various meters, charts, indicator lamps, and so on, which are detectable by control room personnel. However, without specific direction to a given display, the probability of an operator noticing a deviant indication prior to its annunciation is not high.

Outside the control room, deviant conditions can be detected by periodic inspections or by checking on someone's performance. As is the case in the control room, the human reliability associated with inspections and checks outside the control room varies greatly--generally as a direct function of whether a written procedure is used.

19-1

Human Redundancy

In a human reliability analysis (HRA) performed for a probabilistic risk

,assessment (PRA), recovery factors are treated in much the same way as human performance in other contexts. That is, many of the human perfor- mance models used to estimate human error probabilities (HEPs) in perform- ing routine or nonroutine tasks are also used to estimate the HEPs in dealing with recovery actions for those tasks. In addition, some new models dealing specifically with recovery factors are presented in this chapter. The probability of recovery brought about by any type of recovery factor is a function of the extent to which there are opportunities to detect the deviant condition, the extent to which these opportunities are exercised, and the effectiveness of the recovery action.

The first four major topic headings in this chapter deal with four general classes of recovery factors: human redundancy, annunciated indications, active inspections, and passive inspections. The recovery factor of human redundancy is a special case because of the psychological aspects of directly checking someone else's work. Annunciated indications are usually too compelling to ignore and are very potent recovery factors, Active inspections are those in which a person is directed to inspect specific items of equipment, usually via written procedures. The human reliability associated with active inspections is generally much higher than that associated with passive inspections in which the search for deviant condi- tions is considerably more casual, e.g., "looking around the control room to see if anything is out of line."

The last major topic in the chapter discusses how recovery factors are typically handled in a PRA.

Human Redundancy

If a person performs a task and makes an error that he does not detect, the

error may remain undetected until the results of that error impede some system function. If a second person checks the task performed by the first person, there is some probability that he will detect the error and correct it. If this happens, the recovery factor of human redundancy has operated. We take the conservative view that not much, if any, recovery credit should be given for a person checking his own work. Such cases should be handled individually and some assessment made of the level of dependence between a person's errors and the probability that he will catch these errors (see Chapter 10). This chapter deals only with cases in which one person checks another's work.

Some Psychological Considerations

One of the limitations in the use of human redundancy as a recovery factor is that the checker often knows whose work it is that he is checking, or at least he knows the technical level of the person who has done the work. Therefore, the behaviors of an operator* a nd a checker are not independent.

In this chapter, it is convenient to use the work "operator" in a generic sense, i.e., anyone who does work that is to be checked by someone else.

19-2

Human Redundancy

If the checker believes that the operator's work is reliable, he tends to assume that the operator's performance will be correct. This assumption and the resultant perceptual set or expectancy (what one expects to see) generally reduce the checker's effectiveness; he may miss an operator's error because he does not expect it. Even when the error is clearly visible and involves no interpretation, the checker will often fail to "see" it.

There are cases in which the opposite influence between an operator and a checker occurs. For example, if the person being checked is relatively inexperienced, or if he is from a different department, the checker may take extra care because he has doubts about the other person. If such a case is known to exist when performing an HRA, the analyst may assign a lower HEP than is included in the tables in this chapter. (For such cases, we recommend dividing by 2 the tabled HEPs for the checker.) In a PRA, one would estimate the probability that this special case would apply, and this information would be incorporated in the HRA event trees.

There is a frequently expressed misconception about the value of human redundancy: the belief that an operator's HEP will be substantially increased if he knows that his work will be checked and that greater human reliability might be achieved without a checker. From the HEPs we assign to operators and checkers, it is clear that such a state of affairs will rarely, if ever, occur. The increase in the HEP of the operator would have to be such that the joint HEP of the operator and checker is greater than the HEP of an unchecked operator. We do not know of any such cases in the type of work addressed in this Handbook. To illustrate the misconception, assume that the basic human error probability (BHEP) of an operator is .003, our general HEP taken from Appendix G. Now assume that this opera- tor's performance would be adversely affected by his knowledge that someone else will check him. How much might his HEP increase? Let us assume an unrealistically high factor of 5, resulting in a new HEP of .015 for the operator. The basic HEP for a checker is .1, so the joint HEP for the two is .015 x .1 = .0015. But without the check, the operator's original HEP is .003. So, even if we accepted the hypothetical negative effect on the operator, the overall result would still favor the use of a checker.

A final psychological consideration deals with the nature of the inter- action of control room personnel in coping with an abnormal event. We specifically exclude this type of interaction from our concept of human redundancy. The personnel dynamics in coping with an abnormal event are completely different from those involved in the usual routine of checking someone's work in standard operations. In the abnormal situation, the operators are actively pursuing various activities to respond to the event. To estimate conditional HEPs of persons who are supporting the primary operator in an abnormal situation, use the methods for estimating inter- personnel dependence described in Chapter 10.

Estimated HEPs for Checking

Table 19-1 lists estimated HEPs for a person who checks another's work in an NPP. Most of these probabilities include errors of commission and

19-3

Table 19-1

Table 19-1 Estimated probabilities that a checker will fail to

detect errors made by others*

Item Checking Operation HEP EF

(1) Checking routine tasks, checker using written .1 5 materials (includes over-the-shoulder inspections, verifying position of locally operated valves, switches, circuit breakers, connectors, etc., and checking written lists, tags, or procedures for accuracy)

(2) Same as above, but without written materials .2 5

(3) Special short-term, one-of-a-kind checking with .05 5 alerting factors

(4) Checking that involves active participation, such as .01 5 special measurements

Given that the position of a locally operated valve .5 5 is checked (item 1 above), noticing that it is not completely opened or closed:

(5) Position indicator** only .1 5

(6) Position indicator** and a rising stem .5 5

(7) Neither a position indicator** nor a rising stem .9 5

(8) Checking by reader/checker of the task performer in .5 5 a two-man team, or checking by a second checker, routine task (no credit for more than 2 checkers)

(9) Checking the status of equipment if that status .001 5 affects one's safety when performing his tasks

(10) An operator checks change or restoration tasks Above 5 performed by a maintainer HEPs

+ 2

This table applies to cases during normal operating conditions in which a person is directed to check the work performed by others either as the work is being performed or after its completion.

19-4

Human Redundancy

omission, as indicated in the title of the table. In all cases, we assume that the state of affairs to be checked is directly observable--no inter- pretation is required. The probabilities that the checking tasks will not be initiated are discussed in Chapter 16 (Table 16-1).

With no objective data on the HEPs of checking tasks in NPPs, we base the estimates in Table 19-1 on extrapolations from a series of experiments and studies of inspectors in industrial processes (cf Harris and Chaney, 1967 and 1969; McCornack, 1961; McKenzie, 1958; and Rigby and Swain, 1975) and on our experience in military production systems. In the studies cited, inspectors detected from 30% to 90% of existing defects, depending on many variables. Lower defect detection percentages are associated with low defect rates (1% or lower), passive inspection, and inspection for several types of defects in each unit. The highest detection percentages are associated with higher defect probabilities, more active participation in the production process, and inspection for only one or a very few well- defined defects.

Inspection and checking in NPPs are generally not as passive as in typical industrial assembly tasks, and the kinds of signals the checker is looking for are usually well defined. In general, an HEP of .1 for tasks involving checking another's activities while using a written procedure is used as an approximation in the absence of data (item 1 in the table). This HEP should be applied to estimate the possibility of error associated with most NPP checking tasks, including (1) over-the-shoulder checking tasks, (2) verifying the position of locally operated valves, switches for motor- operated valves (MOVs), and circuit breakers, and (3) checking the accuracy of written lists or tags. These tasks are routine, that is, there are no

special alerting features relevant to them that would cause the checker's performance to be more reliable. If these routine checking tasks are performed without written procedures, the HEP of .1 is doubled (item 2 in the table).

There are cases in which the type of checking is "one-of-a-kind", e.g., when an operator is specifically requested to check something and this checking task is not part of his normal day-to-day duties. This special checking constitutes a departure from general work procedures, and the checker can be expected to approach the task with a higher level of alert- ness and attention. For such nonroutine checking tasks, we divide the HEP of .1 by 2, for an HEP of .05 (item 3 in the table). A frequent use of the .05 HEP in HRA is for cases in which someone goes down to the diesel room to check that an operator has restored a diesel to the automatic start standby mode after a test of the diesel. Although the task is a simple one and the alerting factor for the task is good, the estimated HEP is as high as .05 because of the checker's strong expectancy that he will find that the operator had properly restored the diesel to the AUTO position. (In some NPPs, the probability of the unrecovered error of leaving the diesel in other than the AUTO start standby condition is very low because of special displays in the control room that alert the operators.)

19-5

Human Redundancy

Some checking tasks may involve hands-on activities such as using an instrument for some measurement. In such cases, the performance level of the checker approaches that of the original operator. The tabled value of .01 (item 4) is an estimate of the probability that the checker will make an error in an inspection task that includes active involvement. Merely looking to see that someone's work has been done properly, without perform- ing some work of one's own, does not constitute active involvement.

In some checking tasks involving locally operated valves, the discrimina- tions made of valve state often are not complete. The checker will usually assume that the valve has been fully opened or closed, even though the operator may not have completed his task. Generally, this error in valve positioning would be due to sticking of the valve as it was manipulated. Because of the very high expectancy on the part of the checker that the valve was properly manipulated, we assess a high probability of failure for the checker to note that a valve is not completely open or closed after it has been changed or restored.* The highest probability of failure (.9) is for the case in which the valve has neither a position indicator** nor a rising stem (item 7 in the table). In this case, the checker would have to place his hands on the turning wheel and check for the opened or closed position, and it is very likely he would make the same error as the person who originally set the valve. If the valve has a position indicator and a rising stem, we estimate a .5 HEP (item 6 in the table), since he is most likely to pay attention to the rising stem, which can provide an ambiguous indication. If the valve has a position indicator only, he is forced to look at it if he is doing his task, and he will likely see that the valve is not fully opened or closed. The .1 HEP (item 5 in the table) is as- sessed as the usual HEP for a relatively passive type of checking task.

Sometimes, as when calibration procedures are performed, two people act as a team, with one reading aloud from a set of written instructions and the other performing the tasks. The reader also acts as checker of the per- former's work. Because the checker's task is passive, we assign him a high level of dependence. For HRA, assume an HEP of .5 for the reader/checker (item 8 in the table).

The use of several checkers in succession to check someone's performance has limitations because the second, third, and later checkers do not expect to find anything wrong and may see this as a "make-work" type of assign- ment. There are no studies directly related to the loss of effectiveness

Equipment change means a change from normal status to permit maintenance, testing, etc. Equipment restoration means to restore the equipment to its normal status for the power generating mode of the plant.

19-C

Human Redundancy

in repeated checking activities in an NPP. In one experiment with elec- tronics assembly plant personnel (Harris and Chaney, 1969, pp 79-80), 45% of the defects were detected in the first inspection, with about 15% more defects found in each of the next two inspections. After the third inspec- tion, additional inspections were much less effective, and after the sixth inspection, essentially no more defects were found. It would be inappro- priate to generalize these results without modification to the task of checking in NPP operations. The above experiment was artificial in that the subjects knew it was an experiment and saw it as a challenge (the subjects were experienced inspectors). In the usual NPP work situation, the motivation of the second and subsequent checkers would not facilitate effective checking. Although we assess high dependence (HD) for successive checkers, we recommend that no recovery credit be allowed for more than two checkers for a routine task. For HRA, use an HEP of .5 for the second checker of a routine operation (also item 8 in the table).

The last two items in the table deal with special knowledge that the checker has about his job. Item 9 deals with a person's natural concern for his personal safety. If the status of some item of equipment (valve, circuit breaker, etc.) is such that it could endanger him when he worked on the system, we estimate a probability of .001 that he would fail to check the status of the equipment even in the absence of written instructions to do so. This is similar to the situation of an electrician checking the status of a power switch before working on a circuit--the same HEP of .001 applies.

The last item (10) is based on interviews with auxiliary and licensed reactor operators who indicate that they take special care when they check change or restoration tasks performed by maintainers. Normally, only operations personnel perform such tasks, but occasionally some may be done by maintenance personnel. We divide the operator's usual checking HEPs by 2 in consideration of this extra care.

Cautions in Assessing Recovery Credit for a Checker

The estimates in Table 19-1 are predicated on the expectancy of a checker to find things as they should be; but even these relatively high HEPs can be optimistic if the checker's task is carried out in such a way that the human redundancy is minimal or nonexistent. Before assessing recovery credit for human redundancy, theanalyst should observe actual plant opera- tions to determine whether plant-mandated human redundancy is really being followed and what percentage of time it is being followed.

The following example is offered as an illustration of "human redundancy" in action and points out areas of concern to the analyst. The example comes from an assembly plant where great emphasis was placed on the quality of the product with considerably less emphasis on the quantity produced.

At certain stages of assembly in one system, the assembler was instructed by the written procedures to call for an "over-the-shoulder" inspection of a torquing operation. At this point, the inspector was supposed to consult

19-7

Annunciated Indications

his own procedures to determine the value of torque to be applied. Then he was to look over the shoulder of the assembler while the latter applied

torque by means of a torque wrench with a built-in meter. When the assem- bler was finished, the inspector was to indicate if he agreed with the

assembler regarding the torque that had been applied.

If carried out correctly, the above procedure is estimated to result in a

90% recovery factor, i.e., 1 time in 10 the inspector would fail to note an incorrect torque application. However, the manner in which the procedure

was actually carried out removed all human redundancy. First, the assem- bler called to his friend, the inspector, "Hey, Joe, I need an over-the- shoulder." Joe replied, "What's the torque supposed to be?" The assembler informed Joe from his (the assembler's) procedures. Then he proceeded to

perform the torquing operation with Joe looking over his shoulder. The assembler did not even look at the meter; he looked back at Joe. Even- tually, Joe said, "OK, that's it," and the assembler relaxed the torque wrench. This procedure was the equivalent of one person performing the task. If a human reliability analyst improperly allowed the 90% credit normally assigned to this recovery factor, the joint probability of failure

would be underestimated by a factor of 10., This points out the necessity for observing actual plant operations when possible to determine the per- centage of time plant policies are carried out as they are intended to be. Chapter 16 includes more examples of similar problems.

Annunciated Indications

Chapter 11 defines an annunciated display as a special type of legend light

that is "annunciated by some type of compelling sound, such as a horn, and also by a flasher circuit, which causes the legend light to flash on and off." The attention-getting value of such a display is very high--it is very unlikely that an operator will ignore (or will be able to ignore) an

annunciated signal in situations in which it is the only demand on his immediate attention. As his information load increases, e.g., as more and more annunciators compete for his attention, the probability of responding to any given annunciator decreases as a function of that load (Table 11-13).

If an operating parameter approaches or passes an operating tolerance

limit, usually an annunciator associated with that parameter will sound. The potential for recovery exists in that the operator may respond to the annunciator, determine its cause, and take action to mitigate its effect. The probabilities of an operator failing to respond to one or more an- nunciators are listed in Table 11-13, ",The Annunciator Response Model:

estimated HEPs for multiple annunciating indicators." In the context of annunciators, the word "respond" refers to the initiation of some kind of

intended corrective action. The action may be correct or incorrect.

Part of the response to an annunciator will often be the reading of meters, charts, and other displays that are related to the annunciator tile. For example, if the annunciator tile says MAIN FEEDWATER TRIP, experienced operators will also look at displays that will indicate whether there is

19-8

Active Inspections

flow in the auxiliary feedwater system. In this sense, the annunciator tile and its related displays constitute a perceptual unit. These kinds of "trained in" associations must be taken into account in an HRA to avoid undue pessimism.

As with any recovery factor, the effectiveness of the annunciated signals in bringing about recovery from human error must be assessed. Some signals are assessed as 100% effective, that is, each time they occur and are detected, it is certain that recovery from the error will result.* If this assumption cannot be made, the probability of detection must be estimated and also the percentage of the time that detection will result in recovery. Some annunciations will always lead operators to check specific indicators, while others may or may not, depending on the situation.

Finally, in the modeling of human response to annunciations in Chapter 11, it was noted that if an operator turns off the sound and flashing of the signal, and then fails to take action within a fairly short period of time, the attention-getting value of the annunciator tile is lost. The probabil- ity that the operator will notice a steady-on annunciator tile in the usual plethora of such indications is very low. This aspect of human behavior is modeled in the section on "Passive Inspections."

Active Inspections

Active inspections are those in which a person is directed to inspect specific items of equipment. This direction may consist of an oral instruction, a written schedule, or some other directive. The probabili- ties that such tasks will be initiated are discussed in Chapters 15 and 16.

If an active inspection is performed in response to an oral instruction, see Table 15-1 for HEPs for errors of omission. If written materials will be used, see Tables 15-2 and 15-3 for HEPs in preparing the materials and in using them. Various tables in Chapters 11, 13, and 14 deal with related errors of commission.

Active inspections include the following types of tasks:

(1) Every 2 hours during a shift, an operator is required to read and record quantitative information from selected meters, graphs, and other displays.

(2) At the beginning of each shift, the control room operator (or an assistant) uses a written checklist to verify that the status of each safety-related display in the control room is appropriate. We desig- nate this as a written control room audit to differentiate between it and more passive control room inspections not using written checklists.

"Certainty" in the context of HRA means that exceptions are so rare that it is reasonable to assess a probability of 1.0.

19-C

Passive Inspections

(3) During each shift, an auxiliary operator takes a written list and checks about 100 safety-related valves and other items of equipment outside the control room for proper status. We designate this as a written walk-around inspection.

In terms of human reliability, all of these tasks are considered similar for errors per item inspected. Differences in estimated HEPs for errors of

omission and commission will depend on the task and equipment characteris- tics discussed in Chapters 11 and 13 through 16.

The probability that detection of a deviant condition will always result in recovery must be estimated. If the probability of recovery after detection is less than 1.0, the percentage of the time that it will result in recov- ery must be estimated. These latter estimates must be entered into the HRA

event tree to determine their effects on the estimated probability of system failure, as defined by system analysts.

Passive Inspections

Passive inspections involve a more casual search for deviant conditions

than is characteristic of active inspections. The two primary forms of passive inspections are the periodic scanning of control room displays and the basic walk-around inspection, which consists of a scheduled inspection tour of a specified area in the plant to note any deviant conditions. No written materials are employed in either type of passive inspection, nor have the inspectors been instructed to note the status of any specific items of equipment. If such is not the case, the models for active inspec- tions must be employed for the items of equipment involved.

Periodic Scanning

Periodic scanning in the control room includes the initial audit, which is the first periodic scan at the beginning of the shift. Periodic scans are described in detail in Chapter 11. They are intended to provide the con-

trol room operator with opportunity to detect deviant indications on unan- nunciated displays--those which have no associated sound or flashing signal to increase their attention-getting values.

The frequency with which periodic scans are to be performed is usually specified in a plant's administrative policy. Our experience is that this

policy is seldom followed fully. Although a plant policy may specify hourly scanning, scanning may be more or less frequent than hourly. More-

over, not all types of displays are likely to be scanned. The analyst must determine through interviews and observation the scanning periodicity for each display of importance to the HRA.

After the scanning periodicity and specificity have been assessed, the display scanning model (Chapter 11) can be used to estimate HEPs for detecting deviant indications on displays. Table 11-7, "Estimated proba-

bilities of failure to detect one (of one) unannunciated deviant display at each scan, when scanned hourly," provides estimates of failure to detect the incorrect state of different display types. With this estimate, one

1 9-10

Passive Inspections

can enter Table 11-6, "Estimated probabilities of failing to detect at least one of one to five unannunciated deviant displays as a function of the BHEP for detection of a single deviant display during periodic scan- ning," to obtain estimates of the failure of recovery when more than one display is involved.

Basic Walk-Around Inspection

The basic walk-around inspection is a special case of verifying the state of equipment items. This type of walk-around inspection consists of a scheduled inspection tour of a specified area of the plant, without the use of written materials. The inspector is merely told to report anything unusual or any deviant condition of equipment. If he is given explicit instructions (e.g., "Be sure to check the main isolation valve on the RWST."), the oral instructions model (Table 15-1) is used to estimate HEPs for these items. If a written procedure is used in a walk-around inspection, it is no longer a basic walk-around inspection; it is a written walk-around inspection. At some plants, both types of walk-around inspec- tions are employed. For the written walk-around, use the HEPs from Chapter 15 dealing with following a written directive, and the HEPs from Chapter 16 dealing with the probabilities of use and correct use of procedures. The following discussion is for the basic walk-around only.

The basic walk-around is usually performed by an auxiliary operator. It provides an opportunity to detect obvious deviant conditions, such as an oil spill or a water leak, or less obvious deviant conditions, such as a locally operated valve that is in the wrong position. Our modeling of the basic walk-around inspection can be used to estimate the probability of detecting a safety-related item of equipment that was left in an unavail- able state following maintenance or testing.

While the basic walk-around offers a possibility of recovery from a human error, this recovery factor is not as effective as others because of the relatively passive nature of the inspection coupled with the inspector's low expectancy that he will find anything wrong. This section presents some performance models and estimates of the recovery afforded by different applications of the basic walk-around, beginning with basic assumptions and continuing with variations.

our human performance model for the basic walk-around inspection includes the following assumptions:

(1) The inspection is made once per shift.

(2) The inspector performing the walk-around knows the plant well and can recognize deviant conditions if he notices them.

(3) The inspector covers the same area each time (although not necessarily in the same sequence), so all deviant items have almost equal proba- bilities of being observed during each walk-around.

19-11

Passive Inspections

(4) No written procedure is used during the walk-around.

(5) No special oral instructions have been given to the inspector to attend to some particular item of equipment.

(6) Any deviation is fairly obvious if the inspector knows what condition is normal. In this respect, the walk-around task is different from that of the inspector who looks for minor imperfections in product manufacturing, since the product inspector's judgments are more sub- jective. On the other hand, we are not addressing deviations as obvious as a large pool of water on the floor, which we assume will always be noticed.

The assumptions for this basic walk-around imply that the inspection is entirely visual and that the inspector is concerned primarily with things that are clearly deviant. They also imply that discovery of some un- desirable situation (say, an oil slick on a floor) may interfere with his recognition of some other less obvious condition. If any one of the above six assumptions is not valid, the HEPs in this section must be modified.

With the above assumptions, we estimate a probability of .9 per exposure that an inspector will fail to notice a deviation (such as a locally oper- ated valve in the wrong state) if there is no provision for indicating a disparity between the actual and correct states. This high probability of failure is due to a combination of performance shaping factors (PSFs) described below.

The first PSF is the inspector's set, or expectancy, to find important things as they should be. Expectancy is a very powerful PSF and a major cause of error in routine inspections when the probability of a deviant situation is low (e.g., 10-2 or less). Expectancy is based on past experi- ence and the interaction between the inspector and other personnel. This interaction includes the knowledge that other personnel are responsible and competent and seldom make mistakes. In NPPs, the probability that an item of the type that is looked for in a walk-around will be deviant should be less than 10-2 per shift, and considerably less than 10-2 per item in-

spected. Therefore, the inspector performing the walk-around has a very strong expectancy of finding everything in order.

A second PSF derives from the decline of memory with time. If the valves present no indication of their normal status, the inspector must rely on memory. Even though the inspector understands the functions of all major components in the plant, he tends to rely on visual memory when he makes his walk-around; if things look the way they looked the previous day, he will usually accept them as being correct (unless alerted by a deviation from an easily recognized pattern).

When a person has to inspect hundreds or thousands of items, his memory for individual items declines very rapidly. The curve of retention in Figure 19-1 is based on laboratory studies of visual recognition of words across time. The shape of the curve typifies other such studies and can be applied to inspection tasks.

19-12

Figure 19-1

100

i i i

a iUj 60

0 0UuJ

~40

1~ __ __ __ 0.

201--

fttf

0 I I I I 0 1 2 3 4 5 6 7

TIME AFTER LEARNING (DAYS)

Figure 19-1 Curve of retention as determined by the Recognition Method (Strong, 1913).

Passive Inspections

As shown, recognition accuracy on successive days after viewing a large

number of items is 25% for Day 1, 20% for Day 2, 17.5% for Day 3, 16% for Day 4, and <15% for Day 5. Use of this experimental data to estimate the probability of recognizing that an item is deviant would result in esti- mates that would be too optimistic. The basic walk-around situation is different in several respects from the experimental situation, one of the most important differences being the strong expectancy of the inspector to find all things normal during his walk-around. We therefore have modified the data by subtracting 15 percentage points from the experimental percent- ages. However, for Day 5, this subtraction would lead to a value of zero, the lower limit. For Day 5 and subsequent days, we have assigned a minimum Pr[S] of .001 to reflect a low-level recovery factor that has been observed in common experience in other activities. Let us assume that the inspector has seen a valve in the wrong state after it was placed that way but did not note this deviation. His memory trace for the correct status declines as in Figure 19-1. At the same time, he develops memory traces for the valve in the wrong position, so that in time the new (incorrect) position becomes increasingly likely to be seen as correct (or usual). However, even under these conditions, people occasionally recognize an incorrect situation after having accepted it several times. We do not know why this happens; possibly the person is a little more alert than usual or he may just happen to be thinking of the logic underlying the valve position, but occasionally a recognition of the irregularity does occur after a succes- sion of oversights.

Column (a) in Table 19-2 presents the above modified experimental data for the Pr[S] for the recognition of the same deviant item during basic walk- around inspections performed on successive calendar days by the same person. The Pr[S] of .1 for Day I assumes, of course, that this is the first opportunity for the inspector to see the deviant item, and the Pr[S] values for succeeding days are based on the nondetection of that item on all previous days.

Column (b) shows the basic loss (L ) in detection probability subsequent toB Day 1. L is calculated by subtracting the Pr[S] for any subsequent walk-

around from the basic .1 Pr[S] for Day 1.

Column (c) presents the Pr[F] estimates, which are used to estimate the probabilities of detecting a given deviant item within 30 days, under various assumptions about the walk-around inspection.

Over a 30-day period, beginning with the first opportunity to detect a deviation, the total Pr[S] for an inspector who performs a walk-around once per day is:

Pr[S 3 0 days 1 1 shift, 1 inspector] (19-1)

= I - Pr[FDay 1] Pr[FDay 23 Pr[FDay 3] Pr[FDay 4] Pr[FDay 5I26

= I - (.9 x .95 x .975 x .99 x .99926

= .1959 = .20

1 ý- 14

Table 19-2

Table 19-2 Probabilities of successful and unsuccessful recognition of the same deviant item during basic walk-around inspections performed on successive calendar days by the same person*

** t* Day Pr[S] L Pr[F] EF

B s

Item (a) (b) (c)

(1) 1 .1 -- .9 5

(2) 2 .05 .05 .95 5

(3) 3 .025 .075 .975 5

(4) 4 .01 .09 .99 5

(5) 5-30 .001 .099 .999 5

(6) >30 0 -- 1.0 --

The above estimates do not incorporate the special alerting effect of the detection of one deviant item on the detection of other deviant items during the walk-around, as discussed in the text.

L ,the basic loss in detection effectiveness from the first walk-around to other walk-arounds when they are performed on successive calendar days. It is calculated as Pr[SDay 1 Pr[SDay n]

The Pr[F] on any day is the probability of fail- ure on that day given failure on all previous days.

If a deviant item has been undetected for as long as 30 days, it is assumed that it will not be detected on subsequent walk-arounds unless some other indication alerts the operator.

19-15

Passive Inspections

In view of the usual work schedule, which includes days off during a 30-day period, it is recognized that one inspector will not really perform the daily walk-around inspection for 30 successive calendar days. However, for modeling purposes, it is convenient to assume that such is the case. If we assumed a customary work week of 5 days on and 2 off, the Pr[S] calculated above would increase only slightly. For modeling, we will use the more conservative assumption of 30 successive calendar days.

If different inspectors perform walk-arounds, the same Pr[S] applies to

each. Thus, with the usual situation of one walk-around per shift for each of three shifts and the assumption of zero dependence (ZD) between the

shifts, the total probability of detection of some particular deviation over a 30-day period is calculated as follows, using the unrounded Pr[S] of .1959 that was calculated from Equation 19-1:

Pr[S 30 days13 shifts, 3 inspectorsl ZD between shifts] (19-2)

= 1 - {I - Pr[S 3 0 days 11 shift, 1 inspector]}3

= 1 - (1 - .1959)3 = .480 = .5

It can be argued that the assumption of ZD between the three shifts is

overly optimistic. Some of our reviewers with operating experience suggest that the walk-around model should be modified for the evening and night shifts to take into account the likely dependence resulting from the atti- tude that "those guys on the day shift have probably found anything that I could find." This is a reasonable argument. However, since the walk- around HEPs are very large, the application of the dependence model will make very little difference. For example, if moderate dependence (MD) is assumed for the evening shift and HD for the night shift, the Pr[S] is recalculated below, using Equations 10-16 and 10-19 from Table 10-2 to

calculate HEPs based on MD and HD, and using the tabled HEPs for successive days of inspection from Table 19-2. In the following calculation, the term in the first set of brackets is the probability that the inspector on the first shift will fail to detect the deviant item within 30 days. The terms in the second and third sets of brackets represent the Pr(4 3 0 days

1 terms for the inspectors on those shifts.

Pr[S 430 days 13 shifts, 3 inspectorsj MD for 2nd shift, HD for 3rd shift]

1 -1.19591 [1 + 6(.9) -1 + 6(.95) x1 + 6(.975) x1 + 6.9

/i \ (.9 )261 1 9 1 + .95 1 + .975 x 7 12- x 2 x 2

1 + .99 9 1 + .999 +26] 72 [ 2

= 1 - (.8041 x .8302901 x .8983408)

= 1 - .5997649 = .400 = .4

19-16

Passive Inspections

This example illustrates a case in which the assumption of ZD, although not valid from a psychological point of view, is accurate enough. In the re- mainder of this section, we assume ZD between shifts, since the calcula- tions are simplified and no important accuracy is lost.

With the usual walk-around, there is about a 50-50 chance that any one deviation will be detected in a 30-day period. We assume that if a devia- tion remains undetected for 30 days, it will not be detected in subsequent walk-arounds. Our rationale is that if a deviant item remains undetected as long as 30 days, some factor is operating in the walk-arounds that pre- vents it from being detected. Therefore,

ES>3 0 days IF30 days 3 = 0 (19-3)

It is apparent from the above analysis that if each inspection could be made with a fresh inspector the probability of detecting a deviant item within a 30-day period would be very high. Assume, for example, the ideal situation in which each shift can draw inspectors from 30 operators, and in a 30-day period each operator performs only one walk-around, with a 30-day recovery period between walk-arounds for each operator. In such a situa- tion, each operator has a first-day probability of detection of .1 each time he does the walk-around. Therefore, for any shift, the probability of successfully detecting a deviant item within 30 days becomes

Pr[S4 3 0 daysl1 shift,.30 inspectors] (19-4)

= 1 - Pr[FDay 1)30

30= 1 - .9 = 1 - .042 .96

compared with a Pr[S] of .2 for the same person performing the walk-around

every day.

If we extend the above ideal to three shifts, requiring 90 different inspectors, the estimated probability of success is increased as follows, using the unrounded value of .042 from Equation 19-4:

Pr[S 30 days13 shifts, 90 inspectors] (19-5)

= 1- {1 - Pr[S 3 0 days 11 shift, 30 inspectors]} 3

3 = 1 - .042 .9999

(Note: If we assumed MD for the second shift and HD for the third shift, the .9999 estimate would change as follows:

Pr[S] = 1 - x ( + 6(.9)) x + .930 = .99938 .9994

Considering the inexactitude of the estimates, the difference between .9994 and .9999 may be disregarded.)

1S-17

Passive Inspections

Of course, we do not assume that an NPP could afford to have enough people available so that each of the 90 walk-arounds per month would be performed by a different person. Furthermore, it is not necessary from a risk-benefit standpoint. We estimate that if a person performs one walk-around per week, he will recover his recognition accuracy fully between inspections. Because of this recovery, we have a virtually independent inspection each week by that person. Our rationale for the full recovery in 1 week's time is based on com- mon experience with the typical workweek. The performance of a chore once a week (with 6 intervening days of not performing the chore) results in a vir- tually fresh start for that chore each week. Thus, the estimated Pr[S] over a 30-day period of .9999, calculated from Equation 19-5, could be approached if each shift had seven people assigned to the walk-around so that any one person inspects only once every 7 days. This assignment would provide a 1-week re- covery period for each operator.

The same type of analysis can be done for other intervals between walk-arounds if the relation between recovery and walk-around intervals is known. We as- sume that the recovery curve can be approximated by the cumulative normal dis- tribution (with a mean of 4 and a standard deviation of 1), as shown in Figure 19-2.* The values on the ordinate represent the percent recovery, r, of. the detection effectiveness. For example, if a second walk-around is performed by a person 5 days after he performed the first walk-around, his basic loss (L) in detection effectiveness for any deviant item previously undetected will not be as much as if he performed the second walk-around on the very next day. From Figure 19-2, 5 days represents an 84% recovery. Therefore, his loss in detection effectiveness is calculated by taking L = .05 (item 2b, Table 19-2)B and multiplying it by the complement of the recovery percentage, (1 - .84) = .16. Therefore, his loss is .05 x .16 = .008. This loss is subtracted from the basic Pr[S] = .1 for Day 1, yielding a Pr[S] of .092 for his second walk- around (5 days after the first one). As would be expected, the Pr[S] for walk-arounds at 5-day intervals is higher than for walk-arounds on successive calendar days: .092 vs. .05.

Using the same approach, one can calculate the L , Pr[S], and Pr[F] values for successive walk-arounds performed by the same inspector on every fifth day throughout the month. These values are shown below; the calculations for these values and for other intervals between walk-arounds are shown in Appen- dix D.

Walk-Around L Number B Pr[S] Pr[F]

1 0 .1 .9

2 .008 .092 .908 3 .012 .088 .912 4 .014 .086 .914 5 & 6 .016 .084 .916

In the draft version of the Handbook (Swain and Guttmann, 1980), we evalu- ated the assumption of a straight-line recovery curve, a negatively accel- erated recovery curve, and a positively accelerated curve. We found that none of the different assumptions made any material difference in the calculations based on the recovery curve in Figure 19-2.

Figure 19-2

99.9 97.7

84.0

> 50.0 0

16.0

2.3 0.1v-

1 2 3 4 5 6

DAYS BETWEEN WALK-AROUNDS

Figure 19-2 Percent recovery, r, of detection effectiveness as a function of the number of days between basic walk- around inspections by the same person.

19-19

Passive Inspections

If there are five inspectors available during a shift for carrying out the walk-around inspection and each inspector makes a walk-around every 5 days, there will be a total of six walk-arounds per inspector. The Pr[S] for the detection of any particular deviant item in a 30-day period is

Pr[S 30 daysl1 shift, 5 inspectors]

2 5 = 1 - (.9 x .908 x .912 x .914 x .916 ) = .94

where the terms within the parentheses are the Pr[F]s for the six walk- arounds performed by each of the five inspectors. For three shifts, each with five inspectors, the probability of detecting a particular deviant item within 30 days is

Pr[S4 3 0 days13 shifts, 15 inspectors] = 1 - (1 - .94)3 = .9998

Table 19-3 lists the probabilities of detecting any particular deviation over a 30-day period as a function of the number of days between walk- arounds for each inspector. One walk-around per shift is assumed. For HRA convenience, Table 19-4 converts the Pr[S) values to Pr[F]s for the usual case of a basic walk-around inspection each shift and three shifts per day.

Modifications of Basic Walk-Around for More Than One Inspection per Shift

If different inspectors perform a walk-around in the same shift, their detection probabilities are combined. Thus, if two inspections per shift are performed by different inspectors (each performing one inspection per shift for 30 days), the probability of successful detection of a deviant condition within 30 days for one shift is calculated as follows, assuming ZD between inspections and using the unrounded .1959 value from Equation 19-1:

Pr[S430 days 11 shift, 2 inspectors, each making (19-6)

1 inspection per shift]

= 1 - Pr[Finspector 1130 days] Pr[Finspector 2130 days]

2 = 1 - (1 - .1959) = .3534 = .35

This value is the same as that for two shifts with one inspector each (Table 19-3).

If each of the three shifts uses the same procedure, the resultant Pr[S] for a 30-day period is calculated as follows, using the unrounded .3534 from Equation 19-6:

19-20

Table 19-3

Table 19-3 Estimated probability of detecting any particular deviant condition within 30 days for various numbers of days between basic walk-around inspections

Days Between Walk-Arounds for Each Inspector* Pr[S430 days]

One Shift** (a)I tem

(1)

(2)

(3)

(4)

(5)

(6)

1 (daily walk-around)

.20

.37

.63

.86

.94

.96

Three Shifts** (b)

.48

.75

.95

.997

.9998

.9999

(7) 7 (weekly walk-around) .9999

It is assumed that all inspectors have the same number of walk-arounds. For other assumptions, modify the relevant the text.

days between equations in

The entries are derived from the Pr[S] values in Table 19-2, which indi- cate a Pr[S] of .1 for the initial walk-around following the occurrence of any particular deviant condition, and a systematic reduction in Pr[S] for recognition of that deviant condition for subsequent walk-arounds as a function of the days between successive walk-arounds by the same inspector. See Appendix D for calculations.

19-21

Table 19-4

Table 19-4 Estimated probabilities that the basic walk-around inspection*

will fail to detect a particular deviant indication of equipment outside the control room within 30 days**

Cumulative Pr[F] Within

Number of Days Between t 30 Days Given Onetinspection

Item Walk-Arounds per Inspector Per Shift

(1) 1 (daily walk-around .52 for each inspector)

(2) 2 .25

(3) 3 .05

(4) 4 .003

(5) 5 .0002

(6) 6 .0001

(7) 7 (weekly walk-around .0001

for each inspector)

See text for the assumptions for the basic walk-around inspection. One

of these assumptions is that no written procedure is used; if a written

procedure is used for a walk-around, use the tables related to errors of

omission and commission for performance of rule-based tasks (Chapters 11

and 13 through 16).

Three shifts per day are assumed. If not, use the appropriate equations

in the text.

It is assumed that all inspectors have the same number of days between

walk-arounds. For other assumptions, modify the relevant equations in

the text.

For EFs, use the procedure in Appendix A or use EF = 10 as an

approximation.

Passive Inspections

Pr[S4 3 0 days13 shifts, 6 inspectors, 2 per shift, (19-7)

each making 1 inspection per shift]

= 1 - Pr[F4 3 0 daysi1 shift, 2 inspectors, each

making 1 inspection per shift) 3

-1- (1 - .3534)3 * .73

This probability of .73 for two inspectors per shift can be compared with the probability of .5 for one inspector per shift for the detection of a deviation within 30 days (Equation 19-2). If the assumption of ZD is not valid, the estimates of Pr[S] within 30 days will be lower. Assuming MD and using Equations 19-1 and 10-16 (from Table 10-2), the estimates of .35 for one shift and .73 for three shifts become .33 and .70. The calculation of the .33 estimate is illustrated below:

Pr[S430 daysji shift, 2 inspectors, each making 1 inspection

per shift JMD between inspectors]

1- [(9 x1 + 6(.9)) x 95 x1 + 6(.95) (.975 x1 + 6(.975) 1( . 7 "7 7

(99 x1 + 6(.99)) (.999 x1 + 6(.999) 261

= 1 - .667634 = .33

It has been suggested that if an inspector performs a walk-around more than once per shift, there will be a net gain in the probability of his detect- ing a deviant condition. However, even if we make the unrealistic assump- tion that the inspector's motivation will not suffer, there will be no material gain in the probability of detection over a 30-day period because the addition of 30 extra walk-arounds, with a probability of detection of .001 per walk-around, will change the overall .20 probability of success (from Equation 19-1) by only 10%. The calculation is:

Pr[S( 3 0 daysil shift, 1 inspector making 2 inspections

per shift]

= 1 - (.9 x .95 x .975 x .99 x .9995) 1 - .78 = .22

and (.22 - .20) - .20 = 10%

Effectiveness of the Basic Walk-Around Inspection

It is apparent that the use of the basic walk-around inspection to detect less obvious deviant conditions, such as valves in the wrong state, is not

19-23

PRA Treatment of Recovery

nearly as effective as the use of the written walk-around inspection. To illustrate this difference, compare the Pr[S] of .1 per deviant item (Table 19-2, Day 1) for the first basic walk-around inspection with the Pr[S] of .99 per deviant item for a written walk-around inspection, assuming a long list and no checkoff (Table 15-3, item 4). However, the basic walk-around inspection is a quick and reliable method for early detection of obvious deviant conditions such as leaky pumps or pipes.

PRA Treatment of Recovery

Many of the models for estimating probabilities of incorrect performance of routine or nonroutine tasks are also used for estimating probabilities of the failure of recovery factors for those tasks. The probability of the failure of a recovery factor may consist of the HEP for a single action or it may be the total-failure term of some combination of recovery actions, as diagrammed in an HRA event tree. In either case, the probability of recovery failure must be combined with the HEP for the original error to produce an estimate of an unrecovered human error. For example, the proba- bility that an operator fails to restore a valve to the proper position (the original error) and the probability that a checker fails to detect that error (the failure of the recovery factor) must be combined to obtain the estimated probability of the error occurring and remaining unrecovered.

The same recovery factor may apply to several tasks, e.g., the onset of an annunciator, which should tell an operator that he has failed to perform correctly all three steps in a procedure. In this case, the .HEP for fail- ure of recovery should be multiplicatively combined with the original HEP for each step to arrive at the joint HEP. In most cases, this combination would be represented by two successive failure limbs in the same failure path in an HRA event tree, the first failure limb for the original error and the successive failure limb for the failure of the recovery factor. In other cases, the recovery factor by itself may represent another HRA event tree, which could be appended to the original HRA event tree after each original error to which it applies.

When the probability of failure of a recovery factor is included in an HRA event tree, the end-failure probability for the tree represents the proba- bility of some set of unrecovered human errors. This end-failure probabil- ity (and uncertainty bounds) can be entered into the system fault or system event trees.

The above treatment of recovery factors is acceptable only when the recov- ery factors operate immediately after the initial errors or when the time delays between the initial errors and the recovery factors are0 insignifi- cant. If the time delays are significant, we use an alternative approach based on the unavailability equations from Chapter 9. For example, assume a case in which valves were restored soon after maintenance, were checked soon afterwards by a second person, and then were checked again a week later. The effects of the two checks on unavailability of the items would not be identical; the unavailability equations should be used to estimate unavailability over time. These equations hold for cases in which the

19-24

PRA Treatment of Recovery

probability of recovery is independent of the time at which the recovery factor is employed, i.e., a time-independent recovery factor. For a time-dependent recovery factor, that is, for cases in which the probability of recovery is dependent on the time at which the recovery factor is em- ployed, the equations must take this dependence into account, as is done in the equations for the basic walk-around inspection.

Recovery factors are not analyzed for every HRA. If the estimated proba- bilities of human errors without recovery factors are insignificant with respect to those of other contributors, there is no need to include an evaluation of recovery factors. For this reason, in some PRAs the effects of recovery factors are analyzed for only those original errors that by themselves make a significant contribution to the system analysis. This method for considering recovery factors is described in Chapter 20, in the first problem in Chapter 21, and in NUREG/CR-2254.

19-25

PART IV. USE OF HANDBOOK TABLES OF HUMAN ERROR PROBABILITIES FOR HUMAN RELIABILITY ANALYSIS, AND SOME APPLICATIONS

This part of the Handbook consists of two chapters: Chapter 20, "Tables of Estimated Human Error Probabilities," and Chapter 21, "Examples and Case Studies." Chapter 20 includes most of the data tables from Part III along with a search scheme to aid the analyst in accessing the correct table for a specific application. Chapter 21 includes a number of sample problems, including some actual human reliability analyses performed for the U.S.

Nuclear Regulatory Commission.

IV-I

Ch. 20. Tables of Estimated HEPs Overview

CHAPTER 20. TABLES OF ESTIMATED HUMAN ERROR PROBABILITIES

Overview

This chapter summarizes the estimated human error probabilities (HEPs) and their uncertainty bounds (UCBs) (or error factors [EFs]) presented in Part III. The tables in this chapter are duplicates of data tables in Part III except for changes to footnotes and table references to make them appro- priate to Chapter 20. Not all data tables in Part III are included in this chapter; those that are included are sufficient for most human reliability analyses (HRAs) conducted as part of a probabilistic risk assessment (PRA). These tables are intended for use as quick references and are cross-refer- enced to the chapters from which they are drawn. The user is urged to familiarize himself with the source chapters for the proper use of the error terms and the assumptions on which they are based.

This chapter begins with a brief discussion of performance shaping factors (PSFs), followed by a search scheme for the use of the tables, with an explanatory talk-through of the search scheme. The chapter concludes with a list of tables, a quick-reference guide to the tables, and the set of tables.

For users conducting HRAs, the search scheme provides guidance to the ap- propriate tables at each stage of the analysis. The quick-reference guide is intended for general use and will help the analyst locate any table of interest.

Performance Shaping Factors

All of the estimated HEPs in the data tables are nominal HEPs, i.e., they represent HEPs before plant-specific PSFs have been taken into account. When these latter are evaluated, a nominal HEP may be modified upward or downward.

Chapter 3 describes the usual PSFs that influence HEPs in industrial settings. PSFs specific to classes of activities are discussed in detail in Part III. As a rule, the HEPs in the Handbook are based on "average" industrial conditions. We define average industrial conditions as those that do not subject a worker to an unusual degree of discomfort and that are fairly representative of the industry. The user may modify the tabled HEPs if the PSFs for his specific application are not average. Some guid- ance is given to help the analyst to determine the average conditions applicable to each group of HEPs, but most of this information is presented in Part III.

PSFs such as temperature, noise level, lighting, and others related to the comfort or health of the worker will usually be average (or better) in nuclear power plants (NPPs). This is because regulatory agencies such as the Nuclear Regulatory Commission and the Occupational Safety and Health Administration have developed "guidelines" or "recommended limits" for most controllable factors affecting workers. The plants' managements will work

20-1

Search Scheme for Use of

Chapter 20 Tables

to meet the standards set by such agencies, and organizational units such 0 as employee unions and professional organizations will usually report any

deviations from these standards.

The PSFs related to ergonomics considerations are not subject to regula- tion. Hence, considerable variations exist from plant to plant as well as

within any given plant. The estimated HEPs summarized here are based on conditions observed in a number of operating U.S. and foreign plants. In

some cases, differences in PSFs have been estimated in the breakdown of the HEPs. For example, modifications to HEPs based on the PSFs of display type

and information displayed have been defined in the data tables. Display

types such as analog meters, digital indicators, chart recorders, etc.,

have been analyzed for the effect they have on human performance; the HEPs for errors made in dealing with displays have been modified to account for

these effects. Very small differences in performance that might result from relatively minor differences in human factors engineering of displays,

e.g., indicator needle length and width, are not represented in the esti- mated HEPs.

In other cases, it is not possible to provide quantitative estimates of

substantial differences in levels of a PSF. For example, for the PSF of

the quality of administrative control, the user will have to be content with rating this PSF as "good," "average," or "poor," making a subjective decision about the effect of this PSF on any particular task. Guidance is given for evaluating the effects of these types of PSFs, but considerable

judgment by the analyst will be required.

The UCBs (or EFs) for an HEP reflect the estimated range of variability in

performance attributable to differences in relevant PSFs, differences between and within people, differences in analysis, modeling uncertainty,

and uncertainty about the actual HEPs. The tabled UCBs are speculative; the analyst may wish to expand them to indicate greater uncertainty. The

tables list the EFs or UCBs for most of the HEPs, and Table 20-20 presents guidelines for estimating them for the other HEPs and for adjusting the

tabled UCBs for stress and type of task, e.g., dynamic rather than step-by-step, as defined in Table 20-16.

Search Scheme for Use of Chapter 20 Tables

A search scheme is presented in Figure 20-1 to aid the analyst in con-

sidering all tables of HEPs that he should consult in an HRA. This search scheme is organized according to the outline of a Technique for Human Error

Prediction (THERP) procedure for HRA, as presented in Figure 5-6 and dis- cussed in Chapter 5. The heavy lines in the search scheme represent the

paths of HRA activities we have most often employed in HRAs of NPP opera- tions. Ordinarily, the analyst will have completed an initial task analy-

sis and a set of first-cut HRA event trees before using the search scheme. He is now ready to assign HEPs to the failure limbs in the trees. The

search scheme uses the flowchart format to guide the analyst through the essential steps in the conduct of an HRA, indicating the appropriate tables

to which to refer at each stage of the analysis. It is assumed that if the ___

20-2

Figure 20-1 (1/3)

Figure 20-1 Search scheme for use of Chapter 20 tables (p 1 of 3).

20-3

Figure 20-1 (2/3)

Figure 20-1 Search scheme for use of Chapter 20 tables (p 2 of 3).

F L 0 w

D I R E C T 0 0) N

YES

Figure 20-1 Search scheme for use of Chapter 20 tables (p 3 of 3).

A' Talk-Through of the Search Scheme

analyst is directed to the appropriate table, he can select the item in the

table that most closely approximates the task and conditions being evalu- ated.' However, any tabled HEP may have to be modified according to plant- specific PSFs.

If the table to which the analyst is directed does not list an item that'

closely approximates the analysis task, he may select an item from some other table that matches the underlying behavioral processes identified in the task analysis. Alternatively, he may rely on judgment or seek other data sources. Some guidance is presented later, in the section entitled, "The Data Tables."

Figure 20-1 is presented here and also at the end of this chapter for the convenience of the analyst.

A Talk-Through of the Search Scheme

The search scheme in Figure 20-1 represents an iterative process, and the

analyst may enter the figure at any point in the logic. The ellipses represent reference points, the hexagons represent decision nodes, and the rectangles represent action items.

To illustrate the use of the search scheme, we will enter at the "Start" ellipse and proceed through a hypothetical, complete HRA of the type de- scribed in NUREG/CR-2254. Every table will be considered in the following sequence. This talk-through is, of course, generic. To illustrate appli-

cation of the search scheme for a specific sample HRA, see the first exam- ple problem in Chapter 21.

(1) ABNORMAL EVENT? This is the first decision node after "Start." Generally, the abnormal events of major interest in a HRA for a PRA

are loss-of-coolant accidents (LOCAs) and transients. If addressing a LOCA or transient, follow the YES path.

(2) SCREENING REQUIRED? As described in Chapter 5, this is the next decision node on the YES path. Screening involves the assignment of very high failure probabilities to each human task. If the very high HEPs do not have a material effect on the system analysis, the

task(s) may be dropped from further consideration. The decision as to whether screening is required will be made in conjunction with

the system analysts. Assume YES.

(3) -Screening values may be obtained for diagnostic performance and for subsequent rule-based actions (RBAs), using Tables 20-1 and 20-2.

(4) SENSITIVITY ANALYSIS OR END? For some purposes,-the analysis will

end with a screening analysis, or it may be followed by a sensitiv- ity analysis (SA). For either of these cases, follow the YES path. The "Go to SA" ellipse transfers the analyst to the bottom of page 3 of the figure, where he may perform a sensitivity analysis or exit from the flowchart. If postscreening HRA is required, follow the NO path. Assume NO.

20-6

A Talk-Through of the Search Scheme

(5) NOMINAL DIAGNOSIS REQUIRED? The nominal model for diagnostic per-

formance lists HEPs that are more realistic than the HEPs in the screening model. In most PRAs, the nominal HEPs for diagnostic performance are of interest. Assume YES.

(6) The HEPs for the nominal diagnosis model are listed in Table 20-3

and are used to estimate the probability of control room (CR) per- sonnel failing to properly diagnose one or more abnormal events

within the time constraints given by the system analysts.

(7) Table 20-4 lists the CR staffing assumptions as a function of time

after recognition of an abnormal event. These assumptions enable th& analyst to consider the effects of personnel interaction in

modifying the nominal HEPs for postevent activities (e.g., rule- based actions).

(8) RULE-BASED ACTIONS? Usually, RBAs will be evaluated in an HRA. Assume YES and go to the RBA ellipse.

(9) TYPE OF ERROR? This decision node does not have a YES/NO division. The section of the flowchart branching from this decision node and

reuniting at the PSF ellipse encompasses all the rule-based tasks usually addressed in an HRA. Tables 20-5 through 20-14 list the HEPs for all the rule-based tasks specified by the action rectangles in this section. The analyst will follow the appropriate path

through this section for each rule-based task being evaluated. In many HRAs, all the paths will be used. We will assume that this is the case for this HRA. All the paths flowing from the TYPE OR ERROR? hexagon will be considered before going to the "PSF" ellipse

to adjust the nominal HEPs for relevant PSFs. We will address errors of omission first.

(9A) WRITTEN MATERIALS? This decision node applies to whether written materials are mandated for the task. Written materials include

formal procedures, ad hoc procedures, and oral instructions that are

written down by the recipient as he receives them.

If YES, Tables 20-5, 20-6, and 20-7 list the HEPs for the prepar- ation of written materials, for the initiation of the task and

for the misuse of procedures, and for the omission of procedural items when using written materials. (Note that Table 20-5 in- cludes errors of commission as well as errors of omission, but for convenience is placed only in the OMISSION path from the TYPE

OF ERROR? hexagon.)

If NO, the worker is relying on memory. Table 20-6 provides the

HEPs for initiation of the task and Table 20-8 the HEPs in carry- ing out oral instructions as a function of the number of items to be remembered.

- Returning to the TYPE OF ERROR? hexagon, we will now consider errors of commission.

2C-7

A Talk-Through of the Search Scheme

(9B) INTERFACE TYPE? Displays, controls (including switches for motor- operated valves [MOVs]), and locally operated valves are the three types of man-machine interfaces studied in HRAs.

- For some frequently practiced tasks, the analyst may judge that the probabilities of errors of commission are negligible.. See the fourth example in Chapter 21.

- If DISPLAYS, the following tables list the HEPs for selection of displays (20-9), for reading and recording quantitative informa- tion from displays (20-10), and for noting the general state of displays (20-12).

- If CONTROLS or MOVs, Table 20-12 lists HEPs for selection and use of switches, connectors, and other manual controls.

- If LOCALLY OPERATED VALVES, Table 20-13 lists HEPs for selecting these valves, and Table 20-14 lists HEPs for recognizing that a valve is not fully open or closed because it sticks.

(10) Transfer to the "PSF" ellipse on page 2 of Figure 20-1. These rec- tangles list the PSFs that should be considered when evaluating the HEPs for RBAs. The nominal HEPs in any table may not accurately represent a plant-specific situation. Depending on the quality of PSFs observed, the nominal HEP may be raised or lowered by the analyst.

(10A) Table 20-15 indicates the modifiers to be applied to HEPs for chang- ing or restoring the normal states of safety-related components as a function of the tagging level in use. No modification of HEPs is required if the plant uses the usual Level 2 tagging system.

(10B) Table 20-16 lists modifiers to be applied to HEPs for different stress levels under which a task is to be performed, according to the experience level of the personnel on duty. If a task will be performed under different levels of stress at different times, or if different experience levels of personnel will be on duty at differ- ent times, the HRA event trees must represent such fractionation, as described in Chapter 5.

(10C) The "Other PSFs" rectangle is a reminder to consider the many other PSFs mentioned in the Handbook that are not listed in the tables. In addition, almost always there are plant-specific PSFs that the analyst will observe in the course of his site visits, which should be included at this point, using judgment to estimate their effects.

(10D) Tables 20-17, 20-18, and 20-19 present equations and tabled HEPs to be applied to the nominal HEPs to allow for the effects of different levels of dependence that may be assessed between tasks performed by

one person or for the effects of dependence between people working jointly. (Table 20-4 provides initial estimates of dependence among CR personnel in carrying out procedures after an abnormal event.)

A Talk-Through of the Search Scheme

(11) At this stage, the analyst following the HRA sequence shown in Figure 5-6 is ready to perform his first cut at quantifying the total-failure term, Pr[FT], for each HRA event tree. It is at this point in a PRA that certain human error terms may be dropped from further consideration if, as determined by the system analysts, they have no material impact on the system failure events of interest.

(12) UCBs NEEDED? If point estimates of HEPs without any UCBs are ade- quate, follow the NO path. Usually, the YES path will be followed:

- Table 20-20 provides guidelines for assigning UCBs (or EFs) to individual HEPs in the analysis. The upper and lower UCBs may be used as one form of SA, as described in Chapter 7.

- Table 20-21 provides UCBs for conditional HEPs based on use of the dependence model.

- Appendix A presents the methodology for propagation of UCBs through an HRA event tree so that UCBs may be assigned to the total-failure term, Pr[F _, for each HRA event tree. This termT plus its UCBs constitute the usual input to the system analyst for inclusion in the overall PRA.

(13) RECOVERY FACTORS? Usually recovery factors (RF) will be considered at this point in the HRA. Assume YES. Transfer to the top of page 3 of the search scheme to the "Recovery from Deviant Conditions" ellipse.

(14) CHECKING of ANOTHER'S WORK? The recovery factor from any deviant condition under normal operating conditions may depend on the direct checking of someone's work (the YES path) or on inspections of plant indications of deviant conditions. In an HRA, both paths are gen- erally followed. We will begin with the YES path.

(15) The YES path leads to Table 20-6, which provides HEPs for the ini- tiation of the task of the checker, and to Table 20-22, which lists HEPs for errors of omission and commission in the checker's task.

(16) The NO path leads to the ANNUNCIATED? hexagon. The recovery cues may be annunciated or unannunciated. We will address both modes.

(16A) If YES, the decision node, TYPE OF ERROR?, leads to one of two -tables:

- Table 20-23 presents the Annunciator Response Model listing the HEPs for an operator to initiate intended corrective action to one or more annunciators.

- Table 20-24 lists HEPs for remembering to respond to a steady-on annunciator tile after an interruption or for noticing an impor- tant steady-on annunciator tile during the initial audit or sub- sequent hourly scans.

20-9

List of Chapter 20 Data Tables

(16B) If NO, proceed to the decision node, SPECIAL STATUS CHECK OF IN- DIVIDUAL EQUIPMENT ITEMS? if certain displays are read according to

a schedule, or if the operator is otherwise directed to read some display, follow the YES path to the "RBA" ellipse on page 1 of the flowchart. If there is no specific requirement to check the status of individual equipment items, that is, the checking is more of. a general inspection, the NO path leads to four tables:

- Table 20-6 lists the HEP for initiation of a scheduled checking

or inspection function.

- Table 20-25 lists HEPs for detecting deviant unannunciated indi- cations on different types of displays during the initial audit

and on subsequent hourly scans.

- Table 20-26 modifies the HEPs from Table 20-25 when more than one (up to 5) displays are presenting deviant indications.

- Table 20-27 lists HEPs for failure of the basic walk-around in- spection to detect unannunciated deviant indications of equipment within 30 days.

(17) At this point, having considered all important recovery factors, the analyst will proceed to the "PSF" ellipse to consider modifications of the recovery HEPs by relevant PSFs. After the PSFs have been considered, follow the NO path from the RECOVERY FACTORS? decision node at the bottom of page 1 of the flowchart and proceed to the "SA" ellipse on page 3.

(18) SENSITIVITY ANALYSIS REQUIRED? The last thing done in a complete HRA is an SA, although it may be done at other times in the HRA also. The SA is important since it provides a means of ascertaining whether different assumptions or estimates result in materially different effects in the overall PRA. Assume YES.

(18A) As indicated in the rectangle, the analyst may use SA to modify any

assumptions or HEPs, following the procedure described in Chapters 5 and 7. He may then reenter the search scheme at any point to assess changes resulting from these modifications. Reentry will take him back to the "PSF" ellipse on page 2 of the flowchart and to the recalculation of the end-failure term, Pr[F T], using new values.

(18B) The search scheme will always take the analyst back to the SENSITIV-

ITY ANALYSIS REQUIRED? decision node on page 3 of the flowchart. When sufficient SA has been accomplished for purposes of the PRA, the NO path from this decision node leads to the "END" ellipse, signifying the completion of the HRA.

List of Chapter 20 Data Tables

The data tables from Part III that are repeated in this chapter are listed

below. Note that at the end of the title of each table, there appears in

20-10

List of Chapter 20 Data Tables

parentheses the table number in Part III to which the Chapter 20 table corresponds. This reference to Part III table numbers will enable the reader to quickly find background discussion of PSFs that does not appear in Chapter 20. For users familiar with the draft Handbook, Table F-2 in Appendix F provides a cross-index of the table numbers in the revised Chapter 20 with the table numbers from the same chapter in the draft Hand- book (Swain and Guttmann, 1980).

Ch. 20 Table No. Title of Table

20-1 Initial-screening model of estimated HEPs and EFs for diag- nosis within time T by control room personnel of abnormal events annunciated closely in time (from Table 12-2)

20-2 Initial-screening model of estimated HEPs and EFs for rule- based actions by control room personnel after diagnosis of an abnormal event (from Table 12-3)

20-3 Nominal model of estimated HEPs and EFs for diagnosis within time T by control room personnel of abnormal events annunci- ated closely in time (from Table 12-4)

20-4 Number of reactor operators and advisors available to cope with an abnormal event and their related levels of dependence: assumptions for PRA (from Table 18-2)

20-5 Estimated HEP per item (or perceptual unit) in preparation of written material (from Table 15-2)

20-6 Estimated HEPs related to failure of administrative control (from Table 16-1)

20-7 Estimated pr6babilities of errors of omission per item of instruction when use of written procedures is specified (from Table 15-3)

20-8 Estimated probabilities of errors in recalling oral instruc- tion items not written down (from Table 15-1)

20-9 Estimated probabilities of errors in selecting unannunciated displays for quantitative or qualitative readings (from Table 11-2)

20-10 Estimated HEPs for errors of commission in reading and record- ing quantitative information from unannunciated displays (from Table 11-3)

20-11 Estimated HEPs for errors of commission in checking-reading displays (from Table 11-4)

20-11

List of Chapter 20 Data Tables

Ch. 20 Table No. Title of Table

20-12 Estimated probabilities of errors of commission in operating

manual controls (from Table 13-3)

20-13 Estimated HEPs for selection errors for locally operated

valves (from Table 14-1)

20-14 Estimated HEPs in detecting stuck locally operated valves

(from Table 14-2)

20-15 The four levels of tagging or locking systems (from Table 16-2)

20-16 Modifications of estimated HEPs for stress and-experience levels (from Table 18-1)

20-17 Equations for conditional probabilities of success and failure on Task "N," given success or failure on preceding Task "N-i," for different levels of dependence (from Table 10-2)

20-18 Conditional probabilities of success or failure for Task "N"

for the five levels of dependence, given FAILURE on preceding Task "N-I" (from Table 10-3)

20-19 Conditional probabilities of success or failure for Task "N" for the five levels of dependence, given SUCCESS on preceding Task "N-i" (from Table 10-4)

20-20 Guidelines for estimating uncertainty bounds for estimated

HEPs (from Table 7-2)

20-21 Approximate CHEPs and their UCBs for dependence levels given FAILURE on the preceding task (from Table 7-3)

20-22 Estimated probabilities.that a checker will fail to detect errors made by others (from Table 19-1)

20-23 The Annunciator Response Model: estimated HEPs for multiple

annunciators alarming closely in time (from Table 11-13)

20-24 Estimated HEPs for annunciated legend lights (from Table 11-12)

20-25 Estimated probabilities of failure to detect one (of one) unannunciated deviant display at each scan, when scanned

hourly (from Table 11-7)

The Data Tables

Ch. 20 Table No. Title of Table

.20-26 Estimated probabilities of. failing to detect at least one of one to five unannunciated deviant displays as a function of the BHEP for detection of a single deviant display during periodic scanning (from Table 11-6)

20-27 Estimated probabilities that the basic walk-around inspection will fail to, detect a particular deviant indication of equip- ment outside the control room within 30 days (from Table 19-4)

The Data Tables

This section presents the 27 data tables extracted from Part III. To facilitate rapid. access to these tables, a table designator for each table is shown in large print in the outer upper corner of the page on which the table appears. The table designators are expressed without the chapter prefix (e.g., Table 20-6.is expressed as 6).

Figure 20-2, which-precedes the first table, is a quick reference guide to the tables, organized under the seven major headings that are used in the search scheme (.Figure 20-1). For convenience, Figure 20-2 also appears as the last page in Chapter 20.

We remind the user that the tables in this chapter do not stand alone. They must be considered in association with the descriptive material in those chapters that include the original versions of the tables. It is not possible to include all of the relevant PSFs in each table; the complete Handbook must be used.

Obviously, the tables cannot list every act or task that could take place in an NPP--only the most frequently observed tasks are listed. When a task is being evaluated for which we have no tabled HEPs, we assign a nominal HEP of .003 as a general error of omission or commission if we judge there is some probability of either type of error. When evaluating abnormal events, we assign a nominal HEP of *001 to those tasks for which the tables or text indicate that the HEP is "negligible" under normal conditions. The nominal HEP of .001 allows for the effects of stress that are associated with abnormal events.

Most of the tables list the EFs or UCBs for the HEPs. For cases in which the EFs or UCBs are not listed, Table 20-20 presents guidelines for esti- mating them. In the course of an SA, the nominal HEP for some task may change significantly as different assumptions are evaluated. Note that the EFs may change when a nominal HEP is changed; for example, under certain assumptions, some task may have a tabled HEP of, say, .008, with an EF of 3. If the assumptions are modified so that the HEP is doubled (to .016), the EF would change from 3 to 5 (see the second and third items in Table 20-20). Also remember that stress and other PSFs may increase the EFs, as indicated in Table 20-20.

20-13

Figure 20-2

Screening - Diagnosis [I Rule-Based Actions [-2]

Nominal Diagnosis [j Diagnosis -L Postevent CR Staffing 4.

Written Materials Mandated Preparation M Administrative Control jjJ

Errors of Omission Procedural Items

No Written Materials

Administrative Control R-1

Oral Instruction Items FJ

Displays Display Selection F Read/Record Quantitative 10ý

Check-Read Quantitative F11 Errors of Commission Control & MOV Selection & Use F121

Locally Operated Valves Valve Selection 13

Stuck Valve Detection 14-

Tagging Levels 15

PSFs Stress/Experience 16

Dependence 17 f18 19H Other PSFs (see text)

Estimate UCBs 20 Uncertainty Bounds "•'EConditional HEPs and UCBs 21

Errors by Checker 2

Annunciated Cues ý73 F2242 Recovery Factors - Control Room Scanning 99 26

Basic Walk-Around Inspection 27

Figure 20-2 Quick reference guide to Chapter 20 tables.

20- !4

The Data Tables

For record-keeping convenience in an HRA, the left-most column for most of the tables is headed by the word, "Item." In keeping a record of which tabled entries are used in an HRA, reference can be made to a particular table and item number, e.g., T20-1, #1. In some of the tables, e.g., Table 20-8, it is convenient to use small letters to designate separate columns of estimated HEPs. For example, in Table 20-8, Item la refers to the HEP of .001 (EF = 3), which is the top listing in the first column of HEPs. Record keeping for an HRA is illustrated in the first case study in Chapter 21.

20-15

THIS PAGE INTENTIONALLY LEFT BLANK

20-16

Table 20-1 Initial-screening model of estimated HEPs and

EFS for diagnosis within time T by control room personnel of abnormal events annunciated closely

in time* (from Table 12-2)

Median Median joint HEP for joint HEP

T diagnosis of T for diagnosis (Minutes** a single or (Minutes"* of the

Item after TO ) the first event EF Item after TO ) second event EF

(1) 1 1.0 -- (7) 1 1.0 --

(2) 10 .5 5 (8) 10 1.0 --

(3) 20 .1 10 (9) 20 .5 5

(4) 30 .01 10 (10) 30 .1 10

(11) 40 .01 10

(5) 60 .001 10

(12) 70 .001 10

(6) 1500 (= 1 day) .0001 30

(13) 1510 .0001 30

Note that this model pertains to the CR crew rather than to one individual

For points between the times shown, the medians and EFs may be chosen from Figure 12-3.

t To is a compelling signal of an abnormal situation and is usually taken as a pattern of annunciators. A probability of 1.0 is assumed for observing that there is some abnormal situation.

ttAssign HEP = 1.0 for the diagnosis of the third and subsequent abnormal events

annunciated closely in time.

20-17

2 Table 20-2 Initial-screening model of estimated HEPs and EFS for

rule-based actions by control room personnel after diagnosis of an abnormal event* (from Table 12-3)

Item Potential Errors HEP EF

Failure to perform rule-based actions correctly when written procedures are available and used:

(1) Errors per critical step without .05 10 recovery factors

(2) Errors per critical step with .025 10 recovery factors

Failure to perform rule-based actions correctly when written procedures are not available or used:

(3) Errors per critical step with or 1.0 without recovery factors

Note that this model pertains to the CR crew rather than to one individual.

20-18

0 Table 20-3 Nominal model of estimated HEPs and EFs for diagnosis

within time T by control room personnel of abnormal events annunciated closely in time* (from Table 12-4)

Median joint HEPt Median joint Median

for diagnosis HEPtt for joint HEPtt T of a single T diagnosis of T for diagnosis

(Minutes* * or the first (minutes** the second (Minutes** of the Item after T ) event EF Item after T ) event EF Item after T ) third event EF

0 0 0

(1) 1 1.0 -- (7) 1 1.0 -- (14) 1 1.0 --

(2) 10 .1 10 (8) 10 1.0 -- (15) 10 1.0 --

(3) 20 .01 10 (9) 20 .1 10 (16) 20 1.0 --

(4) 30 .001 10 (10) 30 .01 10 (17) 30 .1 10

(11) 40 .001 10 (18) 40 .01 10

(19) 50 .001 10

(5) 60 .0001 30

(12) 70 .0001 30

(20) 80 .0001 30

(6) 1500 .00001 30

(13) 1510 .00001 30

(21) 1520 .00001 30

Note that this model pertains to the CR crew rather than to one individual.

The nominal model for diagnosis includes the activities listed in Table 12-1 as "perceive," "discriminate," "in- terpret," "diagnosis," and the first level of "decision-making." The modeling includes those aspects of behavior included in the Annunciator Response Model in Table 20-23; therefore, when the nominal model for diagnosis is used, the annunciator model should not be used for the initial diagnosis. The annunciator model may be used for esti- mating recovery factors for an incorrect diagnosis.

For points between the times shown, the medians and EFs may be chosen from Figure 12-4.

ST0 is a compelling signal of an abnormal situation and is usually taken as a pattern of annunciators. A probability of 1.0 is assumed for observing that there is some abnormal situation.

%tTable 12-5 presents some guidelines to use in adjusting or retaining the nominal HEPs presented above.

4Table 20-4 Number of reactor operators and advisors available to cope with an abnormal event and their related levels of dependence: assumptions for PRA* (from Table 18-2)

.Time after recognition**

of an abnormal event

Operators or advisors handling reactr unit affected

Dependence levels with

others

I.tem (a) (b)

(1)

(2)

(3)

0 to 1 minute at 1 minute

at 5 minutes

on-duty RO

on-duty RO, SRO (assigned SRO or shift supervisor, an SRO)

on-duty RO, assigned SRO, shift supervisor-

1 or more AOs

high with RO

- - - - high with RO low to moderate with other operators

(4) at 15 minutes on-duty RO, assigned SRO, high with RO shift supervisor --------- low to moderate

with other operators

shift technical advisor- - - low to moderate with others for diagnosis & major events; high to complete for detailed operations

1 or more AOs

These assumptions are nominal and can be modified for plant- and situa- tion-specific conditions.

For PRA, "recognition" is usually defined as the response to a compelling signal, such as the alarming of one or more annunciators.

t No credit is given for additional operators or advisors (see text, Chapter 18).

t This column indicates the dependence between each additional person and those already on station. The levels of dependence are assumed to remain constant with time and may be modified in a plant-specific analysis.

* Availability of other AOs after 5 minutes and related levels of de- pendence should be estimated on a plant- and situation-specific basis.

20-20

5 Table 20-5 Estimated HEP per item (or perceptual unit) in

preparation of written material* (from Table 15-2)

Item Potential Errors HEP EF

(1) Omitting a step or important instruction from a formal .003 5 or ad hoc procedure** or a tag from a set of tags

(2) Omitting a step or important instruction from written Negligible notes taken in response to oral instructionst

(3) Writing an item incorrectly in a formal or ad hoc pro- .003 5 cedure or on a tag

(4) Writing an item incorrectly in written notes made in Negligible response to oral instructionst

Except for simple reading and writing errors, errors of providing incom- plete or misleading technical information are not addressed in the Handbook.

The estimates are exclusive of recovery factors, which may greatly reduce the nominal HEPs.

Formal written procedures are those intended for long-time use; ad hoc written procedures are one-of-a-kind, informally prepared procedures for some special purpose. t A maximum of five items is assumed. If more than five items are to be written down, use .001 (EF = 5) for each item in the list.

20-21

6 Table 20-6 Estimated HEPs related to failure of

administrative control (from Table 16-1)

Item Task HEP EF

(1) Carry out a plant policy or scheduled tasks .01 5 such as periodic tests or maintenance per- formed weekly, monthly, or at longer intervals

(2) Initiate a scheduled shiftly checking or .001 3 inspection function*

Use written operations procedures under

(3) normal operating conditions .01 3

(4) abnormal operating conditions .005 10

(5) Use a valve change or restoration list .01 3

(6) Use written test or calibration procedures .05 5

(7) Use written maintenance procedures .3 5

(8) Use a checklist properly** .5 5

Assumptions for the periodicity-and type of control room scans are discussed in Chapter 11 in the section, "A General Display Scanning Model." Assumptions for the periodicity of the basic walk-around- inspection are discussed in Chapter 19 in the section, "Basic Walk- Around Inspection."

Read a single item, perform the task, check off the item on the list. For any item in which a display reading or other entry must be written, assume correct use of the checklist for that item.

20-22

7 Table 20-7 Estimated probabilities of errors of omission per item of

instruction when use of written procedures is specified* (from Table 15-3)

Item** Omission of item: HEP EF

When procedures with checkoff, provisions are correctly used

(1) 'Short list, <10 items .001 3

(2) Long list, >10 items .003 3

When procedures without checkoff provisions are tf

used, or when checkoff provisions are incorrectly used

.(3) Short list, (10 items .003 3

(4) Long list, >10 items .01 3

(5) When written procedures are avail- .05 5tt able and should be used but are not used

The estimates for each item (or perceptual unit) presume zero dependence among the items (or units) and must be modified by using the dependence model when a nonzero level of dependence is assumed.

The term "item" for this column is the usual designator for tabled .entries and does not refer to an item of instruction in a procedure.

.Correct use of checkoff provisions is assumed for items in which written entries such as numerical values are required of the user.

#tTable 20-6 lists the estimated probabilities of incorrect use of checkoff

provisions and of nonuse of available written procedures.

If the task is judged to be "second nature," use the lower uncertainty bound for .05, i.e., use .01 (EF = 5).

20-23

8 Table 20-8 Estimated probabilities of errors in recalling oral

instruction items not written down* (from Table 15-1)

HEPs as a function of number of items to be remembered**

Number of Oral Instruction Items

or Perceptual Units

t Item

Pr[F] to recall item "N, " order of recall not important

Pr[F] to recall all items, order of recall not important

Pr[F] to recall all items, order of recall is important

(a) HEP EF

(b) HEP EF

Oral instructions are detailed:

(1) (2) (3) (4) (5)

2 3 4 5

.001 .003 .01 .03 .1

3 3 3 5 5

.001

.004

.02

.04

3 3 5 5

.001

.006

.03

3 3

5 5 5

Oral instructions are general:

(6)

(7) (8)

(9) (10)

.001 .006 .02 .06 .2

3 3 5 5 5

.001

.007

.03

.09

3 3 5 5 5

.001

.01

.06

5 5 5

*It is assumed that if more than five oral instruction items or perceptual

units are to be remembered, the recipient will write them down. If oral instructions are written down, use Table 20-5 for errors in preparation of written procedures and Table 20-7 for errors in their use.

**The first column of HEPs (a) is for individual oral instruction items,

e.g., the second entry, .003 (item 2a), is the Pr[F) to recall the second of two items, given that one item was recalled, and order is not im- portant. The HEPs in the other columns for two or more oral instruction items are joint HEPs, e.g., the .004 in the second column of HEPs is the Pr[F] to recall both of two items to be remembered, when order is not important. The .006 in the third column of HEPs is the Pr[F] to recall both of two items to be remembered in the order of performance specified. For all columns, the EFs are taken from Table 20-20 as explained in Chapter 15.

The term "item" for this column is the usual designator for tabled entries and does not refer to an oral instruction item.

*ttThe Pr[F]s in rows 1 and 6 are the same as the Pr[F] to initiate the

task.

Z0-2'4

9Table 20-9 Estimated probabilities of errors in selecting unannunciated displays for quantitative or qualitative readings (from Table 11-2)

Item Selection of Wrong Display: HEP* EF

(1) when it is dissimilar to adjacent displays** Negligible

(2) from similar-appearing displays when they are .0005 10 on a panel with clearly drawn mimic lines

that include the displays

(3) from similar-appearing displays that are part .001 3

of well-delineated functional groups on a panel

(4) from an array of similar-appearing displays .003 3

identified by labels only

The listed HEPs are independent of recovery factors. In some cases, the content of the quantitative or qualitative indication from an in-

correct display may provide immediate feedback of the selection error, and the total error can be assessed as negligible.

This assumes the operator knows the characteristics of the display for which he is searching.

20-25

10 Table 20-10 Estimated HEPs for errors of commission in reading and recording quantitative information from unannunciated displays (from Table 11-3)

Item Display or Task HEP* EF

(1) Analog meter .003 3

(2) Digital readout (< 4 digits) .001 3

(3) Chart recorder .006 3

(4) Printing recorder with large .05 5

number of parameters

(5) Graphs .01 3

(6) Values from indicator lamps .001 3 that are used as quanti- tative displays

(7) Recognize that an instrument .1 5 being read is jammed, if

there are no indicators to alert the user

Recording task: Number of

digits or letters** to be recorded

(8) < 3 Negligible -

(9) > 3 .001 (per 3

symbol)

(10) Simple arithmetic calcula- .01 3 tions with or without calculators

(11) Detect out-of-range .05 5 arithmetic calculations

Multiply HEPs by 10 for reading quantitative values under a

high level of stress if the design violates a strong popula- tional stereotype; e.g., a horizontal analog meter in which values increase from right to left.

In this case, "letters" refer to those that convey no mean- ing. Groups of letters such as MOV do convey meaning, and the recording HEP is considered to be negligible.

20-26

11 Table 20-11 Estimated HEPs for errors of commission in

check-reading displays* (from Table 11-4)

Item Display or Task HEP EF

(1)

(2)

(3)

(4)

Digital indicators (these must be read - there is no true check-reading function for digital displays)

Analog meters:

with easily seen limit marks

with difficult-to-see limit marks, such as scribe lines

without limit marks

Analog-type chart recorders:

with limit marks

without limit marks

Confirming a status change

on a status lamp

Misinterpreting the indi- cation on the indicator lamps

.001 3

.001

.002

.003

(5)

(6)

(7)

(8)

.002

.006

Negligible**

t Negligible

If operator must hold a switch in a spring-loaded position until a status lamp lights, use HEP = .003 (EF = 3), from Table 20-12, item 10.

tFor levels of stress higher than optimal, use .001 (EF = 3).

20-27

Table 20-12 Estimated probabilities of errors of commission2in operating manual controls* (from Table 13-3)

Item Potential Errors. HEP EF

(1) Inadvertent activation of a control see text, Ch. 13

Select wrong control on a panel *from an array of similar-appearing controls**:

(2) identified by labels only .003 3

(3). arranged in well-delineated functional groups .001 3

(4) which are part of a well-defined mimic layout .0005 10

Turn rotary control in wrong direction (for two- position switches, see item 8):

(5) when there is no violation of populational .0005 10

stereotypes

(6) when design violates a strong populational .05 5 stereotype and operating conditions are normal

(7) when design violates a strong populational .5 5 stereotype and operation is under high stress

(8) Turn a two-position switch in wrong direction or

leave it in the wrong setting

(9) Set a rotary control to an incorrect setting .001 10 t

(for two-position switches, see item 8)

(10) Failure to complete change of state of a .003 3 component if switch must be held until change is completed

Select wrong circuit breaker in a group of circuit breakers**:

(11) densely grouped and identified by labels only .005 3

(12) in which the PSFs are more favorable .003 3 (see Ch. 13)

(13) Improperly mate a connector (this includes .003 3 failures to seat connectors completely and failure to test locking features of connectors for engagement)

The HEPs are for errors of commission only and do not include any errors of decision as to which controls to activate.

**If controls or circuit breakers are to be restored and are tagged, adjust

the tabled HEPs according to Table 20-15. tDivide HEPs for rotary controls (items 5-7) by 5 (use same EFs).

tt This error is a function of the clarity with which indicator position can be determined: designs of control knobs and their position indications vary greatly. For plant-specific analyses, an EF of 3 may be used.

20-28

Table 20-13 Estimated HEPs for selection errors for locally 1:3

operated valves (from Table 14-1)

Item Potential Errors HEP EF

Making an error of selection in changing or restoring a locally operated valve when the valve to be manipulated is

(1) Clearly and unambiguously labeled, set. apart .001 3 from valves that are similar in all of the following: size and shape, state, and pres- ence of tags*

(2) Clearly and unambiguously labeled, part of .003 3 a group of two or more valves that are simi- lar in one of the following: size and shape, state, or presence of tags*

(3) Unclearly or ambiguously labeled, set apart .005 3 from valves that are similar in all of the' following: size and shape, state, and presence of tags*

(4) Unclearly or ambiguously labeled, part of a .008 3 group of two or more valves that are simi- lar in one of the following: size and shape, state, or presence of tags*

(5) Unclearly or ambiguously labeled, part of a .01 3 group of two or more valves that are simi- lar in all of the following: size and shape, state, and presence of tags*

Unless otherwise specified, Level 2 tagging is presumed. If other levels of tagging are assessed, adjust the tabled HEPs according to Table 20-15.

20-29

14 Table 20-14 Estimated HEPs in detecting stuck locally

operated valves (from Table 14-2)

Item Potential Errors HEP EF

Given that a locally operated valve sticks as it is being changed or restored,* the operator fails to notice the sticking valve, when it has

(1) A position indicator** only .001 3

(2) A position indicator** and a rising stem .002 3

(3) A rising stem but no position indicator** .005 3

(4) Neither rising stem nor position indicator** .01 3

Equipment reliability specialists have estimated that the probability of a valve's sticking in this manner is approxi- mately .001 per manipulation, with an error factor of 10.

20-30

Table 20-15 The four levels of tagging or locking systems (from Table 16-2) 15

Modifications

to Nominal Level Description HEPs*

2 Tags are not accounted for individually--the operator may take an unspecified number and use them as re- quired. In such a case, the number of tags in his possession does not provide any cues as to the number of items remaining to be tagged. For restoration, the record keeping does not provide a thorough check- ing for errors of omission or selection. If an operator is assigned as tagging controller, it is a collateral duty, or the position is rotated among operators too frequently for them to maintain ade- quate control tags &nd records and to retain skill in detecting errors of omission or selection. OR The shift supervisor retains control of the keys and records their issuance but does not use visual aids such as signout boards or tags.

4 No tagging system exists. OR No locks and keys are used.

Use lower UCBs

Use nominal HEPs

Use upper UCBs

Perform separate analysis

The nominal HEPs are those in the Handbook that relate to tasks involving the application and removal of tags and, unless otherwise specified, are based on Level 2 tagging.

20-31

Table 20-16 Modifications of estimated HEPs for the effects

of stress and experience levels (from Table 18-1)

Modifiers for Nominal HEPs* Stress Level Skilled** Novice**

Item (a) (b)

(1) Very low x2 x2 (Very low task load)

Optimum (Optimum task load):

(2), Step-by-stept xl xl t

(3) Dynamic xl x2

Moderately high (Heavy task load):

(4) Step-by-stept x2 x4

(5) Dynamic x5 x10

Extremely High (Threat stress).t

(6) Step-by-step x5 xl0

(7) Dynamic %% .25 (EF = 5) .50 (EF = 5) Diagnosis These are the actual HEPs to use

with dynamic tasks or diagnosis-- they are NOT modifiers.

The nominal HEPs are those in the data tables in Part III and in Chapter 20. Error factors (EFs) are listed in Table 20-20.

tStep-by-step tasks are routine, procedurally guided tasks, such as carry- ing out written calibration procedures. Dynamic tasks require a higher degree of man-machine interaction, such as decision-making, keeping track of several functions, controlling several functions, or any combination of these. These requirements are the basis of the distinction between step-by-step tasks and dynamic tasks, which are often involved in re- sponding to an abnormal event.

ttDiagnosis may be carried out under varying degrees of stress, ranging

from optimum to extremely high (threat stress). For threat stress, the HEP of .25 is used to estimate performance of an individual. Ordinarily, more than one person will be involved. Tables 20-1 and 20-3 list joint HEPs based on the number of control room personnel presumed to be involved in the diagnosis of an abnormal event for various times after annunciation of the event, and their presumed dependence levels, as presented in the staffing model in Table 20-4.

20-32

Table 20-17 Equations for conditional probabilities of success and failure on Task "N," given success or failure on previous Task "N-1," for different levels of dependence (from Table 10-2)

Level of Dependence Success Equations Equation No. Failure Equations Equation No.

LD •0

Pr[s, . Is,._1 I ZD]

Pr([S.NI, I " .I LD]

Pr[S,,N. IS.N_1, IMD]

Pr[S,,N,, I S,,N_ I, HD]

Pr([S,,, I S,,N_ 1 CD]

1 + 19n 20

1 + 6n 7

1+ n 2

= 1.0

(10-9)

(10-10)

(10-11)

(10-12)

(10-13)

Pr [F,,N,, I F,,N-1, I ZD]

Pr (F,,N, I F,,N-111 ILD]

Pr[F IF I N ND]

Pr [F,,N,, IFos N 1.IHD]

Pr[F I F,,N- 1, 1CD]

1 + 19N

1 + 6N 7

1+ N 2

= 1.0

(10-14)

(10-15)

(10-16)

(10-17)

(10-18)

Table 20-18 Conditional probabilities of success or failure for Task "N" for the five levels of dependence, given FAILURE on preceding Task "N-i" (from Table 10-3)

Task "N" Conditional Probabilities*

tj 0 U

I tem

(1)

(2)

(3)

(4)

(5)

(6)

(7)

(8)

(9)

ZD**

S F

(a) (b)

.75 .25

.9 .1

.95 .05

.99 .01t

.995 .005

.999 .001

.9995 .0005

.9999 .0001

.99999 .00001

S F

.71 .29

.85 .15

.9 .1

.94 .06

.95 .05

(e)

.64

.77

.81

.85

.86

(f)

.36

.23

.19

.15

.14

(g)

.37

.45

.47

.49

.50

(h)

.63

.55

.53

.51

.50

(i)

1.0

HD CD

All conditional probabilities are rounded. Equations 10-14 through 10-18 (Table used to calculate the values in the F columns. The values in the S columns were subtraction.

The conditional probabilities given ZD are the basic probabilities for Task "N."

For PRA purposes, it is adequate to use CHEPs of .05 (for LD), .15 (for MD), and when BHEP 4 .01.

20-17) were obtained by

.5 (for HD)

Table 20-19 Conditional probabilities of success or failure for Task "N" for the five levels of dependence, given SUCCESS on preceding Task "N-I" (from Table 10-4)

Task "N" Conditional Probabilities*

I tem

(1)

(2)

(3)

(4)

(5)

(6)

(7)

(8)

(9)

ZD**

S F

(a) (b)

.75 .25

.9 .1

.95 .05

.99 .01

.995 .005

.999 .001

.9995 .0005

.9999 .0001

.99999 .00001

S F

.76 .24

.9 .1

.95 .05

.99 .01

.995 .005

.999 .001

.9995 .0005

.9999 .0001

.99999 .00001

S F

(e) (f)

.79 .21

.91 .09

.94 .06

.991 .009

.996 .004

.999 .001

.9996 .0004

.99991 .00009

.999991 .000009

S F

(g) (h)

.87 .13

.95 .05

.97 .03

.995 .005

.997 .003

.9995 .0005

.9997 .0003

.99995 .00005

.999995 .000005

S F

10 0i)

1.0 0

*All conditional probabilities are rounded. Equations 10-9 through 10-13 (Table 20-17) were used to calculate the values in the S columns. The values in the F columns were obtained by subtraction.

**The conditional probabilities, given ZD, are also the basic probabilities for Task "N."

__L

20 Table 20-20 General guidelines for estimating uncertainty bounds

for estimated HEPs* (from Table 7-2)

t Item Task and HEP Guidelines** EF

Task consists of performance of step-by-step procedure%% con- ducted under routine circumstances (e.g., a test, maintenance, or calibration task); stress level is optimal:

(1) Estimated HEP < .001 10

(2) Estimated HEP .001 to .01 3

(3) Estimated HEP > .01 5

Task consists of performance of step-by-step procedure % but carried out in nonroutine circumstances such as those involving a potential turbine/reactor trip; stress level is moderately high:

(4) Estimated HEP < .001 10

(5) Estimated HEP ) .001 5

Task consists of relatively dynamic interplay between operator and system indications, under routine conditions, e.g., increas- ing or reducing power; stress level is optimal

(6) Estimated HEP < .001 10

(7) Estimated HEP > .001 5

(8) Task consists of relatively dynamic interplay between operator 10 and system indications but carried out in nonroutine circum- stances; stress level is moderately high

The estimates in this table apply to experienced personnel. The perfor- mance of novices is discussed in Chapter 18.

For UCBs for HEPs based on the dependence model, see Table 20-21. tThe highest upper bound is 1.0.

See Appendix A to calculate the UCBs for Pr[F T, the total-failure term of an HRA event tree.

See Table 20-16 for definitions of step-by-step and'dynamic procedures.

2C-3C

21 Table 20-21 Approximate CHEPs and their UCBs for dependence levels*

given FAILURE on the preceding task (from Table 7-3)

Levels of

Dependence BHEPs

Item (a) (b) (c)

(1) ZD** ( .01 .05 (EF=5) .1 (EF=5)

(d) (e) (f)

.15 (EF=5) .2 (EF=5) .25 (EF=5)

Levels of t

Dependence Nominal CHEPs and (Lower to Upper UCBs)

Item (a) (b) (c)

(2) LD .05 (.015 to .15) .1 (.04 to .25) .15 (.05 to .5)

(3) MD .15 (.04 to .5) .19 (.07 to .53) .23 (.1 to .55)

(4) HD .5 (.25 to 1.0) .53 (.28 to 1.0) .55 (.3 to 1.0)

(5) CD 1.0 (.5 to 1.0) 1.0 (.53 to 1.0) 1.0 (.55 to 1.0)

(d) (e) (f)

(2) LD .19 (.05 to .75) .24 (.06 to 1.0) .29 (.08 to 1.0)

(3) MD .27 (.1 to .75) .31 (.1 to 1.0) .36 (.13 to 1.0)

(4) HD .58 (.34 to 1.0) .6 (.36 to 1.0) .63 (.4 to 1.0)

(5) CD 1.0 (.58 to 1.0) 1.0 (.6 to 1.0) 1.0 (.63 to 1.0)

Values are rounded from calculations based on Appendix A. All values are based on skilled personnel (i.e., those with )6 months experience on the tasks being analyzed.

ZD = BHEP. EFs for BHEPs should be based on Table 20-20.

t Linear interpolation between stated CHEPs (and UCBs) for values of BHEPs

between those listed is adequate for most PRA studies.

20-37

22 Table 20-22 Estimated probabilities that a checker will fail to

detect errors made by others* (from Table 19-1)

Item Checking Operation HEP EF

(1) Checking routine tasks, checker using written . 5

materials (includes over-the-shoulder inspections, verifying position of locally operated valves, switches, circuit breakers, connectors, etc., and checking written lists, tags, or procedures for accuracy)

(2) Same as above, but without written materials .2 5

(3) Special short-term, one-of-a-kind checking with .05 5 alerting factors

(4) Checking that involves active participation, such as .01 5 special measurements

Given that the position of a locally operated valve .5 5 is checked (item 1 above), noticing that it is not completely opened or closed:

(5) Position indicator** only .I 5

(6) Position indicator** and a rising stem .5 5

(7) Neither a position indicator** nor a rising stem .9 5

(8) Checking by reader/checker of the task performer in .5 5

a two-man team, or checking by a second checker, routine task (no credit for more than 2 checkers)

(9) Checking the status of equipment if that status .001 5 affects one's safety when performing his tasks

(10) An operator checks change or restoration tasks Above 5 performed by a maintainer HEPs

This table applies to cases during normal operating conditions in which a person is directed to check the work performed by others either as the work is being performed or after its completion.

20-36

r'J

Table 20-23 The Annunciator Response.Model: estimated HEPs* for multiple annunciators alarming closely in time** (from Table 11-13)

Number Pr[F.] for each annunciator (ANN) (or completely dependent set of of AANs) successively addressed by the operator ___Pr[F.J*

ANNs 1 2 3 4 5 6 7 8 9 10 I

Item (a) (b) (c) (d) (e) (f) (g) (h) (i) Ci) (k) (1) 1 .0001 -------------------------------------------------------------------------. 0001

(2) 2 .0001 .001 -,---------------------------------------------------------------.0006

(3) 3 .0001 .001 .002 ----------------- -------------------. 001

(4) 4 .0001 .001 .002 .004 --------------------------------------------------. 002

(5) 5 .0001 .001 .002 .004 .008 ------------------------------------------. 003

(6) 6 .0001 .001 .002 .004 .008 .016 ----------------------------------. 005

(7) 7 .0001 .004 .002 .004 .008 .016 .032 --------------------------. 009

(8) 8 .0001 .001 .002 .004 .008 .016 .032 .064 ------------------. 02

(9) 9 .0001 .001 .002 .004 .008 .016 .032 .064 .13 -------. 03

(10) 10 .0001 .001 .002 .004 .008 .016 .032 .064 .13 .25 - .05

(11) 11-15 .10

(12) 16-20 .15 Pr[F.] for each additional ANN beyond 10 = .25

(13) 21-40 .20

(14) >40 .25

The HEPs are for the failure to initiate some kind of intended corrective action as required. The action carried out may be correct or incorrect and is analyzed using other tables. The HEPs include the effects of stress and should not be increased in consideration of stress effects.

EF of 10 is assigned to each Pr[F.] or Pr[F.]. Based on computer simulation, use of an EF of 10 for

Pr[Fi] yields approximately correct upper bounds for the 95th percentile. The corresponding lower

bounds are too high; they are roughly equivalent to 20th-percentile rather than the usual 5th-percen-

tile bounds. Thus, use of an EF of 10 for the mean Pr[Fi] values provides a conservative estimate

since the lower bounds are biased high.

"Closely in time" refers to cases in which two or more annunciators alarm within several seconds or within a time period such that the operator perceives them as a group of signals to which he must selectively respond.

Pr[Fi] is the expected Pr[F] to initiate action in response to a randomly selected ANN (or completely

dependent set of ANNs) in a group of ANNs competing for the operator's attention. It is the arithmetic

mean of the Pr[F.]s in a row, with an upper limit of .25. 1

24 Table 20-24 Estimated HEPs for annunciated legend lights*

(from Table 11-12)

Item Task HEP EF

(1) Respond** to one or more annunciated See Table 20-23 legend lights

(2) Resume attention to a legend light .001 3 within 1 minute after an inter- ruption (sound and blinking cancelled before interruption)

(3) Respond to a legend light if more .95 5 than 1 minute elapses after an interruption (sound and blinking cancelled before interruption)

(4) Respond to a steady-on legend .90 5 light during initial audit

(5) Respond to a steady-on legend light during other hourly scans .95 5

No written materials are used.

20-40

25 Table 20-25 Estimated probabilities of failure to detect one

(of one) unannunciated deviant display* at each scan, when scanned hourly** (from Table 11-7)

(Initial t Audit) Hourly Scans

Display Type 1 2 3 4 5 6 7 8

Item (a) (b) (c) (d) (e) (f) (g) (h)

Analog meters:

(1) with limit marks .05 .31 .50 .64 .74 .81 .86 .90

(2) without limit marks .15 .47 .67 .80 .87 .92 .95 .97

Analog-type chart recorders:

(3) with limit marks .10 .40 .61 .74 .83 .89 .92 .95

(4) without limit marks .30 .58 .75 .85 .91 .94 .97 .98

(5) Annunciator light no .9 .95 .95 .95 .95 .95 .95 .95 longer annunciating

(6) Legend light other .98 .98 .98 .98 .98 .98 .98 .98 than annunciator light

(7) Indicator lamp .99 .99 .99 .99 .99 .99 .99 .99

"One display" refers to a single display or a group of completely dependent displays, i.e., a perceptual unit.

For error factors, refer to Table 20-20.

tWritten materials not used.

tl*These displays are rarely scanned more than once per shift, if at all.

Hourly HEPs for each are listed for completeness only.

20-41

26 Table 20-26 Estimated probabilities of failing to detect at least one*

of one to five unannunciated deviant displays as a function of the BHEP for detection of a single deviant display during periodic scanning** (from Table 11-6) 0

Number of Deviant Indications

1 2 3 4 5

BHEP Pr[F] to detect at least one deviant displayt

I tem (a) (b) (c) (d) (e)

(1)

(2)

(3)

(4)

(5)

(6)

(7)

(8)

(9)

(10)

(11)

(12)

(13)

.99

.95

.90

.80

.70

.60

.50

.40

.30

.20

.10

.05

.01

.985

.93

.85

.72

.59

.48

.37

.28

.19

.12

.05

.03

.005

. 98

.90

.81

.65

.51

.39

.28

.20

.13

.07

.03

.01

.003

.975

.88

.77

.58

.43

.31

.21

.14

.08

.04

.02

.007

.001

.97

.86

.73

.52

.37

.25

.16

.10

.05

.03

.01

.004

.001

To estimate the HEP for failure to detect other concurrent unannunciated deviant displays when one has been detected,

use the HEP for the initial audit for those displays that are not functionally related to the display detected (from

Table 20-25) and use the annunciator response model for those displays that are functionally related to the dis- play detected (from Table 20-23). The HEPs apply when no written materials are used.

Except for column (a), the entries above are the com- plements of the entries in Table 11-5. For EFs, refer to Table 20-20.

20-42

27 Table 20-27 Estimated probabilities that the basic walk-around

inspection* will fail to detect a particular deviant indication of equipment outside the control room within 30 days** (from Table 19-4)

Number of days Cumulative Pr[F] between within 30 days

walk-arounds given one Item per inspector inspection per shifttf

(1) 1 (daily walk-around .52 for each inspector.)

(2) 2 .25

(3) 3 .05

(4) 4 .003

(5) 5 .0002

(6) 6 .0001

(7) 7 (weekly walk-around .0001 for each inspector)

See Chapter 19 for the assumptions for the basic walk-around in- spection. One of these assumptions is that no written procedure is used; if a written procedure is used for a walk-around, use the tables related to errors of omission and commission for perfor- mance of rule-based tasks (Figure 20-1, p 1).

Three shifts per day are assumed. If not, use the appropriate

equations in Chapter 19.

It is assumed that all inspectors have the same number of days between walk-arounds. For other assumptions, modify the relevant equations in Chapter 19.

For EFs, use the procedure in Appendix A, or use EF = 10 as an approximation.

20-43

Figure 20-1 (1/3)

Figure 20-1 Search scheme for use of Chapter 20 tables (p 1 of 3).

Figure 20-1 (2/3)

Figure 20-1 Search scheme for use of Chapter 20 tables (p 2 of 3).

20-45

Figure 20-1 (3/3)

F L 0 w D

R E C T

01N•

YES

Figure 20-1 Search scheme for use of Chapter 20 tables (p 3 of 3).

2C -,4

Figure 20-2

Screening

Diagnosis

-- Diagnosis W -- Rule-Based Actions 2

Nominal Diagnosis jjý E Postevent CR Staffing wT

Errors of Omission

- Written Materials Mandated Preparation [5]

Administrative Control Procedural Items F

- No Written Materials

Administrative Control Oral Instruction Items

FE61 FE81

- I - -

D-splays Display Selection F-]

Read/Record Quantitative

Check-Read Quantitative Control & MOV Selection & Use 12 Locally Operated Valves

Valve Selection Ft3 Stuck Valve Detection

Tagging Levels 15

PSFs ____Stress/Experience F16

Dependence FTJ 8- 19 Other PSFs (see text)

~ Estimate UCBs 20 Uncertainty Bounds LConditional HEPs and UCBs F251

Errors by Checker 22

Annunciated Cues F2 4 Recovery Factors Control Room Scanning 25 26

Basic Walk-Around Inspection 27

Figure 20-2 Quick reference guide to Chapter 20 tables.

Ch. 21. Examples and Case Studies

Overview

CHAPTER 21. EXAMPLES AND CASE STUDIES

Overview

This chapter presents examples of the application of some of the human performance models, estimated human error probabilities (HEPs), and uncer- tainty bounds (UCBs) (or error factors [EFs]) used in human reliability analysis (HRA). Although several such examples appear throughout the Handbook, the ones in this chapter illustrate some different applications and are presented in some detail so that the user can follow the various rationales we employed in actual studies.

Five studies are described:

(1) A calculation of failure to initiate steam generator feed-and-bleed procedures following a loss of both normal and emergency feed to the steam generators in a pressurized water reactor (PWR)

(2) A recalculation of a WASH-1400 HRA using HEPs from the Handbook

(3) An evaluation of the effectiveness of the loss-of-coolant accident (LOCA) procedures at a plant

(4) A case study of the availability of auxiliary feedwater at plants where manual switchover from main to auxiliary feedwater was required

(5) A retrospective "prediction" of the probability of detecting the two status lamps at Three Mile Island Unit 2 (TMI), which indicated the unavailability of auxiliary feedwater

In addition to these studies, several other applications of the method-

ology, models, and derived data in the Handbook are presented in the companion document to the Handbook, NUREG/CR-2254 (Bell and Swain, 1983). The examples in the appendix to NUREG/CR-2254 have been used in training courses in HRA conducted for the Nuclear Regulatory Commission (NRC) and for other organizations. We believe that if one works out these examples plus those in the present chapter, he will have a much better understanding of HRA as a part of probabilistic risk assessment (PRA).

Most of the HEPs used in the examples are taken from the tables in Chapter 20, but some estimates are based solely on expert judgment because the tables do not include all the HEPs required for every PRA. In the recalcu- lation of some WASH-1400 estimates, our new modeling of diagnosis results in a relatively large change in the end probability of the critical set of human errors.

Problem 1: rFailure To Initiate Steam Generator Feed-and-Bleed Procedures

This hypothetical example is used to illustrate the HRA event tree, the assignment of HEPs from Chapter 20 tables, and the use of several of the

.Z/ -I

Problem 1: Failure To Initiate Steam Generator Feed-and-Bleed Procedures

human performance models from Part III. The HRA is first worked out in the manner described in Chapter 5, and then the use of the search scheme shown in Figure 20-1 is illustrated.

Background and Assumptions

In Chapter 5, we presented an HRA event tree (Figure 5-3) showing the major human activities and errors in the establishment of the steam generator (SG) feed-and-bleed procedure following a loss of both normal and emergency feed to both SGs associated with a PWR. The loss of all feedwater results in failure to maintain cooling of the nuclear reactor core unless the operating crew establishes feed and bleed.

The present example provides a more detailed analysis of the example given in Chapter 5 and makes some changes to illustrate use of Handbook models and HEPs. Therefore, the reader should not try to draw parallels between the two examples; the one in Chapter 5 was merely to illustrate the mecha- nics of the HRA event tree and the arithmetic associated with calculating Pr[F T], the total-failure term for the tree.

Two of the major changes in the present example are (1) the use of the nominal diagnosis model in Chapter 12 and (2) the inclusion of the shift technical adviser (STA) as a recovery factor for operator errors. We use the staffing and dependence assumptions made in Table 20-4 in which one reactor operator (RO) only is present in the control room in the first minute after the annunciation of the event, a second operator (a senior reactor operator [SRO]) is present after the first minute, the shift super- visor (SS), also an SRO, is present after 5 minutes, and the STA is present at 15 minutes. Table 20-4 provides assumptions for levels of dependence among these personnel: high dependence (HD) between the SRO and the RO, low to moderate dependence (LD to MD) between the SS and the other opera- tors, and low to complete dependence (LD to CD) between the STA and the licensed operators, depending on the nature of the task. For convenience, we will assume that the RO on duty will remain the primary operator,

assisted or advised by the SRO or SS as necessary and advised by the STA for certain major functions. This assumption is an important one. If one assumed that the SRO would take over from the on-duty operator, the depen- dence assumptions might change. For example, if the SRO takes over from the on-duty operator, HD might be assessed as the appropriate level of interaction between this RO and the SRO. For purposes of this example, we will assume that such is not the case and that the SRO is interacting with the on-duty operator, who remains at his station. We will use the depen- dence equations from Table 20-17 to calculate conditional HEPs due to dependence between people; whereas, in a real analysis, we would use the approximate values based on Table 20-21.

The following assumptions also apply to this example:

(1) All automatic safety systems function properly.

21 -2

Problem 1: Failure To Initiate Steam Generator Feed-and-Bleed Procedures

(2) High-pressure injection (HPI) must be initiated within 30 minutes of loss of SG feed, or core melt will begin. Core melt is the system failure event.

(3) A task analysis has established that the abnormal event (loss of all feedwater) must be diagnosed within 20 minutes to allow time to carry out the feed-and-bleed procedure.

(4) The task analysis has also established that some annunciations provide potential recovery cues from a misdiagnosis as well as from failure to carry out critical follow-on actions.

(5) Operators and other technical personnel have provided estimates of the number of annunciators alarming at any time during the abnormal event, and also of the possibility of grouping two or more annunciators into one perceptual unit.

(6) The operators are well versed in carrying out the feed-and-bleed procedures, so that errors of commission are negligible. The only errors to be considered are errors of omission.

(7) The task analysis has established that no adjustments should be made to the nominal HEPs in the Handbook for diagnosis and subsequent rule-based actions (RBAs).

In association with system analysts, it has been determined that the criti- cal human errors are:

(1) Failure to diagnose the event correctly within 20 minutes, which includes failure to respond appropriately to an annunciator that warns of saturation of the pressurizer.

(2) Failure to perform steps 2.4, 2.5, and 2.6 of the "immediate actions" and 3.1 of the "follow-up actions" from a written list of procedures, parts of which are shown in Figure 21-1. Steps 2.5 and 2.6 have annunciator recovery factors. For analysis purposes, the plant written procedures constitute a long list without checkoff provisions, as listed in Table 20-7, item 4.

The Analysis

Figure 21-2 presents the HRA event tree based on the background information and assumptions and on a task analysis. Table 21-1 explains each symbol in the figure and provides a summary of the task analysis so that the reader can see how the HEPs for each failure limb were derived.

Of the five failure paths in the HRA event tree in Figure 21-2, one failure path (F 2 ) is clearly dominant. The other paths do not contribute materi- ally to the overall failure probability, FTo of 1.6 x 10 3. In a real PRA, a dominant path such as F 2 , with only one error and not many recovery

21-3

Figure 21-1

Relevant steps from "Immediate Actions":

2.4 Verify that the steam-driven and electric emergency feedwater pumps start.

CAUTION

P-7B HAS A TIME DELAY OF =100 SECONDS. DO NOT MANUALLY START PUMP UNTIL ALL ES LOADS, IF REQUIRED, ARE SEQUENCED TO BUS A-3.

2.5 Verify that the emergency feedwater block valves open; or open their

bypass valves. Verify emergency feed flow to OTSGs.

2.6 Ifneither EFW nor normal feedwater is available, follow Section III of this procedure.

Relevant steps from "Follow-Up Action Sections":

2.8 Monitor margin to saturation; increase RCS pressure to maintain >50*F subcooling.

CAUTION

IF MARGIN TO SATURATION CANNOT BE MAINTAINED >100F, OR IF RCS TEMPERA- TURE GOES OFF-SCALE HIGH, INITIATE HPI COOLING PER FOLLOW-UP ACTIONS.

3.1 If margin to saturation cannot be maintained >10 0 F, or if RCS tem- perature goes off-scale high, initiate HPI cooling as follows:

3.1.1 Open BWST outlets CV-1407 and CV-1408.

3.1.2 Open all HPI MOVs.

3.1.3 Start the standby ES makeup pump.

Etc.

Key to Abbreviations in above procedures:

P-TB - Pump 7B RCS - reactor coolant system ES - emergency system BWST - borated water storage system

OTSG - once-through steam generators CV - control valve EFW - emergency feedwater MOV - motor-operated valve

Note: At this plant, "verify" includes performing the operation if necessary.

Figure 21-1 Hypothetical plant procedures related to Problem 1.

21'-4

Figure 21-2

.99 = a, A1 = .01

A2 = .0006

.98 = bI

•A 3 = .5

.98 = c 1 A 4 = .14

(3 ' 2 3 c 3 =.16(g A =•=1.0OlO.0 C cX lC3l.98=d , D1= 02 e C4 C 4 =1.0 b4 F I 4 10 -5 D 2 =.51 3 C

C 5 o.0001

4 4 d 3 D3 = .16 3 C6 C6 =.5 F2 1.6 x10

9 9 9 el El =.0014d4 D=103 7 C=.4 d7 D C 8 1.0

E2 4 d6 6= 5 3

S3 .0

F5 << 10-5

FT = F 1 + F2+ F3 + F4 + F5

= AIA 2 A3 A4 A5 + alBIB2 B3 B4 + alblC1 C2 C3 C4 C5 C6 C 7C 8

+ alblclDID2 D3 D4 DsD 6 D7 D8 + alblCldlEIE 2 E3 E4

= ( 10-5) + (1.6 x 10-3) + (4 10-6) + (10-5) + ( 10"5)

-1. x 10-3

Note: Except for the terminal success limbs e 2 , e 3 , and e4, all success limbs following a failure limb lead to the complete success path, e.g., all success limbs ending in ( lead to the C on the com- plete success path.

Figure 21-2 HRA event tree for Problem 1.

Table 21-1

Table 21-1 Explanation of terms in Figure 21-2 (p 1 of 4)

Failure Limb & Estimated HEP UCB or EF

(Person) and Source and Source

A1 .01 10 (all 4) T20-3, #3 T20-3, #3

Control room personnel fail to correctly diagnose the abnormal event within 20 minutes after annunciation, based on presence of RO, SRO, SS, and STA.

A2 .0006 10 (RO) T20-23, #2k T20-23, first footnote

RO fails to respond to annunciator cues of the misdiagnosis. A potential recovery factor for the false diagnosis is an annun- ciator (ANN) that shows that the pressurizer has reached the boiling point, i.e., the saturation point. Interviews have established that if the saturation alarm comes on and is properly noted, the probability of correct rediagnosis approaches 1.0. Interviews have also determined that at this time into the event, approximately six ANNs will be annunciating, of which three relate to the situation. These three ANNs constitute one perceptual unit. This means that if any of the three are responded to, it is equivalent to responding to all three; the RO will recognize the saturation condition and will realize that the initial diagnosis was incorrect. Using the ANN response model, three out of six ANNs converts to the probability of noticing one of two ANNs.

A3 .5 .25 to 1.0 (SRO) T20-17, Eq. 10-17 T20-21, #4a

SRO fails to respond to the annunciator cues. Per Table 20-4, #2b, HD is assessed for the second operator, the SRO.

A4 .14 .04 to .5 (SS) T20-17, Eq. 10-16 T20-21, #3a

SS fails to respond to the annunciator cues. Per Table 20-4, #3b, MD is assessed for the third person, the SS.

A5 1.0 (STA) T20-17, Eq. 10-18 --

STA fails to respond to the annunciator cues. The STA will not pay attention to the saturation ANNs since this is a "detailed operation" (Table 20-4, #4b), and he believes that a correct diagnosis has been made. (If less than CD were assessed, the value of F would not change materially.)T

21-6

Table 21-1

Table 21-1 Explanation of terms in Figure 21-2 (p 2 of 4)

Failure

Limb & Estimated HEP UCB or EF (Person) and Source and Source

B1 (RO)

.01 x2

.02

T20-7, #4 T20-16, #4a

5 T20-20, #5

RO fails to carry out step 2.4 in the "Immediate Actions" (Figure 21-1). Once a correct diagnosis has been made, the RO must carry out step 2.4 in a written procedure constituting a long list without a checkoff provision, so the BHEP for omitting the step is .01. This value is doubled to reflect the effects of a moderately high level of stress. Table 20-20, #5, lists an EF of 5 for step-by-step tasks when performed in nonroutine circumstances and the BHEP > .001.

B2 (SRO)

.51 T20-17, Eq. 10-17

.26 to 1 .0 T20-21 , interpolation between #4a and b

SRO fails to correct RO's error, given HD.

B3 (SS)

.16 T20-17, Eq. 10-16

.05 to .51 T20-21, interpolation between #3a and b

SS fails to correct RO's error, given MD.

1.0 T20-17, Eq. 10-18 --

B4 (STA)

The STA would not T20-4, #4b.

be involved in this procedural detail, per

Cl (RO)

.02 (same analysis as BI)

C2 (SRO)

C3 (SS)

C4 (STA)

RO fails to carry out step 2.5 in the "Immediate Actions" (Figure 21-1)

.51 .26 to 1.0

(same analysis as B2 )

.16 .05 to .51 (same analysis as B3 )

1.0 (same analysis as BO4)

21-7

Table 21-1

Table 21-1 Explanation of terms in Figure 21-2 (p 3 of 4)

Failure

Limb & Estimated HEP UCB or EF

(Person) and Source and Source

C5 (RO)

.0001 T20-23, #1k

10 T20-23, first footnote

RO fails to respond to the loss-of-feed ANN. Interviews establish

that when this ANN comes on, it should be the only one sounding at the time.

C6 (SRO)

.5 T20-17, Eq. 10-17

.25 to 1 .0 T20-21 , #4a

SRO fails to note the loss-of-feed ANN.

C7 (SS)

.14

T20-17, Eq. 10-16

.04 to .5

T20-21 , #3a

C8 (STA)

SS fails to note the loss-of-feed ANN.

1.0 (same analysis as B4 )

The STA would not be involved since, at this time, he thinks the

operators have matters under control.

D1 (RO)

D2 (SRO)

D3 (SS)

(STA)

D5 (RO)

.02 (same analysis as CI)

RO fails to carry out

Figure 21-1.

.51

(same analysis as C2)

.16 (same analysis as C3)

1.0 (same anaysis as C4)

.0001 (same analysis as C5)

step 2.6 in the "Immediate Actions" from

.26 to 1.0

.05 to .51

The loss-of-feed ANN provides a recovery cue for this step also.

21-8

Table 21-1

Table 21-1 Explanation of terms in Figure 21-2 (p 4 of 4)

Failure Limb & Estimated HEP UCB or EF (Person) and Source and Source

D6 (SRO)

(SS)

D8 (STA)

E( (RO)

.5 (same analysis as C6 )

.14 (same analysis as C7 )

1.0

(same analysis as C8 )

.001 See discussion below

.25 to 1.0

.04 to .5

5 T20-20 , #5

RO fails to initiate HPI (step 3.1, Figure 21-1). If the problem

has been diagnosed correctly, it is very unlikely that the operator would fail to realize that help was needed. A conserva- tive HEP of .001 is assessed because an abnormal event is involved. (This rationale is explained in Chapter 20 in the section, "The Data Tables.")

E2 (SRO)

.5 T20-17, Eq. 10-17

.25 to 1.0 T20-21 , #4a

SRO fails to initiate HPI.

E 3 .14

T20-17, Eq. 10-16 .04 to .5 T20-21 , #3a

SS fails to initiate HPI.

ET (STA)

.05 T20-17, Eq. 10-15

.015 to .15 T20-21 , #2a

Per Table 20-4, #4b, the STA is assessed LD to MD for major events. The initiation is THE major event to be accomplished for this abnormal situation. The STA, if he does nothing else, should ensure that HPI is initiated. (If the analyst is concerned that this assessment is optimistic, he may assign a higher level of dependence to see if it impacts the F . In this case, even an

Tassessment of CD would not change F materially.) T

z7-q

Problem 1: Failure To Initiate Steam Generator Feed-and-Bleed Procedures

factors, would signal a need for changes in the system to provide more recovery factors. In commenting upon the HRA event tree methodology, users of the draft Handbook report that the ability to uncover such weak links in the human actions required in coping with abnormal events, or in highlight- ing conditions that could lead to abnormal events, is a major advantage of the method. As stated in Chapter 5, the correct and complete formulation of HRA event trees is usually more important than the exact HEPs used in the trees.

Although the example given above is hypothetical, it is a realistic repre- sentation of the type of analysis encountered in performing the HRA por- tions of a PRA. Obviously, an HRA is not as straightforward as following the procedures for a calibration task. Considerable judgment is required-- with experience, the analyst will make realistic judgments readily. Ini- tially, the analyst will tend to make very conservative judgments, which is what we recommend.

The FT from the HRA event tree is based on single-point estimates of HEPs. When incorporated in the overall system analysis, an estimate of uncer- tainty is required for the F . The uncertainty bounds are calculated usingT the procedures outlined in Appendix A.

Application of the Chapter 20 Search Scheme

This section illustrates the use of the search scheme in Figure 20-1 as an aid for entering the appropriate tables in Chapter 20 and for ensuring that the HRA event tree has the relevant paths and failure limbs. We will assume that we have drawn an HRA event tree such as that shown in Figure 21-2 but have not yet assigned HEPs to the failure limbs. Since we are dealing with a specific HRA problem, not all paths in the search scheme will be used. Unlike the above analysis of Problem 1, we will show the use of the search scheme for the initial screening analysis. It is assumed that the reader has read the earlier section in this chapter entitled "Background and Assumptions."

Because we are illustrating the use of the search scheme as a different approach to solving Problem 1, this section will repeat some of the infor- mation given earlier. We believe that for beginning analysts, the search scheme provides a valuable indoctrination to HRA. For analysts who are very familiar with the tables in Chapter 20, the search scheme offers a final check to ensure that all aspects of the HRA have been addressed.

Following is a talk-through of the search scheme based on Problem 1:

(1) Beginning with the "START" ellipse in Figure 20-1, follow the YES path from the ABNORMAL EVENT? decision node to the SCREENING REQUIRED? node. Assume YES.

21-10

Problem 1: Failure To Initiate Steam Generator Feed-and-Bleed Procedures

(2) The YES path leads to Table 20-1 for the screening diagnosis model. Item 3 in the table gives an HEP of .1 (HF = 10) as the median joint HEP for the control room personnel to diagnose the abnormal event within 20 minutes. Table 20-2 provides HEPs for carrying out of critical steps in Figure 21-1 following a correct diagnosis. Item 1 from the table gives an HEP of .05 (EF = 10) for Steps 2.4 and 3.1; these steps have no recovery factors. Item 2 gives an HEP of .025 (EF = 10) for Steps 2.5 and 2.6 because they do have recovery factors (annunciations). Assume that when these estimates are entered into the system analysis, they show a material impact on the overall failure probability, and it is therefore decided that a more detailed HRA is required.

(3) The SENSITIVITY ANALYSIS OR END? decision node is answered NO; that is, no sensitivity analysis is required at this time, but a detailed HRA is required.

(4) The NO path leads to the next decision node, NOMINAL DIAGNOSIS REQUIRED?. The YES path leads to Table 20-3. Item 3 in the table provides an HEP of .01 (EF = 10) for the control room personnel to diagnose the events within 20 minutes. The last footnote in the table directs the analyst to Table 12-5 to see if the nominal HEP of .01 should be adjusted upward or downward. Assume that plant- specific information is used to retain the .01 estimate.

(5) The next table (20-4) on the path being followed is used to determine how many control room personnel should be considered in estimating HEPs for the carrying out of those activities subsequent to the diagnosis of the event. The problem definition assumes that these activities will be initiated at about 20 minutes into the event and completed by 30 minutes into the event. At 20 minutes into the event, item 4 in the table indicates that all four persons (RO, SRO, SS, and STA) will be present. However, the analyst still must assess which of these persons will participate in each critical activity and the levels of dependence among them.

(6) Follow the YES path from the next decision node, RULE-BASED ACTIONS?. The "Go to RBA" ellipse transfers the analyst to the "RBA" ellipse and the next decision node, TYPE OF ERROR?.

(7) As stated in our problem definition, errors of commission in carrying out the procedures in Figure 21-1 are considered to be negligible, so we can ignore the COMMISSION path.

(8) The OMISSION path leads to the next decision node, WRITTEN MATERI- ALS?. The YES path goes to Table 20-5, which deals with the proba- bilities of errors in the written procedures. Assume that the formal written procedures in Figure 21-1 are used for training and that the analyst is confident there are no important errors.

21-11

Problem 1: Failure To Initiate Steam Generator Feed-and-Bleed Procedures

(9) Table 20-6 is used to estimate the probability that the necessary tasks will not be initiated. Since the event has been diagnosed, a zero probability of failure is assessed.

(10) Table 20-7, item 4, lists a basic human error probability (BHEP) of .01 (EF = 3) for omitting any step in the procedure in Figure 21-1.

(11) The path leads to the "Go to PSF" transfer ellipse, which takes us to the performance shaping factor (PSF) ellipse on page 2 of Figure 20-1. We are now ready to adjust the .01 BHEP for use of written procedures in the rule-based actions for the effects of PSFs. Note that we do not adjust the .01 BHEP assigned to the diagnosis of the event. As explained in Chapter 12, this BHEP already includes the effects of stress, experience, dependence, etc.

(12) Tagging levels are not relevant to this HRA, so we proceed to Table 20-16 to assess the effects of stress and experience on the above .01 BHEP. From Chapter 17, we assess the stress level as moderately high. Plant-specific information is that all the operators have at least 6 months experience in the job to which they are assigned, so we are dealing with the "skilled" experience level. Table 20-16, item 4a, lists a modifier of x2 to apply to the BHEP for step-by-step activities under moderately high stress. Thus, the .01 now becomes .02 for errors of omission for each relevant step in the procedures in Figure 21-1.

(13) The next rectangle asks the analyst to consider other PSFs. We decide that no other PSFs are relevant and proceed to the next rectangle.

(14) We must now assess the appropriate levels of dependence among the four persons to whom credit for carrying out the RBAs might be given. Table 20-4 lists generic levels of dependence, and guidance for plant-specific modffication is given in Chapter 10. The assessment requires considerable judgment. We judge that the levels of depen- dence for the performance of the steps in Figure 21-1 are as follows: HD between the SRO and RO, MD between the SS and the others, and CD- between the STA and the others, except for the critical Step 3.1, which is the initiation of HPI. For this step, we assess LD.

(15) Having decided on the levels of dependence, we may now use Table 20-17, 20-18, or 20-21 to assess the values for the conditional human error probabilities (CHEPs) for each person for errors of omission for each critical step in Figure 21-1. The BHEP is .02, as deter- mined above. If we use the equations in Table 20-17 to calculate the CHEPs, Equation 10-17 for the SRO yields a CHEP of .51, Equation 10-16 yields a CHEP of .16 for the SS, and Equation 10-18 yields a CHEP of 1.0 for the STA, except for Step 3.1, for which the assess- ment of LD yields a CHEP of .05 (Equation 10-15).

21-f2

Problem I: Failure To Initiate Steam Generator Feed-and-Bleed Procedures

(16) At this point, we have enough information to calculate the Pr[F I for the HRA event tree without considering the effects of recovery Tac- tors. Assume that system analysts have already told us that we must consider recovery factors since the .01 BHEP for the diagnosis is a first-order failure term, and it is clearly driving the system analysis.

(17) The next decision node, UCBs NEEDED?, is answered YES, so Table 20-20, item 5, is used to assign an EF of 5 for the .02 HEP, and Table 20-21, items 4 and 3, are used to assign EFs for the .51 and .16 CHEPs. Some interpolation is required between columns (a) and (b), resulting in complete HEP expressions as follows: .51 (.26 to 1.0) and .16 (.05 to .5).

(18) The next rectangle notes that Appendix A can be used to calculate the UCBs around the Pr[F 1. We are not ready for this step since we must include recovery faclors in our HRA.

(19) The RECOVERY FACTORS? decision node is answered YES, and the "Go to RF" ellipse transfers us to the top of page 3 of Figure 20-1.

(20) The CHECKING OF ANOTHER'S WORK? decision node is answered NO because the job of a "checker" is a special category for routine operations, described in Chapter 19, and does not apply to tasks being performed under abnormal operating conditions in which control room personnel are all interacting to some degree.

(21) The ANNUNCIATED? decision node is answered YES, because there are some critical annunciations that can lead to recovery from an incor- rect diagnosis and from failure to perform Steps 2.5 and 2.6 of the "Immediate Actions" in the procedures in Figure 21-1. The NO path is not used since the only recovery factors of importance are cues from annunciators.

(22) The next decision node is TYPE OF ERROR?, and only the OMISSION path is relevant. Table 20-23 is used to obtain estimates of the failure of the RO to be properly cued by annunciators. The first recovery factor is one for a false diagnosis and consists of an annunciator that indicates that the pressurizer has reached the saturation point. As indicated in Table 21-1 (failure limb A2 ), an HEP of .0006 is assigned to the RO (Table 20-23, #2k). The other annunciator recov- ery cues are for Steps 2.5 and 2.6 of the immediate actions. As indicated in Table 21-1 (failure limbs C5 and D5 ), an HEP of .0001 is assigned to the RO (Table 20-23, #1k).

(23) The path leads to the "Go to PSF" transfer ellipse and thus to the "PSF" ellipse at the top of page 2 of the search scheme so that the above annunciator recovery factors can be modified for the presence of the other control room personnel. The modification consists of the use of the CHEPs of (approximately) .5 for the SRO, .15 for the

Problem 2: Reanalysis of a WASH-1400 HRA

SS, and 1.0 for the STA except for Step 3.1, in which the CHEP is .05. These additional recovery factors are then entered into the HRA event tree in the same manner described earlier (see Figure 21-2).

(24) Now we are in a position to quantify the Pr[F I for the HRA event tree, as indicated in the next rectangle on twe search scheme. Our answer would be that indicated in Figure 21-2.

(25) The next decision node is UCBs NEEDED?. All we need now are UCBs for the SRO and SS responses to the annunciator cues for Steps 2.5 and 2.6. They are calculated using Table 20-21. Now that UCBs have been assigned to each HEP in the HRA event tree, the next rectangle calls for the calculation of the UCBs about the Pr[F T, using the procedure described in Appendix A.

(26) The path leads to RECOVERY FACTORS?. Since we have already consi- dered all relevant recovery factors, we take the NO path to the "Go to SA" transfer ellipse and then to the "SA" ellipse at the bottom of page 3 of the search scheme.

(27) The next decision node is SENSITIVITY ANALYSIS REQUIRED?. If we are satisfied with the HRA, the NO path leads the "END" ellipse, and the HRA has been finished. We might, however, wish to vary certain assumptions or assessments we made in the HRA to see what the impact would be on the HRA and on the overall PRA. In this case, the YES path enables us to reenter the search scheme at any point and modify estimates and recalculate the Pr[F T].

Problem 2: Reanalysis of a WASH-1400 HRA

Background

The original analysis is taken from pages 111-67 through 111-69 of Appendix III to WASH-1400, which describes "a sample human reliability analysis" dealing with the probability of the control room personnel failing to correctly carry out some written emergency operating procedures about 30 minutes after a large LOCA. In the original analysis, zero dependence (ZD) was assumed among the people in the control room because it was known that the use of nonzero levels of dependence would not result in major changes in the failure paths of interest.

In the reanalysis, we use the nominal diagnosis model (Table 20-3), the new staffing model (Table 20-4), and more realistic estimates of dependence levels.

We first present the original WASH-1400 analysis and then show the changes that follow from the use of the new models and assumptions. The overall failure probability in the original analysis was .01, exclusive of recovery factors; whereas, the new probability is a factor of 33 lower, .0003. This reduction is due to the new assumptions about human redundancy in the control room, as explained later.

21-14

Problem 2: Reanalysis of a WASH-1400 HRA

The-Original Analysis

The analysis addresses human reliability in shifting from the injection mode to the recirculation mode about 30 minutes after the occurrence of a large LOCA at a PWR. In the example, this shiftover is done manually. If it is not done correctly or on a timely basis, the consequences could be very serious, since the pumps required for long-term cooling could be destroyed by attempting to pump down an empty refueling water storage tank (RWST). The coolant in the RWST is used in the initial injection mode to keep the reactor covered. Before this coolant is completely depleted, it is necessary to perform the actions below to pump water from the contain- ment sump and recirculate it through the reactor vessel.

The analysis is based on the following two paragraphs of the written pro- cedure entitled "Loss of Reactor Coolant":

4.8 When the RWST reaches the low level setpoint (14.5%) and Conse- quence Limiting System (CLS) initiation has been reset (RESET PERMISSIVE 4.5 psig), complete the following actions:

4.8.1 Open MOV-860A and B, suction to the low head Safety Injection (SI) pumps from the containment sump.

4.8.2 Stop the containment spray pump motors and close spray pump turbine steam supply valves MS-103A, B, C, and D.

4.8.3 Close spray pump suction and discharge valves MOV-CS-100A, 100B, 101A, B, C, and D.

4.9 When the RWST reaches the low-low level setpoint (7%), complete the following actions:

4.9.1 Close MOV-862, suction to the low head safety injection pumps from the RWST.

4.9.2 Open the charging pump suctions from the discharge of the low head pumps by opening MOV-863A and B.

Our analysis is restricted to Steps 4.8.1, 4.9.1, and 4.9.2. The MOV switches involved are MOV-1860A and B, MOV-1862, and MOV-1863A and B,* all in the bottom row in Figure 21-3. The two rows of switches shown are the bottom two of seven rows on the leftmost panel of four segments of a large switchboard.

The written procedures dropped the initial digit since it was used only to designate to which of the two reactor units the switch referred. On the Number 1 reactor, it was MOV-1860; on the Number 2 reactor it was MOV-2860A. This is an example of poor writing practice; separate emer- gency operating procedures should be prepared for each unit (Brune and Weinstein, 1983).

21-15

Figure 21-3

@00( 000 0 G 0 0

LO HEAD SI PUMP A

I - SI - P - IA

0 0 00® 00® 00®

SLOHEAD SI PP B DISC ISO VV LO HEAD Si PP A SUMP SUCT VV LO HEAD SI PP A DISC ISO VV LO HEAD SI PP A+B RWST SUCT VV LO HEAD SI PP B SUMP SUCT VV

MOV - 1860A MOV - 1863A MOV 1862 MOV- 1860B ISO DISC FROM HI HEAD SUCT

KEY: The letters G, Y, and R refer to indicator lamps with GREEN, YELLOW, and RED filters, respectively.

Figure 21-3 MOV switches on part of the ESF panel at the PWR used in the WASH-1400 study. (Note: This is a repeat of Figure 3-9 and is based on Figure III 6-2 in WASH-1400.)

Problem 2: Reanalysis of a WASH-1400 HRA

There are two indicator lamps above each switch: G stands for green (closed condition of MOV or pump stopped), R stands for red (open condition of MOV or pump running), and Y stands for yellow (an intermediate condi- tion). Before the low level setpoint is reached, MOV-1862 is open (red lamp) and the other four are closed (green lamps).

Not shown in the sketch, but relevant to the analysis, is the third row from the bottom of MOV switches. The third row consists of five switches identical in shape, size, and arrangement to the switches on the bottom row. The five switches are labeled from left to right as follows*:

LO HEAD S.I. PP A DISC ISO VV MOV-1864A

ISO DISC FROM COLD LEGS

LO HEAD S.I. PP A RECIRC ISO VV MOV-1885A

LO HEAD S.I. PP A&B RECIRC ISO VV

MOV-1885C

LO HEAD S.I. PP B RECIRC ISO VV MOV-1885B

LO HEAD S.I. PP B DISC ISO VV

MOV-1864B ISO DISC FROM COLD LEGS

(The red lamps are lit, indicating

the normally open conditions of the valves.)

The low level setpoint (14.5%) will be reached about 20 to 30 minutes after a large LOCA (in WASH-1400, 30 minutes was assumed). When the low level setpoint is reached, all actions called for in paragraph 4.8 must be accom- plished within 2 minutes, because the low-low level setpoint (7%) may be reached by then, and the operators must be ready to take the actions called for in paragraph 4.9. The approach of each setpoint is indicated by meters that display the water level in the RWST. Annunciators sound when the setpoints are reached.

Two questions were addressed in the analysis:

(1) What is the probability that no action would be taken at the low level setpoint (an error of omission)?

In the procedures, the abbreviations are: LO - low, SI - safety injection, PP - pump, A - channel A, DISC - discharge, ISO - isolation, VV - valve, RECIRC - recirculation, B - channel B.

g I -r?

Problem 2: Reanalysis of a WASH-1400 HRA

(2) What is the probability that some pair of switches other than MOV-1860A and B would be manipulated (an error of commission)?

In addressing the first question, we used the values in the WASH-1400 table of human error rates (Appendix G), which table indicates a basic HEP of 10-1 30 minutes after a-large LOCA. It was assumed that at least three people would be present in the control room by 30 minutes after a LOCA and that action would be taken promptly unless all three failed to anticipate the low level setpoint. It was judged that the meter indication of a falling RWST level should remind those present to get ready to perform Step 4.8.1. If no preparations were made before the annunciator alarmed at the low level setpoint, the chances of completing the procedure correctly in the required 2 minutes would be greatly reduced.

We estimated that each of the three people would have a .5 probability of failing to notice the meter indications. Our rationale was that, although they know that the coolant from the RWST is being used up during a large LOCA, under the stressful condition of many alarms, danger to the plant, and potential danger to the environment, the best one could expect is about a 50-50 chance per person that he would remember to check the RWST level meter before the annunciator alarmed. The meter is located on a vertical panel several feet behind the desk-type panels on which the switches for Step 4.8.1 are located. This is obviously not an ideal location for a cue to remind someone to perform an action. The HEP of .5 represented our judgment based on this situation; no other data related to this task were available.

We judged the joint probability that all three people would fail to notice the meter indication to be .5 .125, rounded to .1. We also judged that there would be some probability that the control room personnel would be anticipating the low level alarm even if they did not monitor the RWST level. We took the BHEP of .1 (from Appendix G) and cubed it to arrive at the joint probability that all three peo le would fail to prepare for the low level setpoint procedures (.13 = 107). Thus, the joint probability of failure to anticipate the low level setpoint and failure to notice the meter readings was taken as .1 x 10-3 = 10-4. This 10-4 is assigned to the first failure limb in the event tree in Figure 21-4.

When the low level annunciator has sounded, the operators have only 2 minutes to perform the steps in paragraph 4.8. It was reasoned that if no action had been planned until the alarm sounded, some degree of disorgani- zation was indicated, and the BHEP of .1 was assigned to each of the three operators. A joint probability of .13 = 10-3 was estimated for the failure of all three operators to take action within 2 minutes after the alarm at the low level setpoint. This probability is shown on the second failure limb in Figure 21-4 leading to failure event F 3 (Step 4.8.1 not done in time).

The next step in the original analysis was to assume that at least one of the three operators did prepare to manipulate switches MOV-1860A and B.

21-18

Figure ,-4

10-4 NO ACTION UNTIL. , ALARM (3 PEOPLE)

.99 CORRECT PAIR OF SWITCHES,

10- 3 FAILURE TO INITIATE ACTION WITHIN 2 MINUTES AFTER ALARM (3 PEOPLE)

F3 < 10-5

STEP 4.8.1 NOT • DONE IN TIME

F2 10- 6

FT = F 1 + F 2 + F 3 _10-2

Figure 21-4 HRA event tree for Step 4.8.1 in LOCA procedure (taken from Figure III

6-3 in WASH-1400).

21-19

Problem 2: Reanalysis of a WASH-1400 HRA

This is depicted in Figure 21-4 by the two limbs designated as "TAKE ACTION." This leads to the second question: What is the probability that some pair of switches other than MOV-1860A and B would be manipulated? This is represented in two places in the event tree--in the terminal limbs leading to F1 and F2. The conditional HEP for this task was estimated as 10- 2. Our rationale was that itlwould be highly probable that responsi- bility for operating the valves would be assigned to one person, that is, no human redundancy would be available to recover an error by the person performing this task. This judgment was based on observation of operators at work. Misselection of switches is the type of error that few operators regard as a credible error. Therefore, it was deemed unlikely that anyone would check the operator who actually manipulated the MOVs. The basic error probability of 101 was assessed to be too large for this act, so 10-2 was selected as the nearest order-of-magnitude estimate. In our reanalysis, we believe the assumption of complete reliance on the reactor operator for so important a step is no longer appropriate, and our recal- Culations include checking by the SS of this critical step in the emergency operating procedure.

Now it is possible to assign HEPs to all of the limbs, since the sum of the probabilities at each branching must equal 1.0. There are three failure paths. A + F3 yields a value of 10-, which was small enough to disregard. Two paths lead to misselection of the switches: Path A + F1 and A + F2 . The probability of going down the second path is as follows:

Path A 4 F2 = 10-4 x .999 x 10-2 = 10-6

In WASH-1400, this small probability was eliminated from further considera- tion. The probability of Path A + F 1 is .9999 x 10-2, which rounds to 10-

Given the 10-2 probability of selecting a wrong pair of switches at the low level setpoint, the question arises as to which incorrect pairs might be selected. The following analysis was performed to estimate the probabili- ties of relevant extraneous errors. It was judged that the most probable candidates are MOV-1863A and B: they are in the same row, are next to the desired switches, and have similar identifying numbers and labels. The probability of selecting a pair of switches from the second row from the bottom is lower because of the dissimilarity of switch nomenclature and the different appearance of the switches themselves (they have an AUTO posi- tion). The switches in the third row from the bottom have labels similar to those on the desired switches, but the outboard switches (the most likely candidates for misselection) are normally open. Their red indicator lamps would furnish a cue that they are not the correct switches. In addition, this third row is somewhat remote from the desired switches.

Given the initial error of selecting some pair of switches other than MOV-1860A and B, it was estimated that there was a probability of .75 that the operator would select MOV-1863A and B and a probability of .25 that some other pair of switches would be selected. (The .75 and .25 split was

21-20

Problem 2: Reanalysis of a WASH-1400 HRA

assessed on the basis of the layout of the switches and represents the kind of judgment that is independent of the HEPs in the Handbook.) The error of misselection of MOV-1863A and B has the following recovery factor at the 7% (low-low level) setpoint. In Step 4.9.2, the operator is supposed to close MOV-1863A and B. If the error of misselection had been committed, the operator would find these MOVs already closed. This should cue him that something is wrong. An HEP of .1 was assumed for failure to note the error. Hence the total estimated failure probability for Step 4.8.1, including failure of the recovery factor, is 10- 2 x .75 x.10-1 = .00075, which is rounded to 10-3. (The HEP of .1 is the HEP assumed in the WASH-1400 analyses for most operator actions 30 minutes after a large LOCA, as shown in the table in Appendix G.)

A similar analysis was performed for Steps 4.9.1 and 4.9.2. The analyses described above involved some subjectivity. This subjectivity was not particularly crucial for the study because the important factor that affects overall results is the order of magnitude of the HEP not its exact value. The UCBs assigned to the final estimate allowed for uncertainties and errors in the analysis. As a tool in itself, the detailed analysis is valuable for the following reasons:

(1) The exercise of outlining all plausible modes of operator action de- creases the probability of overlooking some important failure path.

(2) Due to the lack of error probability data for nuclear power plant tasks, it is necessary to break down operator actions to a level at which existing data can be used.

(3) The detailed approach makes it easier for analysts making independent estimates to check on the source of any disagreement and to resolve it.

A Reanalysis

The following exercise shows how the dependence model, the nominal diag- nosis model, and the staffing model in the current Handbook would change the original WASH-1400 estimates. This reanalysis is for illustration only; in a real analysis, we would base our assumptions on a detailed task analysis and related studies as described in Chapter 4.

Figure 21-5 is a redrawing of the HRA event tree in Figure 21-4, with differences in assumptions. In this new analysis, we use the nominal diagnosis model (Table 20-3, item 4) to obtain the estimated probability of .001 that the control room personnel will not have properly diagnosed the event by 30 minutes after the large LOCA. This failure in diagnosis means that the control room personnel will not be anticipating the low level alarm. When it does sound, we judge that all three operators will be involved. Assuming a total of five competing annunciators at this time, Table 20-23, item 5k, provides a BHEP of .003 for the failure of the primary operator to note what needs to be done. We assign a high level of

21-21

Figure 21-5

CORRECT DIAGNOSIS .999

"READY FOR LOW-LEVEL ALARM-

FAILURE TO NOTICE LOW-LEVEL ANN,

ý, PEOPLE, 5 ANNs

.003 x .5 x .15 = .0002

F4 << 10-5

FAILURE TO INITIATE STEP 4.8.1 IN 2 min AFTER ALARM, .25 x .63 x .29 = .046

Fs ý 5 x 10-5 WRONG PAIR SWITCHES .015 x .5 x .05 =.0004

FT = F 1 + F 2 + F 3 + F4 + F 5 .-.0003

Figure 21-5 Changes to HRA event tree in Figure 21-4 to reflect changes in Handbook models and HEPs.

21-22

Problem 2: Reanalysis of a

WASH-1400 HRA

dependence to the SRO (Table 20-4, item 2) and a low level of dependence to the SS (Table 20-4, item 3). From Table 20-21, items 4a and 3a, their conditional HEPs are .5 and .15. Thus, the joint HEP for failing to take note of the critical annunciator is .003 x .5 x .15 = .0002. Failure path

F4 is 2 x 10 x 10 << 10-, a value that would not be significant in a PRA.

If the annunciator is noted properly (a probability of .9998), we judge that the operators would now be under an extremely high stress level, threat stress, because they now suddenly realize that they are in serious danger of failing to cope with a most serious accident. Accordingly, we assess a BHEP of .25 (from Table 20-16, item 7) for the primary operator to fail to initiate Step 4.8.1 within 2 minutes after the low level alarm. This BHEP is multiplied by .63 and .29 for the SRO and SS (from Table 20-21, items 4f and 2f). The joint HEP is .046, and the end-failure term for F5 is about 5 x 10-5, not a significant contributor to the total fail- ure probability.

If the operators did initiate Step 4.8.1 in time (probability of .954), there is still the possibility of manipulating the wrong pair of switches, which constitutes a step-by-step task. Since they are under extremely high stress, the multiplier for the BHEP of .003 (Table 20-12, item 2) is 5 (from Table 20-16, item 6a), or a BHEP modified for stress of .015. From Table 20-21, items 4a and 2a, the conditional HEPs for the other two opera- tors are .5 and .05, so the joint HEP of selecting the wrong pair of switches is the product of the three estimates, or .0004. The end-failure term F6 is much less than 10-5, not an important contributor.

Proceeding down the correct diagnosis path, the .999 estimate indicates that the event has been properly diagnosed and that the operator is antici- pating the low level alarm. As soon as it sounds, he will initiate the appropriate procedures, beginning with Step 4.8.1. At this point, matters are so well under control that we judge only the primary operator and the SS will be involved. The SRO and STA are presumed to be engaged in other important activities.

The primary operator will be monitoring the RWST level meter, as implied by his written procedures. This is a dynamic task, as defined in Table 20-16. The basic HEP is .01 for errors of omission (Table 20-7, item 4), multi- plied by 5 for the case of a dynamic task performed under moderately high stress (Table 20-16, item 5). The stress level is assessed as moderately high, not extremely high, because the operators have the situation under control. Thus, the modified HEP considering stress is .05. It is unlikely that the SS will be actively involved in monitoring this meter. We assume that only the RO will be so occupied. For this task we consider errors of commission to be negligible.

Even if the RO fails to respond to the low level indication on the RWST meter, he is alert for the low level annunciator. The RO and SS are actively anticipating this particular alarm, regardless of how many alarms

21-23

Problem 3: Case Study of a LOCA Procedure

may sound concurrently, so we assess an HEP of .0001 as the BHEP (Table 20-23, item 1k). We assess a low level of dependence for the SS, so the joint HEP is .0001 x .05 = 5 x 10-6. Multiplying .999 x .05 x 5 x 10-6 yields an insignificantly small value, which is disregarded.

If the operator does monitor the RWST meter level, we assume complete dependence for noticing the low level alarm and initiating Step 4.8.1. Therefore, the conditional failure to notice the alarm is 0, and F2 repre- sents a null path.

Continuing on the complete success path, the only error left is the selec- tion of the wrong pair of switches. The basic HEP of .003 (Table 12, item 2) is multiplied by 2 for moderately high stress (Table 20-16, item 4a) and then by .05 for the low level of dependence of the SS (Table 20-21, item 2a). F3 involves two failure paths through the left side of the HRA event tree, but only one of those paths contributes materially to the total failure probability. The noncontributing path is the product of .999 x .05 x .999995 x .0003 = .000015. The contributing path is .999 x .95 x 1.0 x .0003 = .0002847, which rounds to .0003. To be exactly correct, F3 is the sum of the two paths, or .000015 + .0002847 = .0002997, which rounds to .0003.

The end-failure term of .0003 is the only failure term of consequence, and represents a total failure probability of a factor of 33 less than the one calculated in the WASH-1400 analysis. In view of the changes in the last 8 years in training, practice, staffing, and emphasis on coping with abnormal events, a lower estimated total failure probability for the frequently practiced simulations of large LOCAs is not surprising.

Most of the reduction in the total-failure term in Figure 21-5 follows from our assumption that the SS would be directly involved in selecting the correct pair of switches for recirculation; whereas, in the WASH-1400 analysis, we judged that he would not be involved. This change in assump- tions is based on interviews with operators and supervisors since the Three-Mile Island incident. If this new assumption is incorrect, the revised F in Figure 21-5 would be about .0003 + .05 = .006, or only aT factor of about 1.7 lower than the WASH-1400 estimate.

Problem 3: Case Study of a LOCA Procedure

This example is based on a study performed for the NRC as part of their continuing evaluation of emergency operating procedures. The study was primarily a qualitative analysis, and we have included it to show how the Handbook can be used for qualitative as well as quantitative assessments. The plant conditions that prompted the NRC evaluation no longer exist.

In this plant (a PWR), human participation would be required early in the post-LOCA sequence of recovery actions. The most critical human action would be that of switching from the safety injection (SI) of emergency

21-24

Problem 3: Case Study of a LOCA Procedure

coolant to the recirculation of coolant from the sump to the reactor. This action is similar to the action described in the previous example, except that the switching task would have to be done within 8 minutes after the LOCA instead of within the 30 minutes allowed in the first example. The worst-case analysis addresses a design-basis LOCA. It is based on the assumption of the same type of guillotine break in a large coolant pipe as was assumed in WASH-1400, with further assumptions that the plant is at full power and that the water level in the RWST is at the lowest level allowed by the technical specifications.

The large LOCA curve (Figure 17-2) shows that at about 8 minutes into a large LOCA, an operator would be performing correctly only one time in four. Rather than rely on this speculative curve, we conducted an informal experiment in the plant in which the SS walked through all the actions called for by the written operating instructions. The SS was considered the best operator at the plant, yet starting from a "go" signal, he was barely able to complete the procedures in 8 minutes. This ideal response situation did not allow for any time lost due to indecision about the existence of a LOCA, nor was any allowance made for secondary failures of instrumentation in conjunction with a LOCA. For example, if the contain- ment sump level indicating lamps failed, the first warning to the operator of the need for changeover to the recirculation mode would occur at 471 seconds (nearly 8 minutes) after the LOCA, the second would occur at 500 seconds (8-1/3 minutes), and a third at 503 seconds. If the operator

failed to initiate switchover at the third indication, all of the available coolant would be gone in another 92 seconds, and the feedwater pumps would be damaged. During these 92 seconds, he would have to perform eight switching actions at three panels several feet apart.

We concluded that we could not be confident that the switchover would be

accomplished in time in the event of such a design-basis LOCA. We identi- fied the basic human factors problems as those of time stress, poorly written procedures, and poor human factors engineering of the control room. Specific suggestions were made to reduce the effects of these three prob- lems on human reliability.

In the first area, time stress, some reduction had already been achieved at the plant by eliminating some of the decision-making required of the operator in determining whether a LOCA had occurred. Their training pro- gram directed an operator to assume a LOCA when certain annunciated symp- toms appeared (pressurizer low level, pressurizer low pressure, containment high pressure, etc.). We recommended that talk-throughs like the one we conducted be held frequently and that the onsite NRC inspector periodically check the readiness of an operator to respond.

In the second area, written procedures, we noted that a number of operator actions were required primarily for economic considerations and not for the switchover. We recommended that all such steps be postponed until the switchover had been completed. This would considerably reduce the number of actions an operator would have to take in the critical first 8 minutes.*

21-25

Problem 4: Case Study of Manual Switchover to AFWS

We further suggested changing the sequence of written steps in the proce- dure to reduce travel between different panels--in link analysis terms, to reduce the number of required links (Figure 3-8). Finally, we suggested rewriting the procedures in the columnar format shown in Figure 3-3.

In the third area, human factors engineering, we suggested relabeling displays and controls in a consistent fashion, using the type of location aid shown in Figure 3-11. With these three changes, and the NRC inspec- tor's frequent verification that operators selected at random could indeed carry out the procedure, we estimated that substantial gains in human reliability would result. We did not attempt to quantify the projected human reliability at the time, since we felt that such an evaluation could be carried out more meaningfully after the changes had been implemented.

Problem 4: Case Study of Manual Switchover to AFWS

As part of an NRC post-TMI study of the availability of the auxiliary feedwater system (AFWS), we visited a PWR where the switchover from main feedwater to auxiliary feedwater was done manually. Plans were under way at this plant to incorporate automatic switchover, subsequently completed, and the question was whether manual switchover was a safe interim proce-. dure, not only at this plant but also at other plants. At some plants, the switchover must be made within 5 minutes or the steam generator might run dry. In other plants, as much as 15 minutes is available, and at still others, 30 minutes is available. We were asked to prepare human reliabil- ity estimates for manual switchover under conditions that allowed 5, 15, 30, and 60 minutes to accomplish the switchover. (The interest in the 60- minute period was not related to this particular problem.)

At the plant we visited, a second operator was assigned to the control room whose sole function was to maintain sufficient water inventory in the event of a transient. This operator was designated a dedicated operator (DO), as distinct from the regular RO, who monitored the rest of the control room. The DO at this plant was required to stay within a small area of the con- trol room close to his control panel. Because this job is confining and not very challenging, the plant assigned operators who had not yet achieved status as ROs, but who were working toward this status. It was judged that their motivation would be sufficient for this confining job. All DOs recognized that this was an interim assignment (until the automatic provi- sions could be incorporated) and that management took notice of their will- ingness and ability to serve.

The plant had also adopted a procedure to eliminate the need for decision- making to initiate the AFWS. This procedure called for the DO to switch from main to auxiliary feedwater whenever a reactor trip occurred if the plant was operating at more than 15% power. This plant policy had impor- tant bearing on our HEP estimates.

The switchovers from main to auxiliary feedwater were performed frequently at the plant, in both real and simulated situations, so that the manual

21-26

Problem 4: Case Study of Manual Switchover to AFWS

actions involved in switchover were very well learned by the DOs. There- fore, the only significant source of human error would be the failure to begin the switchover procedures. On the basis of interviews with and observations of operators at the plant, we estimated the probabilities of errors of oversight for plants with and without a DO, as shown in Table 21-2.

We will first consider the situation without a DO, in which the RO would

have to initiate AFWS in addition to his other duties. The HEP of .05 for the first 5 minutes is drawn from the model for annunciated displays (Table 20-23, item 10k). In the event of the need for AFWS, 40 or more annuncia- tors may be sounding, and the RO has to integrate all the information and make appropriate decisions. Note that Table 20-23, item 14k, indicated an expected Pr[F] of .25 for 40 or more annunciators. Our rationale for using

the HEP of .05 instead of .25 is based on the following: in plants requir- ing manual changeover to AFWS, plant policy usually requires the changeover to be initiated any time a turbine/reactor trip occurs. Although each of the annunciators conveys a unique message, the messages are not indepen- dent, and we estimate that at least 10% of them would convey indications that a trip has occurred. The HEP of .05 is the estimated probability of failure to initiate action in response to 1 randomly selected annunciator of 10.

We judged that if the time constraints were loosened from 5 to 15 minutes,

the probability of the RO's failure would be reduced by a factor of 5; hence, the .01 estimate. With about 30 minutes, we allowed another factor of 2 reduction in the HEP, to .005. (These estimated HEPs represent judg- ments not included in the tabled values in the Handbook.)

The RO's failure to initiate AFWS on time could be compensated for by the SS as a backup. We assumed that for the first 5 minutes, the SS would not be available as a backup operator in the control room (Table 20-4, item 3a).* For the period between 5 and 15 minutes, we estimated that he would be available but still "coming up to speed." In our original analysis, we estimated that the conditional probability of the SS's failure to compen- sate for the RO's failures was .5, equivalent to a high level of depen- dence. For the period up to 30 minutes, we reduced the SS's HEP by a

factor of 2 to .25, based on our judgment.

Note that we did not estimate any further improvement at 60 minutes. It

was our judgment that if the AFWS had not been turned on by the end of 30 minutes, the operators would be heavily occupied and performance on the AFWS task would not improve until things were under control.

For the situation with the DO, we started with the oral instructions model

(Table 20-8,item la). We considered the standing instruction for the DO to initiate AFWS when there is a reactor trip to constitute one oral

The staffing at the plant did not then require an SRO to be present in the control room area, as is the current requirement.

21-27

Table 21-2

Table 21-2. Estimated probabilities of failure to initiate AFWS*

At End of X min

Reactor Operator

.05

.01

.005

No change

Situation without Dedicated Operator

Shift Supervisor

.5.(MD)

.25 (LD)

No change

Joint HEPs

.05

.005

.001

No change

Dedicated Operator

.002

.001

.0005

No change

Situation with Dedicated Operator

Shift Supervisor

.5 (MD)

.25 (LD)

No change

Joint HEPs

.002

.0005

.0001

No change

*An error factor of 10 is assigned to each estimate in the "total" columns.

All HEPs are rounded.

21-28

Problem 5: The Two Status Lamps at TMI

instruction, and the HEP assigned to this is .001, which we assigned to the 15-minute time period. We doubled it for the first 5 minutes because we judged that for part of those 5 minutes the DO would still be subject to the incredulity response and the general reaction to many alarms sounding in a short time. For the 30-minute period, we reduced the BHEP by a factor of 2, to .0005. The backup estimates for the SS are the same as for the situation with the RO, and the values in the "joint HEP" column are calcu- lated as before.

Our rationale for the low estimated HEP for the DO was based on the prac- tices at the plant we visited. First, his duties are limited. He is responsible for only a very small portion of the control boards. Second, he performs periodic talk-throughs of the procedures involved in shutting down the main feedwater and initiating the auxiliary feedwater. Third, he has standing instructions to initiate the AFWS in the event of a turbine/ reactor trip. Thus, he has very little interpretation to do and his response should be almost automatic. On the basis of this qualitative analysis, we decided that the oral instructions model was appropriate.

Note that our estimates are pegged at the ends of certain time intervals. We did not attempt to plot a curve of HEPs vs. time, since this would have been pseudoprecision in view of the subjective nature of the estimates.

Problem 5: The Two Status Lamps at TMI

Part of the problem in the TMI-2 incident was that two blocking valves had not been returned to their normally open status after maintenance. These two valves are MOVs controlled by two switches designated as EF-V12A and B. Each has two status lamps (red for open and green for closed). The switches were positioned on the control panel with EF-V12B immediately above EF-V12A. A yellow caution tag attached to an instrument above EF-V12B covered both status lamps of that switch. If only one of the pair of lamps had been covered, it would have made no difference, since the two lamps comprise a perceptual unit.

Some people have expressed surprise that control room personnel did not detect the wrong state of the status lamps during the five shifts up to and including the shift in which the accident occurred. (Stello, 1979, p IA-18, surmises that the valves were not restored after a surveillance test at approximately 10 a.m. on March 26. The demand for auxiliary feedwater occurred at 4 a.m. on March 28.) We estimate a .99 probabiiity of failure to detect a status lamp in the incorrect state during the initial audit (Table 20-25, item 7), assuming that the initial audit includes the require- ment to scan the given status lamps. If the original error of forgetting to restore the valves occurred at 10 a.m., after the initial audit on March 26 for the day shift, there would have been four initial audits in which this anomaly could have been detected. The estimated probability of such a detection is only 1 - .994 = .04.

In actual practice, these status lamps are not checked on any routine basis. According to operating personnel we have interviewed, it is assumed

21-29

Problem 5: The Two Status Lamps at TMI

that the lamps are in the correct position unless something causes opera- tors to question this assumption. There are hundreds of such lamps that display static status.

We can reasonably conclude that incorrect status of these lamps operations in the control room. failing to restore the blocking matter.)

it would have been highly unlikely for the to have been noticed during the routine

(The causes for the original error of valves after maintenance are another

21-30

Part V. Concluding Comments and Appendices

PART V. CONCLUDING COMMENTS AND APPENDICES

This part of the Handbook consists of Chapter 22, "Concluding Comments," and seven appendices:

A - Methods for Propagating Uncertainty Bounds in a Human Reliability Analysis and for Determining Uncertainty Bounds for Dependent Human

Activities

B - An Alternative Method for Estimating the Effects of Dependence

C - Calculations of Mean and Median Numbers of Trials to Detection Given in Table 11-9

D - Calculations for Basic Walk-Around Inspections as a Function of Period Between Successive Walk-Arounds

E - Reviews of the Draft Handbook

F - A Comparison of the October 1980 and Present Versions of the Handbook

G - General Human Error Rates from WASH-1400

V-i

Chapter 22. Concluding Comments Overview; What the Handbook Is

CHAPTER 22. CONCLUDING COMMENTS

Overview

This chapter briefly assesses the state of human reliability analysis (HRA) for probabilistic risk assessment (PRA) and for design and development work. The possible uses and limitations in using the Handbook are described.

What the Handbook Is

The Handbook presents a human reliability assessment method, THERP (Tech- nique for Human Error Rate Prediction), and performance models and esti- mated human error probabilities (HEPs) for estimating the probabilities and effects of human errors on reliability and safety in large systems like nuclear power plants (NPPs). THERP is a conventional reliability analysis method that uses HRA event trees. The models and HEPs are based on experi- ence, theory, and limited data.

The Handbook presents an eclectic approach--admittedly imperfect but effec- tive and useful in PRAs. Since many of the values listed in the Handbook must be regarded as hypotheses, users are urged to test the models and HEPs empirically. If their findings are reported in the open literature, to the authors, or to the Office of Nuclear Regulatory Research, U.S. Nuclear Regulatory Commission (NRC), the Handbook can be updated over time.

Need for the Handbook

We prepared the Handbook at the request of the NRC because no such tool for the performance of in-depth- HRAs existed. The Rogovin Report states, "The best way to improve the existing design process [in NPPs] is by relying in a major way upon quantitative risk analyses, and by emphasizing those acci- dent sequences that contribute significantly to risk. The design review can then focus on those plant systems that contribute to risk, identify weak points, and upgrade various requirements (maintenance, for example) to eliminate them" (Rogovin and Frampton, 1980, p 150).

Ideally, a handbook such as this should be based on data gathered from actual operating experience in NPPs. Although the need for such a data base is well-recognized, the data have not been gathered. Because of the pressing need for human reliability analyses in NPPs, we prepared the Handbook as a working guide until data based on NPP experience becomes available. There are gaps in the coverage of NPP tasks, and even experi- enced analysts will have uncertainties about predicting performance under all the conditions that could develop in NPPs. We hope to fill such gaps as more experience and data are gained.

22-1

Need for the Handbook Limitations in Use of the Handbook

The Handbook addresses real-world problems. To be useful in HRA, certain assumptions had to be made about the models and estimated HEPs, and some corners had to be cut. We don't believe that the shortcomings of this document will offend those with practical problems to solve, since our most important objective is to introduce the user to the methodology of quanti- tative and qualitative evaluation of human reliability.

We are familiar with academic models of human behavior, beginning with the work of Hull (1943 and 1950) and including more recent efforts in human reliability (reviewed by Meister, 1964, 1971, 1973, 1983a and b); Embrey, 1976; Pew et al, 1977; and Swain, 1964b, 1969b, and 1977b). We find that most of these models are not very useful. It is still a standard joke in university psychology departments that the theories and models can predict anything after the results are in. Regrettably, most models of behavior substitute "postdiction" for prediction or are addressed to problems of very narrow scope.

Our approach is different. The performance models and estimated HEPs are unusual in the field of behavior technology in that they can be used to predict probabilities of human errors for identifiable real-world tasks and in that these predictions are verifiable. However, there are limitations.

Limitations in Use of the Handbook

The limitations of the Handbook were described in Chapter 1. At this time we reemphasize two of those limitations: the dearth of "hard" data and the possible misuse of the Handbook.

The lack of data refers to the scarcity of error relative frequencies col- lected in applied settings or in realistic simulations of these settings. This problem defines the major limitation to accuracy in HRA. The reviews of HRA cited previously have indicated that there is no scarcity of models but that data on which to base probability estimates are hard to find. For this reason, the nominal HEPs in the Handbook are based to a large extent on our experience in related areas, our knowledge of psychological theory, and what little data could be found that related to NPP tasks.

The lack of hard data and the necessity to use judgment in deriving the

nominal HEPs is one reason why the HEPs have wide bounds of uncertainty. In some cases, the HEPs might be off by as much as a factor of 10 either way. There is another standard joke in psychology departments that "cor- rect within an order of magnitude" means "wrong." Yet, for some applica- tions, even this degree of latitude is tolerable. One reason the uncer- tainty bounds are wide is that they are generic; not plant-specific. When the models and HEPs are applied to a specific plant where the analyst can identify and evaluate the relevant performance shaping factors (PSFs), the uncertainty bounds can be reassessed.

The second major limitation, or risk, is that the Handbook may easily be misused by experts in human preformance technology who have insufficient

22-2

When To Use the Handbook Needs for Further Research

To Improve HEP Estimates

knowledge in the area of reliability technology or may be misused by ex-

perts in PRA who have insufficient knowledge of human performance tech- nology. It is inevitable that the Handbook will be misused by some or used by others to "justify" inadequate designs or procedures. Any text or hand- book requiring judgment and interpretation can be misused. There is no easy remedy for this.

If the reader wishes to use the Handbook in an evaluation but feels that his preparation is inadequate, there are a number of short courses offered in human factors and in reliability technology that will help him to develop an awareness of the methods used by specialists. A most valuable benefit derived from these courses is the opportunity to meet people expe- rienced in HRA and PRA whom one may contact for help with problems.

When To Use the Handbook

HRA should be used in all stages in the life of a large system, from incep- tion to retirement. Early in the design stages of a new system, one will not have specific human performance requirements, and very wide uncertainty bounds must be used. However, even in the earliest design stages, HRA can help identify any human factors weaknesses and provide gross estimates of risk in alternative design concepts, e.g., control room layout or automatic versus manual modes of operation. This approach has been used in military and space systems and is strongly recommended for the nuclear power in- dustry.

When a plant is operating, it is much easier to obtain accurate information on plant-specific PSFs. Often, a new design is based on an existing plant. The personnel at that plant can provide valuable suggestions regarding the overall design, plant policies, practices, written procedures, training, etc., that can improve the human reliability of the new plant. In such a situation, the Handbook can serve as a guide to asking the right questions and to evaluating the relative importance of suggestions received.

The uses of the Handbook in evaluation of existing facilities are obvious. Utility management personnel can use it to conduct in-house evaluations to aid in determining the adequacy of their established policies, procedures, and designs.

Although the Handbook was written primarily for the nuclear power industry, it applies to many other industries as well. For most industrial cases, the material can be applied directly or with only minor changes.

Needs for Further Research To Improve HEP Estimates

From the preceding, it is apparent that research is needed to collect both

objective and subjective human performance data that are applicable to NPP tasks. Some years ago a plan for developing a human performance data bank was developed for the U.S. Navy, but the plan was not implemented (Swain, 1971). The elements of this plan were again presented in the context of

22-3

Needs for Further Research To Improve HEP Estimates

NPP operations (Swain, 1975). Beginning in 1981, the NRC has sponsored a major research effort to develop a human performance data bank for NPP operations. The data bank itself is being developed under research managed by Sandia National Laboratories (SNL) and has to date resulted in a review of existing human performance data banks (Topmiller et al, 1982, 1983) and a. data bank concept and system description (Comer et al, 1983). This research is continuing with a full development of the data bank concept and a limited tryout.

The NRC. is also sponsoring research on the problem of data acquisition for a human performance data bank. Research managed by the Brookhaven National Laboratory (BNL) has attempted to develop methods for deriving estimates of human error rates from the NRC's Licensee Event Reporting (LER) system and supplemental information. This continuing research is reported by Luckas and Hall (1981); Luckas et al (1982); and Speaker et al (1982). The Aero- space Corporation is carrying out another approach to human error data acquisition, one that is similar to the Aviation Safety Reporting System supported by the National Aeronautics and Space Administration and the Federal Aviation Administration. The Aerospace approach involves a volun- tary, nonpunitive Nuclear Power Safety Reporting System. This continuing research is reported in Findlayson and Ims (1983) and Findlayson (1983). The Institute for Nuclear Power Operations (INPO) is also studying the feasibility of obtaining information and data on human performance of NPP personnel. Reports on this activity are currently in preparation.

Although the status of human performance data collection should improve-in the future, there are problems at present. In addition to the need for controlled experiments to answer specific questions about behavior dynam- ics, there is a need for data based on the experiences of NPP personnel. The present LER system does not provide such data. The LERs do provide valuable information about errors that are reported, but rarely are the important, relevant PSFs described in sufficient detail for complete ana- lysis. Also, many errors are not reported at all--such as the ones that did not result in events designated by the NRC as reportable. Thus, the numerator information needed for HEPs is inadequate (HEPs ideally are based on the number of errors made divided by the number of opportunities for error). The denominator information is totally lacking, although in some cases it can be approximated from knowledge of schedules or interviews of personnel.

When abnormal events have serious consequences (e.g., the Brown's Ferry and Three-Mile Island incidents), plant personnel should be interviewed by people skilled in interviewing as well as in human factors. This was not done in either of the above incidents until long after memories began to fade or were influenced by other considerations. Without proper interviews by persons qualified in the human factors area, valuable information is likely to be lost. If we are to predict how NPP personnel will perform under high stress levels, it is essential to obtain this information as soon as possible after an abnormal event. Without such data to calibrate the HEPs collected in simulations of abnormal events, the use of simulator data in reliability analysis will be suspect (Swain, 1967a).

22-4

Needs for Further Research To Improve HEP Estimates

For some tasks, the best available source of objective data is the dynamic simulator. Until recently, this source was neglected. The potential use of simulators for step-by-step and dynamic tasks has been recognized in the last few years, and a study for this purpose has been sponsored by the NRC and managed by SNL. This simulator research was specifically directed at measures of error relative frequencies to derive HEPs (Beare, Dorris et al, 1983b). In addition, the NRC has sponsored simulator studies managed by the Oak Ridge National Laboratory (ORNL) that have been directed largely to measurement of response time (Haas and Bott, 1979; Bott, Hutto et al, 1981; Bott, Kozinsky et al, 1981; Barks et al, 1982; Beare, Crow et al, 1982; and Beare, Dorris et al, 1982, 1983a).

Studies that simulate tasks performed outside the control room are also needed. WASH-1400 and subsequent PRAs indicated that much of the human error impact on the availability of safety systems arose from maintenance and calibration tasks and the errors associated with restoring these systems to their normal operating states, rather than from control room activities (Zion Probabilistic Safety Study, 1981; Indian Point Probabilis- tic Safety Study, 1982; Arkansas Nuclear One Unit #1 [Kolb et al, 1982a]). This area of research has not yet been addressed.

Clearly, it will take time to collect the large body of human performance data needed. In the meantime, the NRC is investigating an interim solution that involves the use of expert judgment. In his article on the use of ordinal scaling for HRA, Rook (1964) noted that although people are not good estimators of absolute error probabilities, they can reliably rank- order human tasks in terms of some single dimension such as task diffi- culty, error-likeliness, or danger. Psychological scaling, as this method is called, is not new. The basic techniques have been in use since the 1920s. Under sponsorship by the NRC, SNL has been managing research in this area, as described in Chapter 8. Two reports are available. Stillwell et al (1982) reviews the field of psychological scaling, and Seaver and Stillwell (1983) describe four recommended methods of psycho- logical scaling of possible use for HRA. This research is continuing with a tryout of two of the methods.

For those skeptical of using subjective data in risk assessments, Meister (1978, p 383), makes two relevant points:

"(I) One should not think of subjective and objective data as an irreconcilable dichotomy. The subjective . . . data bank will complement the objective one and may be able to solve design problems for which presently available data are not suitable. (2) Efforts to validate and revise the subjective . . . data bank should proceed concurrently with its use. Ultimately the subjective data bank may be transformed into an acceptably objective form. Until that time comes, it will help us to do what needs to be done."

22-5

Future Prospects of HRA/PRA

What is needed, then, are objective and subjective data banks of human per- formance. The objective data bank will consist of HEPs observed "on the 0 job" and the PSFs associated with the tasks. The subjective data bank will

consist of expert opinions quantified by psychological scaling techniques.

Each of the above research areas can provide data that relate errors to different ergonomic features of equipment and to different types of proce- dures, thereby reducing the uncertainty of our estimates. Some people have said that ergonomic changes are frosting on the cake and represent an un- affordable luxury. We hope that no one will use our imperfect data to justify less than the best available ergonomics in a future plant or to justify failure to implement reasonable ergonomic improvements to existing plants. Our human performance models are not sufficiently fine-tuned to quantify the reduction in error to be derived from incorporating every recommended principle of ergonomics. But we know from experience that a system that does not incorporate standard ergonomic practices is a system at risk.

Prospects for the Future of HRA/PRA

The Handbook is oriented toward PRAs of NPP operations. When we prepared the draft issue in October 1980, the only major PRA completed on U.S. NPPs was WASH-1400. Since that time, several PRAs, each with HRAs of varying complexity, have been completed. The draft. Handbook has been used in nearly all of them. In talking with those who used the Handbook, in our reviews of HRAs in several of the published and unpublished PRAs (see Kolb et al, 1982b, and Berry et al, 1983, for reviews of the Indian Point and Zion PRAs), and in an SNL-managed evaluation of the Handbook (Brune et al, 1983), several things became apparent:

(1) The Handbook needed considerable revision. Although nearly all of the reviewers praised the Handbook concept and agreed with its modeling and derived data, many difficulties in using the draft Handbook were identified. HRAs performed by different analysts could differ con- siderably. The present issue of the Handbook, especially the new Chapter 20 with its search scheme and reorganized data tables, is an attempt to reduce gross differences in analysis. Appropriate training in HRA will be necessary to reduce differences to a minimum.

(2) The lack of modeling of the cognitive aspect of behavior in an abnormal event was rightly considered by several reviewers to be a serious gap in our modeling of human behavior. We have answered this criticism only to a limited degree. Our diagnosis models are but a first step towards modeling this difficult area of human behavior. Considerable work is needed in the area of cognitive psychology to develop more detailed and accurate models that can be used to solve practical problems in HRA. Models that require unobtainable data obviously are of no use for dealing with today's HRA problems.

(3) A detailed HRA can identify any serious man-machine deficiencies, which can usually be corrected by improving the related PSFs, by

222-6

Future Prospects of HRA/PRA

changes to the equipment, procedures, or training and practice pro- visions. Regardless of the accuracy of the estimates of HEPs, the very nature of a detailed HRA forces the analyst to address the spe- cifics of man-machine and man-man interfaces as he attempts to iden- tify potential error problems in both normal and abnormal operating conditions in NPPs. When the outputs of the HRA are incorporated into the overall system analysis in a PRA, the relative importance of the potential errors can be assessed. Some participants in PRAs have commented favorably on the use of HRA as a tool to indicate human factors problems that need to be corrected in existing plants or in plants under design. Usually the cost of correcting such problems,. even in-operating plants, is minimal compared with the potential costs of not correcting them.

(4) Accuracy in estimates of error-likely situations and the HEPs associ- ated with them is, of course, highly desirable. Yet in the context of a PRA, considerable quantitative inaccuracy is often tolerable. The use of sensitivity analysis enables the analyst to determine how important major errors in analysis are for system consequences.

(5). Several participants in PRAs have stated that they are concerned that the influence of human errors on system safety can be underestimated by those not knowledgable in the technology of human performance in general and in the types of tasks that NPP personnel must perform. It is apparent that the best HRAs are the result of the efforts of a team of experts in various fields: human performance technology (the psychology of human behavior in an applied setting), HRA techniques, system analysis (including conventional reliability technology), and NPP operations (the plant personnel are the best source of knowledge in this field). An HRA that is weak in any of these areas of exper- tise can result in erroneous assumptions about human behavior and the extent to which human errors are committed in carrying out system- required tasks. Carlson et al (1983) provide recommendations for the makeup of a PRA team, specifically including the above sources of expertise.

(6) Based on our reviews of several PRAs, it is necessary for analysts to keep a detailed record of the assumptions and judgments they make in modeling human performance and in estimating HEPs for a PRA. Without such a record, the HRA sections may lack credibility, and the credi- bility of HRA itself suffers. It is not enough to say, as is stated in some PRAs, that the HRA was based on the Handbook. Recommendations for documentation of an HRA are provided in NUREG/CR-2254.

Despite the above problems and limitations of HRA, it is a very powerful analytic tool. When used by qualified analysts, the HRA will contribute materially to assessing risk in a PRA, and can enable managers to make cost-effective decisions on the priority of human factors problems that need to be corrected during any phase in the design-to-retirement life of a complex system such as an NPP.

22-7

Future Prospects of HRA/PRA

Finally, as we said in the Foreword, it is our intent that the Handbook be a living document. Our next effort will be to apply the human performance models and estimates of HEPs in a full-scale PRA, emphasizing system- required human actions and extraneous actions that are associated with responding to different.kinds of abnormal events. We plan to put the Handbook to a severe test and to prepare any necessary revisions to the models and HEPs. We sincerely hope that the users of the Handbook will inform us of their experiences in using this document.

22-8

Appendix A

Overview

APPENDIX A

METHODS FOR PROPAGATING UNCERTAINTY BOUNDS IN A HUMAN RELIABILITY ANALYSIS AND FOR DETERMINING

UNCERTAINTY BOUNDS FOR DEPENDENT HUMAN ACTIVITIES

The methods presented were developed by R. R. Prairie, Reliability Department, Sandia National Laboratories.

Overview

In probabilistic risk assessments (PRAs) for nuclear power plants (NPPs), the assignment and propagation of uncertainty bounds (UCBs) are performed

using methods described in Chapter 12 and NUREG/CR-2300. In this Handbook, the assignment and propagation of UCBs for human error probabilities (HEPs) in the human reliability analysis (HRA) portions of a PRA are discussed in Chapter 7. This appendix presents two methods with different objectives for handling UCBs in HRA. The first, a UCBs propagation method for HRA, is an approximate method for propagating the UCBs associated with each HEP in an HRA. The second, a dependence UCBs assessment method, is a method for determining UCBs for conditional HEPs based on the dependence model pre- sented in Chapter 10.

The first. method represents a technique to propagate the individual UCBs

for each HEP in an HRA event tree to obtain the UCBs around the total failure probability represented by the tree. The second method corrects an oversight in the draft version of the Handbook, in which no guidance was provided for the assessment of UCBs for conditional HEPs derived on the basis of our dependence model.

This appendix includes sections describing the development of the UCBs

propagation method, a set of rules for its application, a set of rules for assessing UCBs for dependent human actions, and an example that illustrates the application of both methods. The last section in this appendix shows a very close correspondence between a sample application of the UCBs propaga- tion method and results obtained via computer simulation using a Monte Carlo procedure.

Although the discussion of the UCBs propagation method is based on UCBs for HEPs, the method can apply equally well to the propagation of UCBs for equipment failure terms. Based in part on the method of moments described in Chapter 12 of NUREG/CR-2300, the method in this appendix has been de- veloped for lognormal distributions of failure events but can be modified to fit other distributions. It is not intended that this method supplant UCBs that are developed from actual data, which are, of course, preferable. For meaningful application of the UCBs propagation method, it is necessary

that the distribution of each HEP be accurate and truly representative. If the UCBs about the individualHEPs are grossly inaccurate, the resulting UCBs on the total failure probability may be suspect. One should not

A~-1

Development of the UCBs Propagation Method

assume that large UCBs compensate for gross inaccuracies in estimates of nominal HEPs and their distributions.

Development of the UCBs Propagation Method for HRA

In a PRA, the output of an HRA event tree is inserted at the appropriate

level of the system analysis, as described in Chapter 5. For example, in some box in a system fault tree, there may be an HEP that itself is the

result of several actions, including recovery factors. Each such HEP will consist of a best estimate, the "nominal HEP," with an error factor (EF) describing its uncertainty bounds. For example, an EF of 10 about a nomi- nal HEP of .01 means that the lower bound of uncertainty is HEP/10 = .001, and the upper bound is HEP x 10 = .1. The typical application is to con- sider the best estimate of the HEP as a median and the lower uncertainty bound (L) to be the 5th percentile HEP and the upper uncertainty bound (U) to be the 95th percentile HEP of a lognormal distribution. If additional

conservatism is desired, L and U can be defined to be different percen- tiles, such as the 10th and the 90th.

As discussed in Chapter 7, there are various approaches to estimating the

UCBs about the HEPs that are inserted into a system analysis. We believe that the method presented here provides bounds on the final estimate that closely approximate the true bounds that could be closely determined with a

Monte Carlo procedure but without the expense and time of simulator runs that this procedure requires. A similar approach has been used in the West German Reactor Safety Study (TUV Rheinland, 1981, Section 3.2.5).

Figure A-i illustrates a typical HRA event tree, as described in Chapter 5. The failure limbs would normally be designated with capital letters A, B, ... N. For this discussion, however, it is convenient to take exception to our usual Handbook symbology and designate F. as failure on any task, F as the end-failure term for any of the threeliailure paths in the tree, and

FT as the total-failure term whose probability would be entered into the system analysis in a PRA. For example, failure limbs F , F , and F

1 1 21 13 make up the end-failure term F ; F , etc., make up the'end- 1lure terib F; and so on. 1 2,12

In the tree, the dotted lines from each unlabeled success limb indicate the

operation of several recovery factors. For the situation as diagrammed by the HRA event tree in Figure A-I, system failure (F ) occurs if F or F or F3 occurs. Failure F occurs if F and F and T all occur1 Simi-3 1 1,1 1 213 ocu; sn. larly for F2 and F 3. The F.j may represent Basic or •onditional human errors.

The probability of end-failure term Fi is, in general,

n. 1

Pr[F.] = HPr[F..] (Al) I j 13

where n. is the number of failure limbs in failure path i. 1

A-2

Figure A-i

IF1 ,1

1,3

r 2,1 F1

F2 ,2, F2 ,3

F2, 4

F 30,2 F2

S 1 F3

FT =- F 1 + F2 + F3

Figure A-I An HRA event tree with three complete-failure paths.

A-3

Development of the UCBs Propagation Method

For this example, the probability of end-failure term F is

Pr[FI] = Pr[F1 ,1 and F2 and F, 3 ]

= Pr[F1] x Pr[F,] x Pr[F

111,21,

and the probability of system failure for this example is

Pr[FT] = Pr[F or F2 or F3

Stated in general terms,

Pr[FT] = Pr[F or F2 ... or ... F

A good approximation to Pr[F I in Figure A-i is T

Pr[FT] = Pr[F I] + Pr[F 2] + Pr[F3

This approximation will be conservative and is very good when each Pr[F..]

is .01 or lower and the number of terms is not large. This is because He joint probabilities that are ignored contribute very little to the total probability. That is, only the single additive terms shown contribute a material amount to the total probability of failure. Even for [F..] > .01,

the approximation is often used with the associated increase in coIser- vatism when Pr[F..] .1.

The approximation is

Pr[F Pr[F.] (A2) T

where n is the total number of failure paths through the HRA event tree.

It is possible to obtain the approximate UCBs on PrF TI by determining the distribution of Pr[F ] and then obtaining the lower and upper 5th percen-T tiles. The general procedure is described below. In this development, ln refers to the natural logarithm.

The Pr[F..] are assumed to be distributed lognormally as discussed in

Chapter i The validity of this assumption is not addressed here. There- fore, Pr[F.] as given by equation Al is also lognormal. Pr[FT I as given by equation Aý is assumed to be approximately lognormal. This assumption is

supported by analytical studies by Mitchell (1968) who stated, "The distri- bution of the sum of log-normal variates is shown for most cases of inter- est to be very accurately represented by a log-normal distribution instead of a normal or Rayleigh distribution that might be expected from the central-limit theorem." In addition, our conclusion is supported by several simulation studies at Sandia National Laboratories.

A-4

Development of the UCBs Propagation Methoc

Obviously, if n is very large, the distribution will approach the normal. However, for typical HRA cases, n will be small, usually < 10, and the lognormal approximation will hold.

It follows that the mean and variance of ln Pr[F I can be derived based on the mean and variance of the Pr[F.]I and UCBs obtained on Pr[F T. The details follow.

The basic UCBs are on the Pr[F..]. Each HEP, whether it be a basic HEP or a conditional HEP, is expresse•3as

M ij (L ij, Uij)

where M.. is the "best" estimate and is taken to be the median, L.. is the lower 56t percentile and U.. the upper 5th percentile of the uncertainty distribution on Pr[F..]. Nually, M.., L.., and U.. are determined by the human reliability analyst, using theliable in Chaier 20 or other sources of information to estimate these parameters. Further, as previously stated it is assumed that the uncertainty distribution is lognormal. In some cases, the expression M..(L. ., U..) is not compatible with a lognormal dis- tribution.. This occurs'Zhen the'aistribution of the HEPs is truncated at U.. = 1.0 or when the bounds are not symmetric on the log scale. In such caaes, the "best" estimate is retained as the median, and the ratio U j/Lij is taken as the ratio of the 95th to 5th percentile for calculationali convenience. The case of complete dependence is where the main violation occurs of this lognormal assumption and assigned U. . and L. . parameters. It is conjectured that the significance of this vio ation 1 minor because the number of -completely dependent.events is small and the contribution to the variance of the HEPs of these events is minor. It is not known how this violation affects the UCBs. If there is a large number of completely dependent terms, this method should be replaced by a simulation. However, in a typical HRA when the assignment of complete dependence is made between human activities, they are usually treated as a single activity (or task).

From the lognormal assumption on Pr[F..] and the definition of M..(L.., U..), the -mean and standard deviation'af ln Pr[F..] can be calcu±teA?

13 Tis follows since ln HEP is normal and therefore ln L.. = - 1.645 a.., and ln U + 1.645 j., where 4 ij and ai are the mean bd standar•3 deviation of in HEP. Thus,

= in M ij (A3)

ln U.. - ln L.. 13 13

=ij 3.29

-ln -3(A4) 3.29 Lij

A-5

Development of the UCBs Propagation Method

It should be noted that the latter equation will be a biased estimate for the case where the conditional HEP equals 1.0, but the contribution to un- certainty from this case is small for the present applications.

Since Pr[F..] is lognormal, Pr[F.] is also lognormal with mean and standard 'deviation 1 in Pr[F.] given by

in Pr[F i]

0 ln Pr[F.] 1

ln M= • In 1) (A5)

(A6) 3.29/

I in

U >2

Lij

Uncertainty bounds on Pr[F.], if desired, are given by1

Lpr[F = I exp I iln Pr[ Fi] - 1.645 aln Pr[Fi]I (A7)

(A8)Up r[F'I = expI 'ln Pr[F.] + 1.645 ain Pr[Fi]]

To get UCBs on Pr[F 1, one could obtain the distribution of Pr[F I and then select the lower aný upper 5th percentiles as the bounds. Based on some

earlier simulation studies on the distribution of the sum of lognormal variables, it appears that the distribution of Pr[F ] can be adequately approximated by the lognormal rather than the normaT. Thus, the UCBs can be obtained by using the mean and standard deviation of in Pr[FT].

T Recall that

Thus,

PrlF T ] Pr[F ]

11Pr[lF ] P lPr[lF. T1

(A2)

(A9)

and

2 0 Pr[FT]

Pr[F.]1 (AWO)

A-C

Rules To Determine UCBs on Pr[FT]

The mean and variance of Pr(F.] can be determined from

I1Pr[Fj = expP{'in Pr[F.] + -1 n Pr(F i](Al 1 12

r[F~ - I [n Pr[F.] in Pr[F .] n Pr[F+]

where p in Pr[F.] and y in Pr[F] are defined by equations A5 and A6.

Then, PPrIF T] Pr[FT] are obtained by substituting W i , as given .T T 2 Pr[.11by equation All, into equation A9 and substituting a P as given by

equation A12, into equation A10.

The mean and variance of in Pr[F T are

UPr[ FT

Pin Pr[F in (A13) T G Pr[FT

+ 2pr [T 1+ [ T

2 Pr 1 T in PrF = in 1 + 2 (A14)

2 T Pr[FT

Thus, the median and UCBs on Pr[F T are

11in PrF [FT MPr[F T e T (Al5)

LPr[FT] T exp [ Jin Pr[FTI - 1. 6 4 5 ain Pr[FT]] (A16)

UPr[FT] exp [P'in Pr[FT] + 1.645 a in Pr[FT]1 (A17)

Summary of Rules To Determine UCBs on Pr[FI

The rules to determine the UCBs on Pr[FT I are summarized in the following six steps.

A-7

Rules To Determine UCBs on Pr[F T

Step 1

Prepare a five-column table of the following form:

1 2 3 4 5

Failure Limb (F..)

End-Failure Term (F.)11

Dependence Level

Median HEP

1. Input to the Failure Limb column is the symbol given to the particular F.. failure term being considered. In the usual Handbook symbology, eah limb is designated with a capital Roman letter. In this appendix, we have used FI, F1 ,2, etc. The symbology used is immaterial.

2. The End-Failure Term, F., refers to F F 2, and any other terminal points in a failure patA through the ARA event tree.

3. The Dependence Level refers to one of five dependence levels, as de- fined in Chapter 10. These are zero dependence (ZD) (i.e., the basic HEP), low dependence (LD), moderate dependence (MD), high dependence (HD), and complete dependence (CD). The next section describes how the UCBs are determined for each dependence level.

4. The median HEP is the "best" estimate taken from the Handbook or other data source and in this appendix is designated as Pr[F. .], a median of a lognormal distribution of HEPs.

5. U ij/L ij is the ratio of the upper bound HEP to

The first line of a typical table would be:

the lower bound HEP.

Failure Limb (F..)

F1 F 1,1

End-Failure Term (F.)

F 1

Dependence Level

Median HEP

.001

U. ./L.. 13 03

100

Step 2

For each Fi, compute

Pr[F. 1n Mij1 j

2 0 ln Pr([Fi] 1)2[ nBi Uj) 2]3 . 2 9 i I / L j

A-8

Rules To Determine UCBs on Pr[F T]

Step 3

For each F. compute1

11~ =ix +c n Pr [Fi

'Pr[F]J exp [yin Pr[F.] 2

1 1

Pr[F] exp I21n Pr[F] + 2iln Pr[F] [expan Pr[F ]

Steo pu

Compute

11Pr[F :i= IPr[F.] T I

Compute

Pr[F T]

lin Pr[FT

= a2 Pr[Fi]

= in

2 0 1n Pr[FT] = In

Step 6

Compute

e11 n Pr[F T prTF ]

A-9

UCBs for CHEPs

L pr[F T exp k'ln Pr[FT - 1. 6 4 5 Oln Pr[FT]

UPr[FT exp In Pr[FT] + 1.645 ain Pr[FT]

Steps 1 through 6 are illustrated with an example in a later section. First, however, it is necessary to introduce our method for determining UCBs for the levels of dependence from the dependence model presented in Chapter 10.

A Method for Determining UCBs for Conditional

HEPs Based on the Dependence Model

Review of Dependence Model

In Chapter 10, a dependence model is described that divides the continuum of positive dependence into five discrete points: ZD, LD, MD, HD, and CD.

In the typical use of the model, the analyst judges what level of de- pendence is appropriate for some task following an initiating task. The

initiating task normally is assigned a basic HEP, that is, an HEP without considering the effects of dependence. The HEPs for succeeding tasks

represent conditional probabilities and may reflect dependence levels.

For convenience, the equations for conditional HEPs

are listed here, where Task "N" follows Task "N-I."

Level of Dependence

Zero

Low

Moderate

High

Complete

Equation

Pr[IF,,N,,IF,,N-1,,IZD] = N

1+ 19N Pr[F,,N,,IF,,N

111ILD] = 20

Pr[F,,N,1 IFN-1_ ,IMD) + 6N

Pr[F,, IF,, ,IHD] = I +N2

N'i 'N-_1

PrIIF,,N,,IF "N-1,ICD] = 1.0

any one of the five

(CHEPs) from Table 10-2

CHEP for N

BHEP = CHEP

.05 for BHEP .01"

.15 for BHEP ( .01

.5 for BHEP • .01

1.0

The conditional HEPs for the three intermediate levels of dependence can be

approximated as indicated above. For BHEP > .01, it is recommended that the equations or the values in Table 7-3 be used to determine the condi- tional HEPs.

Substitution of BHEP = .01 in the equation for LD results in a calculated

CHEP = .06. For BHEPs < .005, the equation will result in CHEP = .05 (rounded). However, to avoid the appearance of pseudoaccuracy, we employ CHEP = .05 for BHEP < .01.

A-1O

UCBs for CHEPs

Qualitative Aspects of UCBs for Dependence Levels

When a qualified analyst assesses a particular level of dependence between two tasks (or people), he will have some degree of uncertainty. However, if he bases his assessment on a thorough task analysis (Chapter 4), he should not be in error by more than one level of dependence in either direction. For example, if he makes an assessment of MD, it is likely that the true level of dependence would have a high probability of falling, within the range defined by LD and HD. A probability of .9 is assigned. The present method for assigning UCBs for dependence levels makes this general assumption, with slight modifications.

The Method

The method for determining UCBs for conditional HEPs based on levels of dependence has four major rules:

(1) The HEPs for any task are assumed to be lognormally distributed. In all cases, U is assumed to be a 95th percentile of the HEPs for a task, the best estimate of the HEP (i.e., the nominal BHEP or CHEP) is the 50th percentile, and L is the 5th percentile. When U = 1.0, it is assumed that Pr[CHEP = 1.01 = .05, but when the CHEP = 1.0 (which is the case for complete dependence), it is assumed that Pr[CHEP = 1.0] = .5. In this case, U = CHEP = 1.0. An alternative approach for the case of complete dependence, suggested by P. Kafka and G. Zipf of the West German Institute for Reactor Safety, is to assume no uncertainty is associated with CD and to use the single value of CHEP = 1.0, i.e., EF = 1.0. This approach will give somewhat pessimistic answers in the usual HRA since half of the CHEPs given CD could range between .5 and 1.0, whereas with the suggested approach, all of the CHEPs given CD would equal 1.0. In the following discussion, we use our approach in which L given CD is < 1.0.

(2) Calculation of upper uncertainty bounds:

a. For CHEPs based on LD, MD, and HD, use the CHEP for the-next higher level of dependence unless that value is smaller than the U for the next lower level of dependence, in which case, use the latter bound.

b. For ZD (the BHEP), use the U for the BHEP as given in the Handbook (Chapter 20).

c. For the CHEP based on CD, define U = CHEP = 1.0.

(3) Calculation of lower uncertainty bounds:

a. For CHEPs based on LD, MD, and HD, use the equation L = (CHEp) 2/U. (This is based on the lognormal distribution.)

b. For ZD (the BHEP), use the L for the BHEP as given in the Handbook (Chapter 20).

A-1 1

Application of the UCBs Propagation Method

c. For the CHEP based on CD, use the nominal CHEP for HD.

(4) When BHEP < .01, for purposes for practical applications, it is ade- 0 quate to use the following CHEPs and UCBs:

LD = .05 (.015 to .15)

MD = .15 (.04 to .5)

HD = .5 (.25 to 1.0)

CD = 1.0 (.5 to 1.0)

When BHEP > .01, use rules 2 and 3 as stated above.

An Illustration of the Method

To illustrate the use of the rules for setting UCBs, three examples are shown in Table A-i. These examples represent BHEPs that range from very low to high. For purposes of this illustration, rule 4, above, was not used; the CHEPs for the three intermediate levels of dependence were calculated using the equations presented earlier in this section.

It can be seen that the values for the UCBs in the table are consistent with our statement that 90% of the estimates should not be off by more than about one level of dependence either way. In the case of HD, this gen- eralization is weaker for BHEPs of 10-5 or 10-K. However, in both cases, the .25 HEP for L is only about a factor of 1.7 larger than the HEP for the MD level. Considering the subjectivity of our dependence model and the fact that L is higher, thus yielding more conservative results, this is not a bad match.

The fit to the above generalization appears to be very poor for LD, but this is because of the large distance from the BHEP to the CHEP based on LD. This is not a major problem because, when a skilled analyst estimates LD, it is not very likely that he would err so much that the true level of dependence was actually zero. He has made the judgment that there is some nonzero level of dependence, and he should not likely be wrong in this assessment.

An Example Application of the UCBs Propagation Method

In Figure 5-3, an HRA event tree was used to illustrate the assignment of BHEPs and conditional HEPs to the limbs in a tree and for the calculation of the point-estimate associated with the total failure probability. The present section expands that tree to illustrate the propagation of UCBs through an HRA event tree to produce a set of values that could be incor- porated into the appropriate level of a system analysis. In the example presented here, the final output might be inserted into some box in a system fault tree so that the human action could be treated analytically in the PRA just as the other system events.

A-1 2

Table A-I

Table A-i Examples of UCBs for dependence levels determined by the method in this appendix

Levels of

Dependence .00001 (EF = 10)

.00001 (.000001 to .0001)

.050 (.017 to .143)

.143 (.041 to .5)

.5 (.25 to 1.0)

1.0 (.5 to 1.0)

BHEPs

.001 (EF = 3)

CHEPs and UCBs

.001 (.0003 to .003)

.051 (.018 to .144)

.144 (.041 to .5)

.5 (.25 to 1.0)

1.0 (.5 to 1.0)

.1 (EF = 5)

.1 (.02 to .5)

.145 (.042

.229 (.095

.55 (.3 to

1.0 (.55 to

to .5)

to .55)

1.0)

A-13

Application of the UCBs Propagation Method

Figure A-2 shows the expanded HRA event tree. The primary difference between this tree and that shown as Figure 5-3 is that the present tree includes limbs for the recovery factors afforded by extra people in the control room. This particular example is based in part on a real-world

analysis in which there were two reactor operators present in the control room, with a shift supervisor nearby. For the six failure paths, all three personnel have to fail. This tree is explained in greater detail in Chapter 21, where the derivation of the estimates of HEPs is presented. For the current illustration, however, the reader should accept these numbers without explanation.

To facilitate the use of Table A-3 (which constitutes Step 2 of the UCBs propagation method), we have numbered the failure limbs in each F se- quentially and have designated the appropriate level of dependence beside the BHEP or CHEP assigned to each limb. Table A-2, using the terminology developed early in this appendix, describes what each human action refers to. In general, the first failure limb in each failure path refers to an error made by the control board operator, the second failure limb to a recovery error made by his backup, and the third failure limb to a recovery error made by the shift supervisor. The dotted lines represent success probabilities that recover the previous person's error. Thus, the top dotted line in Figure A-2 indicates that the backup control board operator caught the error made by the first operator (failure limb #1). The dotted line takes us back to the complete-success path in the tree.

In F 2 , F 4 , and F5, there are annunciator (ANN) recovery factors that must be overlooked by all three personnel in the control room. In Table A-2, the phrase "fails to note relevant ANNs" means that one or more ANNs came on, and the operating crew cancelled the auditory portion of the signal and turned off the flashing of each ANN tile but then failed to initiate some intended corrective response to the message on each tile. This could have happened because they were busy doing something else during the transient or because of other possible distractions.

Equation A2 can be used to calculate an estimate of the single-point total failure of the system shown in Figure A-I. For those interested in cal- culating this Pr[F ], following are the Pr[F.] terms that are added simply

T I to obtain an approximation of the total failure probability. As usual, we round at the end.

Pr[F 1 .009 x .15 x .05 = .0000675

Pr[F 2 .1 x .55 x .23 x .0006 x .5 x .5 = .0000019

Pr[F 3 .02 x .5 x .15 = .0015

-8 Pr[F 4 .02 x .5 x .15 x .0001 x .5 x .15 = 10

-8Pr[F5 I .02 x .5 x .15 x .0001 x .5 x .15 = 10

A-1 4

Figure A-2

.009 ZD

.15 MD

F 3 .5 HD

1 ZD

15 MD

F 4

LEVEL OF DEPENDENCE IS SHOWN AFTER EACH HEP

Figure A-2 Expanded HRA event tree for loss of steam generator feed.

(Note: This tree is an expansion of Figure 5-3.)

A-1 5

Table A-2

Table A-2 Explanation of failure limbs in Figure A-2

Failure Limb (F. -

. . Description

FI,1 R01 (reactor operator #1) fails to note the relevant ANNs that indicate the need to establish feed and bleed.

F1,2 R02 (reactor operator #2) fails to correct RO1's error.

F1,3 Shift supervisor (SS) fails to correct ROl's error.

F 2 ,1

F 2 ,2

F 2 ,3

F 2 ,4

F 2 ,5

F 2 ,6

F3, 1 F 3 2 F3 3,

F4 ,1

F4, 2 F 4 3 F 4 ,3

F 4,5

F4 ,6

F5 ,1 F5,2

F 5 ,3

F 5 ,4

F5,5

F5,6

F6 ,1

F 6 ,2

F6 ,3

R01 noted the relevant ANNs but makes an incorrect diagnosis and does not initiate feed and bleed.