ML20206S393

From kanterella
Jump to navigation Jump to search
Summary of 980121 ACRS Subcommitee on Human Factors Meeting in Rockville,Md Re Human Performance & Reliability Plan & Integration of Human Factors Insights Into Insp Process
ML20206S393
Person / Time
Issue date: 01/28/1998
From:
Advisory Committee on Reactor Safeguards
To:
Advisory Committee on Reactor Safeguards
References
ACRS-3087, NUDOCS 9905210121
Download: ML20206S393 (10)


Text

W 6 YRS 3001 fDR Issued: January 28, 1998 4

CERTIFIED: February 3, 1998 ADVISORY COMMITTEE ON REACTOR SAFEGUARDS HUMAN FACTORS SUBCOMMITTEE MEETING MINUTES

. JANUARY 21, 1998 ROCKVILLE, MARYLAND The Advisory Committee on Reactor Safeguards (ACRS) Subcommittee on Human Factors held a meeting on January 21, 1998, in Room T-2 B3, 11545 Rockville Pike, Rockville, Maryland, with representatives of the U.S. Nuclear Regulatory Commission (NRC).

The purpose of this meeting was to hear presentations by and hold discussions with the NRC staff regarding the Human Performance and Reliability Plan, integration of human factors insights into the inspection process, results of studies that used the human performance database, and the status of the ATHEANA pilot demonstration at Seabrook Nuclear Station. Mr.

Noel Dudley was the cognizant ACRS staff engineer for this meeting. The meeting was convened at 8:30 a.m. and adjourned at 5:50 p.m.

ATTENDEES 6CEi G. Apostolakis, Chairman J. Carroll, Consultant T. Kress, Member A. Mosleh, Invited Expert D. Miller, Member M. Stutzke, Invited Expert D. Powers, Member N. Dudley, ACRS Staff R. Seale, Member NRC Staff W. Hodges, RES T King, RES QSO\\o g J. Persensky, RES N. Siu, RES M. Cunningham, RES E. Trager, AE0D S. Rubin, NRR J. Rosenthal, AE00 rq go G. Meyer, RI J. Bongarra, NRR gg No written comments or requests for time to make oral statements were received from members of the public. A list of attendees is available in the ACRS office and will be made available upon request.

9905210121 900128 "

E7 ACRS PDR DESIcrATED ORIGINAL

~

c.m-um =_ M s

yy-3

Human factors Subcomittee 2

January 21. 1998 Y

OPININGREMARKSBYTHESUBCOMMITTEECHAIRMAN Dr. George Apostolakis, Chairman of the Human Factors. Subcommittee, convened the meeting at 8:30 a.m.

He introduced a consultant Mr. James Carroll, and invited experts Dr.. Ali Mosleh, Professor at the University of Maryland and Mr. Martin Stutzke, Science Applications International Corporation (SAIC).

Dr. Apostolakis note that the consultant and invited experts would participate as members of the Subcommittee.

NRC STAFF PRESENTATIONS Human Performance and Reliability Plan - Dr. Julius Persensky, RES i

Dr. Persensky, RES, presented the background associated with the development of the Human Performance and Reliability Plan and its relationship to the NRC

. Strategic Plan, Performance Plan, and Office Operating Plans. The staff is

. revisiting the plan's structure, model, and prioritization scheme.

Dr.

Persensky explained the linkage among planning activities, the strategic arenas, the conceptual model for the planning process, and possible prioritization methods.

He presented the staff's schedule for completing the plan, meeting with the ACRS, and briefing the Commission.

The Subcommittee Members and the NRC staff discussed the transitivity of the

. pair wise comparison technique and the analytical hierarchy process (AHP) being investigated by the Office of Nuclear Regulatory Research for prioritizing research activities. They also discussed tools, such as the human performance inspection process, that were available to the staff in making regulatory and enforcement decisions. Other issues discussed included:

training related to tools used to assess human performance.

identifying the need for each plan activity, developing a consistent framework to link substrategies and activities, international involvement in NRC human factors activities, staff's inclination to delete activities based on its revised plan, factors that goes into building a plan.

event reporting requirements, how anticipatory research is identified, and adequacy of the staffing and resources available to accomplish the planned human performance activities.

p o

a r

Human factors Subcomittee 3

January 21, 1998 use of Human Factors Considerations in the Insoection Process -

Mr. Stuart Rubin, NRR, and Mr. Glenn Meyer, RI Mr. Rubin, NRR, explained the attached simplified process diagram. The diagram illustrates how human factors considerations are integrated into the

)

' inspection process. Mr. Meyer, RI, explained the human performance aspects of i

facility inspection plans, plant performance reviews, plant issues matrices.

and NRC inspection procedures. Mr. Rubin presented how plant operational data related to human performance is collected, disseminated and analyzed. He listed the NRC training programs associated with inspecting for human performance and the staff resources available for conducting human performance inspections.

Mr; Rubin described how the human factors information system data are_ compiled and used in the regulatory process.

The Subcommittee Members and the staff discussed the following issues:

.the importance of inspecting human performance:

the appropriate level of detail for performance-based inspection guidance:

linkage between lower level performance inspectico guidance and higher level performance-based criteria:

lack of guidance for evaluating crew performance, low-power and shutdown operations, and activities outside the main control room:

j how probabilistic risk assessment (PRA) results are used in designing

. inspection guidelines:

how the plan issues matrix is used:

why the inspection process did not identify human performance problems that contributed to significant events-i NRC' support for human factors activities:

.the impact.of human performance inspections on plant safety; and use'of risk-worth in and cost effectiveness of inspection activities.

Human Factors Subcomittee 4

January 21. 1998 Hudian Performance Event Database - Mr. Eugene Trager, AEOD Mr. Trager. AE0D. explained how the human performance event database (HPED) was developed and identified the sources of data.

He presented the results of a report concerning the performance factors that contributed to events.

He described the human performance event information available in the HPED. Mr.

Trager concluded by explaining the plans to:

update the HPED.

perform studies using the HPED, develop information for studying human reliability, and-assess the adequacy of the HPED for modeling human reliability.

Mr. Jack Rosenthal. AE0D. noted that the report results indicate that for the most safety significant events:

management and organizational factors are most frequently found, procedural factors are not frequently found, and human factors engineering. workload, and work environment factors are least frequently found.

The Subcommittee Member and the staff discussed the following issues:

trends seen in the data.

4 definition of the performance factors that contributed to events.

use of human reliability models in developing the HPED structure,

=

improving licensee event reporting guidelines to require better data on human performance, and the failure to inspect events that were of high safety significant based on the accident sequence precursor program.

Status of the ATHEANA Pilot Demonstration - Mr. Mark Cunningham, RES Mr. Cunningham, RES, provided an overview of the development of a new technique for human event analysis (ATHEANA) and the associated documents.

ATHEANA is a human reliability assessment (HRA) tool.

He presented the goals, success criteria, and approach used for the ATHEANA demonstration project conducted at the Seabrook Nuclear Station.

Using ATHEANA, the Seabrook staff developed a scenario consisting of a medium sized loss of coolant accident as I

h l

4.'

Human factors Subcommittee 5

January 21. 1998 an ' initiating event and two instrument failures that provided performance

. forcing _ functions for staff responses. Mr. Cunningham stated that the demonstration success criteria had been met. He presented plans for completing a peer review of ATHEANA. linking ATHEANA with the management and organization. effort at Idaho National Environmental and Engineering Laboratory (INEEL), and translation of ATHEANA into a computer-based analysis tool.

.The Subcommittee Members and the staff discussed the following issues:

how the peer review process relates to the ATHEANA model.

the' operating events to which the model can be applied.

the appropriateness of using the term " error forcing function."

use of simulator exercises to identify error forcing contexts.

the level of expertise required to search event trees.

plans for validating the ATHEANA model, and plans for evaluating other second generation HRA tools.

COMMENTS BY INVITED EXPERT Dr: Ali Mosleh. University of Maryland Dr Apostolakis noted that Dr. Mosleh,had developed an HRA tool called Information. Decision. and Action (IDA). which uses cognitive models.

Dr.

Mosleh stated that the.following questions are common.to all human performance issues:

What factors affect human performance?

How will performance be affected (i.e., error modes)?

Are the factors risk or safety significant?

How can the factors be prevented or the impact of the factors be mitigated?

He observed that human performance assessment and prediction are very

-difficult problems, which require creating a strategy that reflects the available research resources.

Dr. Mosleh stated that an HRA tool needs a systematic method of identifying and integrating operator responses into risk models and a credible method for quantification of human performance. He then presented the obstacles and challenges associated with developing an HRA tool.

He recommended that any HRA tool:

)

be developed though observation, definition of the problem, and a validation process that results in a solution:

)

u

?;

t

. Human Factors Subcomittee 6

January 21, 1998 not be over extended'beyond.its design domain; and

-be developed using the! broadest possible inputs and participation by l

domain experts.

The Subcommittee Members and.the staff discussed the risk worth associated with human performance and whether human performance needed to be improved in order to protect the public health and safety.

In Dr. Mosleh's opinion, the risk worth of the human is probably over estimated in PRAs. Mr. Stutzke very 4

.much agreed with Mosleh.

'In response to questions. Dr. Mosleh explained that ATHEANA was being developed to treat operator errors of commission..since PRAs are not suitable for evaluating dynamic situations. He noted that.the.use of simulators was necessary to validate an HRA tool and to develop completeness of HRA models.

Dr. Nathan Siu. RES, added that ATHEANA does not address time nor the thermal i

hydraulic variables related to events, and that engineering judgement by experts was necessary in evaluating the' effects of these variables. Dr.

Mosleh noted that since no linkage could be made between context parameters and human performance.. assessments used conditional failure probabilities based on the context of the scenario.

Mr. Martin Stutzke. SAIC y

Mr. Stutzke. SAIC noted that-the NRC has adopted a risk-informed regulatory

- philosophy and identified the following questions that.should be answered:

Is there a problem with the existing level of human reliability?

Can the effect of influencing factors be explained quantitatively?

How can human reliability be predicted in.a risk context?

Mr. Stutzke; stated that PRAs overstate the risk associated with human

. performance, He noted that management, regulatory, and industry 1

organizational' activities contribute to the risk of operating plants. Mr.

Stutzke made the following recommendations-Instead of developing'models the NRC should perform fundamental data and collection and analysis.

The NRC should measure the quantitative risk impact that post-THI

. modifications have had on human performance.

g i

4

p Human Factors Subcomittee 7

January 21. 1998

,s The.NRC should develop an integrated plan to direct future HRA research.

Research should be driven by observed risk-significant events.

- The Subcommittee Members discussed how the AEOD database could be expanded and how NRC HRA tools should be reviewed by independent third parties.

l SUBCOMITTEE COMMENTS. CONCERNS. AND RECOMMENDATIONS Dr. Powe'rs stated that the Human Performance and Reliability Plan has changed-little over the last year. He asserted that human actions are important to the safe operation of nuclear power plants.

He maintained, however, that there is no quantified evidence that the NRC should be approaching human performance issues differently. Dr. Powers comented that the most pressing need stated by the staff was,the opportunity to digest and learn from the application of the available tools.

Dr. ' Powers stated that he was. impressed with the apparent consensus between the invited experts that PRAs over estimate the risk associated with human i

performance. He recommended that the staff identify a mission for the NRC.

human factors activities and a vision for the desired end state.

He asserted

' that any research, which reduced the uncertainty of quantifying human

. performance and render PRAs more robust, would be' good.

Dr, Powers noted that the Office of-Nuclear Material Safety and Safeguards could apply human performance tools in the area of health physics to identify error forcing contexts in hospitals and other material licensee facilities.

Mr. Carroll commented that the staff had expressed the need for tools to evaluate management and organizational factors.

Dr. Powers contended that it

'_is difficult for the staff to make progress regarding research in the field of management and organizational. factors.

Mr. Carroll recommended that the licensee event reporting guidelines be modified to enable the collection of better human performance data.

.Dr. Seale stated that he was impressed with the attempt,to use the ATHEANA model and that the presentation related to inspectors was very good.

He commented that he was not as pessimistic as Dr. Powers concerning the progress f'

-being made on developing a plan.

i t

l

~Dr. Miller stated.that the presentations concerning the use of human

[

. performance insights during inspections and the HPED were enlightening. He I

l l

Human Factors Subcomittee 8

January 21, 1998 recommended that the NRC use consistent terminology for human performance concepts in its reports.

Dr. Kress stated that it was premature to bring the issues discussed at the i

Subcommittee meeting to the full Committee. He asserted that more emphasis and resources should be given to developing a plan, which would lead to a better understanding of human factors and to improved PRAs.

Dr. Mosleh stated that human performance data need to be analyzed with an explicit model.

He noted that the industry would benefit from the NRC sharing its information and data on human performance.

Mr. Stutzke stated that the HPED should be shared with the world.

The Subcommittee discussed the dearth of resources being directed toward the development of the plan and how the Committee could raise the priority given to the plan by the staff.

STAFF COMMITMENTS l

The staff committed to inviting the ACRS members to the peer review training session, for ATHEANA. The staff stated that it would consider recommendations j

from the Subcommittee regarding experts, who could serve on the peer review panel.

i The staff agreed to provide the Subcommittee the following documents:

an updated Human Performance and Reliability Plan, the INEEL reports associated with the HPED, and a paper by Dr. Nathan Siu, RES, related to an operator's model of a j

plant and-how it changes during a dynamic event.

SUBCOMMITTEE DECISIONS Dr. Apostolakis planned to present a Subcommittee Report at the February 1998 ACRS meeting. The Subcommittee decided not to draft a letter concerning the Human Performance and Reliability Plan.

FOLLOW UP ACTIONS The Committee plans to review the Human Performance and Reliability Plan, i

reports related to the HPED, and ATHEANA when these documents become 2available.

1

U i

Human Factors Subcommittee 9

January 21. 1998 f

Dr.' Mosleh agreed to provide the Subcommittee copies of two papers he has written concerning human performance assessment methods.

j l

The Safety Research Program Subcommittee plans to discuss the use of the analytical hierarchy process (AHP) for prioritizing research activities.

J BACKGROUND MATERIAL PROVIDED TO THE SUBCOMMITTEE PRIOR TO THIS MEETING i

Memorandum dated November 24, 1997, from Noel Dudley, ACRS Senior Staff Engineer, to ACRS Planing and Procedures Subcommittee,

Subject:

Meeting Summary Concerning the Human Performance and Human Reliability

(

Implementation Plan.

U.S. Nuclear Regulatory Commission Inspection Manual Inspection Procedure 71707, " Plant Operations," issued September 9,1997.

g Human Performance Categories Areas and Individual Issues.Found in Inspection Reports and Logged in HFIS.

J. Wreathall, The Wreath Wood Group, and A. Ramey-Smith, NRC, "ATHEANA:

A Technique for Human Error Analysis: An Overview of Its Methodological Basis."

J.

Forester and D, Whitehead, Sandia National Laboratories, et. al.,

"A Nuclear Power Plant Application of a New Technique for Human Event Analysis - ATHEANA," presented at the OECD/NEA Specialist Meeting on Human Performance in Operational Events, Chattanooga, Tennessee. October 13-17, 1997.

PRESENTATION SLIDES AND HAND 0lJTS PROVIDED DURING THE SUBCOMMITTEE MEETING The presentation slides and handouts used during the meeting are available in the ACRS office files or as attachments to the meeting transcript.

NOTE: Additional details of this meeting can be obtained from a transcript available in the NRC Public Document Room, 2120 L Street, N.W.,

l-Washington, D.C. 20006, (202) 634-3274, or can be purchased from Ann l

Riley & Associates LTD., 1250 I Street, NW, Suite 300, Washington, D.C.

I 20005, (202) 842-0034.

I i

m3M w

3

.N s

e s e

ro n u t, n c

os,e n n tcis,ug a

a a e

n ati,ci n

H Fat mn m n s. e n P

uo P nras n

Hf rM aol e a

fM mfP mP r

l e

o nr.&

h P

8 uI o

H f

s

+

o, n

sr n

o s

n on o t, o r

s o

t t o n,i o ne n

ca at c o m,e icieN i

t. cts

,,a i c e cc t

at t&.m I

a

.o F e F rin en,p nH c

,pu.ee cu,s

,ee r

nnn v

e g

r aaon Gmin s e e

o mfi i

,s S

r r

mfF

,n n

ds ooh

,IP p

f ui m

4 H

F et mis ecef C

i f

se

+

sgnga t

a nc rde oerMt i

S goo

@f r

l e c w pC Q a oxR H aCr

@f FnEN P

K &o&

f c

iDs n s

r chu..W r

o o

aoo t

a ce F

i rt t F

s )s e c c n

r nsoT 3

I a

a e m

rt c

a ocE oFp u

=

mt a crD H

u at rns HFngo.

Pan

+

=

C (e dmi e

n s e uh s

i o e i

F i

s t r s

iHt t

cu f

roien.

s r

et n

t t=

o pa

@}

ns e s

o cecmm co t

l a rmf i

fl pgt F n.,a u F-sc ac e a

o a

no mt sai rd cst c i r n e eP a

u aeSo minn a

ge mmwRo n d.oc v

HFs P

ai.r o a

im i

@(

s S

it mu.Pr t

A ca Sa uGo P

ua ar H

eg r

H C g

G r

& P n

)e e

i c

e n n s F

t a a

_1 l s

t n

nsfoo mn n r

u.unl

'm '

a r

u a

t I

osc R o (H o P mt o

cAe ft u aS p re foP A

HFKs e

n Ac) a i

+

+

c o,

l sn E

r m&

fpq o

o i

mml t

a g

e v

n r

i, e

u su Hpa t

r O

T

(

I 3^

I I

i

.