ML20149L546

Guidance for Development of Simulation Facility to Meet Requirements of 10CFR55.45

ML20149L546
Person / Time
Site:	Yankee Rowe
Issue date:	10/31/1987
From:	UTILITY SIMULATION FACILITY GROUP
To:
Shared Package
ML20149L501	List: ML20149L498 ML20149L546
References
NUDOCS 8802240283
Download: ML20149L546 (81)

Text

.- . . - -. . . . . - . ,

1 l

ENCLOSURE 3 GUIDANCE FOR DEVELOPMENT OF A SIMULATION FACILITY TO MEET THE REQUIREMENTS OF 10CFR55.45 8

M a

OCTOBER 1987 UTILITY SIMULATION FACILITY GROUP 8802240283 880212 .

DR ADOCK 05 09 l REVISION 1 l

TABLE OF CONTENTS SECTION EAGf LIST OF TABLES iii LIST OF FIGURES iii LIST OF EXAMPLES iii FORHARD iv

1.0 INTRODUCTION

1 1.1 Purpose 1 1.2 Background i 1.3 Definitions 2 1.4 Acronyms 6

~

2.0 CRITERIA 8 2.1 Human Factors 9 2.2 Procedures 12 2.3 Steady State and Transient Models 15 ,

2.4 Performance Testing 16 2.5 Operating Test Methodology 17 3.0 DEVELOPMENT OF A SIMULATION FACILITY 21 3.1 System Function / Task Analysis 24 3.2 Operational Cue Analysis 30 3.3 Fidelity Evaluation 30 3.4 Simulation Device Integration 34 3.5 Simulation Facility 36 3.6 Implementation 36 3.7 Configuration Management Plan 37 4.0 SIMULATION DEVICES ,

44 4.1 Non-Plant Referenced Simulator 45 1 4.1.1 Human Factors 45 4.1.2 Procedures 46 l 4.1.3 Steady State and Transient Models 46 l

4.1.4 Performance Testing 48 4.1.5 Operating Test Methodology 48 i

i t .

1 i

TABLE OF CONTENTS SECTION PAGE 4.2 Control Room Hock-Up 50 4.2.1 Human Factors 50 4.2.2 Procedures 51 4.2.3 Steady State and Transient Models 52 4.2.4 Performance Testing 52 4.2.5 Operating Test Methodology 52 4.3 Reduced Scope Simulator 54 4.3.1 Human Factors 54 4.3.2 Procedures 55 4.3.3 Steady State and Transient Models 56 4.3.4 Performance Testing 57 4.3.5 Operating Test Methodology 58 4.4 Part Task Simulator 59

~ '

4.4.1 Human Factors 59 4.4.2 Procedures 60 4.4.3 Steady State and Transient Models 61 4.4.4 Performance Testing 62 4.4.5 Operating Test Methodology 63 4.5 CRT Simulator 64 4.5.1 Human Factors 64 4.5.2 Procedures 64 4.5.3 Steady State and Transient Models 65 4.5.4 Performance Testing 66 4.5.5 Operating Test Methodology 67 4.6 Reference Plant 68 4.6.1 Human Factors 68 4.6.2 Procedures ,

68 4.6.3 Steady State and Transient Models 69 4.6.4 Performance Testing 69 4.6.5 Operating Test Methodology 69

5.0 CONCLUSION

S, OBSERVATIONS, AND RECOMMENDATIONS 70

6.0 REFERENCES

71 11

LIST OF TABLES  :

TABLE fifd 1 Kuman factors Considerations 10 2 Simulation Facility Development Process Steps 23 3 System function / Task Analysis Steps and Process 39 4 System Function / Task Analysis Criteria 41 5 Fidelity Evaluation Criteria 43 LIST OF FIGURES FIGURE Pfff 1 Simulation Facility Development Pro-oss Flow 22 2 Han/Hachine Interface Model 29 3 Fidelity Evaluation 32 LIST OF EXAMPLES EXAMPLE Egg 1 Breakdown of an Operator Task 26 l

l I

l

~

iii i

FORWARD There are definite, measurable benefits to be gained from the use of non-plant specific simulation devices, both in the areas of training and operator evaluation. The guidance of this document proposes a methodology for realizing those benefits.

This document was developed to provide consistent methodology for use by licensees in implementing plans to meet the requirements of 10CFR55.45(b)(1)(1). This document is also intended for use by the NRC in evaluating 10CFR55.45(b)(1)(i) simulation facilities. While this document provides generic guidance, the authors recognize that plant specific plans to meet 10CFR55.45 requirements may result in deviation from the guidance contained herein. This document also identifies the methods and generic limitations of operating tests on a simulation facility that is made up of the simulation devices discussed herein.

This document was prepared by representatives from the four author l l

utilities that joined to form the Utility Simulation Facility Group l (USFG). Specifically, the four USFG utilities and respective plants l are:

Consumers Power Company (Big Rock Point),

Public Service Company of Colorado (Ft. St. Vrain),

Southern California Edison Company (San Onofre 1), and Yankee Atomic Electric Company (Yankee Nuclear Power Station).

I i

i iv l 1

l

The above licensees will utilize this document to outline their plans to develop a simulation facility to meet 10CFR55.45 using existing or upgraded operator training hardware, enhanced, as necessary, by some use of Controllers, pen and ink procedure changes, and walk-through methods discussed herein. The plans will describe the simulation device (s) to be used as a simulation facility to evaluate the operators' generic skills and knowledge necessary to satisfy the thirteen operating test criteria of 10CFR55.45. Evaluation of the generic skills and knowledge is fundamental to providing assurance that the operators are prepared to perform the duties and respons',bilities required of reactor and senior reactor operators.

It is the conclusion of the author utilities that effective operator training and examination is realized if any one of the simulation devices used alone, or in combination with other devices, constitutes the licensee's simulation facility. Use of these simulation facilities provides for operators that are trained and evaluated to standards necessary to assure safe operation of nuclear power plants.

Utility Simulation Facility Group NOTE: Revision 1 of this document reflects revisions to incorporate NRC comments and NRC/USFG agreements and l i

understandings reached during an NRC/USFG meeting on l

September 15 and 16, 1987.  !

. l l

y

1.0 INTRODUCTION

1.1 Purcose-i This document provides sufficient generic guidance for the development and use of simulation facilities for those Nuclear Power Plant Licensees that plan to meet the requirements of 10CFR55.45(b)(1)(1). Plant specific plans will describe the licensee's simulation facility and chosen methods of conducting operating tests on the simulation facility.

1.2 Backaround 10CFR55, "Operator Licenses," Paragraph 55.45, "Operating Tests" requires that an applicant for a reactor operator or senior reactor operator license demonstrate both an understanding of and the ability to perform certain essential job tasks. It specifies that the demonstration will be done through the administration of operating tests in a plant walkthrough and in a simulation facility. The simulation facility may be one that consists solely of a plant-referenced simulator certified to the NRC by the facility licensee, or it may be one which has been approved by the NRC, after appitcation for such approval has been made by the fac"111ty licensee.

I This document addresses the preparation and implementation of a plan i

for the development of a simulation facility and guidance for use and approval of that simulation facility.

1.3 Definitions Best Estimate - Reference plant response data based upon engineering evaluation or operational assessment.

Candidate - An indlyidual being evaluated for a reactor operator or senior reactor operator license.

Controller - An individual responsible for clarifying deviations between a simulation device and the reference plant.

Critical Parameters - 1) Those parameters that require direct and continuous ob:ervation to operate a nuclear power plant under manual control.

2) Input parameters to nuclear power plant safety systems.

I Cue - Information available for use in evaluating' plant status.

Deviation - An identified difference between a simulation device and the reference plant.

Deficiency - A deviation that the fidelity evaluation identifies as a "need-to-fix" item.

Examiner - An NRC representative who conducts operating tests.

Fidelity - Reference plant replication in either system model, physical appearance or system function.

Instructor - A utility representative responsible for the operation of the simulation device.

Multidisciplinary Review - Review by disciplines of appropriate background and experience.

Operational Cue Analysis - An analysis to determine the cues available on a simulation device and the cues required by the referenced plant operating procedures.

Operator - An individual who possesses a reactor operator or senior reactor operator license.

~

Pen and Ink Procedure Changes - Changes made to contfolled reference plant procedures for the purpose of mitigating simulation device deficiencies, before those procedures are used to conduct operating tests. *

-3

Potential to Confuse - Deviation potential to perplex the operator and disrupt his mental and physical actions.

Potential to Error - Deviation potential to lead the operator t0 not perform any action or to perform an incorrect action.

Potential to Impede - Deviation potential to delay or hinder significantly the operator's correct response.

Procedures - Reference plant normal operating procedures, abnormal operating procedures, emergency operating procedures, and emergency plan implementing procedures that an operator or candidate would be required to implement.

Procedure Performance Time - The realistic or actual time for a candidate to perform a procedure cr task in the reference plant control l l

room.  !

Real Time - Computer simulation of dynamic performante in the same time base relationships, sequences, durations, rates and accelerations as the dynamic performance of the reference plant.

I

- j

. _ _ _ - ~ - - .

l Reference Plant - The specific nuclear power plant from which a simulation facility's control rocm configuration, system control arrangement, and design data are derived.

Simulation Device - A component of a simulation facility that simulates a portion or all of the reference plant.

Simulation Facility - One or more of the following simulation devices, alone or in combination, used for the conduct of operating tests:

1) Non-Plant Referenced Simulator

2) Control Room Hock-Up

3) Reduced Scope Simulator

4) Part Task Simulator

5) CRT Simulator

6) Reference Plant I

i l

~5- l

System Function / Task Analysis - A systematic analysis of the reference plant procedures that yield the cue and I&C requirements.

Task - A unit of control room operator work which may require plant information collec. tion, systems operation, or both.

"ask Element - A unit of human activity comprising a task.

Task Statement - An independent unit of control room operator work.

1.4 Acronyms CFR - Code of Federal Regulations CRDR - Control Room Design Review CRT - Cathode Ray Tube CRM - Control Room Hock-Up HF - Human Factors '

I&C - Instrumentation and Control NPRS - Non-Plant Referenced Simulator I NRC - Nuclear Regulatory Commission

~

PRS - Plant Referenced Simulator <

PTS - Part-Task Simulator RO - Reactor Operator

-s- ,

4 1

RSS - Reduced Scope Simulator SFTA - System Function / Task Analysis SME - Subject Matter Expert ,

SRO - Senior Reactor Operator  :

USFG - Utility Simulation Facility Group I 2

1 l

k 4

e-

• k 1

6 i

l

. t k

i  :

I I  !

4 e

J

! . i i < 1

'i I

r b t\

I i j -

$ I

2.0 CRITERIA i

1 The following section provides generic criteria to be applied in the evaluation and use of a simulation device. All or part of each of these criteria is applied to specific simulation devices in the manner described in Sections 3 and 4.

O 1

W l

. .g.

I 2.1 Human Factors l

Human factors addresses the comparability of the simulation device with the reference plant in the areas of control room and panel layout, I&C configuration and ambient operating environment. The primary human factor consideration in a plant referenced simulator (PRS) is for the simulator to have fidelity with the reference plant. PRS fidelity means duplication in physical appearance, physical layout, system function and system model. For a simulation device other than a PRS, 1007. fidelity may not be achievable in system model and physical layout. Therefore, the only highly achievable fidelity component would be duplication in system

~

function. A simulation device has to meet a high degree of duplication in system function criteria to be considered to be viable for the conduct of an operating test.

The degree to which a deviation does not meet physical / functional fidelity becomes a human factors concern. In a simulation device, the principle goal of human factors is to assure fidelity deviations have no negative impact on operator task performance. Deviations that hinder the operator should be documented and evaluated in a systematic fashion. Cognitive and behavioral operator actions should be considered. Operator per:eptions (color, faimics, patterns, etc.) are to be considered if essential to task l

performance. Other human factors considerations deal with the specific components in Table 1. The simulation facility will

-g. l I

l l

l 1

i HUMAN FACTORS CONSIDERATIONS  !

Maior Area Comoonents Control Room Layout - Physical Orientation Operator Station Panel Layout - Systems Orientation Control Panels Annunciator Panel Instruments and Controls - Displays Controls -

Instrument Range Instrument Accuracy Engineering Units Ambient Operating Environment -

Normal and Emergency Lighting Humidity Temperature Noise Communications I

l l

Table 1 i

l

l contain controls, instruments, alarms and other man-machine interfaces for the operator to demonstrate his capability. The ambient environment in the simulation facility should be replicated to the extent possible. To the extent practicable, the following generic criteria should be applied to any simulation device. The specific application of this generic criteria to each simulation device is discussed in Section 4.

Cgn. trol Room and Panel Layout The simulation device should approximate the reference plant physical orientation and appearance. The simulation device should be the same physical size as the reference plant although reduced

~

scale reproductions are acceptable provided the SFTA determines that the reduction does not significantly detract from the operating test.

The operator's station and other working space should be i replicated. Deviations from the reference plant shall be evaluated -

as discussed in Section 3.

The control panels should be posltioned on the simulation device in the same physical location as the reference plant. The systems orientation within the panel should replicate the reference plant.

Deviations from the reference plant panel layout shatl"be evaluated as discussed in Section 3.

l i

i l

I&C Conficuration The simulation device controls, indications, etc., on the control panels should approximate the same physical location as in the ,

reference plant. The instrument displays, controls, range, accuracy and units should replicate the reference plant. Deviations from the reference plant I&C configuration shall be evaluated as discussed in Section 3.

Ambient Oeeratina Environment The ambient operating environment shall permit the operator to perform his duties. The ambient operating environment factors to be considered are lighting, humidity, temperature, noise and communication. Significant deviations from the reference plant environment shall be evaluated as discussed in Section 3.

2.2 Procedures The procedure considerations are: 1) the scope, 2) the manner of use and, 3) methods of modifying the procedures used in the administration of operating tests. Controlled copies of reference plant procedures are used in the conduct of an operating test.

Procedures performed on a simulation device allow candidates to demonstrate their "ability to perform" the operations rIquired by those procedures. The following generic criteria should be applied when reference plant procedures are used during the course of an

operating test on the simulation devices described herein. The specific application of these criteria to each simulation device is discussed in Section 4.

Procedure Scoce Types of procedures exercised on the simulation device include:

Normal Operating Procedures, Abnormal Operating Procedures.

. Emergency Operating Procedures, and Emergency Plan Implementing Procedures.

The scope of procedures to be exercised on any simulation device shall be determined using the methods discussed in Section 3.

Procedure Use Reference plant procedures will be used on the simulation device (s). The reference plant may be used to exercise the ' Normal Operating Procedures' that can be performed as part of normal operations. Those procedures or tasks requiring control room interaction should be performed on a single simulation device (or ,

appropriately integrated simulation devices).

e 13

Procedure Modifications The data obtained during the SFTA (described in Section 3) will be used to identify any deviations between the simulation device and the procedures that will be tested. The determination will then be made which, if any, of these deviations should be resolved by pen and ink procedure changes. The procedure steps may then be modified to accommodate simulation device deficiencies, provided the modifications do not significantly detract from the conduct of the operating test. This may include partial completion or deletion of

~

procedure steps or assuming the successful or u7 successful completion of operator tasks that cannot be performed on the simulation device.

Before procedures are used in the conduct of operating tests, any necessary pen and ink procedure changes identified by the SFTA will be made. Pen and ink procedure changes may possibly affect Procedure Performance Time. Procedure Performance Tirie will be taken into consideration in the development, upgrade or use of existing devices or in the implementation of the pen and ink procedure changes.

All recommended pen and ink procedure changes will be forwarded to the Multidisciplinary Review Team (see Section 3.1) Tor review.

e e

Such changes will be made only after the following has been considered:

1. Determination has been made that the controlled procedure cannot be performed on existing simulation devices.

2. Upgrades to existing simulation devices, or the development of new simulation devices for the procedure (s) or part of the procedure (s) which cannot be conducted require an excessive effort or burden in relation to the benefit gained.

3. Determination has been made that pen and ink changes are preferable to other alternatives.

Any procedure modifications will be controlled by the simulation facility Configuration Hanagement Program described in Section 3.7.

2.3 Steady State and Transient Models The steady state and transient modeling used as part of the simulation device (s) shall adequately model the operating behavior of the reference plant. The following generic criteria should be applied, as applicable, to any simulation device. Tfie specific j application of this criteria to each simulation device is discussed in Section 4. i i -,- - _ . . .

S.LQAft Simulation device output should approximate and display expected plant responses. The responses should be based upon plant operating data or best estimate analyses, as appropriate.

Fidelity The models should be of a level of sophistication necessary to assure the adequacy of th? output information being presented to the operator.

Time All simulation devices should approximate real time, to the extent practicable.

2.4 Performance Testina Performance testing is conducted to verify the simulation device performance as compared to actuni or predicted reference plant performance. The initial performance testing hall serve to verify and validate the adequacy of the completed :Imulation Facility, including any proceoure modifications. The specific application of .

performance testing criteria to each simulation device is discussed in Section 4.

Performance testing should be conducted on a schedule consistent with 10CFR55.45.

1

2.5 ODeratina Test Hethodoloav This section provides a generic process for conducting examinations on simulation devices for the purpose of evaluation in accordance with 10CFR55.45(a). The following generic criteria should be implemented, as applicable, on any simulation device. The specific application of this criteria to each simulation device is discussed in Section 4.

Those portions of the operating test conducted on each simulation -

device shall be limited to the procedure scope determined by the Operational Cue Analysis (described in Section 3.2) for that simulation device.

Examinations should be conducted in accordance with NUREG-1021, "Operator Licensing Examiner Standards."

Procedure Performance Time on a simulation device should be followed as closely as possible during operating tests.

During the conduct of the operating test, Controllers may be used to mitigate identified simulation. device deft:1encies. The use of Controllers should follow the guidelines described b11ow.

_17

Guidelines for Controller Interaction The role of the Controller is to provide an added dimension to the simulation device to enable the device to more closely approximate the reference plant during the conduct of operating tests. In this sense, controller usage is similar in nature to those activities conducted by utility instructors who, during conduct of operating tests on Plant Reference Simulators, provide information as outside operators I&C technicians, etc. In this case the Controller is used to augment the simulation devices. Therefore, the purpose of the Controller is to provide, under direction, those cues unavailable from the device that may be needed to carry out actions durinc the performance of the operating test.

Controllers used during implementation of operating tests on various simulation devices shall follow specific guidelines established herein.

b Examination Intearity o Examination integrity is paramount to the success of the operating test. The Controller will not compromise examination integrity.

E o Controller actions shall be conducted under the direct supervision and control of the examiner.

6

o Controllers shall not prompt the operators in the performance of their duties. PrompTimj ttniti rntrit k the invalintion of the operattng test.

Controller Oualifications o Controllers shall be trained on their duties and responsibilities. Details of this training program shall be included as part of the simulation facility plan submitted by each utility.

_ o Qualifications of controllers shall be supplied as part of the simulation facility plans submitted by each individual utility and should meet the minimum criteria listed below.

1) The controller should be employed by the utility or as a vendor under contract to the utility.

2) The controller should posess the level of training and qualifications required by the utility for simulation facility instructors as outlined in their respective accredited training programs. Controllers should hold or have held an SRO license or certifiertion on the plant for which the operating test is being conducted.

I l

l 1

Controllers should also be knowledgeable on the simulation device or which the operating test is being conducted.

3) Controllers should receive additional training, as required by the utility, on the conduct of operating tests. As a minimum, this training shall cover the criteria listed above.

1 Controller Functions

. o Controllers shall function to provide cues to the operators that are not available from the simulation device.

o Controllers shall provide cues only as answers to specific questions from the operators or as directed in the operating test scenario. These cues are only for the put.9ose of providing information not available from the simulation device or as necessary to clarify a deviation of the simulation device from the reference plant.

o Controllers shall perform any other actions as identified I

and directed by the examiner in the conduct 6f the operating l

test.

{

l l

3.0 DEVELOPMENT OF A SIMULATION FACILITY This section provides the methodology used to develop a simulation facility to meet 10CFR55.45(b)(1)(1) using one or more of the simulation devices that is described in Section 4. The purpose of the simulation facility is to enhance the conduct of operating tests.

It is recognized that operating tests are better performed on devices that promote an active man / machine interface, such as Non-Plant Reference Simulators, Reduced Scope Simulators, or Part Task Simulators. This interface facilitates evaluation of the individual (s) in an actual operating environment. However, in cases where there is limited implementation of an active man / machine  ;

interface, adequate qualifications can be demonstrated by alternate devices. As further discussed in Section 4, these methods use Control Room Hock-ups, CRT Simulation and the Re'v 'e Plant alone or in combination.  :

The following systematic evaluation methodology description below is ,

based upon an SFTA. The process briefly identified in Figure 1 and Table 2 and further described in Section 3 is one method for developing a Simulation Facility based upon an SFTA. Each utility may have completed portions or all of the SFTA, but Iot in the exact

{

format described below. Therefore, this section describes the USFG's understanding of the detail needed in such an analysis. It l

{

SIMULATION FACILITY DEVELOPMENT PROCESS FLOH Analysis of reference plant procedures to deduce cue and I&C information requirements.

SFTA Comoarison of simulation device with information requirements.

Evaluation of fidelity deviations between simulation device and the reference plant.

SIMULATION DEVICE

1. NPRS

- 2. CRM FIDELITY

3. RSS EVALUATION

4. PTS

5. CRT SIM. _

6. REF. PLANT Selection of simulation device to satisfy information/ fidelity requirements.

Intearation of simulation device (s) into a simulation qp facility. ,

i SIMULATION Utility Plan of simulation FACILITY devices used to meet 10CFR55.45 rule.

Figure 1 9

l

SIMULATION FACILITY DEVELOPMENT PROCESS STEPS System Function / Task Analysis

1. Task Identification

2. Task Element Identification

3. Cue Information Requirements

4. I&C/ Physical Characteristics

, Simulation Device Comearison and Selection

1. Comparison of SFTA/ Reference Plant with Simulation Device

2. Refer Deviation to Fidelity Evaluation Fidelity Evaluation I

1. Identification of Deviation

2. Assessment of Deviations

3. Olsposition of Deviations Simulation Facility Plan

1. Description of Simulation Devices ,

2. Match Procedures with Devices Table 2 23

is expected that plant specific implementation of an SFTA will include parts of or all of any previously performed SFTA or related type work.

3.1 System Function / Task Analysis (SFTA)

The SFTA is a systematic analysis of the reference plant procedures that yield the cue and I&C characteristics. From the reference plant control room instrumentation, the operator obtains the required cues to help him in performing his tasks. The systematic determination of these man / machine interface characteristics provides a basis for evaluating the adequacy of a simulation device to support the conduct of operating tests. Tables 3 and 4 (located at the end of Section 3) detail the SFTA steps and process and SFTA criteria, respectively.

The method is characterized as a top-down analysis which begins with the reference plant procedures. The procedures are partitioned into units of activities identified as tasks. A task is a unit of control room operator work which may require information collection, systems operation, or both. A task is characterized by being a relatively small unit of work which is comprised of approximately the same sequence of elementary human actions regardPe'ss of the operational sequence in which the task appears. The main criterion l for the identification of a task is that the task should define the information and/or control functions needed by the operator to perform that unit of work.

Each task 'is further partitioned into the units of human activity, called task elements, which need to be sequentially accomplished in order to execute the task. The main criterion for identifying these task elements is that each should further refine and identify the information requirements needed by the operator to execute the task in the context of all the operational sequences in which the task appears.

A set of cues is developed from the task element requirements. The

, cue in this context is defined as the significant information acquired by the control room operator that prompts him to act. The cue is taken directly from the task element, taking into account the information required from the specific system.

A set of characteristics describing the functional requirements for each task element is identified. The task element may be associated with a specific instrument. In that case, the characteristics include range and units. The physical requirements will be specified if the particular characteristic is critical to performing the task. In cases where only one type of equipment will satisfy the physical and functional requirement, the specific manufacturer i and component will be identified. Example 1 shows the breakdown of an operator task.

i l

l I

BREAKDOWN OF AN OPERATOR TASK Operation Procedure - E0P XXX Task Statement - Determine if RCPs are operating Task Element - Read RCP ON/0FF Status, System - RCP Read RCP current, System - RCP Read RCP flow, System - RCP Read RCP Delta P, System - RCS Read Voltage to RCP, System - RCP Read.RCP Speed System - RCP Read RCP Electric Power, System - EDS U

~

Cue Information - RCP Speed > 500 rpm Annunciator Hindow Off No Annunciator Audio Alarm 4

ILC/ Physical Requirement - Display Value Range, 0-1200 Units, RPH Accuracy, N/A Display on Control Room Panel 56 Abbreviations RCP - Reactor Coolant Pump ,

EDS - Electrical Distribution System Example 1

-26

The SFTA will be conducted by a Multidisciplinary Review Team composed of subject matter experts (SHEs) of appropriate background and whose responsibilities are discussed below. The SHEs will utilize their expertise to process the task breakdown. The SFTA process steps will be documented and supported with appropriate sources.

Multidisciolinaev Review Team The Multidisciplinary Review Team shall consist of utility or consultant personnel that are SHEs of appropriate disciplines. The

.. areas of expertise and experience of the personnel on the Multidisciplinary Review Team will be detailed in each utility's plant specific plans. The role of the Multidisciplinary Review Team is to:

Review the SFTA process and documentation generated by the SFTA.

I Evaluate the Operational Cue Analysis data and disposition the deviations identified in this process.

- Determine the applicability of each procedur1 to each simulation device.

- Recommend the makeup of the Simulation Facility.

l l

SFTA Philosochv The System Function / Task Analysis (SFTA) is a logical link to the other analyses that have been conducted to support training programs. The starting point for SFTA is the procedures which were also the basis for the job analysis previously done for the training programs. The job analysis provides the skills and knowledge requirements needed for the operator to perform his job. The task analysis provides the task information requirements for the operator to perform his tasks. The system function analysis provides the functional operator controls requirements needed to perform procedural steps.

The job analysis, task analysis, and system function analysis connect to the three points of a man / machine interface model that exists at the utilities and discussed in INPO Document 83-047. The operator is the center of the triangular model shown in Figure 3.

The model addresses the interfaces and interrelationships that exist among the operator, the plant (system function analysis), the procedure (task analysis), and the training (job analysis). The SFTA process ensures that the component interfaces have been considered adequately and that the operators can operate the plant safely and efficiently during all operating situations'.

l

MAN-MACHINE INTERFACE tDDlk Job Analysis in i

"!i!!

Operator '!!

!i t ;..

5'15;

.inii .

ns-liFn i in i nu .

n;;; T  !!! i 'u:! 4

~"'..... iisi Procedures a "5 i: Plant Task Analysis System Function Analysis Figure 2 29-

3.2 Ooerational Cue Analysis The Operational Cue Analysis consists of the SFTA and the simulation device comparison and selection process. An I&C inventory will be conducted on each simulation device. As shown in Figure 1, the I&C inventory is then compared to the SFTA results to identify the I&C set and cues available to execute the reference plant procedure set. Each procedure task listing is evaluated to determine the ability to adequately perform the procedure on the simulation device. This process yields a procedure set applicable for examination on that particular simulation device. Procedures that cannot be implemented on any other simulation device can be examined on the reference plant. Accordingly, it is expected that all procedures can be examined on the Simulation Facility.

The process of collecting data for the Operational Cue Analysis shall be similar to that identified in NUREG-1258, Appendix B. The data fields to be evaluated shall be determined by the SFTA. The discrepancies between the reference plant and the simulation device i shall be dispositioned by the Multidisciplinary Review Team using j the process identified in Section 3.3.

1 3.3 Fidelity Evaluation After a simulation device goes through the SFTA comparison process, a fidelity evaluation should be performed. The purpose of the l

i l

fidelity evaluation is to identify and assess the deviations of a simulation device from the reference plant. A type of fidelity evaluation process and criteria that could be used is detailed in Figure 3 and Table 5 (located at the end of Section 3). The fidelity evaluation identifies the potential areas where comparability deviations could cause an operator difficulties in performing procedure tasks. Deviations are identified from three different sources: 1) the SFTA results, 2) the control room as  ;

noted in Table 1 and 3) the Simulation Facility Development l process. The evaluation process will consist of identifying the deviation, assessing the deviation, and dispositioning the

_ deviation. The evaluation process will be done by the Multidisciplinary Review Team.

Deviations vary to degrees of impact on the operator. Three factors will be used to assess the deviation impact; (1) Potential to Impede, (2) Potential to Error, and (3) Potential to Confuse. The Potential to Impede is the potential that the deviation has to delay or hinder significantly the operators correct response. The Potential to Error is the potential that the deviation has to lead the operator to not perform any action or to perform an incorrect action. The Potential to Confuse is the potential tha,t the deviation has to perplex the operator and disrupt his mental and 1

physical actions. The Potential to Impede is the most important factor since this factor can prevent the operator from performing the tasks. The Potentials to Error and 'co Confuse are of equal e

i

FIDELITY EVALUATION h

SFTA Reference Plant Simulation Facility (Table 1) Development 5

! t I I E t E

$ Identi fy Deviations 1r 4

g i f W

N Evaluate d and Rate c

E Deviation i t i l Hodify Device.

Another Device, I I Controller, Modify Procedure, Etc.

, JL S Sum Rate Yes

% > 5.00 2

m 5 , No Device, Procedure

• Left "As Is" 11 -

Figure 3 value. The three factors are weighted with the Potential to le;ede having a factor of 0.50 and the Potential to Error and to Confuse having a factor of 0.25 each.

Each deviation will be rated on each potential from 1-10. A low probability is a measure of 1-3. A medium probability is a measure of 4-7. A high probability is a measure of 8-10. The formula to derive the sum rate is:

Sum Rate P) (.2E) + P2 ( 90) + P3 (.25) where: Pj is the Probability to Impede Pp is the Probability to Error P3 is the Probability to Confuse The sum rate will determine the degree of deviation impact on the operator and direct the solution to the fix. Any sum rate less than or equal to 5.00 denotes the deviation can be left "as is." Any sum rate greater than 5.00 denotes a deviation that must be addressed by modification.

l The weighing factor is derived from the D. Meister method of l assigning weights to criteria factors. The method of assigning 1-10 measurements is derived from the CRDR experience of rating Human Engineering Descrepancies (HEDs) for priority.

! l l e-l  !

3.4 Simulation Device Intearation The integration of the SFTA and the simulation device capabilities is necessary to ascertain the optimum simulation facility for the performance of operating tests. The goal of the selection process is to provide a simulated control room environment that presents the highest level of active man / machine interface, in which the

> reference plant procedures can be exercised. The Simulation Facility shall provide the opportunity to test all of the operator responses to the cues identified by the operational cue analysis of the reference plant procedures. The ability of the operator to use controlled copies of the reference plant procedures and produce the desired responses on the simulation device, so as to place the plant in the desired operating configuration, shall be a determining factor in selecting the simulation device for the operating test.

The simulation device shall be capable of producing the desired response to the operator actions identified in the reference plant procedures to the extent nscessary to assure that the operator can ,

determine from the available cuts that the plant is responding in the direction predicted in the reference plant transient analysis.

The process of selecting the simulation device (s) that will make up the simuletion facility begins by identifying the device (s) that

- provides the most active man / machine interface. An I&C inventory would be conducted on the selected simulation device (s). As shown j in Figure 1, the I&C inventory is then compared to the SFTA results I .

~

34

, _ . . _e

i to identify the ILC set and cues available to execute the reference plant procedures. Each procedure task listing will be evaluated to ,

determine the ability to adequately perform the procedure (s) on the '

simulation device (s) without consideration of modifications to the device (s).

Following consideration of simulation device modifications, the process would then be repeated for those procedures which previously could no's be satisfactorily performed on that particular device.

The end result of this process will yield a procedure set applicable for examination on that simulation device and a list of simulation

- device modifications that are appropriate, i A similar process would be conducted on other simulation devices in descending order of their capabilities for active man / machine interface.

The emphasis during this process is to identify the simulation device that is most useful for examiners and at the same time provides a cost effective simulation device that will provide for effective training. Before any passive simulation device is ,

selected consideration will be given to the use of controllers and pen and ink changes to the procedures with the intent to always try to use the device that promotes the most active man / machine 1

interface.

l l

1 -

The final selection of devices and the integration into a simulation 4 facility is an iterative process. The benefits of a device that promotes active man / machine interface must be continuously evaluated against the negative aspects of the device. Consideration must be given to the number of pen and ink changes, intrusion of the controller on the operators activities, human factors issues, etc.

3.5 Simulation Facility The last step of the flow process illustrated in Figure 1, is to describe all the devices that makeup the Simulation Facility. Plant specific plans will describe the procedures to be run on each simulation device. The supporting documentation for each plant's plan will include all the dispositioned deviations along with the evaluations and backup information.

3.6 Imolementation The development of a Simulation Facility should be undertaken in a systematic manner, giving consideration to the benefit of any upgrades to a simulation device. When considering or upgrading any ,

simulation devices, the Operational Cue Analysis (Section 3.2) l should be used to assess the significance and importance of the l l

modification. The implementation of the Simulation Facility sho'J1d give consideration to any device that promotes active man / machine interface, not necessarily just those included in this document. It l

. l l

l i

is understood that there are other simulation devices available now or will be available with advancing technology that may be cost effective for particular applications. The intent of this document is to describe a systematic approach to the development of a facility; not to restrict the facility to using any particular simulation device (s). It is important that the simulation facility provide the opportunity to examine all of the operator responses to the cues listed in the Operational Cue Anaysis.

. 3.7 Confiauration Manacement Procram A program to provide accountability of Simuintion Facility physical and functional capabilities will be developed and implemented. This program will consist of procedures and/or guidelines and will be -

controlled by established administrative procedure controls for each utility. The program will perform the following functions:

i

! 1. Identify, document, track and test discrepancies.

l

2. Identify, document and track deviations between the 1 Simulation Facility and the reference plant. l

3. Identify, document and track reference plant' changes denoting effects on operating tests.

< l

. j

The details of the program will be described in the individual utilities plan and will include the following:

1. An outline of the administrative procedures and responsibilities for maintaining the Simulation Facility current in accordance with the guidelines of this document.

2. A description of the organization (s) responsible for maintaining the Simulation Facility.

3. The administrative procedures, including time requirements, for updating the Simulation Facility upon a reference plant modification.

4. The administrative procedure (s), including time requirements, for completing, if necessary, a review of the ,

4 SFTA upon a modification to the Simulation Facility.

5. The administrative procedure (s) for review / evaluation of the performance test baseline data upon, modifications to the reference plant, modifications to the Simulation Facility or ,

on a periodic bases (i.e., once every four years).

j h

e

SYSTEM FUNCTION / TASK ANALYSIS STEPS AND PROCES3

$1[PS PROCESS I. Task Identification A. Select Station Procedure -

Select one station procedure at a time to process.

B. Breakdown Station Procedure -

Breakdown the procedure according to operators actions (task).

II. Task Element Identification A. Breakdown Operator Task -

Breakdown the operator tasks into small units of work.

(task element)

8. Identify Cognitive / -

Associate task element with Behavioral Action operator's cognitive and behavioral actions.

III. Cue Information Recuirements A. Specify Task Element -

Determine information operator Information needs to do task elements.

B. Specify Perceptual -

Determine information operator Information Required is cognizant about before, after and durirg task element performance.

C. Identify Time Dependency -

Determins information that Requirements will assist operator in performing the task element.

Determine information that is very critical relative to operator actions.

Table 3 SYSTEM FUNCTION / TASK ANALYSIS STEPS AND PROCESS STEPS PROCESS IV. ILC/ Physical Characteristics A. Identify !&C - Deduce the functional I&C Characteristics characteristics required to carry out the task.

B. Identify Physical - Identify physical character-Characteristics istics if required to perform task element and function.

- Specify type and model of component, if one-of-a-kind.

't i

4 Table 3 (Continued)

I l

40 ,

- - - _ - - - - - ~ , - -.

SYSTEM FUNCTION / TASK ANALYSIS CRITERIA T_ask Identification 1.

A task is a unit of control room work that can be easily analyzed.

2.

A task completion is independent of the preceding or following task.

3.

A task is comprised of a limited number of human action units (elements).

Task Element Identification 1.

Task elements are elementary human actions needed to accomplish a particular task.

2.

Task elements can be divided into cognitive and behavioral

- divisions.

3.

Each task element refers to information or control requirements.

4.

Task elements are written only for operator actions which require control room information or control function.

Cue Information Recuirements 1.

The cues required enable operator task performance.

2.

The cues assist the operator in assessing the control room status relative to the immediate tasks.

3.

The cues are a perceptual process that include audio, visual and sensory inputs.

4.

Cuts that are time dependent must denote the time window .

required.

• Table 4 e

SYSTEM FUNCTION / TASK ANALYSIS CRITERIA ILC Physical Characteristics

1. The I&C characteristics must specify the:

4

- Display required.

- Range required.

- Units of the quantity.

2. State physical requirements, if critical to performing the i

immediate task and function.

I i

j 2

l J

^

l .

Table 4 (Continued) 42

FIDELITY EVALUATION CRITERIA Identification of Deviation

1. A deviation is a physical difference that exists between the simulation device and the reference plant for the areas in Table 1.

2. A deviation is a difference in function between the SFTA and the simulation device.

3. A deviation is any other significant differences found between referente plant and simulation device during the development of the simulation facility.

Assessment of Devic +l2D

1. The deviation must not prevent the operator from performing tasks.

2. The deviation must not confuse the operator to the point of affecting task performance.

3. The deviation must not lead the operator to an error in task performance.

Diseosition of Deviation

1. Any modification to the simulation device in question will require approval by the Multidisciplinary Review Team.

2. Any deviation referred to another simulation device for resolution will require feedback through the Fidelity Evaluation process. ,

3. The selected disposition will require verification.

4. Any deviation left "as is" must have all supporting documentation and analysis and an adequate description.

S. Pen and ink procedure changes to the procedure will be the last option to resolve deviations.

Table 5 43 l

)

i l

4.0 SIMULATION DEVICES

The following sections discuss the various simulation devices used i in the conduct of operating tests. These simulation devices are currently available simulation technology that can be used in the

! development of a simulation facility. Each section discusses the l application of the previously described criteria to each simulation device, the recognized limitations of each simulation device and the advantages of each simulation device, b -

i O

i 0

) .

I -

i 1

) '

i

. l l

4.1 NON-PLANT REFERENCED SIMULATOR A Non-Plant Referenced Simulator (NPRS) is a simulation device that models plant systems. The following criteria apply to the use of an NPRS as a simulation device.

4.1.1 Human Factors Control Room and Panel layout An NPRS may not replicate the reference plant control room and panel layout. Deviations will be evaluated for impact on performance of

. operator tasks in accordance with the Section 3 methodology.

I&C Confiauration Hardware and location differences are allowed. Plant specific labels, overlays, scaling modifications or other surface enhancements should be used to make NPRS controls and panels more closely approximate the reference plant. Deviations will be evaluated for impact on performance of operator tasks in accordance with the Section 3 methodology.

Ambient Oeeratina Environment An NPRS may not replicate the reference plant controT room ambient operating environment. Deviations will be evaluated for impact on performance of operator tasks in accordance with the Section 3 methodology.

4.1.2 Procedures As previously noted, the types of procedures exercised on the NPRS may include:

Normal Operating Procedures Abnormal Operating Procedures Emergency Operating Procedures, and Emergency Plan Implementing Procedures.

The scope of procedures to be used on the NPRS will be determined by 11e SFTA process described in Section 3. For each procedure

~

performed on the NPRS, the Procedure Performance Time will be matched as closely as possible. The remainder of the generic Procedures (Section 2.2) criteria applies to the NPRS.

4.1.3 Steadv State and Transient Models

• W 51 sat The output of the NPRS should approximate reference plant response.

However, the similarity will be limited by considerations such as core size and numbers of redundant or auxiliary systems. As a minimum, the NPRS should be capable of producing the oper'ator cues required to enable implementation of those procedures identified by e f

the process described in 4.1.2. Software modifications should be implemented, as practicable, to achieve fuli exercise of the precadures on the NPRS.

Fidelity The steady state values for critical parameters shall be stable and not vary significantly from the initial values over a 60-minute period. Changes in critical parameters should correspond in direction to those expected from operating data or a best estimate If analysis and should not violate any physical laws of nature.

parameters deviate significantly and software modeling changes cannot be reasonably pursued, these parameters may exceed the criteria, provided:

o They are specifically corrected using a Controller QB o Cue cards are substituted for these parameters QB o Appropriate pen and ink procedure changes are implemented.

lim NPRS plant operating and transient time responses shou,ld approximate i Any deviations will be handled by the SFTA real time simulation.

process described in Section 3.

4.1.4 Performance Testina 1CQ2ft The NPRS performance testing should be limited to those procedures identified by the process described in 4.1.2.

Methodoloav Those procedures identified in 4.1.2 should be performed on the NPRS after all identified software modifications have been incorporated.

An initial set of data shall be collected and evaluated. After review and approval by the Multidisciplinary Review Team, these transient results become the baseline data set for subsequent performance testing evaluation.

Acceotance Criteria Determination of acceptability of performance test results shall be performed by the Multidisciplinary Review Team.

4.1.5 Ooeratina Test Methodoloav The NPRS permits evaluation in a control room team environment. The operating test conducted on the NPRS shall be limited to the procedures identified by the process described in 4.f.2. The generic guidelines for Controller interaction (Section 2.5) shall apply. Those procedures not accomplished on the NPRS should be evaluated separately on another simulation device.

l Examiners will be able to select various NPRS initial conditions and plant malfunctions to evaluate operators' or candidates' responses.

The NPRS should have the capability to stop and restart the simulation, as necessary, at any point.

The remainder of the generic Operating Test Methodology (Section 2.5) applies to the NPRS.

m d

4.2 CONTROL ROOH HOCK-UP A Control Room Hock-up (CRH) is a simulation device that consists of a display of the reference plant control room panels, including the switches, indications, and alarms, arranged in a configuration sirailar to the reference plant. The CRM may consist of photographs, three-dimensional mock-ups or a combination of both. The following criteria apply to the use of a CRM as a simulation device.

4.2.1 Human Factors Control Room and Panel Layout The CRM should replicate the reference plant physical orientation and appearance. The CRM should be the same physical size as the reference plant although reduced scale reproductions are acceptable provided the SFTA determines that the reduction does not significantly detract from the operating test. Any reduction effort should be limited such that labels, controls, indications, alarms, etc., remain clearly legible. Deviations will be evaluated for impact on performance of operator tasks in accordance with the Section 3 methodology.

I&C Confiouration The CRM controls, indications, etc., should be in the same physical location as in the reference plant. They should be replicated in sufficient detail to enable the desired operator capabilities to b'e

j successfully demonstrated. Functional fidelity of a CRM cannot be achieved due to its passive nature. Deviations will be evaluated for impact on performance of operator tasks in accordance with the Section 3 methodology.

Ambient Ooeratina Environment The CRM may not replicate the ambient operating environment of the reference plant. Deviations will be evaluated for impact on performance of operator tasks in accordance with the Section 3 '

methodology.

l 4.2.2 Procedures l As previously noted, thi types of procedures exercised on a CRM may include:

1 Normal Operating Procedures, Abnormal Operating Procedures, Emergency Operating Procedures, and Emergency Plan Implementing Procedures l

l l

The scope of procedures to be used on the CRM will be determined by ,

the SFTA process described in Section 3. For each p'rocedure performed on the CRM, the Procedure Performance Time will be matched as closely as possible. The remainder of the generic Procedures (Section 2.2) criteria applies to the CRM.

}

4

4.2.3 Steady State and Transient Models S.C.QD1 Operator cues required for those procedures identified by the process described in 4.2.2 will be given on the CRH using cue cards or by use of a Controller. These cues should be based on reference plant operating data or steady state and transient best estimate analysis data.

Fidelity The CRM should be a representation of the reference plant.

.. Controllers or cue cards may be used to enhance procedure usage.

Time CRH usage is not expected to be in real time.

4.2.4 Performance Testina Performance testing to verify a simulation facility's performance as compared to actual or predicted reference plant performance is not applicable for a CRM.

~

4.2.5 Ooeratina Test Methodoloav Operating tests conducted on a CRM are to consist of walkthroughs for each procedure to be tested. Task performance can only be I

I

l i

discussed and may require extensive use of Controllers and/or cue j cards to provide operational cues. The remainder of the generic Operating Test Methodology (Section 2.5) applies to the CRM.

e d

e

l l

4.3 REDUCE 0 SCOPE SIMULATOR i

A Reduced Scope Simulator (RSS) is a simulation device that physically and functionally models significant portions of the major systems of the reference plant. A RSS demonstrates expected plant response to operator input and to normal and transient conditions to which the simulator has been designed to respond. However, the number of initial conditions, normal functions, and malfunctions available will be less than the standard defined by ANS-3.5-1985.

The following criteria apply to the use of a RSS as a simulation device.

4.3.1 H_uman Factors Control Room and Panel Layout The RSS should be positioned to approximate the reference plant physical orientation and appearance. The RSS should ce the same physical size as the reference plant although reduced scale reproductions are acceptable provided the SFTA determines that the reduction does not significantly detract from the operating test. ,

Any reduction effort should be limited such that labels, controls, indications, alarms, etc., remain clearly legiole. Deviations will be evaluated for impact on performance of operator tasks in accordance with the Section 3 methodology.

l

~

l

I&C Confiauration The RSS controls, indications, alarms, etc., should be in the same physical location as in the reference plant. They should be replicated in sufficient detail to enable the desired operator function (s) to be successfully demonstrated.

Photographic images of non-functional control board components are acceptable. Operator control input / output devices, such as handswitches and recorders, should be similar in operation but need not be identical in manufacturer or model number to those installed in the reference plant. Deviations will be evaluated for impact on

_ performance of operator tasks in accordance with the Section 3 methodology.

Ambient Ooeratina Environment The RSS may not replicate the reference plant control room ambient operating environment. Deviations will be evaluated for impact on performance of operator tasks in accordance with the Section 3 methodology.

4.3.2 Procedures ,

As previously noted, the types of procedures exercised on a RSS may include:

l l

Normal Operating Procedures Abnormal Operating Procedures Emergency Operating Procedures, and Emergency Plan Implementing Procedures.

The scope of procedures to be used on the RSS will be determined by For each procedure the SFTA process described 1.1 Section 3.

performed on the RSS, the Procedure Performance Time will be ma The remainder of the generic Procedures as closely as possible. ,

(Section 2.2) criteria applies to the RSS.

4.3.3 Steady State and Transient Models Scope The e, c' '5e RSS should approximate expected reference plant l responses.

For the system (s) modeled, the majority of the operator cues required for the use of the procedures identified by the process described in 4.3.2 shall be modeled and displayed.

Fidelity The RSS computed values for steady state operation with the reference plant control system configuration (for those. systems modeled) shall be stable anc' not vary more than 12% of the initial i values over a 60-minute period for critical parameters or more than Transient 2

10% of the initial values for non-critical parameters.

l changes in displayed parameters shall not violate any physical laws i

1 l

-. _ _4

of nature and shall be in the same direction as operating data or a best estimate analysis of the reference plant. Expected relationships between parameters should occur in a manner consistent with expected reference plant response. For those alarms modeled and displayed, the RSS shall not fail to cause an alarm or automatic I 1

action that would have been actuated in the reference plant, or cause an alarm or automatic action that would not actuate in the reference plant. The RSS accuracies shall be related to full power values and ir.terim power levels for which valid reference plant information is available.

limg )

To the extent practicable, the RSS should approximate real time, l

4.3.4 Performance Testina

$101t The RSS performance testing should be limited to those procedures 1

identified by the process described in 4.3.2. The performance testing should include validation of the pen and ink changes made to any modified procedures.

Methodoloav Those procedures identified by the process described in 4.3.2 should be performed on the RSS after all identified software modifications have been incorporated. An initial set of data shall be collected s

and evaluated. After review and approval by the Multidisciplinary Review Team, these transient results become the baseline data set for subsequent performance testing evaluation.

Acceotance Criteria Determination of acceptability of performance test results shall be performed by the Multidisciplinary Review Team.

4.3.5 Oceratina Test Methodoloav The RSS permits evaluation in a control room team environment with a major portion of the control boards replicated. Therefore, most procedures can be evaluated even if some components are not software modeled. The operating test on the RSS shall be limited to the procedures identified by the process described in 4.3.2.

Examiners will be able to select various RSS initial conditions and plant malfunctions to evaluate operators' or candidates' responses.

9 The RSS should have the capability to stop and restart the simulation, as necessary, at any point.

The systems that are not modeled in the RSS may require evaluation of operating license candidates on other simulation devices (e.g. reference plant). '

1 The remainder of the generic Operating Test Methodology j (Section 2.5) applies to the RSS.

J 1

e I

4.4 PART TASK SIMULATOR Part Task Simulator (PTS) is a simulation device incorporating detailed modeling of a limited number of specific reference plant ,

components or subsystems. Such a device demonstrates expected response of those components or subsystems. The following criteria t

applies to the use of a PTS as a simulation device.

i 4.4.1 Human Factors i Control Room and Panel layout As necessary, layout of the PTS should be similar to the portion of the panel or system being simulated. The PTS should be the same

]

physical size as the reference plant although reduced scale i l

reproductions are acceptable provided the SFTA determines that the '

reduction does not significantly detract from the operating test.

Deviations will be evaluated for impact on performance of operator tasks in accordance with the Section 3 methodology.

The PTS should replicate the various reference plant surface enhancements, such as the use of color and mimics. The degree of correspondence with the reference plant should be consistent with the capability to replicate the existing surface enhancement with a l given simulation device construction technique.

l 9

I&C Confiauration The PTS controls, indications, alarms, etc., should be in the same physical location as in the reference plant. They should be replicated in sufficient detail to enable the desired operator function (s) to be successfully demonstrated. If the switch, control, indicator, alarm, or recorder is not required for the scope of the PTS, it does not need to be included on the simulator.

Deviations will be evaluated for impact on performance of operator tasks in accordance with the Section 3 methodology.

Ambient Ooeratina Environment A PTS may not replicate the ambient operating environment of the reference plant control room. Deviations will be evaluated for impact on performance of operator tasks in accordance with the Section 3 methodology.

4.4.2 Procedures 1

l As previously noted, the types of procedures exercised on a PTS may include: l Normal Operating Procedures l

Abnormal Operating Procedures i l

Emergency Operating Procedures, and l

Emergency Plan Implementing Procedures.

1

The scope of procedures to be used on the PTS will be determined by the SFTA process described in Section 3. For each procedure performed on the PTS, the Procedure Performance Time will be matched as closely as possible. The remainder of the generic Procedures (Section 2.2) criteria applies to the PTS.

4.4.3 Steady State and Transient Models 3.f.Q22 The output of the PTS should approximate expected reference plant responses. For the system (s) modeled, the operator cues (responses)

_ required for the use of the procedures identified in 4.4.2 should be displayed.

Fidelity The PTS computed values for steady state operation with the reference plant control system configuration (for those systems modeled) shall be stable and not vary more than 12% of the initial values over a 60-minute period for critical values or more than 110%

of the initial values for non-critical values. Transient changes in l l

displayed parameters shall not violate any physical laws of nature i and shall be in the same direction as operating data or a best, estimate analysis of the reference plant. Expected relationships between parameters should occur in a manner consistent with expected plant response. For those alarms modeled and displayed, the PTS l shall not fail to cause an alarm or automatic action that would have I l

been actuated in the reference plant, or cause an alarm or automatic The PTS action that would not actuate in the reference plant.

accu >acies shall be related to full power values and interim power levels for which valid reference plant information is available.

Ilme The PTS should approximate real time, to the extent practicable.

4.4.4 Performance Testina SLQDR The PTS performance testing should be limited to those procedures identified by the process described in 4.4.2.

Bgthodoloov Those procedures identified by the process described in 3.4.2 should be performed on the PTS after all identified software modifications have been incorporated. An initial set of data shall be collected  :

After review and approval by the Mult1 disciplinary and evaluated.

Review Team, these transient results become the baseline data set for subsequent performance testing evaluation.

Acceetance Criteria Determination of acceptability of performance test results shall be  !

performed by the Multidisciplinary Review Team.

l 4

4.4.5 Ooeratina Test Methodoloav Operating Tests shall be limited to those normal operator tasks and resporses to cues that can be accomplished on the PTS. The instructor or examiner should have the ability to stop and restart the simulation, as necessary, at any point.

The remainder of the generic Operating Test Methodology (Section 2.5) applies to the PTS.

1 l

i 6

i l

l 4.5 CRT SIMULATORS I l

A CRT Simulator is a simulation device that is computer based and CRT displayed. The information presented is a model of the reference plant operating behavior. The Input / Output may be limited to a computer keyboard and CRT, and the parameter set limited to a specific scope. The following criteria apply to the use of a CRT 1 Simulator as a simulation device.

4.5.1 Human Factors The human factors that can be properly addressed in using a CRT Simulator is essentially limited to the scope of information cues that are presented. Due to the keyboard and CRT I/O methodology the control panel layout I & C configuration or ambient operating environment cannot be accommodated in a CRT Simulator.

4.5.2 Procedures As previously noted, the types of procedures exercised on a CRT Simulator may include:

Normal Operating Procedures "

Abnormal Operating Procedures l l

Emergency Operating Procedures, and l i

Emergency Plan Implementing Procedures. I i

l l

.st-l

The scope of procedures to be used on the CRT Simulator will be determined by the SFTA process described in Section 3. For each procedure performed on the CRT Simulator, the Procedure Performance Time will be matched as closely as possible. The remainder of the I generic Procedures (Section 2.2) criteria applies to the CRT Simulator.

4.5.3 Steady State and Transient Models Sint The output of the CRT Simulator should approximate expected reference plant responses for those systems modeled. The system modeled and the displayed responses should be based on those cues required to use those procedures identified by the process describeo in 4.5.2.

Fidelity The level of sophistication for CRT Simulator models should assure adequacy of output information. The CRT Simulator computed values for steady state full power operation with the reference plant control system configuration (for those systems modeled) shall be stable and not vary more than 12% of the initial values over a 60-minute period for critical values or more than 11(f1 of the initial values for r,on-critical values. Transient changes in j displayed parameters shall not violate any physical laws of nature  !

l

. l

, - . - - . . -..n, -

- , - ,, _. -, ---.-w. , . . . , , ,,

and shall be in the same direction as operating data or a best estimate analysis of the reference plant. Expected relationships between parameters should occur in a manner consistent with expected plant response.

Time To the extent practicable, CRT simulator responses should approximate real time. Any inadequacies in the models can be compensated by the support of pen and ink procedure changes, Controller use, or other methods, provided that the use of these methods does not detract from the examination.

~

4.5.4 Performance Testino SEAft The CRT Simulator performance testlng should be limited to those procedures identified by the process described in 4.5.2.

Methodoloov Those procedures identified by the process described in 4.5.2 should be performed on the CRT Simulator after all identified software modifications have been incorporated. An initial set of data shall l

be collected and evaluated. After review and approvil by the Multidisciplinary Review Team, these transient results become the baseline data set for subsequent performance testing evaluation.

j

I Acceotance Criteria Determination of acceptability of performance test results shall be performed by the Multidisciplinary Review Team.

l l

4.5.5 Ooeratina Test Methodoloav Examiners will be able to select various initial conditions and plant malfunctions to evaluate operators or candidates. The CRT Simulator should have the capability to stop and restart the

, simulation, as necessary, at any point.

The CRT Simulator can be used in conjunction with a Control Room Mock-up, a Part Task Simulator, or the Reference Plant. The use of the CRT Simulator is both enhanced by and enhances the use of these other simulation devices in the simulation facility. The use of these other devices in conjunction with the CRT Simulator should reduce the amount of pen and ink procedure changes and controller use.

The remainder of the generic Operating Test Methodology (Section 2.5) applies to the CRT Simulator.

s'

l l

l l

l 4.6 REFERENCE PIMil l The Reference Plant, as a simulation device, is the Control Room of the specific nuclear power plant which serves as all or part of a simulation facility. The following criteria apply to the use of the Reference Plant as a simulation device.

4.6.1 Human Factors Hhen using the Reference Plant, equipment layout, instrument and Human control configuration, cue scope and environment is exact.

~

factors need only be addressed as applied to the plant operating conditions for the evolution to be examined.

4.6.2 Procedures Use of the Reference Plant as a simulation device allows usage of all of the procedures. No modifications to these procedures are required. To the extent consistent with existing plant conditions, the operating test may address any or all of these procedures to demonstrate familiarity with the plant. Operator tasks which cannot be actually performed should be accomplished through verbal The discussion and a walkthrough of the evolution being examined.

controls and indication needed to perform the evolution should be physically shown to the examiner and accompanied by a description of what occurs when that control is manipulated.

e 4.6.3 Steadv State and Transient Analysis Models When using the Reference Plant as a simulation device, steady state and transient models are only applicable to the extent necessary to assure that the examiners and controllers possess appropriate information regarding expected plant behavior. This information should be based upon plant operating data or best estimate data.

4.6.4 Performance Testina Performance testing is not applicable when the Reference Plant is utilized as the simulation device.

4.6.5 Oeeratina Test _Hethodoloav Operating tests conducted on a Reference Plant are to consist of walkthroughs and/or operation of selected plant evolutions, for each procedure to be tested. Task performance that cannot actually be performed should be discussed and may require extensive use of controllers and/or cue cards to provide operational cues. The remainder of the generic Operating Test Methodology (Section 2.5) applies to the Reference Plant.

1 l

l

5.0 CONCLUSION

S, OBSERVATIONS, AND RECOMMENDATIONS l

o Operating tests can be performed on simulation devices other than plant referenced simulators.

o Use of the simulation devices described herein enable the evaluation of the generic skills and knowledge necessary to fulfill the responsibilities of a reactor or senior reactor operator.

o Those specific skills and knowledge that cannot be evaluated on

.. other simulation devices can be evaluated in reference plant walkthroughs.

o The development of a simulation facility should consist of a systematic evaluation, such as the method described herein, of the operating test requirements and be responsive to those needs.

o It is preferred that simulation devices present the information in an active man / machine interface.

o It is recommended that the NRC approve the USFG methodology and use it to evaluate the adequacy of the resultant' Simulation Facilities.

e

6.0 REFERENCES

NUREG-1258, "Evaluation Procedure for Simulation facilities Certified Under 10 CFR 55." Draft Report, March 1987 NUREG-1021, "Operator Licensing Examiner Standards," Revision September 1986 ANS-3.5-1985, "American National Standard for Nuclear Power Plant Simulators for Use in Operator Training."

Regulatory Guide 1.149, "Nuclear Power Plant Simulation Facilities for Use in Operator License Examinations," Revision 1, April 1987 "Behavioral Analysis and Heasurement Methods," 0. Meister, Hiley and Sons, 1985 8797F o

. 7). ,

- - -g ---- ,y y, y

9 h

ENCLOSURE 4 NRC CONCERNS WITH REVISED USF6 PROPOSAL AND AGREEMENTS REACHED AT DECEMBER 7, 1988 MEETING NOTE: Those concerns numbered 1-14 are keyed to Enclosure 5 (titled:

"Conclusions and Agreements Reached") of the Meeting Summary from the September 15-16 seeting.

1. USE OF PROCEDURES, AND DEMONSTRATION OF THE 'APILITY TO PERFORM."

A. Although in some parts of the reviscd proposal the USFG refers to controlled copies of procedures, in others (e.g. page 13) they fail to do so.

AGREED - The next revision to the document will make consistent reference to "controlled copies" of procedures.

B. Although ites (b) talks about making pen and ink changes to procedures only after new simulation devices have been considered, the U5F6 ignores this on page v. When they do consider it on page 15, there is no sention of how the required cost-benefit analysis would be performed.

AGREED - Cost-benefit analyses will be contained in each individual utility's plant-specific submittal.

C. The words on page 15 do not parallel those in iten (c) which reterences use of controllers. The statement on page 18 that the use of controllers will not compromise exas integrity is appropriate, but not supported.

D. The final paragraph of Ites i is addressed satisfactorily on page 13.

2. PHYSICAL AND FUNCTIONAL FIDELITY.

A. This remains the major point of contention between NRC and USFG. They continue to make statements, such as those on pages 21 and 30, that a3 operator's "adequate qualifications," and "procedures" can be demonstrated on the actual plant if they cannot be done on an interactive simulation device, and that the plant itself can serve as a full or partial simulation facility. J B. The NRC's second paragraph requested the USFG to perfore research and analysis to support their position - they have ignored this while continuing to support that position. They have neither done, nor agreed tn do, such an analysis.

I

3. EXISTING VS NEW SIMULATION DEVICES.

The USFG seems to have ignored this issue - other than a cursory statement on page 15. They clearly do not give it higher priority than the use of controllers. They seem to be seeking to justify the status quo.

AGREED - Proper conduct of the SFTA could lead to the conclusion that new simulation devices are required; if this proves to be the case, such new devices will not be excluded from consideration.

4. HARDWARE ALTERNATIVES AND INTEGRATION.

This was addressed satisfactorily.

5. BEST ESTIMATE ANALYSIS AND BASELINE DATA.

This was addressed satisfactorily.

. 6. REAL-TIME SIMULATION.

The USFG continues to use the expression "approxisate real time to the extent practicable * (e.g. see pages 57, 62). The f act that they think of the actual plant as potentially serving as the entire simulation facility (page

68) indicates that they don't consider real-time capability seriously.

AGREED - The expression "to the extent practicable" will be deleted whenever it appears with regard to "real time.'

7. SKILLS AND KNOWLEDGES.

This was addressed satisfactorily.

8. EXAMINATION INTEGRITY AND THE USE OF CONTROLLERS.

Although this topic has been reasonably well addressed, several concerns remained. One sentence in our prior agreement ("Mechanises for ensuring the integrity of examinations when using controllers will also be explored")

remains of concern because the USFG simply states that it will not occur - but doesn't support or justify this. The intent of the USFG was that the contro'ler would work under the direction of the examiner. But the staff indicated that the controller should be under the direct supervision of the simulator operator, not the examiner.

AGREED - The next revision to the proposal will incorporate a statement that the duties and actions of the controller during an examination will be agreed upon in advance between the examiner (s) and the simulation facility operator. Also, the statearnt at the top of page 19 of Revision 1 of the proposal will be thanged to read: "Prompting may result in the invalidation of the ver Sting tes4t ty thi dEC."

v . -

,- - , y-, ,

-..m

9. HUMAN FACTORS ISSUES.

This was addressed satisfactorily.

10. OPERATIONAL CUE ANALYSIS.

The level of detail is acceptable, but NRC is concerned about the fact that the USFG has ignored the importance of dynamic cues in the plant. If carried to its logical end, this position would lead to the conclusion that the reference plant is an acceptable simulation facility because it has a high degree of physical fidelity.

AGREED - The USF6 will address cue dynamics in the next revision.

11. MULT!-D!SCIPLINARY TEAM.

This was addressed satisfactorily.

. 12. CONFIGURATION MANAGEMENT There is no specific ref erence to NPRS - perhaps understandable in light of the f act that the f acility licensee may not be in control of NPRS, or that one NPRS may have to serve several plants (e.g. Zion).

AGREED - This issue is best handled in detail in the plant-specific plans.

I

13. MISCELLANEOUS The issues of utility review and approval of NRC examinations, and ref erences to the repeated use of controllers were deleted; thus, this issue was addressed satisfactorily.

I

14. WORKING RELATIONSHIP

. This agreement was reinforced during the meeting, j

GENER AL ISSL'ES .

The stJf f asked the individual USFG sember utilities to identify, in their plant-specific plans due by May 1988, the status of their thinking regarding the specific devices they are centemplating to use for their simulation facilities.

The staif reminded the USFG that it considers the use of the reference plant, when serving as a simulation facility. to mean agiual get f gteange, not simply a walk-throught and that this position has implications for the inability to conduct operating examinations when the plant is shut down.

The staff asked the USFG to add, in its next revision, definitions of terms such asi shall, should, may, must, will, etc.

NEW ISSUE.

A new issue was raised by the USFG revised plan. This is:

FIDELITY EVALUATION AND ASSESSMENT OF DEV!ATIONS.

Considerable discusoion took place about this subject and the formula developed to assess deviations. The staff pointed out, and the USFG agree 1, that the foreula presented on page 33 had its values reversed. The USFG recognl:ed that its chosen criterien of 5.0 was arbitrary; chosen as the mid-point of a 1 10 scale. Since these numbers represent reliability, there is significance to the value chosen. There was discussion about whether the criterion, presently 5.0, should be changed to 3.0 or 2.0; or whether there should be a combination criterion, i.e. one value for an individual probability, and another value for a combination of the three probabilities.

The staff also requested that the USFG consider a fourth factor in its analysis, known as the "potential to enhance" (i.e. eaking the examination easier that real-world operations).

AGREED - The USFG will take a new lock at this section, the formulas used, and the criterion values established.