ML20148Q468

Rev 2 to Guidance for Development of Simulation Facility to Meet Requirements of 10CFR55.45

ML20148Q468
Person / Time
Site:	San Onofre, Yankee Rowe, Big Rock Point, 05000000, Fort Saint Vrain
Issue date:	03/31/1988
From:	CONSUMERS ENERGY CO. (FORMERLY CONSUMERS POWER CO.), PUBLIC SERVICE CO. OF COLORADO, SOUTHERN CALIFORNIA EDISON CO., YANKEE ATOMIC ELECTRIC CO.
To:
Shared Package
ML20148Q222	List: ML20148Q468
References
PROC-880331, TAC-67882, NUDOCS 8804120478
Download: ML20148Q468 (79)

Text

r- ,

GUIDANCE FOR DEVELOPMENT OF A SIMULATION FACILITY TO MEET THE REQUIREMENTS OF 10CFR55.45 4

1

}

i MARCH 1988 i

UTILITY SIMULATION FACILITY GROUP l-l l

l REVISION 2 I 8804120478 88040509

! PDR ADOCK 050 l

V I

r:

TABLE OF CONTENTS SECTION PAGE LIST OF TABLES iii LIST OF FIGURES iii LIST OF EXAMPLES iii FORRARD .iv

1.0 INTRODUCTION

1 1.1 Purpose 1 1.2 Background 1 1.3 Definitions 2 1.4 Acronyms 6 2.0 CRITERIA 8 2.1 Human Factors 9 2.2 Procedures 12 2.3 Steady State and Transient Models 15 2.4 Performance Testing 16 2.5 Operating Test Hethodology 17 3.0 DEVELOPHENT OF A SIMULATION FACILITY 21 3.1 System Function / Task Analysis 24 3.2 Operational _ Cue Analysis 29 3.3 Fidelity Evaluation 31 3.4 Simulation Device Integration 35 3.5 Simulation Facility 38 3.6 Implementation 38 3.7 Configuration Management Program 39 4.0 SIMULATION DEVICES 46 4.1 Non-Plant Referenced Simulator 47 4.1.1 Human Factors 47 4.1.2 Procedures 48 4.1.3 Steady State and Transient Models 48 4.1.4 Performance Testing 50 4.1.5 Operating Test Methodology 50 i

TABLE OF CONTENTS SECTION PAGE 4.2 Control Room Hock-Up 52 4.2.1 Human Factors 52 4.2.2 Procedures 53 4.2.3 Steady State and Transient Models 54 4.2.4 Performance Testing 54 4.2.5 Operating Test Methodology 54 4.3 Reduced Scopa Simulator 56 4.3.1 Human Factors 56 4.3.2 Procedures 57 4.3.3 Steady State and Transient Models 58 4.3.4 Performance Testing 59 4.3.5 Operating Test Methodology 60 4.4 Part Task Simulator 61 4.4.1 Human Factors 61 4.4.2 Procedures 62 4.4.3 Steady State and Transient Models 63 4.4.4 Performance Testing 64 4.4.5 Operating Test Methodology 65 4.5 CRT Simulator 66 4.5.1 Human Factors 66 4.5.2 Procedures 66 4.5.3 Steady State and Transient Models 67

! 4.5.4 Performance Testing 68 4.5.5 Operating Test Methodology 69 4.6 Reference Plant 70 4.6.1 Human Factors 70 4.6.2 Procedures 70 l

4.6.3 Steady State and Transient Models 71 4.6.4 Performance Testing 71 4.6.5 Operating Test Methodology 71 i

l

5.0 CONCLUSION

S, OBSERVATIONS, AND RECOHHENDATIONS 72

6.0 REFERENCES

73 ,

11 l

y_-_-- _ - -- - - - - - __ - - - _ - - - - . - . . - - _

LIST OF TABLES TABLE EASE-1 Human Factors Considerations 10 2 Simulation Facility Development Process Steps 23 3 System Function / Task Analysis Steps and Process 41 4 System Function / Task Analysis Criteria 43 5 Fidelity Evaluation Criteria 45 LIST OF FIGURES FIGURE EASE 1 Simulation Facility Development Process Flow 22 2 Man / Machine Interface Model 30 3 Fidelity Evaluation 32 LIST OF EXAMPLES EXAMPLE EME I Breakdown of an Operator Task 27 l

t' iii .

t FORWARD There are definite, measurable benefits to be gained from the use of non-plant specific simulation devices, both in the areas of training and operator evaluation. The guidance of this document proposes a methodology for realizing those benefits.

This document was developed to provide consistent methodology for use by licensees in implementing plans to meet the requirements of 10CFR55.45(b)(1)(1). This document is also intended for use by the NRC in evaluating 10CFR55.45(b)(1)(1) simulation facilities. While this document provides generic guidance, the authors recognize that plant specific plans to meet 10CFR55.45 requirements may result in deviation from the guidance contained herein. This document also identifies the methods and generic limitations of operating tests on a simulation facility that is made up of the simulation devices discassed herein.

l This document was prepared by representatives from the four author utilities that joined to form the Utility Simulation Facility Group (USFG). Specifically, the four USFG utilities and respective plants

! are:

l Consumers Power Company (Big Rock Point),

l Public Service Company of Colorado (Ft. St Vrain),

i j Southern California Edison Company (San Onofre 1), and l Yankee Atomic Electric Company (Yankee Nuclear Power Station).

iv

r The above licensees will utilize this document to outline their plans to develop a simulation facility to meet 10CFR55.45 using existing, upgraded or new operator training hardware, enhanced, as necessary, by some use of Controllers, pen and ink procedure changes, and walk-through methods discussed herein. The plans will describe the simulation device (s) to be used as a simulation facility to evaluate the operators' generic skills and knowledge necessary to satisfy the thirteen operating test criteria of 10CFR55.45. Evaluation of the generic skills and knowledge is fundamental to providing assurance that the operators are prepared to perform the duties and responsibilities required of reactor and senior reactor operators.

It is the conclusion of the author utilities that effective operator training and examination is realized if any one of the simulation devices used alone, or in combination with other devices, constitutes the licensee's simulation facility. Use of these simulation facilities provides for operators that are trained and evaluated to standards necessary to assure safe operation of nuclear power plants.

Utility Simulation Facility Group NOTE: Revision 1 of this document reflects revisions to incorporate NRC comments and NRC/USFG agreements and understandings reached during an NRC/USFG meeting on September 15 and 16, 1987. Revision 2 reflects comments from a December 7, 1987 meeting.

v

1.0 INTRODUCTION

1.1 Puroose This document provides sufficient generic guidance for the development and use of simulation facilities for those Nuclear Power Plant Licensees that plan to meet the requirements of 10CFR55.45(b)(1)(1). Plant specific plans will describe the licensee's simulation facility and chosen methods of conducting operating tests on the simulation facility.

1.2 Backaround 10CFR55, "Operator Licenses," Paragraph 55.45, "Operating Tests" requires that an applicant for a reactor operator or senior reactor operator license demonstrate both an understanding of and the ability to perform certain essential job tasks. It specifies that the demonstration will be don? through the administration of operating tests in a plant walkthrough and in a simulation facility. The simulation facility may be one that consists solely of a plant-referenced simulator certified to the NRC by the facility licensee, or it may be one which has been approved by the NRC, after application for such approval has been made by the facility licensee.

l

This document addresses the preparation and implementation of a plan for the development of a simulation facility and guidance for use and approval of that simulation facility.

1.3 Definitions Best Estimate - Reference plant response data based upon engineering evaluation or operational assessment.

Candidate - An individual being evaluated for a reactor operator or senior reactor operator license.

Controller - An individual responsible for clarifying deviations between a simulation device and the reference plant.

Crit cal Parameters -

i

1) Those parameters that require direct and continuous obserntion to operate a nuclear power plant under snanual control.

l

2) Input parameters to nuclear power plant l

safety systems.

l l

l Cue - Information available for use in evaluating plant status.

l l Deviation - An identified difference between a simulation device and l the reference plant.

l Deficiency - A deviation that the fidelity evaluation identifies as a "need-to-fix" item.

Examiner - An NRC representative who conducts operating tests.

Fidelity - Reference plant replication in either system model, physical appearance or system function.

Instructor - A utility representative responsible for the operation of the simulation device.

Multidisciplinary Review - Review by disciplines of appropriate background and experience.

Operational Cue Analysis - An analysis to determine the cues available on a simulation device and the cues required by the referenced plant operating procedures.

Operator - An individual who possesses a reactor operator or senior reactor operator license.

Pen and Ink Procedure Changes - Changes made to controlled reference plant procedures for the purpose of mitigating simulation device l

deficiencies, before those procedures l

are used to conduct operating tests.

1 i

l

'l Potential to Confuse - Deviation potential to perplex the operator and disrupt his mental and physical; actions.

Potential'to Error - Deviation potential to lead the.operatcr to not perform any action or to perform an incorrect action.

Potential to Impede - Deviation potential to delay or hinder-significantly the operator's correct response.

Procedures - Reference plant normal operating procedures, abnormal operating procedures, emergency operating procedures, and emergency plan implementing procedures that an operator or candidate would be required to implement. When plant procedures are referenced to throughout this document,1 1t is assumed that they are "controlled copies" of the procedures, unless otherwise specified.

Procedure Performance Time - The realistic or actual time for a candidate to perform a procedure or task in the reference plant control room.

Real Time - Computer simulation of dynamic performance in the same time base relationships, sequences, durations, rates and accelerations as the dynamic performance of the reu rence plant.

Reference Plant - The specific nuclear power plant from which a simulation facility's control room configuration, system control arrangement, and design data are derived. ,

Simulation Device - A component of a simulation facility that simulates a portion or all of the reference plant.

Simulation Facility - One or more of the following simulation devices, alone or in combination, used for the conduct of operating tests:

1) Non-Plant Referenced Simulator

2) Control Room Hock-Up

3) Reduced Scope Simulator

4) Part Task Simulator

5) CRT Simulator

6) Reference Plant

System Function / Task Analysis - A systematic analysis of the reference plant procedures that yield the cue and I&C requirements.

l Task - A unit of control room operator work which may require plant-information collection, Systems operation, or both.

Task Element - A unit of human activity comprising a task.

l l Task Statement - An independent unit of control room operator l  !

work.

l 1.4 Acronyms i

CFR - Code of Federal Regulations CRDR - Control Room Design Review CRT - Cathode Ray Tube CRM - Control Room Hock-Up HF - Human Factors I&C - Instrumentation and Control NPRS - Non-Plant Referenced Simulator NRC - Nuclear Regulatory Commission PRS - Plant Referenced Simulator PTS - Part-Task Simulator RO - Reactor Operator

. .~ ,. .. . - .

~

4 RSS - Reduced' Scope Simulator

• SFTA - System Function / Task Analysis '

SHE - Subject Matter Expert SR0 - Senior Reactor Operator USFG - Utility Simulation Facility. Group t

s 4

L f

(

i i

k i

i f

1 5

f I

1:

i, i

i r

t

$ i I l

n ,

i -[g .

p. L' A n . . . . _ . . _ _ . _ . . . . .....__.....___..____.__...._......___.____.______....__._---m._-_. '

2.0 CRITERIA The following section provides generic criteria to be applied in the evaluation and use of a simulation device. -All or part of each of these criteria is applied to specific simulation devices in the manner described in Sections 3 and 4.

i

}

i i

i 2.1 lhLmjtn Factors

} Human factors addresses the comparability of the simulation device with the reference plant in the areas of control room and panel layout, I&C configuration and ambient operating environment. The primary human factor consideration in a plant referenced simulator (PRS) is for the simulator to have fidelity with the reference plant. PRS fidelity means duplication in physical appearance, physical laycut, system function and system model. For a simulation device other than a PRS,100% fidelity may not be achievable in system model and physical layout. Therefore, the only highly achievable fidelity component would be duplication in system function. A simulation device has to meet a high degree of duplication in system function criteria to be considered to be viable for the conduct of an operating test.

4 i

The degree to which a deviation does not meet physical / functional fidelity i>ecomes a human factors concern. In a simulation device,

the principle goal of human factors is to assure fidelity deviations S.e no negative impact on operator task performance. Deviations that hinder the operator should be documented and evaluated in a systematic fashion. Cognitive and behavioral operator actions should be considered. Operator perceptions (color, mimics, patterns, etc.) are to be considered if essential to task performance. Other human factors considerations deal with the specific components in Table 1. The simulation facility will i

l

- . ~

HUMAN FACTORS CONSIDERATI_0RS MajorArea Comoonents Control Room Layout - Physical Orientation Operator Station Panel Layout - Systems Orientation Control Panels Annunciator Panel Mimics Instruments and Controls - Displays Controls Instrument Range Instrument Accuracy Engineering Units Ambient Operating Environment - Normal and Emergency Lighting Humidity Temperature Noise Communications Auditory Signals Table 1

~

r contain controls, instruments, alarms and other man-machine interfaces for the operator to demonstrate his capability. The ambient environment in the simulation facility should be replicated to the extent possible. To the extent practicable, the following generic criteria should be applied to any simulation device. The specific application of this generic criteria to each simulation device is discussed in Section 4.

Control Room and Panel layout The simulation device should approximate the reference plant physical orientation and appearance. The simulation device should be the same physical size as the reference plant although reduced scale reproductions ar9 acceptable provided the SFTA determines that the reduction does not significantly detract from the operating test.

The operator's station and other working space should be i

replicated. Deviations from the reference plant shall be evaluated as discussed in Section 3.

l l

The control panels should be positioned on the simulation device in the same physical location as the reference plant. The systems orientation within the panel should repliccte the reference plant.

Deviations from the reference plant panel layout shall be evaluated as discussed in Section 3.

(

i 1&C.Confiouration The simulation device controls, indications, etc., on the control panels should approximate the same physical location as in the reference plant. -The instrument displays, controls, range, accuracy and units should replicate the reference plant. Deviations from the reference plant I&C configuration shall be evaluated as discussed in Section 3.

Ambient OoeratiDa Environment The ambient operating environment shall permit the operator to perform his duties. The ambient operating envi~'nment factors to be considered are lighting, hurnidity, temperature, noise and communication. Significant deviations from the reference plant environment shall be evaluated as discussed in Section 3.

2.2 Procedures The procedure considerations are: 1) the scope, 2) the manner of use and, 3) methods of modifying the procedores used in the administration of operating tests. Controlled copies of reference plant procedures are used in the conduct of an operating test.

Procedures performed on a simulation device allow candidates to demonstrate their "ability to perform" the operations required by those procedures. The following generic criteria should be applied when reference plant procedures are used during the course of an

operating test on the simulation devices described herein. The specific application of these criteria to each simulation device is discussed in Section 4.

Procedure Scone Types of procedures exercised on the simulation device include:

Normal Operating Procedures, Abnormal Operating Procedures, Emergency Operating Procedures, and Emergency Plan Implementing Procedures.

The scope of procedures to be exercised on any simulation device shall be determined using the methods discussed in Section 3.

Procedure Use Reference plant procedures will be used on the simulation device (s). The reference plant may be used to exercise th ' Normal Operating Procedures' that can be performed as part of norr.1 operations. Those procedures or tasks requiring control room interaction shoul1 be perr'ormed on a single simulation device (or appropriately integrated simulation devices).

J

t' Procedure Modifications The data obtained during the SFTA (described in Section 3) will be used to identify any deviations between the simulation device and the procedures that will be tested. The determination will then be made which, if any, of these deviations should be resolved by pen.

and ink procedure changes. The procedure steps may then be modified to accommodate simulation device deficiencies, provided the modifications do not significantly detract from the conduct of the operating test. This may include partial completion or deletion of procedure steps or assuming the successful or unsuccessful completion of operator tasks that cannot be performed on the -

t simulation device.

i Before procedures are used in the conduct of operating tests, any  ;

naressary pen and ink procedure changes identified by *:he SFTA will i i.

be made. Pen and ink protadure changes may possibly affed i Procedure Performance Time. Procedure Performance Time will be taken into consideration in the development, upgrade or use of existing devices or in the implementation of the pen and ink procedure changes.

All recommended pen and ink procedure changes will be forwarded to  !

the Multidisciplinary Review Team (see Section 3.1) for review.

i sl4-

l Such changes will be made only after the following has been considered:

1. Determination has been made that the controlled procedure cannot be performed on existing simulation devices.

2. Upgrades to existing simulation devices, or the development of new simulation devices for the procedure (s) or part of the procedure (s) which cannot be conducted require an ,

excessive effort or burden in relation to the benefit gained. The evaluation of this burden 11 benefit shall be documented.

3. Determination has been made that pen and in' changes are  :

preferable to other alternatives (i.e., use of controllers or other similar mechanisms have been considtred, but would result in a degradation to the examination process). ,

l Any procedure modifications will be controlled by the simulation 4

facility Configuration Management Program described ir Section 3.7.

r 2.3 Steadv State and Transient Models The steady state and transient modeling used as part of the i simulation device (s) shall adequately model the operating behavior of the reference plant. The following generic criteria should be -

a.-. ,_ ~ _ .-_ _. . . . - _ , ,, .-, - __

applied, as applicable, to any simulation device. The specific application of this criteria to each simulation device is discussed in Section 4.

SLQ9st Simulation device output should approximate and display expected plant responses. The responses should be based upon plant operating data or best estimate analyses, as appropriate.

Fidelity The models should be of a level of sophistication necessary to assure the adequacy of the output information being presented to the operator.

c Ilmft All simulation devices should approximate real time.

2.4 Performance Testina Performance testing is conducted to verify the simulation device performance as compared to actual or predicted reference plant performance. The initial performance testing shall serve to verify and validate the adequacy of the completed Simulation Facility, including any procedure modifications. The specific application of performance testing criteria to each simulation device is discussed in Section 4.

4- .

Performance testing should be conducted on a schedule consistent with 10CFR55.45. ,

2.5 Ooeratina Test Methodoloay This section provides a generic process for conducting examinations on simulation devices for the purpose of evaluation in accordance with 10CFR55.45(a). The following generic criteria should be implemented, as applicable, on any simulation device. The specific application of this criteria to each simulation device is discussed in Section 4.

Those portions of the operating test conducted on each simulation device shall be limited to the procedure scope determined by the Operational Cue Analysis (described in Section 3.2) for that simulation device.

m Examinations should be conducted in accordance with NUREG-1021, "Operator Licensing Examiner Standards."

Procedure Performance Time on a simulation device should be followed as closely as possible during operating tests.

During the conduct of the operating test, Controllers may be required to mitigate identified simulation device deficiencies. The use of Controllers should follow the guidelines described below.

f Guidelines for Controller Interaction The role of the Controller is to provide an added dimension to the simulation device to enable the device to more closely approximate the reference plant during the conduct of operating tests. In this sense, Controller usage is similar in nature to those activities conducted by utility instructors who, during conduct of operating tests on Plant Reference Simulators, provide information as outside operators, I&C technicians, etc. In this case the Controller is used to augment the simulation devices. Therefore, the purpose of the Controller is to provide, under direction, those cues unavailable from the device that may be needed to carry out actions during the performance of the operating test.

Controllers used during implementation of operating tests on various simulation devices shall follow specific guidelines established herein and as mutually agreed upon by individual examiners and the simulation facility management.

Examination Integrity t

o Examination integrity is paramount to the s CCess J of the operating test. The Controller should not compromise examination integrity.

I i

o Controller actions shall be conducted under the direct supervision and control of the simulation facility operator.

o Controllers shall not prompt the operators in the performance of their duties. Prompting may result in the invalidation of the operating test.

Controller Oualifications o Controllers shall be trained on their duties and responsibilities. Details of this training program shall be included as part of the simulation facility plan submitted by each utility.

o Qualificaticns of controllers shall be supplied as part of the simulation facility plans submitted by each individual utility and should meet the minimum criteria listed below.

1) The controller should be employed by the utility or as a vendor under contract to the utility.

2) The controller should posess the level of training and qualifications required by the utility for simulation facility instructors as outlined in their respective accredited training programs. Controllers should hold or have held an SR0 license or certification on the

plant for which the operating test is being conducted.

Controllers should also be knowledgeable on the simulation device on which the operating test is being conducted.

3) Controllers should receive additional training, as required by the utility, on the conduct of operating tests. As a minimum, this training shall cover the criteria listed above.

Controller Functions o Controllers shall function to provide cues to the operators that are not available from the simulation device, o Controllers shall provide cues only as answers to specific questions from the operators or as directed in the operating test scenario. These cues are only for the purpose of providing information not available from the simulation device or as necessary to clarify a deviation of the simulation device from the reference plant.

o Controllers shall perform any other actions as identified and directed by the examiner in the conduct of the operating test.

3.0 DEVELOPMENT OF A SIMULATION FACILITY This section provides the methodology used to develop a simulation facility to meet 10CFR55.45(b)(1)(1) using one or more of the simulation devices that is described in Section 4. The purpose of the simulation facility is to enhance the conduct of operating tests.

It is recognized that operating tests are better performed on devices that promote an active man / machine interface, such as Non-Plant Reference Simulators, Reduced Scope Simulators, or Part Task Simulators. This interface facilitates evaluation of the individual (s) in an actual operating environment. However, in cases where there is limited implementation of an active man / machine interface, adequate qualifications can be demonstrated by alternate devices. As further discussed in Section 4, these methods use Control Room Mock-ups, CRT Simulation and the Reference Plant alone or in combination.

The following systematic evaluation methodology description below is based upon an SFTA. The process briefly identified in Figure 1 and Table 2 and further described in Section 3 is one method for developing a Simulation Facility based upon an SFTA. Each utility may have completed portions or all of the SFTA, but not in the exact format described below. Therefore, this section describes the USFG's understanding of the detail needed in such an analysis. It

SIMULATION FACILITY DEVELOPMENT PROCESS FLOH Analysis of reference plant procedures to deduce cue and I&C information requirements.

SFTA Comoarison of simulation device with information requirements.

Evaluation of fidelity deviations between simulation device and the reference plant.

f I

SIMULATION DEVICE 8

1. NPRS

2. CRM FIDELITY

3. RSS EVALUATION

4. PTS

5. CRT SIM.

6. REF. PLANT Selection of simulation device to satisfy information/ fidelity requirements.

Intearation of simulation device (s) into a simulation facility, qg t

1 SIHULATION Utility Plan of simulation FACILITY devices used to meet 10CFR55,45 rule.

Figure 1 SMA20N FACILITY DEVELOPMENT-PROCESS STEPS System Function / Task Analysis

1. Task Identification

2. Task Element Identification

3. Cue'Information Requirements

4. I&C/ Physical Characteristics Simulation Device Comoarison and Selection

1. Comparison of SFTA/ Reference Plant with Simulation Device

2. Refer Deviation to Fidelity Evaluation Fidelity Evaluation

1. Identification of Deviation

2. Assessment of Deviations

3. Disposition of Deviations Simulation Facility Plan

1. Description of Simulation Devices

2. Match Procedures with Devices Table 2 is expected that plant specific implementation of an SFTA will include parts of or all of any previously performed SFTA or related type work.

The rigorous implementation of this simulation development methodology will assure the appropriate use of a particular simulation device for the performance of operating tests.

Conversely, the implementation may provide indication that a currently used simulation device is inappropriate to meet the needs of an operating test and a new device should be pursued. Therefore, it is the goal of this simulation facility development methodology to provide an appropriate simulation facility, considering both existing and new simulation devices.

3.1 System Function / Task Analysis (SFTA)

The SFTA is a systematic analysis of the reference plant procedures that yield the cue and I&C characteristics. From the reference plant control room instrumentation, the operator obtains the required cues to help him in performing his tasks. The systematic determination of these man / machine interface characteristics provides a basis for evaluating the adequacy of a simulation device to support the conduct of operating tests. lables 3 and 4 (located at the end of Section 3) detail the SFTA steps and process and SFTA criteria, respectively.

3 The method is characterized as a top-down analysis which begins with the reference plant procedures. The procedures are partitioned into units of activities identified as tasks. A task is a unit of control room operator work which may require information collection, systems operation, or both. A task is characterized by being a relatively small unit of work which is comprised of approximately the same sequence of elementary human actions regardless of the operational sequence in which the task appears. The main criterion for the identification of a task is that the task should define the information and/or control functions needed by the operator to perform that unit of work.

Each task is further partitioned into the units of human activity, called task elements, which need to be sequentially accomplished in order to execute the task. The main criterion for identifying these task elements is that each snould further refine and identify the information requirements needed by the operator to execute the task in the context of all the operational sequences in which the task appears.

A set of cues is developed from the task element requirements. The cue in this context is defined as the significant information acquired by the control room operator that prompts him to act. The cue is taken directly from the task element, taking into account the information required from the specific system.

t A set of characteristics describing the functional requirements for ,

each task element is identified. The task element may be associated with a specific instrument. In that case, the characteristics -

include range and units. The physical requirements will be .

j specified if the particular characteristic is critical to performing l the task. In cases where only one type of equipment will satisfy ,

the physical and functional requirement, the specific manufacturer and component will be identified. Example I shows the breakdown of an operator task. ,

The SFTA will be conducted by a Multidisciplinary Review Team composed of subject matter experts (SMEs) of appropriate background and whose responsibilities are discussed below. The SMEs will

! utilize their expertise to process the task breakdown. The SFTA 4 process steps will be documented and supported with appropriate sources.

Multidisciolinary Review Team -

4

! The Multidisciplinary Review Team shall consist of utility or consultant personnel that are SHEs of appropriate disciplines. The areas of expertise and experience of the personnel on the

{

1 Multidisciplinary Review Team will be detailed in each utility's plant specific plans. The role of the Multidisciplinary Review Team

is to

I i

i

BREAKDOHN OF AN OPERATOR TASK J

Operation Procedure - E0P XXX Task Statement - Determine if RCPs are operating Task Element - Read RCP ON/0FF Status, System - RCP Read RCP current, System - RCP Read RCP flow, System - RCP Read RCP Delta P, System - RCS Read Voltage to RCP, System - RCP Read RCP Speed, System - RCP l

Read RCP Electric Power, System - EDS I t Cue Information - RCP Speed > 500 rpm Annunciator Hindow Off No Annunciator Audio Alarm 1f I&C/Phy;ical Requirement - Display Value Range. 0-1200 Units, RPM Accuracy, N/A Display on Control Room Panel 56 Abbreviations 1

l RCP Reactor Coolant Pump i

EDS - Electrical Distribution System l Example 1 i

l

~27-i

- Review the SFTA process and documentation generated by the SFTA.

- Evaluate the Operational Cue Analysis data and disposition the deviations identified in this process,

- Determine the applicability of each procedure to each simulation device.

- Recommend the makeup of the Simulation Facility.

SFTA Philosoohy The System Function / Task Analysis (SFTA) is a logical link to the other analyses that have been conducted to support training programs. The starting point for SFTA is the procedures which were also the basis for the job analysis previously done for the training programs. The job analysis provides the skills and knowledge requirements needed for the operator to perform his job. The task analysis provides the task information requirements for the operator to perform his tasks. The system function analysis provides the functional operator controls requirements needed to perform procedural steps.

The job analysis, task analysis, and system function analysis connect to the three points of a man / machine interface model that exists at the utilities and discussed in INPO Document 83-047. The operator is the center of the triangular model shown in Figure 2.

The model addresses the interfaces and interrelationships that exist among the operator, the plant (system function analysis), the procedure (task analysis), and the training (job analysis). The SFTA process ensures that the component interfaces have been considered adequately and that the operators can operate the plant safely and efficiently during all operating situations.

3.2 Doerational Cue Analysis The Operational Cue Analysis consists of the SFTA and the simulation device comparison and selection process. An I&C inventory will be conducted on each simulation device. The I&C inventory will include a listing of the presence of both static and dynamic cues, each type of cue being important to the operational cue analysis. As shown in Figure 1, the I&C inventory is then compared to the SFTA results to ide.tify the I&C set and cues available to execute the reference plant procedure set. Each procedure task listing is evaluated to determine the ability to adequately perform the procedure on the simulation device. This process yields a procedure set applicable for examination on that particular simulation device. Procedures

MAN-MACHINE INTERFACE MDfi Job Analysis Training

3::

i i'li:

Operator ""

.! l;! . . .

I i"i i '

. i' ii:

!!il.i

.ij.

... :ii!'! !!!!!  :! '

?!!E!!!:i!!

!!i" .!!iiiiiNiiii' . . . . .

i:'

i

!;; "  ::; i': """ n:- ., . . . . . .

.. t.

.  :: !!i t 6

... nn:::: ' !! !:.:. . ' "

"':!"::4 iHi i"i

i!!!!!. !!'".  ;;;;; , ;;g i;; i;.;... filiiii-ri.ii..i

., .iii.i  !.!i..fi. .i.ii !  :

.+

Procedures n

p.  !:i.!

Plant Task Analysis System Function Analysis figure 2 that cannot be implemented on any other simulation device can be examined on the reference plant. Accordingly, it is expected that 4

all procedures can be examined on the Simulation Facility.

The process of collecting data for the Operational Co* ;1ysis shall be similar to that identified in NUREG * .. Appendix 8. The data fields to be evaluated shall be ' .,.-mined by the SFTA. The discrepancies between the ref-' , plant and the simulation device shall be dispositic - , che Multidisciplinary Review Team using

..., . ..s .aentified in Sect *on 3.3.

. 3.3 Fidelity Evaluation

After a simulation device goes through the SFTA comparison process, a fidelity evaluation should be performed. The purpose of the fidelity evaluation is to identify and assess the deviations of a simulation device from the reference plant. A type of fidelity evaluation process and criteria that could be used is detailed in Figure 3 and Table 5 (located at the end of Section 3). The fidelity evaluation identifies the potential areas where comparability deviations could cause an operator 'ficulties in performing procedure tasks. Deviations are identified from three 4

k F

FIDELITY EVALUATION h SFTA Reference Plant Simulation Facility i g (Table 1) Development p I

$ f U F

C U

f 5 i

~

Identify U Deviations Jl 5

y 1 M

M Evaluate Q and Rate C 37 Deviation Modify Device, Another Device, Cont: 011er, Modify H Procedure, Etc.

=

h 0

Sum Rate Jes

> 4.00 0

S N E No p

, e dsk j

l l Figure 3 32-

different sources: 1) the SFTA re:ults, 2) the control room as noted in Table 1 and 3) the Simulation Facility Development process. The evaluation proceu will consist of identifying the deviation, assessing the deviation, and dispositioning the deviation. The evaluation process will be done by the Multidisciplinary Review Team.

Deviations vary to degr:es of impact on the operator. Three factors will be used to assess the deviation impact; (1) Potential to Impede, (2) Potential to Error, and (3) Potential to Confuse. The Potential to Impede is the potential that the deviation has to del.?y or hinder significantly the operators correct response. The Potential to Error is the potential that the deviation has to lead the operator to not perform any action or to perform an incorrect action. The Potential to Confuse is the potential that the deviation has to perplex the operator and disrupt his mental and physical actions. The Potential to Confuse includes the potential for the simulation to enhance the operator's performance by the presentation of information not available in the reference plaat.

The Potential to Impede is the most important factor since this factor can prevent the operator from performing the tasks. The Potentials to Error and to Confuse are of equal value. The three factors are weighted with the Potential to Impede having a factor of 0.50 and the Potential to Error and to Confuse having a factor of 0.25 each.

M Each deviation will be rated on each potential from 1-10. A low q probability is a measure of 1-3. A medium probability is a measure of 4-7. A high probability is a measure of 8-10. The formula to derive the sum rate is:

Sum Rate - Pj (.50) + P2 (.25) + P3 ( 25) a where: P) is the Probability to Impede P2 is the Probability to Error 4

P3 is the Probability to Confuse v

The sum rate will determine the degree of deviation impact on the j operator and direct the solution to the fix. For example, any sum rate less than or equal to 4.00 may denote the deviation can be left "as is."' Conseqsently, any sum rate greater than 4,00 may denote a

- deviation.that must be addressed by modification.

The weighing factor is derived from tne D. Meister method of i

assigning weights to criteria factors. The method of assigning 1-10 l measurements is derived from the CRDR experience of rating Human Engineering Descrepancies (HEDs) for priority.

It is recognized that the process described above is a subjective i one, since the results depend upon the judgment of the personnel l

evaluating the deviation in their application of the low, medium and high probability values discussed above. The assignation of i

l l

mfw i

• Q }% ,

ew l 4,

4

~ probability ratings and determination of modifications disposition is the responsibility of the Multidisciplinary Review Team performing the evaluation (s). Therefore, the guidance of this document suggests the use of a rating criteria such as the 4.00 t criteria suggested above. It is the purpose of this rating guidance to specify that this type of rating of deviations be performed to I determine the need for modifications and/or justification of  ;

existing design. The use of 4.00 allows for the judgment that l deviations of low probability most likely would not require modification and that some medium probability deviations should not require upgrade. It is expected that these types of medium  :

. probability deviations could consist of, but would not be limited  ;

to, deviations that are not frequently encountered by the operator f in his duties and, therefore, would not be expected to significantly degrade the examination process. This rating acknowledges that the majority of the medium probability deviations and all of the high probability deviations should be addressed by a simulation facility l modification, unless significant justification is provided to the contrary. Plant specific plans should describe the rating criteria j and basis for that criteria, if the proposed criteria takes exception to this guidance. (

[

3.4 Simulatien Device Integration The integration of the SFTA and the simulation device capatilitisJ  !

is necessary to ascertain the optimum simulation facility for the performance of operating tests. The goal of the selection process is to provide a simulated control room environment that presents the highest level of active man / machine intarface, in which the reference plant procedures can be exercised. The Sinulation Facility shall provide the opportunity to test all of the operator responses to the cues identified by the operational cue analysis of

^he reference plant procedures. The ability of the operator to use controlled copies of the reference plant procedures and produce the desired responses on the simulation device, so as to place the plant in the desired operating configuration, shall be a determining factor in selecting the simulation device for the operating test.

The simulation device shall be capable of producing the desired response to the operator actions identified in the reference plant procedures to the extent necessary to assure that the operator can determine from the available cues that the plant is responding in the direction predicted in the reference plant transient analysis.

The process of selecting the simulation device (s) that will make up the simulation facilit) oegins by identifying the device (s) that provides the most active man / machine interface, An I&C inventory would be conducted on the selected simulation device (s). As shown in Figure 1, the I&C inventory is then compared to the SFTA results to identify the I&C set and cues available to execute the reference plant procedures. Each procedure task listing will be evaluated to determine the ability to adequately perform the procedure (s) on the simulation device (s) without consideration of modifications to the device (s).

Following consideration of simulation device modifications, the process would then be repeated. The end result of this process will yield a procedure set applicable for examination on that simulation device and a list of simulation device modifications that are appropriate.

A similar process would be conducted on other simulation devices in descending order of their capabilities for active man / machine interface.

The emphasis during this process is to identify the simulation device that is most useful for examiners and at the same time provides a cost effective simulation device that will provide for effective training. Before any passive simulation device is selected consideration will be given to the use of controllers and pen and ink changes to the procedures with the intent to always try to use the device that promotes the most active man / machine interface.

The final selection of devices and the integration into a simulation facility is an iterative process. The benefits of a device that l promotes active man / machine interface must be continuously evaluated against the negative aspects of the device. Consideration must be given to the number of pen and ink changes, intrusion of the l controller on the operators activities, human factors issues, etc.

l l

l l

r 3.5 Simulation Facility The last step of the flow process illustrated in Figure 1, is to describe all the devices that makeup the Simulation Facility. The plant specific application for approval should describe the procedures to be run on each simulation device. The supporting documentation for each plant's application should include all the dispositioned deviations along with the evaluations oi.J backup information.

3.6 Imolementation The development of a Simulation Facility should be undertaken in a systematic manner, giving considuration to the benefit of any upgrades to a simulation device. When considering or upgrading any simulation devices, the Operational Cue Analysis (Section 3.2) should be used to assess the significance and importnnte of the modi fi catior.. The implementation of the Simulation Facility should give consideration to any device that promotes active man / machine interface, not necessarily just those included in this document. It is understood that there are other simulation devices available now or will be available with advancing technology that may be cost effective for particular applications. The intent of this document is to describe a systematic approach to the development of a facility; not to restrict the facility te using any particular

! simulatica device (s). It is important that the simuiation facility provide the opportunity to examine all of the operator responses to the cues listed in the Operational Cue Anaysis.

1 3.7 Confiauration Manaaement Proaram A program to provide accountability of Simulation Facility p?"cical and functional capabilities will be developed and implemented. This program will consist of procedures and/or guidelines and will be controlled by established administrative procedure controls for each ctility. The program will perform the following functions:

1. Identify, document, track and test discrepancies.

2. Identify, document and track deviations between the Simulation Facility and the reference plant.

3. Identify, document and track reference plant changes denoting effects on operating tests.

The details of the program will be described in the individual utilities plan and will include the following:

1. An outline of the administrative procedures and responsibilities for maintaining the Simulation Facility

, current in accordance with the guidelines of this document.

2. A description of the organization (s) responsible for maintaining the Simulation Facility.

7

3. The administrative procedures, including time requirements, for updating the Simulation Facility upon a reference plant modi fication.

4. The administrative procedure (s), including time requirements, for completing, if necessary, a review of the SFTA upon a modification to the Simulation Jacility.

5. The administrative procedure (s) for review /svaluation of the performance test baseline data upon, modifications to the reference plant, modifications to the Simulation Facility or on a periodic bases (i.e., once every four years).

i t

b b

l l

l 1 . .- .. . ._ -.

SYSTEM FUNCTION / TASK ANALYSIS SIEP_S AND PROCESS STEPS PROCESS I. Task Identification A. Select Station Procedure - Select one station procedure at a time to process.

B. Breakdown Station Procedure - Breakdown the procedure according to operators actions (task).

II. Task Element Identification A. Breakdown Operator Task - Breakdown the operator tasks into small units of werk.

(task element)

8. Identify Cognitive / - Associate task elGment with Behavioral Action operator's cognitive and behavioral actions.

III. Cue Information Reauirements A. Specify Task Element - Determine information operator Information needs to do task elements.

l B. Specify Perceptual - Determine information operator l Information Required is cogaizant about before, I

after and during task element performance.

C. Identify Time Dependency -

Determine information that Requirements will assist operator in l

performing the task element.

Determine information that is l

very critical relative to I operator actions.

l l

Table 3 l

t i

t SYSTEM FUNCTION / TASK ANALYSIS STEPS AND PROCESS STEPS PROCESS IV. I&C/ Physical Characteristics A. Identify I&C - Deduce the functional I&C Characteristics characteristics required to carry out the task.

B. Identify Physical - Identify physical character-Characteristics istics if required to perform task element and function.

- Specify type and model of component, if one-of-a-kind.

1 l

l l

Table 3 (Continued) l

SYSTEM FUNCTION / TASK ANALYSIS CRITERIA Task Identification

1. A task is a unit of control room work that can be easily analyzed.

2. A task completion is independent of the preceding or following task.

3. A task is comprised of a limited number of human action units (elements).

Task Element Identification

1. Task elements are elementary human actions needed to accomplish a particular task.

2. Task elements can be divided into cognitive and behavioral '

divisions.

3. Each task element refers to information or control requirements.

4 Task elements are written only for operator actions which require control room information or control function.

Cue Information Reauirements

1. The cues required enable operator task performance.

2. The cups assist the operator in assessing the control room status relative to the immediate tasks.

! 3. The cues are a perceptual process that include audio, visual and sensory inputs.

4. Cues that are time dependent must denote the time window required.

Table 4 i

SYSTEM FUNCTION / TASK ANALYSIS

-CRITERIA I&C Physical Characteristics 4

1. The I&C characteristics must specify the:

- Display required.

- Range required.

- Units of the quantity.

2. State physical requirements, if critical to performing the immediate task and function.

i l

l ,

l l

l l

Table 4 (Continued) l 1

t

l FIDELITY EVALUATION CRITERIA Identification of Deviation

1. A deviation is a physical difference that exists between the simulation device and the reference plant for the areas in Table 1.

2. A deviation is a difference in function between the SFTA and the simulation device.

3. A deviation is any other significant differences found between reference plant and simulation device during the development of the simtilation facility.

Assessment of Deviation

1. The deviation trust not prevent the operator from performing tasks.

2. The deviation must not confuse the operator to the point of affecting task performance.

3. The deviation must not lead the operator to an error in task performance.

Disoosidon of Deviation

1. Any modification to the simulation device in question will require approval by tne Multidisciolinary Review Team.

2. Any deviation referred to another simulation device for l resolution will require feedback through the Fidelity Evaluation process.

3. The selected disposition will require verification.

4. Any deviation left "as is" must have all supporting documentation and analysis and an adequate description.

I

5. Pen and ink procedure changes to the procedure will be the last option to resolve deviations.

l l

Table 5 l

l l

4.0 SIMULATION DEVICES 1

l l The following sections discuss the various simulation devices used in the conduct of operating tests. Thesa simulation devices are curretitly available simulation technology that can be used in the development of a simulation facility. Each section discusses the i application of the previously described criteria to each simulation device, the recognized limitations of each simulation device and tlie advantages of each simulation device.

4.1 NON-PLANT REFERENCED SIMULATOR A Non-Plant Referenced Simu'.: tor (NPRS) is a simulation device that models plant systems. The following criteria apply to the use of an

' NPRS as a simulation device.

4.1.1 Human Factors Control Room and Panel Lavout An NPRS may not replicate the refercntu plant control room and panel layout. Deviations will be evaluated for impact on performance of operator tasks in accordance with the Section 3 methodology.

I&C Confiauration Hardware and location differences are allowed. Plant specific labels, overlays, scaling modifications or other surface enhancements should be used to make NPRS controls and panels more closely approximate the reference plant. Deviations will be evaluated for impact on performance of operator tasks in accordance with the Section 3 methodology.

Ambient Ooerating Environment An NPRS may not replicate the reference plant control room ambient operating environment. Deviations will be evaluated for impact on performance of operator tasks in accordance with the Section 3 methodology.

4.1.2 Procedures As previously noted, the types of procedures exercised on the NPRS may include:

Normal Operating Procedures Abnormal Operating Procedures Emergency Operating Procedures, and Emergency Plan Implementing Procedures.

The scope of procedures to be used on the NPRS will be determined by the SFTA process described in Section 3. For each procedure performed on the NPRS, the Procedure Performance Time will be matched as closely as possible. The remainder of the generic Procedures (Section 2.2) criteria applies to the NPRS.

4.1.3 Steady State and Transient Models Scope The output of the NPRS should approximate reference plant response.

However, the similarity will be limited by considerations such as

! core size and numbers of redundant or auxiliary systems. As a minimum, the NPRS should be capable of producing the operator cues required to enable implementation of those procedures identified by 1

5

-e. , , - -- -

the process described in 4.1.2. Sof tware modifications should be implemented, as practicable, to achieve full exercise of the procedures on the NPRS.

Fidelity The steady state values for critical parameters shall be stable and not vary significantly ' rom the initial values over a 60-minute period. Changes in critical parameters should correspond in direction to those expected from operating data or a best estimate analy:is and should not violate any physical laws of r3ture. If parameters deviate significantly and software modeling changes cannot be reasonably pursued, these parameters may exceed the criteria, provided:

o They are specifically corrected using a Controller DE o Cue cards are substituted for these parameters DE o Appropriate pen and ink procedure changes are implemented.

ILm_st NPRS plant operating and transient time responses should approximate real time simulation. Any deviations will be handled by the SFTA process described in Section 3.

49-

- - - - _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ - _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ - _ n

4.1.4 Performance Testina Scope The NPRS performance testing should be limited to those procedures identified by the process described in 4.1.2.

Methodoloav Those procedures identified in 4.1.2 should be performed on the NPRS after all identified software modifications have been incorporated.

An initial set of data shall be collected and evaluated. After review and approval by the Multidisciplinary Review Team, these transient results become the baseline data set for subsequent performance testing evaluation.

Acceotance Criteria Determination of acceptability of performance test results shall be performed by the Multidisciplinary Review Team.

4.1.5 Ooeratina Test Methodoloav The NPRS permits evaluation in a control room team environment. The operating test conducted on the NPRS shall be limited to the procedures identified by the process described in 4.1.2. The generic guidelines for Controller interaction (Section 2.5) shall apply. Those procedures not accomplished on the NPRS should be evaluated separately on another simulation device.

Examiners will be able to select various NPRS initial conditions and plant malfunctions to evaluate operators' or candidates' responses.

The NPRS should have the capability to stop and restart the simulation, as necessary, at any point.

The remainder of the generic Operating Test Methodology (Section 2.5) applies to the NPRS.

t l

s

(

1 i

4.2 CONTROL ROOM MOCK-UP A Control Room Hock-up (CRM) is a simulation device that consists of a display of the reference plant control room panels, including the switches, indications, and alarms, arranged in a configuration similar tc the reference plant. The CRM may consist of photographs, three-dimensional mock-ups or a combination of both. The following criteria apply to the use of a CRM as a simulation device.

4.2.1 Human Factors Control Room and Panel Layout The CRM should replicate the reference plant physical orientation and appearance. The CRM should be the same physical size as the reference plant although reduced scale reproductions are acceptable provided the SFTA determines that the reduction does not significantly detract from the operating test. Any reduction effort should be limited such that labels, controls, indications, alarms, etc., remain clearly legible. Deviations will be evaluated for impact on performance of operator tasks in accordance with the Section 3 methodology.

I&C Configuration 1

The CRM controls, indications, etc., should be in the same physical location as in the reference plant. They should be replicated in sufficient detail to enable the desired operator capabilities to be

successfully demonstrated. Functional fidelity of a CRM cannot be achieved due to its passive nature. Deviations will be evaluated for impact on performance of operator tasks in accordance with the Section 3 methodology.

Ambient Ooeratina Environment The CRM may not replicate the ambient operating environment of the reference plant. Deviations will be evaluated for impact on performance of operator tasks in accordance w'ith the Section 3 methodology.

4.2.2 Procedures As previously noted, the types of procedures exercised on a CRM may include:

Normal Operating Procedures, Abnormal Operating Procedures.

Emergency Operating Procedures, and Emergency Plan Implementing Procedures The scope of procedures to be used on the CRM will be determined by the SFTA process described in Section 3. For each procedure performed on the CRM, the Procedure Performanco Time will be matched as closely as possible. The remainder of the generic Procedures (Section 2.2) criteria applies to the CRM.

4.2.3 Steadv State and Transient Models Scope Operator cues required for those procedures identif*ed by the process described in 4.2.2 will be given on the CRM using cue cards or by use of a Controller. These cues should be based on reference plant operating data or steady state and transient best estimate analysis data.

Fidelity The CRM should be a representation of the reference plant.

Controllers or cue cards may be used to enhance procedure usage.

Ilma CRM usage is not expected to be in real time.

4.2.4 Performance Testina Performance testing to verify a simulation facility's performance as compared to actual or predicted reference plant performance is not applicable for a CRM.

4.2.5 Ooeratina Test Methodology Operating tests conducted on a CRM are to consist of walkthroughs for each procedure to be tested. Task performance can only be

discussed and may require extensive use of Controllers and/or cue cards to provide operational cues. The remainder of the generic t

~0perating Test Methodology (Section 2.5) applies to the CRM.

e b

h

?

I f

l:

l l-l l

i

?

l 1

l >

i r

! -ss-1 i

4.3 REDUCED SCOPE SIMULATOR A Reduced Scope Simulator (RSS) is a simulation device that physically and functionally models significant portions of the major systems of the reference plant. A RSS demonstrates expected plant response to operator input and to normal and transient conditions to which the simulator has been designed to respond. However, the number of initial conditions, normal functions, and malfunctions available will be less than the standard defined by ANS-3.5-1985.

The following criteria apply to the use of a RSS as a simulation device.

4.3.1 buman Factors Control Room and Panel Layout The RSS sFould be positioned to approximate the reference plant physical orientation and appearance. The RSS should be the same physical size as the reference plant although reduced scale reproductions are acceptable provided the SFTA determines that the reduction does not significantly detract from the operating test.

Any reduction effort should be limited such that labels, controls, indications, alarms, etc., remain clearly legible. Deviations will be evaluated for impact on performance of operator tasks in accordance with the Section 3 methodology.

I&C Confiauration The RSS controls, indications, alarms, etc., should be in the same physical location as in the reference plant. They should be replicated in sufficient detail to enable the desired operator function (s) to be successfully dLnonstrated.

Photographic images of non-functional control board components are acceptable. Operator control input / output devices, such as <

handswitches and recorders, should be similar in operation but need not be identical in manufacturer or model number to those installed in the reference plant. Deviations will be evaluated for impact on performance of ooerator tasks in accordance with the Section 3 methodology.

Ambient Ooeratina Environment The RSS may not replicate the reference plant control room ambient operating environment. Deviations will be evaluated for impact on performance of operator tasks in accordance with the Section 3 methodology.

4.3.2 Procedures As previously noted, the types of procedures exercised on a PSS may include:

+

i L

Normal Operating Procedures Abnormal Operating Procedures Emergency Operating Procedures, and

. Emergency Plan Implementing Procedures.

The scope of procedures to be used on the RSS will be determined by the SFTA process described in Section 3. For each procedure performed on the RSS, the Procedure Performance Time will be matched as closely as possible. The remainder of the generic Procedures-(Section 2.2) criteria applies to the RSS.

4.3.3 Steadv State and Transient Models SCDRit The output of the RSS should approximate expected reference plant responses. For the system (s) modeled, the majority of the operator cues required for the use of the procedures identified by the process described in 4.3.2 shall be modeled and displayed, t N

Fidelity The RSS computed values for steady state operation with the reference plant control system configuration (for those systems modeled) shall be stable and not vary more than 12% of the initial values over a 60-minute period for critical parameters or more than 110% of the initial values for non-critical parameters. Transient changes in displayed parameters shall not violate any physical laws of nature and shall be in the same direction as operating data or a best estimate analysis of the reference plant. Expected relationships between parameters should occur in a manner consistent with expected reference plant response. For those alarms modeled and displayed, the RSS shall not fail to cause an alarm or automatic action that would have been actuated in the reference plant, or cause an alarm or automatic action that would not actuate in the reference plant. The RSS accuracies shall be related to full power values and interim power levels for which valid reference plant 4

information is available, lima The RSS should approximate real time.

4.3.4 Performance Testino SCDRE The RSS performance testing should be limited to those procedures identified by the process described in 4.3.2. The performance testing should include validation of the pen and ink changes made to any modified procedures.

Methodoloav Those procedures identified by the process described in 4.3.2 should be performed on the RSS after all identified software modifications have been incorporated. An initial set of data shall be collected

and evaluated. After review and approval by the Multidisciplinary Review Team, these transient results become the baseline data set for subsequent performance testing evaluation.

Acceotance Criteria Determination of acceptability of performance test results shall be performed by the Multidisciplinary Review Team.

4.3.5 Ooeratina Test Methodology The RSS permits evaluation in a control room team environment with a major portion of the control boards replicated. Therefore, most procedures can be evaluated even if some components are not software modeled. The operating test on the RSS shall be limited to the procedures identified by the process described in 4.3.2.

Examiners will be able to select various RSS initial conditions and plant malfunctions to evaluate operators' or candidates' responses.

The RSS should have the capability to stop and restart the simulation, as necessary, at any point.

. The systems that are not modeled in the RSS may require evaluation of operating license candidates on other simulation devices (e.g. reference plant).

The remainder of the generic Operating Test Methodology (Section 2.5) applies to the RSS.

m

4.4 PART TASK SIMULATOR Part Task Simulator (PTS) is a simulation device incorporating detailed modeling of a limited number of specific reference plant components or subsystems. Such a device demonstrates expected response of those components or subsystems. The following criteria applies to the use of a PTS as a simulation device.

4.4.1 Human Factors Control Room and Panel Layout As necessary, layout of the PTS should be similar to the portion of the panel or system being simulated. The PTS should be the same physical size as the reference plant although reduced scale reproductions are acceptable provided the SFTA determines that the reduction does not significantly detract from the operating test.

Deviations will be evaluated for impact on performance of operator tasks in accordance with the Section 3 methodology.

The PTS should replicate the various reference plant surface enhancements, such as the use of color and mimics. The degree of correspondence with the reference plant should be consistent with the capability to replicate the existing surface enhancement with a given simulation device construction technique.

. . _ . . . _ . . _ . ..............J

1 l

JLC_Confiauration The PT3 controls, indications, alarms, etc., should be in the same physical location as in the reference plant. They should be replicated in sufficient detail to enable the desired operator function (s) to be successfully demonstrated. If the switch, control, indicator, alarm, or recorder is not required for the scope of the PTS, it does not need to be included on the simulator.

Deviations will be evaluated for impact on performance of operator tasks in accordance with the Section 3 methodology.

Ambient Ooeratina Environment A PTS may not replicate the ambient operating environment of the reference plant control room. Deviations will be evaluated for impact on performance of operator tasks in accordance with the Section 3 methodology.

4.4.2 Procedures As previously noted, the types of procedures exercised on a PTS may include:

Normal Operating Procedures Abnormal Operating Procedures Emergency Operating Procedures, and Emergency Plan Implementing Procedures.

l

The scope of procedures to be used on the PTS will be determined by the SFTA process described in Section 3. For each procedure performed on the PTS, the Procedure PerforJance Time will be matched as closely as possible. The remainder of the generic Procedures (Section 2.2) criteria applies to the PTS.

4.4.3 Steady State and Transient Models SCDAft The output of the PTS should approximate expected reference plant responses. For the system (s) modeled, the operator cues (responses) required for the use of the procedures identified in 4.4.2 should be displayed.

Fidelity The PTS computed values for steady state operation with the reference plant control system configuration (for those systems modeled) sh11 be stable and not vary more than 12% of the initial i

values over a 60-minute period for critical values or more than 10%

of the initial values for non-critical values. Transient changes in displayed parameters shall not violate any physical laws of nature

< and shall be in the same direction as operating data or a best estimate analysis of the refe:ence plant. Expected relationships between parameters should occur in a manner consistent with expected plant response. For those alarm, modeled and displayed, the PTS shall not fail to cause an alarm or automatic action that would have I

i

{ ~

been actuated in the reference plant, or cause an alarm or automatic action that would not actuate in the reference plant. The PTS accuracies shall be related to full power values and interim power levels for which valid reference plant information is available.

Ilma The PTS should approximate real time.

4.4.4 Performance Testina Scope The PTS performance testing should be limited to those procedures identified by the process described in 4.4.2.

Methodoloav Those procedures identified by the process described in 3.4.2 should be performed on the PTS after all identified software modifications have been incorporated. An initial set of data shall be collected and evaluated. After review and approval by the Multidisciplinary Review Team, these transient results become the baseline data set for subsequent performance testing evaluation.

Acceotance Criteria Determination of acceptability of performance test results shall be performed by the Multidisciplinary Review Team.

b 4.4.5 Ooerating Test MetFadslagy-Operating Tests shall be limited to those normal operator tasks and responses to cues that can be acco.mplished on the PTS. The instructor or examiner should have the ability to stop and restart the simulation, as necessary, at.any point.

The remainder of the generic Operating Test Methodology (Section 2.5) applies to the PTS.

F I

i l

e

(

1 F

I b

4.5 CRT SIMULATORS A CRT Simulator is a simulation device that is computer based and CRT displayed. The information presented is a model of the reference plant operating behavior. The Input / Output may be limited to a computer keyboard and CRT, and the parameter set limited to a speci fic . scope. The following criteria apply to the use of a CRT Simulator as a simulation device.

i 4.5.1 Human Factors The human factors that can be properly addressed in using a CRT Simulator is essentially limited to the scope of information cues that are presented. Due to the keyboard e.nd CRT I/O methodology the control panel layout, 1 & C ccnfiguration or ambient operating environment cannot be accommodated in a CRT fimulator. Deviations will be evaluated for impact on the performance of operator tasks in accordance with the Section 3 methodology.

4.5.2 Procedure.1 As previously noted, the types of procedures exercised on a CRT Simulator may include:

Normal Goerating Procedures Abnormal Operating Procedures Emergency Operating Procedures, and Emergency Plan Implementing Procedures.

The scope of procedures to be used on the CRT Simulator will be determined by the SFTA process described in Section 3. For each procedure performed on the CRT Simulator, the Procedure Performance Time will be matched as closely as possible. The remainder of the .

gentric Procedures (Section 2.2) criteria applies to the CRT Simulator.

4.5.3 Steadv State and Transient Models Scope The output of the CRT Simulator should approximate expected reference plant responses for those systems modeled. The system

! modeled and the displayed responses should be based on those cues required to use those precedures identified by the process described in 4.5.2.

Eldelity The level of sophistication for CRT Simulator models should assure rJNiacy of output information. The CRT Simulator computed values for steady state full power operation with the reference plant control system configuration (for those s36tems modeled) shall be stable and not vary more than 12% of the initial values over a 60-minute period for critical values or more than 10% of the initial values for non-critical values. Transient changes in '

displayed parameters shall not violate any physical laws of nature s

n and shall be in the same direction as operating data or a best estimate analysis of the reference plant. Expected relationships between parameters should occur in a manner consistent with expected plant response.

l Iin CRT simulhtor responses should approxiraate real time. Any inadequacies in the models can be compensated by the support of pen and ink procedure changes, Controller use, or other methods, provided that the use of t4ase methods does not detract from the examination.

4.5.4 Performance Testina  !

SCDM The CRT Simulator performance testing should be limited to those procedures identified by the process described in 4.5.2.

l .

Methodoloav Those procedures identified by the process described in 4.5.2 should be performed on the CRT Simulator after all identified software l modifications have been incorporated. An initial set of data shall be collected and evaluated. After review and approval by the Multidisciplinary Revies Team, these transient results become the baseline data set for su'asequent performance testing tvaluation, i

I

Acceptance Criterig Determination of acceptability of performance test results shall be performed by +he Multidisciplinary Review Team.

4.5.5 Ooeratina Test Methodology Examiners will be able to select various initial conditions and plant malfunctions to evaluate operators or candidates. The CRT Simulator should have the capability to stop and restart the simulation, as necessary, at any point.

The CRT Simulator can be used in conjunction with a Control Room Hock-up, a Part Task Simulator, or the Reference Plant. The use of the CRT Simulator is both enhanced by and enhances the use of these other simulation devices in the simulation facility. The use of these other devices in conjunction with the CRT Simulator should reduce the amount of pen and ink prc,cedure changes and controller use.

The remainder of the generic Operating Test Methodology l (Section 2.5) applies to the CRT Simulator.

4.6 REFERENCE PLANT The Reference Plant, as a simulatica device, is the Control Room of the specific nuclear power plart which serves as all or part of a simulation facility. The following criteria apply to the use of the Reference Plant as a simulation device.

4.6.1 Human Factors When using the Reference Plant, equipment layout, instrument and control configuration, cue scope and environment is exact. Human factors need only be addressed as applied to the plant operating conditions for the evolution to be examined.

4.6.2 Procedures i

Use of the Reference Plant as a simulation device allows usage of all of the procedures. No modifications to these procedures are required. To the extent consistent with existing plant conditions, j the operaHng test may address any or all of these procedures to demonstrate familiarity with the plant. Operator tasks which cannot be actually performed should be accomplished through verbal discussion and a walkthrough of the evolution being examined. The controls and indication needed to perform the evolution should be physically shown to the examiner and accompanied by a description of what occurs when that control is manipulated. Deviations, such as i

the above discussed static operating environment versus a desired dynamic operating environment, will be evaluated for impact on the performance of operator tasks in accordance with the Section 3 methodology.

4.6.3 Steady State and Transient Analysis Models When using the Reference Plant as a simulation device, steady state and transient models are only applicable to the extent necessary to assure that the examiners and controllers possess appropriate information regarding expected plant behavior. This information should be based upon plant operating data or best estimate data.

4.6.4 Performance Testina Performance testing is not applicable when the Reference Plant is utilized as the simulation device.

4.6.5 Onntjqg_Ict Methodology Operating tests conducted on a Reference Plant may consist of walkthroughs and/or operation of selected plant evolutions, for each procedure to be tested. Task performance that cannot actually be  !

performed should be discussed and may require extensive use of controllers and/or cue cards to provide operational cues. The remainder of the generic Operating Test Methodology (Section 2.5) applies to the Reference Plant.

N

5.0 CONCLUSION

S, OBSERVATIONS, AND RECOMMENDATIONS o Operating tests can be performed on simulation devices other than plant referenced simulators, o Use of the simulation devices described herein enable the evaluation of the generic skills and knowledge necessary to fulfill the responsibilities of a reactor or senior reactor operator.

o Those specific skills and knowledge that cannot be evaluated on other simulation devices can be evaluated in reference plant walkthroughs.

o The development of a simulation facility should consist of a systematic evaluation, such as the method described herein, of the operating test requirements and be responsive to those needs.

o It is preferred that simulation devices present the information in an active man / machine interface.

o It is recommended that the NRC approve the USFG methodology and use it to evaluate the adequacy of the resultant Simulation Facilities.

d

-- , , - - a

6.0 REFERENCES

NUREG-1258, "Evaluation Procedure for Simulation Facilities Certified Under 10 CFR 55." Draft Report, March 1987 NUREG-1021, "Operator Licensing Examiner Standards," Revision September 1986 ANS-3.5-1985, "American National Standard for Nuclear Power Plant Simulators for Use in Operator Training."

Regulatory Guide 1.149, "Nuclear Power Planc Simulation Facilities for Use in Operator License Examinations," Revision 1. April 1987 "Behavioral Analysis and Measurement Methods," D. Meister, Miley and i

Sons, 1985 l

8797F r

E I

e