ML20087G001

Forwards Addl Info Requested Per G Grant Transmitting Insp Repts 50-454/95-06 & 50-455/95-06

ML20087G001
Person / Time
Site:	Byron
Issue date:	08/11/1995
From:	Graesser K COMMONWEALTH EDISON CO.
To:	NRC OFFICE OF INFORMATION RESOURCES MANAGEMENT (IRM)
References
BYRON-95-0281, BYRON-95-281, NUDOCS 9508160173
Download: ML20087G001 (21)

Text

e Osmmonwcalth li1iwn Company flyrrm Generating 5,tation

<i150 North (.crman Churc h Road Ilyron,11. 6101n9791

'101 H15 2 54 5441 -

)

August 11, 1995 l

LTR:

BYRON 95-0281 FILE:

1.10.0101 United States Nuclear Regulatory Commission Washington, DC 20555 ATTENTION:

Document Control Desk

SUBJECT:

Byron Station Nuclear Power Station Units 1 and 2 Response to Request for Additional Information Inspection Report 50-454/95006; 50-455/95006 NRC Docket Numbers 50-454; 50-455

REFERENCES:

G. Grant letter to K. Graesser dated July 14, 1995 transmitting NRC Inspection Report 50-454/95006 and 50-455/95006 Gentlemen:

This letter provides additional information requested per the G. Grant letter to K.

Graesser dated July 14, 1995 transmitting NRC Inspection Report 50-454/95006 and 50-455/95006.

Operations and Training management had been monitoring the increasing number of simulator demonstration failure experienced during the 1995 Annual Operational Examinations. At the end of the examination period (on June 22, 1995), Problem Identification Form (PIF) 454-230-95-0016, " Increasing Rate of License Operator Failure during Simulator Demonstrations," was written.

The PIF was se7.f identified by Training and Operations personnel.

This PIF trend investigation was initiated to determine if there were contributors to the increased number of License Operator failures during simulator demonstrations at Byron Station in 1995.

The investigation covers a review of data for 1993, 1994 and 1995.

The focus of this investigation w&s twofold:

1)

To review and determine if the increased number of failures, as manifested in the 1995 License Requalification Examination process, are supported by an increased number of significant in-plant human performance events.

2)

To review and determine if there are concerns / problems in the Licensed Operator Training Program that could have lead to this l

increased number of failures.

., 7... -

0 (p:\\95byttn\\9502 sis 081195) 9508160173 950811 l

I PDR ADOCK 05000454 I I O

PDR

(

A tinicu m (.vmpan)

i t

1 Byton Ltr. 95-0281 Page 2 August 11, 1995 This investigation produced two major conclusions.

The increase in 1995 simulator demonstration failures is not supported by an increase in significant human performance events by licensed operators in the plant. There was no increase in significant human performance events by licensed operators in the plant.

The results of this investigation are that there was no single root cause among the six simulator failures. The increased number of simulator failures were determined not to have an impact on plant operations. The license operator program is effective overall.

While the increase in simulator exam failures cannot be discounted, only one crew displayed performance that challenged plant safety (i.e. simulator critical task failure).

The remaining crews displayed performance that failed to meet Byron Station standards and expectations. All crews were adequately remediated and re-examined prior to resuming license duties.

The standards established for the Operator Requalification Program are f

in line with Byron Station management expectations for continuous improvement in Operator performance.

f Through the normal progression of the Training System Development (TSD) process the Licensed Operator Training Program at Byron Station continuously undergoes revisions due to a variety of reasons (industry / management standards, operator feedback, in-plant performance, etc.).

Several " changes"

[

that had evolved were found to have impacted the license operator requalification program. The root cause report contains recommendations to address these changes. As per the TSD process, improvement areas were i

previously identified to address programmatic, human performance weaknesses and observations noted during the 1995 Annual Operational Examinations and Inspection. These improvement areas were incorporated into an action plan (attached) which has been updated.

Operations and Training management continually monitor performance of the Operations Department (in Training and in the plant) to ensure that the operators are meeting expectations for continuous improvement. Operations and i

Training management will constantly incorporate revisions / improvements as required in order to ensure safe operations and continuous improvement.

)

(p:\\95byttn\\950281W81195)

Byron Lbr.

95-0281 Page 3 August 11, 1995 If you have any questions or need additional information, please contact Bob Wegner, Shift Operations Supervisor, at (815) 234-5441 ext. 2215 or Steve Pettinger, Operations Training Supervisor, at (815) 234-5441 ext. 3212.

Respectfully, t'

~

L.

r sser s.

Site Vice President Byron Nuclear Power Station KLG/RC/rp Attachment (s) cc:

H.J. Miller, NRC Regional Administrator - RIII G.F. Dick Jr.,

Byron Project Manager - NRR L.F. Miller Jr.,

Reactor Projects Chief - RIII H.

Peterson, Senior Resident Inspector - Byron D.

L.

Farrar, Nuclear Regulatory Services Manager - Downers Grove Safety Review Dept., c/o Document Control Desk, 3rd Floor - Downers Grove i

I (p195byttnT950281081195)

i

-i

u. * :

I

-J Byron Station I,1 cense Reaual Trainina Performance " Action Plan * ~

. Developed for. Upper Management 6/25/95.

U dated 8/08/95 p

'I.

. Investigate the Trend.PIF. (Trend #95-16 written 6/22/95)

J 1)

, Select Team Members from R.A., Ops, Training, PTD RCE Lead - Bill Pirnat Operating - Tim McDougal (NSO)

Training - Jeff Hamilton

l Training-/ Operating - Ray Franklin Production Training Department - Paul Digiovanna i

i (Completed 6/27/95) 2)

Finalize the Root Cause Action Plan

[ Completed 7/03/95]-

3)

Collect License Requal Training AND Operational Plant Performance Data (ie. agather and establish facts")

Suggested Start:

Review 93, 94, 95 Data JPMs, Simulator, Quiz Pass Rate (quarterly and annually)

Individual / Crew Trends Analyze scenario issues Analyze In Plant Operations Performance Perform Interviews / Surveys of operators,-instructors, etc.

t Evaluate ' type of failures'- (Training vs. Plant) f Write the PIF Report and receive approvals

[PZF Report signed by Station Manager on 8/10/95]

II.

Each SE will ' brainstorm' with their crews to collect possible causes/ reasons for training trend. This will be fed into the Root Cause investigation to be compiled with ' change. analysis'.

(Completed.

Six Operating Dept. Crews and Training Dept. Instructors submitted ' brainstorming items' on 7/06/95.)

III.

Identify ALL the NRC Concerns / Weaknesses and issue TRR's:

Examples:

Individual and crew performance should be traced ' Low Power *

[

Simulator Sets should be evaluated Appraisal and Remediation Documentation can be more complete

[

JPMs should be evaluated more frequently Ros should be reminded that Simulator sets can be stopped'in the middle for training (I-view item)

(Completed. Fif teen Training Revision Requests (TRR's) written by l

6'/30/95),

(pi\\bbeteno\\9062zz\\080995) r

m l

I Communicate 1995 NRC Requal Program Inspection Comments / Conclusions to all licensed operators.

[ Completed. All licenses received cc-mail 6/30/95)

V.

Operations Manager & Shift Operations Supervisor to introduce 3rd Quarter Requal Classes.

Discuss:

Facts and concerns from 2nd Quarter Requal Period Personal responsibility in training and on shift Being Self Critical on individual performance Being part of solution

[ Ongoing.

shift Operations Supervisor communicating expectations on first day of requal training.)

VI.

Develop and implement a system to track individual / crew performance (over a period of time). System will feedback to supervisors for Evaluations or other Actions. The Supervisors (eg. SE) should observe all their crew members.

[ Completed 7/28/95.

Computer program will track written exam and simulator performance. BTP 100-16 revision.)

VII.

Ensure a Formal Feedback Mechanism for communicating training or operating performance issues to all license holders (expanded use of Required Reading possibly).

[ Completed 7/28/9S.

TRR 95-153 written to capture student and instructor issues per BTP 100-16.]

VIII. yalidate & yerify (ie, human factor) Procedures for the following JPM concerns:

l QPTR BOS S/G PORV, Manual Operation BOP Inverter to CVT Transfer Special consideration to procedure enhancements and tagging / label improvements.

l

[ Completed 6/28/95. No problems were found procedurally.

Tagging / labeling improvements to occur.)

Yalidate & Yerify Procedures for the Simulator Concerns:

BFR H.1 Feed & Bleed Criteria Use of PZR Heaters BEP-2 SG Isolation

[J. Bowers, B.

Quigley due 8/17/95) (ps\\bbsteno\\9062ZZ\\080995)

^

an

-?

,y.

r IX.

Consider'and develop aids / enhancements for the Procedure Reader /SRO.

l These Include:

e' Method to formally check that all BEP Steps are read and performed j

Method to ensure all Procedure Continuous Action' Steps are followed e'

Method to ensure the Operator Action Summary Sheets'are reviewed during procedural usage l

-(S. Swanson, B. Quigley due 8/17/95)

I i

X.

-Each SE will perform a crew Self Assessment on the " Communications Standard, Ops Policy #400-21".

S/A to address: what works good; what needs discussion; what's not understood; etc.

Enhancements will be incorporated in a revision to'the Policy.

[ Shift Engineers, J. Heaton due 8/17/95)

XI.

Consider Requal Training _'Back to Basics'.

~

5 (eg. Read EOP's and Bases in class, present Systems l

' straightforward', evaluate simulator board time)

(M. Brown, S. Pettinger, B. Wegner due 8/17/95)

.5

-XII.

Evaluate /Re-train on generic weaknesses from 2nd Quarter:

Simulator Examples BFR H.1 (" Hot Dry S/G", Continuous Action Step, _ Feed & Bleed Criteria), BCA 2.1 RCS Temperature Stabilization-Post Faulted S/G j

e PZR Htr Control (RO responsibilities) l e

SG PORV/ Steam Dump Control relative to RCS Heatup or-e Cooldown rate (use of computer point on CRT #3) l l

[S. Fruin evaluation due 8/17/95, training completed by 12/11/95) l XIII. Discuss the Final Conclusions and Completed Actions with.the NRC.

(Completed 8/10/95. Letter, Action Plan, and PIF Report distribated to-NRC) l l

l.;

(pr\\bbsteno\\9062Z2\\080995)

J

a Comed CompOny Trend Investigation Report j

]

l i

Report Number:

Trend 95-016 (PIF 454-230-95-0016)

Evaluator:

Bill Pirnat (Root Cause Analysis Process Expert)

Ray Franklin (Shift Engineer in Training Department)

Jeff Hamilton (Operations. Training Instructor) l Paul Digiovanna (PWR Operations Training Supervisor, PTC) l Steve Pettinger (Byron Operations Training Supervisor) l Tim McDougal (Licensed NSO) i Nick Crawford (Licensed EO) j Item Date:

6/22/95 Rev Number:

00

Title:

Increasing rate of License Operator failures during simulator-demonstrations i

Executive Summary:

This PIP trend investigation was initiated to determine if there were contributors to the increased number of License Operator failures during a

simulator demonstrations at Byron Station in 1995.

The investigation I

covers a review of data for 1993, 1994 and 1995.

l 1

The focus of this investigation was twofold:

i 1).

'To review and determine if the increased number of failures, as manifested in the 1995 License Requalification Examination process, are supported by an increased number of significant in-plant human performance events.

I 2).

To review and determine if there are concerns / problems in the Licensed Operator Training Program'that could have lead to this increased number of failures.

i This investigation produced two major conclusions.

l The increase in 1995 simulator demonstration failures is not supported j

by an increase in significant human performance events by licensed operators in the plant. There was no increase in significant hn==n performance events by licensed operators in the plant.

Procedure usage was the only common _ causal factor between simulator performance and the in-plant events by licensed operators. Twenty-two percent (5 of 23) of the 1995 in-plant licensed operator events (first quarter data) and 33% (2 of 6) of the 1995 simulator demonstration failures could be attributed to procedure usage.

(p:\\regassur\\pifleritrend\\tr95416,wpfW62695-1) i

Executive Summary: (cont.)

,The licensed operator program is effective overall.

Through the normal progression of the Training System bevelopment (TSD)

Process, the Licensed Operator Training Program at' Byron Station continuously undergoes revisions due to a variety of reasons (industry / management standards, operator feedback, in-plant performance, etc.).

Several " changes".that had evolved were found to have impacted

=

the license operator requalification program.

Procedure usage was

-l impacted by two areas of change: classroom training time and

{

methodology (procedure review)

The ability to meet management l

expectations was impacted by two areas of changer simulator standards

?

(enforcement of standards) and the simulator self assessment process.

The results of this investigation are that there was no single root cause among the six simulator failures. The increased number of simulator failures were determined not to have an impact on plant operations, while the increase in simulator exam failures cannot be discounted, only one crew displayed performance that challenged plant safety (le simulator critical task failure). The remaining crews displayed performance that failed to meet Byron Station standards and expectations. The standards established for the Operator Requalification Program are in line with Byron Station management f

expectations for continuous improvement in Operator performance.

All*

j crews were adequately remediated and re-examined prior to resuming i

license duties.

i Backaround:

On June 22, 1995, Problem Identification Form (PIF) 454-230-95-0016

" Increasing rate of License Operator failures during simulator demonstrations" was written. The PIF was self identified by Training and Operations personnel as the result of a noted increase in 1995 annual simulator operational exam failures over 1994 and previous years i

Methodoloov:

In order to do a complete evaluation, a methodology or process is necessary as a "how to" guide for the individual or team performing the investigation. Commonwealth Edison Byron Station uses a Root Cause v

Process consisting of many accepted analytical techniques. This PIF

.l trend analysis was performed using various analytical techniques including document review, change analysis, barrier analysis, and l

in terviews.

i Training performance data reviewed consisted of exam results, classroom and simulator topics for 1994-1995, training methods and techniques in addition to formal written operator and management feedback. Human I

performance event PIFs for 1993-1995 and previous PIF trend reports, both for the overall Station population and specifically for licensed

[

operators, were also analyzed.

t During the change analysis over 20 changes were initially identified.

These 20 were narrowed down to 6 that had a potential impact on the i

training performance observed. These 6 were then substantiated and/or refuted through interviews and data review.

1 i

(p:\\rrgauur\\pifler\\ trend \\tr95416.wpfiO62695-2) a

~

t

+

7 i

'l C

Methodoloav: (cont.)

e In addition, 12 licensed individuals were interviewed to substantiate or

' refute the impact of these changes.

Interviewees were picked to provide

)

a cross section of instructors, managers, and operators who both passed.

and failed the exam. A standardized list of questions was developed to

)

perform the interviews (see attachment). These questions were developed i

based on brainstorm sessions from all 6 Operating Department crews, l

brainstorm sessions from Byron Training Instructors, and from data reviewed.

i Results of Investication:

The investigation covers a review of data for 1993, 1994 and 1995.

During this time frame the following failures were noted:

1. of Sim 1_of Written Test Year Demo Crews JPMs Written Exams Score Ava.

1993 0 of 24 23 of 465 0 of 93 95,5%

i 1994 1 of 24 18 of 470 5 of 94 94.5%

l i

1995 6 of 23 16 of 445 0 of 24*

95.6%*

• Year to Date--written exams not complete for 1995 The focus of this PIF Trend investication was twofold Focus one was to review and determine if the increased number of failures, as manifested in the 1995 License Requalification Examination i

process, are supported by an increased number of significant in-plant human performance events.

The investigation considered written examinations, the six simulator demonstration failures and 16 JPM failures that were experienced during

[

the 1995 requalification exams. To date, written examination and JPM

(

pass rates have been and continue to be consistently greater than 94%.

Therefore, concentration of this investigative effort was in the simulator demonstration failures.

Two causal factors for the 6 simulator failures were determined to be verbal communication errors (lack of information exchange) and work practice errors (procedure i

usage / diagnostics).

This investigation detenu 2ed that the increase in simulator failures is i

not supported by an increase in significant human performance events by licensed operators in the plant.

+

i t

(p :\\tt p uu r\\pifler\\ trend \\tr954)l 6. wpfiO62695-3) i i

.. ~

.,m_.

Results of Investication: (cont.)

Byron Station data shows that in 1993 1285 PIFs were written, in 1994

'2214 PIFs were written and in 1995 the station is at a pace to reach 3000 PIFs.

Significant in this data is the increase starting with 1994.

The increased number of PIFs reported is due to the lower threshold created in early 1994.

Based on a self assessment, an Integrated Reporting Program (IRP) awareness training was conducted. This in conjunction with frequent management communications resulted in a reduced threshold for PIF generation. This generated an increase in nonsignificant events recorded. Studies show that on average, for every 10 non-consequential events, one significant event will occur.

Since the significant (level three) events had not increased, this supports the conclusion that the increase in the non-consequential (level four) events is primarily due to the reduced reporting threshold.

Based on data reviewed, significant PIFs involving licensed operators showed a slight decrease: 20 Human Performance Events in 1994 (4 LERs in 3rd and 4th quarters combined); 7 Human Performance Events in the first 6 months of 1995 (2 LERs in 1st and 2nd quarters combined).

Causal factors were evaluated for commonality between simulator and in-plant events.

Procedure usage was the only common causal factor between simulator performance and in-plant events by licensed operators.

Of these events, it can be concluded that 22% (5 of 23) in-plant licensed operator events and 33% (2 of 6) simulator failures could be attributed to procedure usage. Both the simulator demonstrations and the plant procedure usage errors could be classified as " inattention to detail".

This concern has previously been identified in PIF Trends95-009 (Wrong Unit / Train / Component) and 95-011 (Mispositionings). JPM failures also showing a common causal factor of procedure usage are being addressed in part by the corrective actions identified for in-plant events. The most notable of these actions is to provide Questioning Attitude / Quality verification and Validation (OVV) training (Trend 95-009) and review the self check training presentation to help the operators better internalize self checking (Trend 95-011).

i Focus two was to review and determine if there are concerns / problems in j

the Licensed Jperator Training Program that could have lead to this increased number of failures.

l A thorough review of the simulator failures was conducted. The three j

shift crew and three staff crew fail'ures involved four exam scenarios.

}

The following is the breakdown of these scenarios.

Sim Exam Number Title Times used Failures BY-04 Loss of Heat Sink with 6 crews 2 shift crews subsequent Faulted S/G (FW 1 staff crew Break)

BY-05 ATWS with subsequent Faulted 4 crews 1 shift crew S/G BY-10 Dropped Rod and S/G Tube 6 crews 1 staff crew Rupture BY-29 Faulted / Ruptured S/G with 4 crews 1 staff crew failure of Steam Line Isolation (p:\\regassur\\pifler\\trerd\\rr95 Ol63pfiO62695-4)

Results of Investication (cont.)

, This review resulted in no ~ single common cause being identified among the six failures.

While the increase in simulator exam failures cannot be discounced, no major problem in the licensed operator program was found.

The licensed operator program is effective overall.

Two of the failures associated with BY-04 involved usage of Functional Restoration Procedure BFR H.1.

One of these failures concerned the amount of FW supplied to a " Hot-Dry S/G". The other failure involved the initiation of "RCS Feed and Bleed" when FW had been isolated by operator action. This action was not communicated to the SRO.

The third failure associated with BY-04 was caused by operator performance deficiencies prior to the implementation of the emergency procedures.

The failure associated with BY-05 was based on not closing all of the AF isolation valves for the Faulted S/G.

The failure associated with BY-10 was based on incorrect actions taken by the crew during performance of Abnormal Operating Procedure BOA ROD-3 in response to the Dropped Rod.

The failure associated with BY-29.was based on lack of communications regarding the actions taken for FW flow to the Faulted S/G.

Communication problems were noted in two of the six failures.

These incidents involved failure to exchange information.

They were not associated with the communications standard enhancement made in August, 1994 which involved three-legged communications.

This standard was first formally trained on in the fourth quarter'of 1994 requalification training, continually re-enforced in simulator training, and trained on in the classroom in the second quarter of 1995 using a video tape produced by the Station.

Communications training material and overall-performance was reviewed and found to be appropriate.

This investigation determined that only one failure should have been tied to a Simulator Scenario " critical Task". The evaluation process considers successful completion of all Critical Tasks in addition to several " Competencies".

The remaining five failures were based on station standards (Compen tencies).

These five failures were not tied to unsafe operations.

l l

4 (p:\\ reg assur\\pifler\\ trend \\rr954)l 6. wpfiO62695-5) i

i Results of Investicationt ( cLa t. ) -

,A review of the' simulator exam failures was performed to ensure validity of the, listed. critical tasks and to see if any new critical ~ tasks were identified.' This review identified that the critical taska' listed were valid. However, successful completion criteria was more restrictive than required to mitigate the casualty. Originally two crews were failed l

based on critical tasks..A review of these two failures indicated that

[

these crews had successfully completed the goal of the critical task, although they did not meet station standards. The crews would still have failed based on not meeting management's expectations. However, their actions were sufficient to prevent significant plant degradation.

In scenario BY-05, the critical task was to isolate all Feedwater to'the faulted Steam Generator with the intent to stop the RCS cooldown.- While j

the actions of the crew in throttling Auxiliary Feedwater flow met the intent of the critical task, the failure to close the Aux Feed 13s was a violation of the procedure which resulted in the crew failure.

The crew had throttled AF flow to a value low enough to stop an RCS cooldown which alleviated the safety significance of the event, therefore the i

failure should not have been documented as tied to a critical task.

In scenario BY-29, the critical task was to throttle flow to all S/G's to 25 gpm.

This is to reduce RCS cooldown and maintain S/G tubes wetted for subsequent cooldown once S/G isolation can be completed.

The crew throttled all S/G's to 25 gpm except the faulted S/G which was throttled to O gpm. This is acceptable since the faulted S/G would never be used I

in subsequent cooldown once isolation was completed. The SRO was not informed that flow was throttled to o gpm on the faulted steam generator, resulting in crew failure based on lack of communications.

The crew throttled flow as required to the remaining intact steam generators, stopping the RCS cooldown which alleviated the safety significance of the event. Therefore, the failure should not have been i

tied to a critical task.

In scenario BY-04, one "new" critical task was identified based on the actions taken by one of the crews. A crew member failed to inform the SRO that Feedwater had been manually isolated.

The crew proceeded to initiate "RCS Feed and Bleed" without the benefit of this knowledge.

The new critical task was to reestablish FW flow prior to initiating

" Feed and Bleed".

The initiation of " Feed and Bleed" led to a reduction in the margin of safety other than that introduced by the original scenario.

L Based upon these critical tasks discrepancies, it is recommended that

{

training adopt a formal method of evaluating critical tasks before and after exams. (Recommendation 5)

Procedure usage was impacted by two areas of change: classroom training

\\

time and methodology.

The ability to meet management expectations was j

impacted by two areas of change:

standards (enforcement of' standards) and the self assessment process.

(p:\\regassur\\pifler\\ trend \\t:95416.wpf\\062695-6)

Results of Investication (cont.)

.The change analysis identified six changes to the' training program, I

confirmed through interviews and data review, as having potential impact on the simulator demonstration results. These changes were:

Amount of Training Time Training Methodology / Content Changing Standards (enforcement of standards)

Training Scenario Level of Difficulty Simulator Post Senario Self Assessments.

Instructor Personnel Turnover The procedure training process was changed from formal classroom lectures to a self study method in 1992 due to operator feedback. Based on interviews this current method of procedure review was determined to be less effective than other training methods (such as formal lectures).

Operators do not routinely use background documents or other references to assist in their review of procedures. Also, there is a disconnect between Operations Management expectations and the shift personnel in regards to performing procedure reviews on shift. (Recommendations 1 and 2b)

The following is a breakdown of the training hours for the operating requalification program.

Setting / Year 1993 1994 1995 (projected)

Classroom hours 114 122 104 Simulator hours 60 60 70 Self Study hours 130 82 66 Annual Exams hours 16 16 16 TOTAL HOURS 320 280 256 The total training time has been reduced over the last three years in an effort to improve training and operations efficiency.

It was noted in 1993 that classroom self study time was not fully utilized.

Therefore, self study time and total training hours were reduced from 1993 to 1994 The slight reduction in hours from 1994 to 1995 was made to accommodate the operating department 12 hour1.388889e-4 days <br />0.00333 hours <br />1.984127e-5 weeks <br />4.566e-6 months <br /> schedule. Annual NGET and Leadership III training attended by all operators was not included as part of the training time summarized in the preceding table. Although the licensed material contact time has not substantially changed, the classroom self study time provided during the continuing training program has decreased.

(Recommendation 2a) t 1

(p:\\ reg assur\\pifie ritterw!\\tr954)l 6. wpf\\062695-7)

i Results of Investication: (cont.)

, Operator perception is that training time is reduced by the presentation I

of non-technical training that does not directly prepare them to pass their exam.

Crew feedback identified that the topics presented which j

helped them to pass their annual exam were the best use of allotted training time.

Interviews with management indicated that the purpose of the Licensed Operator Requalification Program is to develop a well rounded fully qualified operator. This resulted in a recommendation that the goals of the licensed operator training program be reiterated to the operators. (Recommendation 2c)

The current process of post scenario self assessment in the simulator needs improvement.

This process for simulator critiques was established in the fall of 1994 and involves the crew (lead by the SRO) evaluating their own performance on the simulator under the facilitation of an instructor. The operator feedback and management observation of the process indicated that the process can be more effective in assessing all operator skills. This is due to the crew members' difficulty in i

observing performance and in being self critical, and the instructors' hesitancy to intervene in the self assessment process.

(Recommendation 4)

The perception is that standards are applied differently from ' training sets' to ' evaluation sets'.

Training critiques have changed over the past few years.

Previously instructors had critiqued the crew on every area that needed improvement. More recently, the instructors concentrate on a few major improvement areas to increase overall critique effectiveness (i.e., operator retention of lessons learned).

Evaluation critiques have always concentrated on all areas needing improvement. (Recommendation 3).

In 1994 and 1995, the evaluation standards were more critically applied.

Previously, the operators would fail if their performance led to a missed critical task. Most recently, a higher emphasis is being placed on satisfactory performance of crew and individual competencies.

The two areas that were evaluated and found to have no impact on the failures experienced ware training turnover (personnel) and training scenario level of difficulty. Training staff turnover was generally viewed as a positive change.

The difficulty of training scenarios has not appreciably changed as evidenced by the review of materials and through interviews.

(p:\\regassur\\pifier\\ trend \\tr954)l6.wpfV)62695-8) t

Pecommendations:

1).

The operations and training departments should evaluate methods to improve procedure training. Formal procedure training should be given more frequently.

2).

The operations and training departments should evaluate methods to s

improve the efficiency of training time.

a).

Increase the effectiveness of classroom self study time, b).

License holders should take advantage of opportunities for Self Study.

c).

The operations and training departments should communicate to the operators that the purpose of the Licensed Operator Requalification Program is to develop a well rounded, fully qualified operator.

3).

The purpose and goals of simulator Training set critiques versus Evaluation set critiques should be communicated to operators.

4).

The simulator post scenario Self Assessment process should be evaluated to increase its effectiveness.

a).

The crews, with the instructor's facilitative assistance, need to ensure that performance is critically assessed as it relates to operating standards, i

b).

To allow the SRO to function in an oversight capacity and evaluate the performance of the crew more effectively, I

evaluate the implementation of 5 person crews for simulator training. This will allow human performance standards (procedure usage, communications, self checking, independent

)

verification, questioning attitude) to be further evaluated and re-enforced by on-shift personnel.

I 5).

Training should adopt a method to perform a formal review of simulator Critical Tasks before and after the exams to ensure the validity of the existing tasks and whether any new critical tasks were created based upon scenario outcome.

Previous Trends and Industry Events:

A computer search using the " STAIRS" program database revealed no events of the same nature or corresponding data. The database search was performed using the key words: operator and failures, licensed and simulator and licensed operator failures to establish applicable previous documents.

Based on the above, no documents were determined to be applicable to this trend.

(p:\\regassur\\pifierittend\\tr95416.wpfiO62695-9)

Comed Company Trend Investigation Report Report Number:

Trend 95-0016 (PIF 454-230-95-0016)

Evaluator:

Bill Pirnat (Root Cause Analysis Process Expert)

Ray Franklin (Shift Engineer in Training Department)

Jeff Hamilton (Operations Training Instructor)

Paul Digiovanna (PWR Operations Training Supervisor (PTC))

Steve Pettinger (Byron Operations Training Supervisor)

Tim idcDougal (Licensed NSO)

Nick Crawf ord (licensed EO)

Item Date:

06/22/95 Rev Number:

00

Title:

Increasing rate of License Operator failures during simulator demonstrations On-Site Review:

OSR Disciplines Required:

b (~

~VQM M7/95 SED DATS REVIEWER DISCIPLINES DATE RA

/6

[b SED ASE[T 7 hI OP

$A/AJ ABS DYsOS Other 8-2 ~77 w

g2F J2r Station Manager:

SN/D![

{

)

(p:\\ reg assur\\pifie r\\ trend \\tr954)l 6.wpf\\062695 10)

SUPERVISORY / MANAGEMENT OUESTIONS What do you feel the licensed training program's biggest i

asset / strength is?

liability / weakness?

-)

Give an example of the 1 or 2 things in the license training program that you would change!

Do you feel that the instructor turnover has affected the quality of licensed operator training?

l i

How much time do you spend in training sets observing crew and instructor performance?

Do you feel simulator instructors during training sets apply the standards to your expectations?

Do you feel the operators understand the purpose / benefits of

" soft skills" training?

i What information do you feel should be available for license training procedure self reviews?

1 What do you feel are the benefits and drawbacks of the Self Assessment (Sim Demo) process?

l l

Do you feel the written exams go beyond the scope of the operators job?

l L

Specifically in what areas (if any)?

i How effective is self study time now?

How effective was self study time in the past?

How would'you rate the difficulty of training sets now i

versus the last 2 or 3 years?

How would you rate the difficulty of evaluation sets now l.

versus the last 2 or 3 years?

l

[

L INSTRUCTOR OUESTIONS What do you feel the licensed training program's biggest asset / strength is?

liability / weakness?

)

Give an example of the 1 or 2 things in the license training program that you would change!

Do you feel that the instructor turnover has affected the quality of licensed operator training?

Do you feel the procedure self review is effective?

l If not why not?

1 Of the license operator training that you present what do you feel does not enhance operator knowledge?

If any.

Why?

Do you feel the operators understand the purpose / benefits of

" soft skills" training?

Do you feel the expectations (B Ws) during evaluation sets differ from your expectations during training sets?

In the course of normal (everyday) simulator training do you see a degradation in the background knowledge or general application of emergency procedures?'

Do you feel the license operator training program is meeting Bob Wegners expectations and the expectations /needs of the licensees?

What do you as an instructor perceive your role to be in the simulator self assessment process?

f1;

.- sh:..

]$(

l

• Do you' feel-that the self assessment process is successful g"

.iniidentifying.and correcting concerns / problems?-

[

Do you feel the written exams.'go beyond the scope of,the L-operators job?

Specifically in what areas.(if any)?

.'How effective is self study time now?

How effective ~was self study time in the past?

How would you rate the difficulty of training sets.now versus the last 2 or.3 years?

How would you rate.the difficulty of evaluation sets now-versus the -last 2 or 3 years?.

read /6g.mpf

c NSO/SRO OUESTIONS What do you feel the licensed training program's biggest asset / strength is?

liability / weakness?

Give an example of the 1 or 2 things in the license training program that you would change!

What information do you use when you do your procedure reviews?

Is the information readily available?

Do you feel the standards are applied differently from training scenarios to formal evaluation scenarios?

Specifically what standards do you feel are different?

(show interviewee standards and get examples)

Do you feel training which enhances human performance traits / abilities is applicable to license training?

What do you feel are the benefits and drawbacks of the Self Assessment (Sim Demo) process?

Do you feel that the written exams go beyond your job scope?

If so; Specifically in what areas?

How effective is self study time now?

How effective was self study time in the past?

How would you rate the difficulty of training sets now versus the last 2 or 3 years?

How would you rate the difficulty of evaluation sets now versus the last 2 or 3 years?

e i

O f' t h e t r a i n i n g t h'a t y o u g e t, what do you feel-does not'

• ~

i.'

- enhance operator knowledge?

i Why?--

1-I Do you feel that the' instructor turnover has affected the quality of licensed operator training?

e b

-1

+

5 l

1 1

E i

b I

e i

i

)

i 1

L.