ML20140C357: Difference between revisions

From kanterella
Jump to navigation Jump to search
(StriderTol Bot insert)
 
(StriderTol Bot change)
 
Line 1: Line 1:
{{Adams
#REDIRECT [[B16253, Provides Response to NRC Initial Operating Licensing Exam Rept 50-245/96-12,dtd 970207]]
| number = ML20140C357
| issue date = 03/03/1997
| title = Provides Response to NRC Initial Operating Licensing Exam Rept 50-245/96-12,dtd 970207
| author name = Thayer J
| author affiliation = NORTHEAST UTILITIES SERVICE CO.
| addressee name =
| addressee affiliation = NRC OFFICE OF INFORMATION RESOURCES MANAGEMENT (IRM)
| docket = 05000213, 05000245, 05000336, 05000423
| license number =
| contact person =
| document report number = B16253, NUDOCS 9704030172
| package number = ML20140C354
| document type = CORRESPONDENCE-LETTERS, INCOMING CORRESPONDENCE
| page count = 90
}}
 
=Text=
{{#Wiki_filter:---  .        -  _ - ._
    .              N::rtheast                                              107 sem saut suu 06037                  ,
MI Utilities System                                                sonheast veties serme company            i P.o. Bos 270                            f Hartford, CT 06141-0270                  I (203) 665-5000 t
t March 3,1997
                                                                                        ' Docket Nos. 50-213 J
50-245 1                                                                                                      50-336      ,
50-423 i
B16253 Attachment 1 - Exempt From Public              l
;                                                                    Disclosure Pursuant to 10CFR 2.790            :
l        U.S. Nuclear Regulatory Commission
:        ATTN: Document Control Desk                                                                                l 5
Washington, DC 20555                                                                                        I Haddam Neck Plant                                                  i Millstone Nuclear Power Station, Units 1,2 & 3                                      ;
Response to initial Operator Licensino Examination Report No. 50-245/96-12 l
The purpose of this letter is to provide a Connecticut Yankee Atomic Power Company                          l
:        (CYAPCO) and Northeast Nuclear Energy Company (NNECO) response to the NRC's                                !
l        Initial Operator Licensing Examination Report No. 50-245/96-12,W dated February 7, 1997. Specifically, during the week of December 2,1996, the NRC administered initial                      ;
examinations to seven senior reactor operator (SRO) applicants for Millstone Unit 1.
Six of the seven applicants failed the examinations. Consequently, the NRC requested i:        NNECO to submit the results of its root cause assessment of this performance during the initial examinations.                                                                                  !
{        As a result of the operator licensing examination failur$s, the Recovery Officer - Nuclear
:        Engineering and Support commissioned an Independent Review Team (IRT) to assess
!        the circumstances surrounding the examination failures. A copy of the final IRT report 4
W          G. W. Meyer letter to Bruce D. Kenyon, " Initial Operator Licensing Examination
;                    Report No. 50-245/96-12." dated February 7,1997.                                                4 9704030172 970327
-        PDR        ADOCK 05000213 V                      PDR osm2 arv. :-94 i
 
  -_~ - -    - - - . . .        . . . . - . . _ - _ . -      _ . - _ _ - .      - - - _-              - - - _
4 t
l..        U.S. Nucinar Regulatory Commission B16253\Page 2                                            _
!          is enclosed as Attachment 1. The IRT report includes an analysis of the event, root and            ;
j          contributing causes, generic implications and recommended corrective actions.                      t
;                                                                                                              t
;          in addition, the NRC Staff requested that NNECO and CYAPCO provide a summary of                    .
j          the investigations conducted to date regarding the inconsistencies in licensed operator            !
!          training records. The summaries for the Haddam Neck Plant and Millstone Units 1,2,                  ;
and 3 are enclosed as Attachment 3.                                                                  l The IRT concluc'ed that the root causes for the examination failures were: 1) a lack of
.          manageme.nt oversight and accountability by management, supervisors, and staff; and                !
j        . 2) the correct lve action program was ineffective in resolving previously identified, long j          standing issues. The IRT report also identifies several other contributing causes. To
:          address the root causes and the various contributing causes, the IRT recommended a                ;
;          number of corrective actions. The complete corrective action plan, including a schedule j          for addressing each of the identified weaknesses, will be approved by CYAPCO and                    l l          NNECO management and submitted to the NRC staff by April 2,1997. As part of the corrective action plan, roles and responsibilities for completing NRC Form 398 will be clearly defined.
l The IRT focused its assessment on the Millstone Unit 1 Operat5r Training Program; however, preliminary evidence suggested the potential for similar problems to exist in the training programs for Northeast Utilities' (NU's) other Connecticut nuclear units.
Therefore, a review of the other units' training programs was initiated.
The deficiencies that contributed to the Millstone Unit i examination failures are identified in the IRT Report. The Nuclear Training Department (NTD) had previously conducted a less extensive review that found, among other things, that several major historical events associated with operator training occurred at times when staffing of the affected Operator Training group had fallen to unacceptable levels. This pattern seemed to hold true for the three Requalification Program failures in NU's past, as well as several lesser events. Inadequate staffing for the existing workload and insufficient experience in conducting operator training programs were significant factors in the Millstone Unit 1 examination failures.
While several of the contributing causes reported by the IRT would appear to be common among two or more of the Connecticut nuclear units, staffing problems went unmitigated only in Millstone Unit 1 Operator Training. Aggressive efforts to achieve a consistent level of line ownership and oversight, and to restore the Operator Training Branch to full staffing levels and to add experienced operator trainers in the process, are currently underway.
                                                                                            .~
 
  ,          U.S. Nucl:ar Regulatory Commission                                                              ,
B16253\Page 3                                                                                    ,
The IRT assessment included a review of the NTD records for each license applicant              !
involved in the NRC examination that occurred during the week of December 2,1996.                l To apply for a reactor operator (RO) or SRO license, the licensee must submit an NRC            '
Form 398, " Personal Qualification Statement - Licensee," for each prospective license          i candidate. The form certifies that the candidate has successfully completed the facility licensee's requirements to be licensed as an Operator / Senior Operator pursuant to 10          i CFR Part 55. Contrary to this, the audit results to date have revealed that the NRC              i Forms 398 were submitted to the NRC based on inaccurate information provided to those signing the forms.
}                                                                                                              !
i              Specifically, the IRT review ideritified several instances where the required facility          l
]      ' training for SRO license candidates had not Leen completed. The requirements to j            complete the Licensed Operator initial Training (LOIT) class included, in part, a set of          i j            on-the-job training (OJT) cards, five reactivity manipulations, and a specified amount of        l
'              hours of under-instruction watch standing. Once these deficiencies were identified at            i Millstone Unit 1, the NTD established a team to investigate the matter further at                l i              Millstone Unit 1, the other Millstone units and the Haddam Neck Plant. The initial              ,
j              review scope was Millstone Unit 1 1996 Licensed Operator Upgrade Training (LOUT),                !
j              1995 LOIT; Millstone Unit 21996' LOIT and Millstone Unit 21996 LOUT; Millstone                  ,
j              Unit 31995 LOIT; and CYAPCO 1996 LOIT/ LOUT. As part of the review process, when a discrepancy is discovered, line management will review the deficiency and                  i determine the necessary immediate corrective action (s), as appropriate, for the affected        l l              RO or SRO license.                                                          "
The results of the initial reviews identified problems in the programs selected. A                l decision was made to expand the reviews to additional classes on all the units. The scope and results of the initial reviews will be submitted to the NRC by March 15,1997.
The final results of the expanded reviews will be submitted by April 2,1997.
Attachment 3 provides a summary of the reviews conducted to date.
Additionally, a review of the NRC Form 396 review process will be condgcted, to ensure          ,
completeness and accuracy. The scope of the review and its expected completion date              '
will be included in our corrective action pim which will be submitted to the NRC by_          ,
:              April 2,1997.                                - - -
As a final matter, the IRT report enclosed as Attachment 1 contains personnel                  ,
information, the public disclosure of which would constitute a clearly unwarranted -          :
invasion of personal privacy. Accordingly, pursuant to 10 CFR 2.790(a)(6), NNECO              li requests that Attachment 1 be withheld from public disclosure in its entirety. An affidavit to support NNECO's request is attached. NNECO has prepared a redacted                  i version of the final report, which is enclosed as Attachment 2. Consistent with 10CFR2.790(a)(6), the redactions from Attachment 2 have been made to protect
                ,r  ---,e ,, ,,:,                e
 
O i
    . U.S. Nuclear Regulatory Commission                                          ._ -
B16253\Page 4 personal privacy,.and
                      ~-    ~~ '
consist primarily of the deletion of names and other identifying information.                                                                                  '
l The following are CYAPCO's and NNECO's commitments made within this letter.                  !
Other statements within the letter are provided for information only.
B16253-01      Submit the complete corrective action plan, including a schedule for          !
addressing each of the identified IRT weaknesses to the NRC staff by          ;
April 2,1997. As part of the corrective action plan, clearly define roles and responsibilities for completing NRC Form 398.
B16253-02 Complete corrective actions for the Millstone Unit 1 LOIT/ LOUT program prior to restarting respective classes.
i B16253-03 Complete corrective actions for the Millstone Unit 2 LOIT program prior to restarting LOIT classes.
B16253-04 Complete corrective actions for the Millstone Unit 3 LOIT/ LOUT program            1 prior to NRC examinations of the current LOIT/ LOUT class.                    ]
l 816253-05 Conduct a thorough review of the Millstone Unit 3 LOIT program against the accredited program requirements prior to submittal of the license applications to the NRC.
B16253-06      Forward the scope of NRC Form 396 process review and its expected completion date for Millstone Units 1,2 and 3 and the Haddam Neck Plant by April 2,1997.
B16253-07 Submit the results of HNP cata reviews of LOIT/ LOUT findings to the
_ NRC by April 2,1997.
B16253-08 Submit the results of initial reviews of additional classes on all the units will be submitted to the NRC by March 15,1997.
B16253-09 Complete specific reviews of the HNP LORT program by April 4,1997.
Please contact Mr. R. T. Laudenat at (860) 444-5248 should you have any questions regarding this matter.
4 M
 
                                                  - __. _ . _ _      _ . _ _ . . _ _ _ . _ , _ . - ~
:.                                                                                                        i i
  . U.S. Nuclear Regulatory Commission B16253\Page 5                                                                                        !
l
.                                Very truly yours,                                                        '
I                                                                                                          l
!j                              CONNECTICUT YANKEE ATOMIC POWER COMPANY                                    :
l                                NORTHEAST NUCLEAR ENERGY COMPANY i
l
                                      .    .            A 1
YK. Thayer, Recovery Officer Nuclear Engineering and Support Services attachments (3)                                                                                        l J
l cc:    H. J. Miller Ragion i Administrator
__ _ G.W. Meyer, Chie.f of Operator Licensing, NRC Region i F. J. Miraglia, Acting Director, NRR W.D. Travers, PhD., Director, Special Projects Office W.D. Lanning, Deputy Director, inspections, Special Projects Office P. F. McKee, Deputy Director, Licensing and Oversight, Special Projects Office J. P. Durr, Branch Chief Inspections, Special Projects Office M.B. Fairtile, NRC Project Manager, Haddam Neck Plant W.J. Raymond, Senior Resident inspector, Haddam Neck Plant                                  .
S. Dembek, NRC Project Manager, Millstone Unit 1 T.A. Easlick, Senior Resident inspector, Millstone Unit 1 D.G. Mcdonald, Jr., NRC Senior Project Manager, Millstone Unit 2 D.P. Beaulieu, Senior Resident inspector, Millstone Unit 2 J.W.- Andersen, NRC Project Manager, Millstone Unit 3 A.C. Cerne, Senior Resident inspector, Millstone Unit 3 MM Y
 
                        .~_        _ _ -      . . _ _ . _ _ . _ -              . _.  .-.  --  .
, . U.S. Nucl2ar Regulatory Commission                                                        2 B16253\Page 6 AFFIDAVIT I, Jay K. Thayer, being duly sworn, depose and state as follows:
: 1.      I am currently the Recovery Officer - Nuclear Engineering and Support with Northeast Nuclear Energy Company (NNECO). I am filing this affidavit in support of NNECO's request that Attachment 1 to the response to .NRC Inspection Report No. 50-245/96-12 be withheld from public disclosure pursuant to 10 CFR S 2.790(a)(6).
l      2.      The information contained within Attachment 1 contains personnel information,
;              the public disclosure of which would constitute a clearly unwarranted invasion of personal privacy. The information identifies individuals who were involved with the matters being investigated. In NNECO's view, public disclosure of the information would invade the personal privacy of these individuals.
l        3. The personal privacy information contained in NNECO's response, to the best of my knowledge and belief, has not been disclosed to the public.
j                          . . . .
The above three paragraphs are true and accurate to the best of my knowledge, information, and belief.
Executed this8 rd day of March,1997.
C
                                                                @y K' Thayer Subscribed and swom to before me this  3 ""    day of      **                ,1997
                      .V                                                              ..
Date Commission Expires:          b 2d> 5"e'                                      ,
i
    - . .                                                                                              ,  i l
                                                                                                            )
 
t Docket No. 50-213
.                                                                                        50-245 50-336 50-423 B16253 f
I i
T
;i Attachment 2 Haddam Neck Plant Millstone Nuclear Power Station, Unit Nos.1,2, and 3 5                                          Redacted Version 4
i l
l                                                                                                1 i'
l l
i I
:                                                                                                l I
I I
i i
March 1997
                                                  ~
y' h    j    [                                            -
 
          #"          - Northeast l        ///g\Y Utilities System
:                                    Independent Review Team Report j                        1996 MP -1 LOUT NRC Examination Failures 1                                                                -.
!                                                      PREPARED BY:
~
1c/n2HHTeam          AnMember                      1/nl L. M. Dooley 4      .
W. H. Barron                                                              @eam Member Mb                v                                                          \
S. R. Frye              Team Member.                  T. P. GilbeYt    Mm Member hbb h C. W. Sawyer nw Ieam Member                                l L> c-                      0 '*/91 K. D. Mu(cino        Team Lead                Date REVIEWED BY:
L .I t a _ /e _ - o/.4
                                            .Ferguson              /                Date ACCEPTED BY:
aMA                  9//Sk'7 3 9 .yer                e                      d.te
 
EXECUTIVE
 
==SUMMARY==
Backeround Early in 1996, Unit 1 Operations management requested a Licensed Operator Upgrade Training (LOUT) program be developed and presented. The original candidates for this program were to be NRC licensed reactor operators (ROs), who were to receive the additional training required to attain NRC senior reactor operator (SRO) licenses. -
The Nuclear Trauung Department (NTD) was requested to deliver this program startmg in June 1996, with a projected NRC exam date in the lauer part of the fourth quarter 1996.
Using these criteria, (candidate background and available time) the Unit 1 Operator Trauung Branch (OTB),
revised the accredited Licensed Operatorinitial Trauung (LOIT) program to accommodate these parameters. This is the first time a speciSc program for " upgrade" candidates was used at Unit 1.
In March _1996, Unit 1 Operations management informed NTD that in addition to the RO candidates previously mentioned, other candidates would be included in the program as well. These candidates were unlicensed and would pursue SRO status as " Instant SROs"          Individual trammg plans were developed for these candidates, however no significant modification to the LOUT trammg program was made to adcommodate the background and skills of these individuals.
The trammg program began in June 1996 and continued through the NRC exam in December 1996. From the start of the program design through its completion, every individual with programmatic responsibilities was either removed or replaced due tojob transfers or promotion. During the first week of the program one student withdrew.
After the company final exam was given in November 1996, another candidate refused remediation and withdrew from the program.
From November 19 through November 21, 1996, the Unit 1 Operations Trammg Branch (OTB) and the line orgmntion conducted a review and validation of the NRC exam. As a result of the initial exam review approximately 70% of the exam was rewritten. On November 22, the exammation was revalidated by two Unit 1          I SROs.
On December 2,1996, the mdih took the written portion of the exam At the three hour point, the NRC exammer authorized an additional 30 minutes to the original four hour time limit due to many candidates taking longer than anticipated. Later in the day, Unit 1 OTB conducted a prelinunary scoring of the written exam and estimated that only one candidate out of seven passed. The candidates were informed of these results.
On December 6,1996, the candidates completed the operations portion of the exam.
At the NRC exit meeting on Dec. 6,1996, questions were raised regardmg the competency of the licensed RO to SRO candidates to perform certain RO tasks.
The results of this exam are as follows (see Attachment VIH):
* One candidate passed both the wTitten and operations examinations e    One candidate passed the wTitten exam only
          . One candidate passed the operations exam only Both the operations and written exams must be passed to obtain an SRO license. Only one candidate out of seven successfullypassed the exam.
      'Ihis Independent Review Team (IRT) was established to investigate the circumstances surrounding these exam failures.
 
.                                                                                                                                  i l
EXECUTIVE
 
==SUMMARY==
(cont.)
Root Cause                                                                                                                        ;
A lack of management oversight and accountability by management, supervision, and staff coupled with the ineffectiveness of the corrective action program to remedy previously identified, long standing issues led to the degradation of training program quality.
Additional aspects of the root cause are discussed below.
Accountability and Responsibility Operations and Trauung management both missed numerous indications and warnmgs concemmg the progress and quality of the LOUT program This information was available prior to and durmg the conduct of the program This event could have been prevented had appropriate actions been taken to resolve long enAng issues. Trammg staff and supervision contributed to this event by ineffectively applying procedures and accreditation criteria in the design and conduct of the program Breakdowns in the Levels of Defense in Ouality                                              _
Failures in the first three levels of the defense of quality were observed. Some ' procedures were not followed or were used ineffectively. Supervision did not provide adequate guidance to the mstructors and failed to act on observed problems. Management failed to provide in-depth oversight and follow through. The oversight organization (QA) failed to identify some of the precursor conditions during audits and surveillances. This event would have been prevented or mitigated had these lower levels of defense been effective. This event was stopped by the NRC exam, the 4th and last barrier in the defense ofquality.
Ineffective Corrective Action Prorram While some aspects of this event were umque due to this bemg the first LOUT program for MP-1, many of the basic causal factors contributing'to the unsatisfactory performance on this exammation were found to be sinular to previously identified problems' across the three Millstone units. These unresolved weaknesses in operator trammg                  j have been identified in numerous self-assessments, QA audits, INPO evaluations, and NRC mspections (see                          l Attachment I) and should have been viewed as precursors to the current MP-1 event. The corrective action process                  l has not been given priority nor used effectively to resolve previously identified weaknesses. Had these previously                I identified issues been remedied in a timely manner, the negative outcome of the LOUT class might have been                      !
prevented. Significant examples include:                                                                                        i e    A self assessment process that does not systematically resiew past trammg problems to identify the conunuing success of remedial r:tions.
* Corrective action plans not being developed for issues idennfied in previous self assessments
* Issues idenufied at one unit not being reviewed for applicability at other units.
* Exam banks did not meet industry and regulator c mJons.
* Incumbent task lists were not updated as required.
* Plant and industry events were not consistently included in trammg
* Simulator trammg was not effectively observed by line and trammg management.
    ' 1989 MP-3 LORT failure; 1991 & 1992 MP-1 LORT failures; 1992 NRC NUREG-1220 inspection; 1994 MP-1 LOIT company exam failures; 1996 MP-2 LOIT company exam failures.
Fua!DusA e il Fehnury.1997                                      a 1996 MP 1 LOUT NRC 1mmusmann Fehmus IltT                        U
 
  . _ . _ _ _.___ . . _ .                                          _ _ _ . _ _ _ . . _              ._ _ ..  .____._..._m              _ . _ _ _ _ .
}
l                        EXECUTIVE
 
==SUMMARY==
(cont.)
t
!                        Lack of Manneement Oversieht and Accountability In addition to the previously unresolved weaknesses in operator trauung, management oversight was ineffective in
:                        identifying and correctmg emerging challenges and deficiencies associated with the 1996 LOUT program
;                        Management +..as not cogmrant of and did not respond adequately to several factors that contributed to this event.
i j                              e      'Ihe magnitude of changes incorporated into the new LOUT program (e.g.,13 weeks of "on-shift"
;                                      component reduced to 4 weeks,31 systems selected for self-study with no formalinstruction).
i                              e      Deficiencies in student and program records regarding prerequisites, class =*-ad==ce, on-the-job (OJT)
!                                      trauung time, "as taught" schedules, program change documentation (Adverse Condition Report - ACR-M1-97-0117 imtiated by Operator Trauung supervision during IRT investigation).
l l
* The cumulative effect of personnel changes on the experience level of the program staff.
l
* Failure to follow procedures and the Systematic Approach to Trammg (SAT) 1                                                                                                                    --
!                                                  - In developing the LOUT program                                  -
1                                                  - To upgrade curriculum for plant design changes (PDCRs), procedure changes, etc.
4
;                                                  - To update task lists and exam banks 1
i                        Ernerience And Staffine l                        The experience level of the key individuals associated with the LOUT program played a'significant role in this i                        event. As a result of frequent personnel changes at all levels during the conduct of the program, the staff experience level was diluted.                                                                                                i l
j                        Systematic Annroach to Trainine (SAT)                                                                                        !
i A lack of utilization and understandmg of the SAT led to errors in judgment in program design and implementation.
i                        Deviations from or failure to follow established procedures contributed to a sigticcant breakdown in the SAT process.
l j                          Candidate Selection Process Due to ao Armal candidate selection process, with n= die selection criteria, some of the candidatae selected for the 1990 GT prog.w.s may have been margmally motivated to obtam an NRC license. One candidate left the program within the first week, another candidate refused remadi=+ ion after fadmg the company final exam.
Exam Review and Validation                                    -
l The process to review and validate the exam was ineffective. This was evidenced by the need to challenge 10 questions after the exam was =dminictared. Additionally, this process was not formahzed or procedurahzed It dapandad on the experience of the staff, of which three out of four reviewers had not previously reviewed an NRC exam. n NRC Written Exam The difficulty of the NRC written exam was evaluated by the IRT and an indanandant consultant (Nuclear Trammg and Educational Services, Inc. - A"=+= ant III). It was determmed that the exam was lengthy and difficult but                ;
within NRC guidelines. In light of the weak performance on the simulator and JPM portions of the exam, the difficulty of the written portion was not viewed by the IRT as a significant factor.
FaalDmit - 11 Fatsumy.1997                                      a-1996 MP lIDUTNRC F====a== Fahses RT                              E
 
  . _ . _ _ _ _ _ . . ~ . _ . _ _ _ _ _ _ _ _ _ . . . _ . . _ . _ _ _ _ _ _ . _ . . _ _ _ _ _ _ _ . .
l 4
l      -
I i.
EXECUTIVE
 
==SUMMARY==
(cont.)
Generic Ime!! cations                                                                                                            ,
i i                          The focus of this investigation centered on Unit 1 Operations Trauung, and the circumstances surroimdmg the                    ;
j                          LOUT program. While a review of the other programs was not conducted , evidence suggests that certam related                    l l                          problems may exist in trammg programs for the other units. Further in-depth review of these programs is i                          warranted For example, ACR M2-96-0657 was issued on Nov. 3,19% for Unit 2 Operations Trainmg fathng to i                          follow the SAT process. The issues identified are simdar to those found by this IRT on Unit 1. The prelinunary investigation of the MP-2 issues indicated that other trnmmg programs are at risk or have sinular problems.
l
!                          Sienificance and Consecuences i
!                          Concern over the ability of MP-1 Operations Training to adequately prepare individuals for NRC license exams l                          may suggest other training related problems and lead to NRC attention and action. Problems concerning i                  ~ comp ance        li with the National Academy for Nuclear Tramme accreditation criteria can result in actions by the j                          accrediting board.
!                          Recommendations                                                                                                                i' i                          Numerous remedial corrective actions are recommended in this report to address the root cause and various j                          contributing factors. Among the most urgent of these are the following interim
* r = tory measures:
.                                    1. Draw upon INPO loanees or industry experts to temporarily function in the MP-1 Operator Trauung Branch until the experience level of the incumbents can be raised to an appropriate level.                    l l
j                                  2. Conduct in-depth reviem for applicabili:y ofissues from this investigation to other trammg programs for
!                                          all Millstone Units. Canider temporary "stop-work" orders for those programs deternuned to be j                                          unacceptable.
1 j                                    3. Evaluate competencies of the licensed RO to SRO emndiA*s based on exammation results. Evaluate how identined weaknesses of the candidates pertam to the rarn=iaAar of the licensed staff.                        ;
: 4. Conduct reviews of previously identified problem areas and expedite close-out of those relatmg to                  !
accreditation, regulations, and trammg quality. This should include re-evaluatmg priorities and resource comnutmaats agamst other Nuclear Trammg Department excellence initiatives.
: 5. ' Implement an effective corrective action program which in ludes accountability for closure in a timely and        ;
effective manner.                                                                                              !
: 6. Senior management should clarify and reinforce expectattu,s of Trammg and Operations management regarding their responsibilities for oversight of trnmmg Implement methods and measures to ensure that oversight is effective.
                                                                                                            +e M
e h'
FeelDat .11 Fehnery.1991                                                          -
tsee MP 8 LOUTNRC Enmusmann Fehmus RT                                            IY
 
]                                                                  TABLE OF CONTENTS i
p,g, Executive Summary                                                                                          i i
Table of Contents                                                                                          I 1.0                      Introduction                                                                      2
}
: j.          2.0                      Refenaces                                                                          6 I
3.0                      Individuals Involved                                      .,                      6 4.0                      Analysis                                                                          7
!                                    4.1      Management Oversight                                                    9 i
4.2      Self Assessment and Corrective Action                                  II 4.3      Systematic Approach to Training                                        15 i                                    4.4      Experience and Staffing                                                19 i                                    4.5      Candidate Selection Criteria                                            24
!                                    4.6      Training Records                                                        26
!                                    4.7      Training Materials                                                      27 j                                    4.8      NRC Funm, Review,and Validation                                        28 5.0          Event Consequences                                                                            30 1
;            6.0          Root Cause, Lessons Learned and Conclusions                                                  31 i
j            7.0          Summary of Recommendations                                                                    34 8.0          Attachments                                                                                  36 i                                      I                Summary of Previously Identified Issues II                References
                                                                                                                  ~~
.                                      III              NETS Review WNRC Examination" -
i                                      IV                Evaluation of LOUT Issues Against NRC & INPO Requirements l                                      V                Analysis of NTD Procedures l                                      VI                Barrier and Failure Mode Analysis i                                      VII              Conclusion and Causal Factor to Recommendation Matrices
]                                      VIII              Previous Pass Rates MP-1 LOIT & 1996 Exam Results
;                                      IX                IRT Charter
}                                      X                Accredited Program Sequence 3
I.
Fani that .11 February,1997
,            1996 MP.1 LOUT NRC famaanneen Falhous IRT                              }
 
==1.0          INTRODUCTION==
 
An Independent Review Team (IRT), winch meluded utility peers and independent consultants, was chartered to assess therollective events leadmg tomd Mu= the Senior Reactor Operator (SRO) exanunation failures of six 4
out of seven candidates (Attachment IX). The events reviewed included trammg program revision and design, program implementation, exam review and valuiation, and the exammation itself.
;      Analysis methods employed in this review mehuie the Phoenix Handbook from NSRC, and concepts from the Failure Prevention incorporated (FPI) !=w rhal
* root cause litera:ure to supplement the company's Root Cause Procedure NGP 3.15.
4 j      1.1          RESPONSE TO THE EVENT                                                                                !
4 Based on the prelimmary written exam scores and NRC comments about simulator performance the Unit 1 Operator Trauung Branch (OTB) initiated ACR M1-96-1024 and ACR M1-96-1025 that documented weaknesses
;      in both LOUT written exams performance and LOUT simulator and JPM performance. Additionally, senior management commissioned the fonnation of an IRT to investigate and determme the root cause and casual factors of this event.
I
.      1.2          ' POST EVENT 1
On January 12,1997, the IRT convened at she Millstone Trammg Center. During the next two weeks, the team conducted interviews and reviewed denmentation relating to this event. The major post event findings, conclusions
:      and recommendations are included in the body of this report and summarized in the Executive Summary During the team investigation ACR M1-97-019 and ACR M1-97-0117 were initiated by Unit 1 OTB documenting j      weaknesses in the application of Systems Approach to Trammg (SAT), and inability to document completion of LOUT trammg.
1.3          COMPARATIVE TIMELINE
!        A comparative time of the event and precursors is found on the following pages. This timeline reviews the actual 3
occurrences of the event, an analysis ofhow and where the occurrences departed from what should have happened,
;        and a discussion of the significance of each occurrence to the event as a whole.
t i
1 N
4 Faul Drst . Il Fehnery.199' 1996 MP-1 LOUT NR.C
* FaGurus RT                    2
_ _ - ~
 
e    .
1996 LOUT Program Failures -Timeline of Events 9p :XWlhet shouldhave ~~happeted                        , ,y                                              4W                    W+g.p
                                                                                                                                                              ~p 5R ~ MK5@6%c                    ~gQ " =%!@(n@ldyffy v
m is . . .. <
e , What settaEy happasted A i Desah          ;
                                    .s                                        m..  ~5'''. 3g" #                                                ' ~**
  '
* n,,e 1        : ; n ~ ':. '                <- --~            e    MW                "        %gp : W MM                                                                                                :
Adherence to SAT should have been maintained.                        His sets the stage for future ,,..' :...                                            l Pre 93            The Systevnetic Approach to Tranung (SAT) breaks down.
(precursor)                                                                          ,
Root cause to SAT problems should have been identified and          He illusion that "all is well" was genersted.
93@4              Symptomatic fines apphed to visdde SAT deficiencies. Operations actions teken.                                                      (precursor).
training programs re-accredited.                                                                                                                                                                                                    .
Extensive tumover should haw occurred or ASOT position              One barrier in the 2nd level defense of quality .... .;"
10/93            ASOT (Assistant Supervisor of Operator Trainmg) accepts position in
* operations No. /              ^ is made due to rey.. ration.              Siled.
I/96              LOLTT progam .            J to be developed by line organization.
Enhanced supervisory oversight ofine@in:ed c inator.                A barrier in the ist level of defense of quality is cha" . ,4 by placing            :
I/96              New I OUT propam coordmater assigned.
an inexperienced individual in the position. Given the importance of                -l a    First time assignment, l.01Tr managementshouldhaverecognizedWiispotentialweakness                          !
e    No significant tumover or ove sight evident.                                                                                        and compensated.                                                                    !
t Accssddstion requirements and pmcedures should have been            De rust level in the defense of quality failed. De basis for all I/96              LOLTT propam designed without reference to accredited program er follomd.                                                            operator training resulting in a new or first time license wee the                  [
NTD procedures Licensed Operator initial Training Program. (LOIT). His propam                      '
had been reviewed by INPO and accredded by the National Academy Failing to consuler the propams requirements severe 8y damaged the revised IJ01TT preyem validity.                                                      ;
Accreddation requirements and NTD rwd res should have                De second level in the defense of qualdy failed. R. 6 by l/96-2/96        LOLIT proyam reviewed by Ops training supervisor and plant                                                                                                                                                                          I been recognized.          .                                          supervision and the line should have revealed deficiencies in this area.
management,                                                                                                                                                                                                                          !
De inability to recognize these problems suggests weakness in line and trmining supervision umlerstanding of the training process.                      f Program should have been reviewed.                                  De .a .J level in the defense of quality failed. M_., -- ^ a..e 1/96-2/96        Lotfr pregam revision not reviewed by Ops Trng Mgr.
of major progam changes is vital .
y l
d
                                                                  .; J as candidates were not  Program structure reviewed to ensure that original design            ne fust and second levels in the defense of qualdy failed. S,.;-..
2/96              Planned self-study component not 1,                                                                                                                                                                                                  !
assumptions remained valid.                                          training represents a large percentage of the knowledge that enust be known. No program modifications made.                                                                                                                                                                                                '
mastered by the candidate. De original plan to accomplish some of this training by self-study before the start of formal classroom could have been an efficient enethod to reduce actual classroom hours. De                  l training staffshould have recognized the impact of adding a                          l significant self-study component to an already challenging classroom schedule.
Detailed orientation of training responsetnuties.      .            De second level in the defense of quality was chattenged. A newly 3/96              New Operations Manager for Unit I appointed.
arpointed marager is often immdated with details and on thejob ll learning. Plant and training management should have paid close
* attention to those aspects relating to training to ensure expectations were being met.
Erdienced management oversight of inesperaJ supervisor.              De mm.J level in the defense of qualdy was cha"              J. A new 4/96              Operations Training Supervisor takes on site position. Replaced by incumbent senior instmctor in " acting status"                                                                                                  supervisor is oAen inundated with details and on thejob learning.
Trmining - _            -
should have assisted the new supervisor to                  -
ensure expectations were being met.
Training Readmess meetingnot held-                                        Meeting between plant and training to review current status and      His meetingwould have been anotherw.; . , to re-check 6/96                                                    '                                  determine readiness of pmyam                                        assumptions and plans.
0
!                                                                                                                                                          I i
Feel then . II Fetwuery.1997                                                                                          3 7
19s6 MP-l t.0UT NRC Ewen - .
* Fedures IRT 6
                                                                                                                                                                                                        --                      . . - -      n .
 
1996 LOUT Program Failures -Timeline of Events (continued)
                                        @ha(actuallyhappenedp ' s                                      9habhould have happgned ?
s          -
eSignincances iDaselv                            ,                                                                                                            ^.                            .
M    '*
[TimU                                              ''OnW                                                                    '''                                                                                      <                                                v 6/96            Candidates are announced "only a few days' before start of program. Sufficient time should have been available to complete training  hs previously mentroned, not announcing candidates prevented Some candidates are selected the day before program start.              equivalency assessments.                                        implementation of the self study component. Additionally,this was not enough time for the training stafTto adequately review the back ground and knowledge level of these new candidates.
One student withdraws in first week of the program.                    De motivation and abihty of the candidates should have been      License candidates represent a significant investment of utitity 6/96 assessed prior to class start.                                  resources. Additionally,the license holder occupies a pWtion of trust with regard to maintaining the plant. De selection process should be tigorous enough to screen out those candidates who are not strongly motivated.
Plant management appoints Shift Manager as " mentor' to class. Mentor  Frequent and fruitful interaction with candidates and feedback  The second level in the defense of quahty was ineffective. De 6/96 only meets with class once during the entire program.                  provided to pfant and training.                                Mentor had the potential of being an valuable source of feedback regarding the quality and conduct of the program.
7/96              LOUT Coordinator accepts different position. Replaced by incumbent    Continuity and coordination of propam.                          De first level in the defense of quahty was challenged. Anew instructor.                                                                                                                            employee is ofte:S inundated with details and on thejob tearning.
Trsining supervision should have assisted the new coordinator to ensure expectations were being met 7/96-8/96        Two candidates attend OFE classes in parallel with classroom, resulting OFE training should be completed prior to starting license      Candidates pere started in the IJOtfr program who had not met their in missed cfassroom training.                                          program.                                                        entrylevelprerequisites. Additionaffyunreasonableexpectations placed on those candidates who were already coping with a self-study and an aggressive classroom schedute.
Exam bank questions provided to NRC do not meet NRC guidelines        Iligh quahty exam b nk provided to and used by NRC.            De inadequate exam bank drove the NRC to author many of the test 7/96 forcing all questions to be developed by NRC                                                                                          questions instead ofusing the utility test bank.
Candidates' feedback to training that they do not have time to read all Given the magnitude of the changes in this program, close      Given the fact that this was the first time at Unit I were a -                                                      J 7/96 self-study texts. Training acknowledges issue but is constrmined by    attention should have been paid to student comments.            Iicemed class was given, strong consideration to the student comments schedule.                                                                                                                              should have been given.
ASOT position fitted but;                                              Experienced ASOT available to provide oversight and guidance    A second barrier in the defense of quahty was not available.
8/96
* Assigned to temporary project.
                      . Not SRO certified.
9/96              Operations Trairung Manager removed, no replacement made.              An experienced Ops Tmg Mgr. appointed and assigned.              A second level in the defense of quality was removed At the time the manager was removed, both the training supervisor and coordinator -
had been on the job for only a few months.
001T0 coordmator requests additional resources to support program. Management assessment of risk in not prr vidmg resources.      Another indication of problems in the program was not acted on 9/96 6/96-            Program status meetings not held.                                      Regular status meetings to review progress and adjust plan as  Additional opportunities for ensuring the quahty of training were needed.                                                  I      n issed.
I1/96
                                                                                                                                                  'I I t/96          Company written exam given;.
* Averagescore 86.3%
e      I failure 11/13/96        001J0 Program Coordinator authors memo requesting academic review      Academic review board meets and appropriate oversight given. De coordinator had reservations about one candidates readiness far board for one candidate based on past performance. A board was not                                                                      the exam. This was an appropriate action for the coordinator to take.
convened.                                                                                                                              It was also a formal action and should have warranted a formal response.
Candidate who faded company exam withdraws from program citing        De motivation and ability of the candidates should have been    I,icense candidates represent a significant investment of utihty 1 I19 6 personal reasons                                                      assessed prior to class start.                                  resources. Additionally,the license holder occupies a position of trust with regard to maintaining the plant. He selection process should be rigorous enough to screen out those candidates who are not strongly motivated.
t FinalDren - 1I Fetmsary F97 1996 MP-1 LOtfr NRC Enemm clon Fedwes tRT 4                                                                        ,
 
e .
1996 LOUT Program Failures -Timeline of Events (continued) y DefeQ                    .
q.Wre(netually htppertidi
                                                          ~        +.        .jq ,
                                                                                                        < ?sWhatshouldhav.elszppened g s
s,
                                                                                                                                                                                                                                                                    ;stgniacances y
II/IS/96          Practice final exam given.                                            .      Test scores similar to company final.                    His test was presented as a practice exam test after the company
                        .      Average score 78.7%.                                      .      Management consideration of results                      exam. He results should have raised questions regarding the e                                                                                                                                student's readiness for the NRC exam.
3 failures.
No action taken by training er Operations management.
Il/19/96          Inexperienced exam reviewers assigned.                                Experienced individuals or appropriate oversight provided.      ne first and second levels in the defense of quality failed ne effects of this inexperience resulted in an inefTective review and validation of the NRC exam.
II/19/96          Exam review and validation occur concursently.                        En si review should have been completed sufficiently in        Combing the review and vahdation phases reduced the utihties ability edi ance of and independent from validation.                    to evaluate the exam construction. Typically reviews are done in advance of the validation. His provides sufficient time for dialog and consensus between the NRC and utility.
I1/19/96-          Exam review results in rewriting over 70% of exam.                    Degree of changes required should have triggered questioning    While exam revision is normal, the degree of revision seen in this 1I/20/96                                                                                attitude over the suitability of the exam.                      exam seemed high.
I1/20/96          Exam tevalidated by non representative SROs. Exam takes 3.5 hours to  Vahdation should be performed by individuals representative of  If more representative SROs had been used, better data about the compfcte.                                                            the expected knowledge and skill level of the candidates. Exam  length of the exam would have been evident.
should take approximately 3 hours.
12/2/96            Written exam taken -
* Exam should have been able to be completed in 4 hours. A less than effective exam review resulted in a challenging but valid e                                                              exam.
                      ,e NRC enminer n!!ows candidates additional half hour at the 3            100 % pass rate.
hour point.
ie      6 out of 7 fail (preliminary results) 12/2/96            Candidates informed of preliminary results.                          Prehminary results should be released after candidates complete Informir*g the candidates of prehminary results may have distracted att portions of the exam.                                      and de-motivated them.
12/3/96            Simulator and JPM portion of exam taken. NRC indicates weak          100 % pass rate.                                                At the NRC crit meeting, questions were raised regarding the performance.                                                                                                                          competency of the licensed RO candidates to perform certain RO tasks.
12/8/96            Unit i OTBinitiated ACR Mi-96-1024 and ACR Mi-96-1025 for weaknesses in totTf exam and simulator and JPM performsnce.
12/13/96          letter sent to NRC containing comments on 10 exam questions.          An cfTective review and validation would entail goal of 0      De number of comments sent to the NRC reflects poorly on the
                                                        '                                                                                                quality of the exam review. The purpose of the exam review is to challenges to NRC.
prevent any comments 12/96-            IRT commissioned.                                                                                                                    Senior Management recognition of potential seriousness of event.
I/97 til5/97            ACR MI-97-0109 imtiated to descr-be weaknesses in S AT for Unit i                                                                    Ops training recognition of SAT issue.
Ops training.                                                                                                                ,
1/17/97            ACR Ml-97-0117 initiated describing inability to document completion                                                                  Ops training recognition of records issue.
oflDtTf training.
Final Dielt - Il February,199:
5996 MP-1 LOL/r f*RC Enemination Feilures litT
 
j a
J                                                                                                        .,.
l 4                                                                                                                                                                    i
 
==2.0          REFERENCES==
.                                                                                                                            l 1-'
Documents reviewed by this IRT are listed in Attachment II.
d i
j.
3.0          INDIVIDUALS INVOLVED e
s
;                21            INDEPENDENT REVIEW TEAM MEMBERS i
* K. C. MuccinO (Team Leader)
                              .            W. H. Barron
* L. H. Dooley                                                                                                              ,
                              +            S. R. Frye                                                                                                                I
;                              .            T. P. Gilben i                            e            C. W. Sawyer                                                                        --
1 1
I
:                  3.2          PERSONS INTERVIEWED OR CONTACTED 1'
Name                                          Desertnient / Position
;                                                                  SRO Ucense Canchdate j                                  ,
SRO Ucensecandate                                              _.    -.
4
                                    ~
MP.1 Opee Tramma Inanacsor
!                                                                  MP.I Assinant Operanons Supervisor I                                  ~.
SRO Usense Candulate f                                                                  MP.1 Shift Mansarr                                                                                i i                                                                  SRO Ucense Can&date                                                                              l l                                              _                  Prewous De Nuc. Trasuna
?                                                                  Mar. Ena Support & Prof Devel. (NTD)                                                              l
}                                        .                          MP.1                                                                                            i
!                                                                  QAS(Contractor)
MP.I Operator Tramuna lanructor (Conractor)
{!
4 Dunctor. Nuclear Tramune Prewous MP.! Manmaar Operator Trasuna f                                                                    MP.! Drector. Una Operanons                                                                      j i
Current LOUT Prosram Coenhnasar
;                                                                    SRO Lacense Candsdale 2
Performance C. -                Inanaciar j                                                                    MP.I Assistant Operssor Trasuna Supernsor Prewous I.DUT Prosram Coonhnator i                                                                    Nuclear Traisnne Serwass
!-                                                                  MP.1 Supervisor Operasar Tramuna MP.I Operauens Manaaer l
i                                                                    Prewous MP.1 Operances Mannaar l                                                                    MP.1 Operator Tramuna lanrumor a                                                                    MP.! Operator Tramma lanrucsor
_                                      Manaaer. Operator Trauuna f                                                                    MP.I Operator Tramuna lasarucsor (Comaractor)
                                                  ~
MP.I Asmstans Operanons Manneer i                                    [.                            SRO License Cansbdine SRO License Canibdese Mar. oAS *-
i                                                                  ' NETS' r%==hmss .NRC F===mation Rewsw 1
i
                                  ~ . .
I e
;                PsalDet . :: Feenery isp 39s6 htP- 14:3N3tCEmmmmmenFemmes RT g
1 p.
4
 
e j        - 4.0      ANALYSIS
  ,        his event is the result of a combination of factors that:
* Made the situation vulnerable and setup the event
!                  e    Triggered the event
                  . Aggravated the event already in motion
* Mitigated the event nis section exanunes the above four factors, followed by a detailed evaluation and discussion of the major issues I          identified.
4.0.1 EVENT SETUP (What allowed the event to happen).
j          The Unit 1 Operator Trauung Branch (OTB) madequately redesigned the Licensed Operator Initial Trauung
_(L_OIT) program to meet Unit 1 Operations management needs for additional SRO licensed personnel. The
;          following factors set the utility up for the event.
4.0.1.A Previous Weaknesses in the Systems Approach to Training (SAT):.:The IRT detemuned through i
interviews with various members of NTD that the SAT process had been breakmg down for a period of time. '%is i          was manifested by not following established procedures and misundere== dine the concepts of SAT.
1
{          4.0.1.B OrganizationalPressures: The original candidates proposed by the plant in early 1996 were currently 4          licensed ROs pursuing upgrades to SROs. De LOUT program redesign was based to a large extent on the time the i          candidates would available for trammg %e trammg staff was given a start date by the plant and avp~+M to develop a program that would fit within this available time. His method contrasts sharply with a systematic
;          approach, that considers fisst what must be taught, the entry skills of the candidates, and then determmes Wad j          course length.
4.0.1.C. InexpertencedProgram Designer (LOUT Coordinator): %e coordmator had not previously pasticipated in a major program design or modification. He was unaware of the requirements of the accredited LOIT program, and did not use available procedures when modifying the accredited program
                                                                                                                                      ]
4.0.1.D Selection Criteria - Entry Level Assumptions: The ongmal proposed "ndid=*~ consisted of currently licensed ROs pursuing upgrades to SROs. The program modifications considered certam entry level knowledge and skills. As the course grew nearer to startmg, mstant SRO andid='~ were added Neither NTD staff or supervision questioned the appropriateness of combining these two distmetly different audiences in an already significantly modified program        Additionally, the original revised program included a f6iir month self-study component. The intent was for the candidates to self-study 31 selected systems and to receive periodic tests on the        i topics studied Rese systems were selected based on the probability of not bemg on the NRC exam. His self-              "
study c.u-st was included to e-r;== for the relatively short time available for formal trammg However,                    i due to delays in the selection process, some candidates were not announced until May 1996, resultmg in the self-          l study waycemt not bemg implemented as planned. %is change to the program design was not considered by NTD staff to be significant.
IN E $I E i nn mr                                                      7 l
 
1
: 4. 0.1.E. Inefective Management Oversight, involvement and Accountability: Prior to the start of the class,            1 numerous opportunities for management intervention were missed. Management (plant and trammg) was more focused on the larger issues facing the Millstone site than on specifics. This coupled with an overconfidence in the    i subordinates' skills and abilities led to a " hands-off" approach to program design by trammg and line management.      l This lack of oversight resulted in several non conservative decisions on the part of the trammg staff including        i reducing the on-shift component from 13 weeks to 4 weeks, and requiring some candidates to attend both General Fundamentals Exam (GFE) training and LOUT at the same time.
4.0.2 EVENT TRIGGER (What set the event in motion).
The start of the LOUT program in June of 1996 was the trigger for the event. Up until this point, reconsideration of all decisions and assumptions regarding the program were possible without significant wasted resources. After the program started, with candidates and staff assigned, the simulator schedule and test date established, the difficulty in effecting major programmatic changes was increased.
4.0.3 AGGRAVATING FACTORS (What made the event worse than it could have been).
4.0.3.A Self-Study; Some systems were selected to be presented by self-study insteEl ofin classroom. The bulk of
,    this self-study was to be done prior to the start of formal trammg. However this self-study material was not given to the candidates until the start of the class. This signhmly increased the burden on the candidates especially when considering the already shortened program.
4.0.3.B GFE Artendance: Two candidates had not completed the NRC admmistered GFE prior to the start of the program. These candidates were required to attend selected portions of GFE concurrently with LOUT attendance, again increasing the individuals burden. Neither the candidates, mstructor or trammg supervisor questioned the appropriateness of this and the self-study expectations. This resulted, in some cases, of candidates missing LOUT training to attend GFE trauung.
4.0.3.C IneffectiveManagementInvolvement: Numerous examples ofineffective involvement were evident during the conduct of the program. The Shift Manager Mentor, assigned by the operations department, met only once with the candidates during the entire program. This valuable source of feedback to management, and resource to the candidates was missing. Additionally, management failed to recognize and or act on several indicators from both the candidates and coordinator of potential trouble. For example, the coordinator requested an academic review board for one student based on the readiness of the candidate to sit for the NRC exam. This board was not convened.
4.0.3.D Resource Issues: In some cases trammg material was not updited as a result of procedure and plant modifications. At times, test questions were given on plant changes that were not referenced in the trammg materials. Additionally, on-shift time with the exception of prescribing certain OJT cards to complete was unstructured. While the need for additional mstructional assistance was requested by the coordinator, no action was taken by management.
4.0.3.E Exam Review and Validation: The overall experience of validators was low with 3 of 4 having never validated an NRC exam prior to this exam. This process is very dependent on staff experience to provide an adequate review and validation. After review and validation, questions about the length, scope, and difficulty of the exam remamed, but the staff was uncertain of what actions could be taken to address the situation. Additionally, the review and validation took place concurrently during the same week thereby reducing the effectiveness of the process. After the exam,10 exam questions ( 10% of exam) were challenged by the unit. This indicates that the review of exam questions was incomplete and that the validators may not have been representative of the candidate population.
ru oma . n rem. im                                          8 1996 MAI14XRNRC Emnsunan Faiksu IKT
 
l l
i l      4.0.4 MITIGATING FACTORS (What prevented the event from becoming worse).
4.0.4.A Management Actions: NTD and Operations management reahzed the severity of this event by the
;      completion of the NRC exam week. ACR's were wntten, and the IRT was commissioned. Additional ACR's were i      wntten as necessary during the course of the IRT.
!      4.0.5 ISSUES In reviewing the casual factors and elements of this event, the team identified the following eight issues. These issues are discussed in detail in the remamder of this section of the report.
            . Management Oversight, Involvement and Acemtahility
            . Self Assessment and Corrective Action e    Systematic Approach to Training
            . Experience and StnfFmg e      Candidate Selection Criteria                                                                          .
* Trammg Records                                                                      .-
* Trauung Materials
* NRC Exam, Review, and Validation i
4.1      MANAGEMENT OVERSIGHT, INVOLVEMENT AND ACCOUNTABILITY 4.1.1    Management oversight of the 1996 LOUT program was ineffective in identifying and correctmg challenges and deficiencies in presenting quality trammg. Throughout the conduct of the program, decisions were made and actions taken without consultation or review by management. Procedures and policies relatmg to oversight activities were not effective. For example:
* Corrective Actions for previously identified issue'swere not implemented.
        . Trammg management did not reahze the inexperience level of the original LOUT program coordmator. This coordinator did not utilize either procedures or the accredited trammg requirements in the design of the LOLTT program 'Ibese omissions were not recogmzed by management.
* Operations Trammg management was not engninnt of the magnitude of changes made to the accredited program For example the Operator Trammg Manager was under the impression that "on-shift" component was reduced to 8 weeks, not the 4 weeks actually presented during trauung e  Required assessments of trammg by Operations and Operations Traming management were not conducted.
* The Operations Department Mentor (Shift Manager) c*ad only one observation of trammE during the entire program Other than this one observation the mentor had no association with the class.
        . The Operations Liaison position was open throughout the time of the class. As a result trammg matenal and exams were not reviewed by a plant subject matter expert e  Senior management failed to reengai= the cumulative effect of personnel changes of the individuals associated with this program        Every position that held programmatic responsibilities for LOUT was replaced or reassigned during the time frame of the program
        . Indications that programmatic difficulties were being encountered was provided by both the candidates and the second LOUT coordinator. No apparent action was taken based on this feedback.
        . Management was not effective in communicating and following up its expectation that procedural compliance ~
and adherence to SAT were a high priority. Instead, many staff have interpreted recent management changes and comments to imply that the first priority of trauung is to meet the customer's needs ELErrDirm. mr                                                              __
9
 
l i
                                                                                                                                  ~
l 1
          .      The LOUT program coordinator formally requested an ="damic review board for one student based on that
: i.                candidates documentaA performance. This academic review did not occur.
1 l
* Trauung supervision and management were unaware that deficiencies were evident in student traming records j                until notified by the IRT,
!          Issues relating to the ineffective management oversight have been identified in numewus other inspections, audits f          and assessments. (Refer to Attachment I)
Examples of previous reports indisting weaknesses in Management Oversight 5'        .      ACR 11037 Adverse Trend in Unit 1 Simulator Ev=1 nation = Root Cause Analysis (October 1996) j                noted that a lack of ownership by both the Shift Managers and Trammg Mentors contributed to
:                degraded crew perfonnance.
J l          e        1995/19% MP-2 LOIT Class Comoany Final Exiuwmion Root Cause Evaluation (September 1996)
:                . indicated insufficient oversight by Unit 2 trauung n anagement was a casual factor to the event.
i
!
* OAS Audit Reoort #A21071 " Unit Staff Trainino"(April 1995) noted that managers and supervisors                  _
1                were not conducting trammg obsenations as required by procedures.                              --
v 1                                                                                                                                      l 1          4.1.2       
 
== Conclusions:==
:                                                                                                                        .            1 2                      4.1.2.A                Management oversight of the 1996 LOUT program was ineffective in identifying            i and correcting challenges and deficiencies in the presenting of quahty trammg
.                                              Management missed numerous opportunities to ensure that trauung delivery was of        I the highest quality.
{                                                                                                                                      l l                      4.1.2.B                Management failed to consider previously identified oversight issues (across all l                                              units) as "warnmg flags" and adjust its actions appropriately.
1 1
i
!          4.1.3        Recommendations:
;                      4.1.3.A                  Review industry guidance and INPO identified "best performers" with respect to management oversight. Adopt best practices into oversight process.
;                        4.1.3.B                Reinforce with Trammg and Operations management their responsibilities regarding j                                                oversight of trammg 4.1.3.C                Establish high expectations regarding oversight and implement methods to ensure l
;                                                that oversight is denmantaA frequent, and useful.
!                        4.1.3.D                Ensure that oversight feedback is communicated and addressed consistently across j                                                all units.                                                                _
9 i
: j.          runt thea . H Fetney.IM 10                                -
,            im w.iuxrme - r m m 1
a
                                                          . - . - - . ,        m                            ,.
 
  -                                                                                                                                  l I
  -                                                                                                                                j 4.2          SELF ASSESSMENT AND CORRECTIVE ACTION PROGRAM 4.2.1        Here were numerous previously identified, long-standing trammg issues and problems. nese areas were identified in NRC inspections, INPO evaluations, QA audits and internal self assessments. He reports evaluated covered training programs at all Millstone units. In many cases, these identified issues were the same as those
~
issues identified as casual factors for the LOUT exam failures. In short, if these issues had been acted on in a timely manner the negative outcome of the LOLTF class might have been prevented. Attachment I contains a                    _
summary of t aining issues from 14 past inspections, audits and assessments. A review of these items indicates that trauung management had sufficient warning that training programs were degradmg.
The following examples contain illustrative comments from past evaluations, inspections and assessments.
Management                            1996 ProcranfCoordinator Assessment of Tramme Procrams indicated that attendance Oversight                            problems at training has been tolerated by both Trammg and Line Management.
Systematic. Approach                NRC Inspection 93-80 indicated that TPCC committee size limits the committees' to Training                          ability to provide checks and balances among all aspects of the trammg program.
Experience and                      The Team Assessment of 1995 Nuclear Trainine Department Activities indicated that Staffing                              instruction is skewed to the instructor's expertise, resultidg~m incomplete learnmg.
Training Records                    Surveillance A-324 " Verification of LORT Tramma Attendance (March 1992) indicated that ammd=ce records were found to be of poor quality.
Selection Criteria                    1996 Procram Coordinators Assessment ofTramma Procrams indicates that the Exam validation process is weak due reputedly due to resource limitations. Several program
                                    ~
coordinators believe that their exams do not disenmmate between competent and non-competent personnel.
Quality ofTraining                  Team Assessment of 1995 Nuclear Tramma Deoartment Activities indicated that Materials                            training focuses on passing tests, not on operating the plant.
4.2.1.1 Effectiveness of self assessment in generating and ensuring change NTD utilizes a multi-level self assessment process that considers many elements of trammg. In some cases these self assessments were in-depth and self critical. Unit 1 Operator Trammg Branch (OTB) conducted a program by program assessment in late 1995. This assessment noted the following weaknesses:
e    ne self assessment process did not systematically review past trammg problems to identify the continuing success of remedial actions.
        . He assessments included line management only as interviewees, not participants.
        . No corrective action plan was developed for those issues identified in the self assessment.
        . Unit 1 OTB was unable to provide any examples of bow student feedback or supervisor assessments recommended improvements in trammg. He lack of such feedback indicates a potential weakness in the evaluation phase of training. This was not noted as an area in need ofimprovement.
        . Unit 1 OTB was unable to offer any examples of how continuing trauung and plant upgrade topics were included in the LOLTT program, despite this being a clear commitment to provide such training to those candidates holding RO licenses.
Frd Draft .1i February,1997 g
1996 MP.1 LCET NRC f.marmnsten Fa&su 1R7
 
      ..- - - - - . ---                                                    .  .-    - - . . - -                --        _    .~.- -.  - .
i i
.                      Unit 1 OTB offered only nunimal feedback in Objective 3 " Development and Quali6 cation of Staff." The staff j    ,                  did not effectively evaluate this objective and failed to identify the lowering of group experience within Unit 1
;                        OTB.
                  . Issues identified at one unit are not always reviewed for applicability at other units. For example Adverse
:                        Condition Report (ACR) M2-%-0657 was issued for Unit 2 Operations tramme failing to follow the SAT 1
process on 11/3/96. Unit I did not issue a sinular ACR until the IRT resiew. Those items listed in this later ACR were in existence at the time of the Unit 2 ACR.
                  . Items identiSed in self assessment are not always assigned appropriate corrective action and tracked to completion. This issue was also noted in ACR M1-96-0339.
l-4.2.1.2 Urgency in resolving corrective actions l                  The team did not sense any significa.nt urgency on the part of trammg or line management in resolving and closing out trammg deficiencies. For example,.
l                  . Task list for Unit I licensed operators has not been reviewed and updated sind1993. By NTD procedure this i                        action is required every two years. Despite being overdue for more than one year, this review is not scheduled I                        until August of 1997.
;                  e    ACR M1-96-0339 identified training issues in need of improvement that were not assigned concrete corrective l                        action to prevent recurrence.
l
}                  4.2.1.4 INPO comments regarding self assessment
;                  On a similar note, the 1996 INPO E&A visit generated the following comments regarding self assessment and j                  corrective actions:-
                  . He long standing nature of simulator instructor perfonnance problems was not identified by self assessment.
j j                  .      Corrective actions for simulator instructor performance problems were not effective.
2 i                  .      Self assessment did not achieve desired results
!                  .      Ongoing self assessments are insufficiently broad in scope to identify trammg weaknesses.
                  .      Ongoing self assessments did not include industry experience.
l j                  .      Self assessments did not include review of station problems to identify poten+2al worker performance issues.
l
!                  .      Trammg self assessments did not review wash = identified from previous inspections and assessments 4
!                  4.2.2     
 
== Conclusions:==
 
                                                                                                                                                    )
4.2.2.A              The Self Assessment and Corrective Action psocess has not been used effectively              j to identify and correct weaknesses in trammg                                                  i i
j                                4.2.2.B              Be Self Assessment and Corrective Action process does not systematically assess i                                                    past problems to ensure the adequacy and sustamability of past corrective actions.
J 4.2.2.C              he Self Assessment and Corrective Action process in many cases focuses on a
:                                                    single plant and does not consider if problems identified in one unit are present j                                                      elsewhere.
;                                4.2.2.D              Some assessments in the pas including the accreditation self evaluation report, did          l not includeline mana,W ownership.
r-iaa . n ww.m                                                      n i                  1996 IO-l iDUT NILC Lammenon Fahres IIt?                          -
k I
 
r i
 
====4.2.3 Recommendations====
]
4.2.3.A              Review industry guidance and INPO identified "best performers" with respect to
,                                                        self-assessment and corrective action. Adopt best practices in the NTD self                                        !
[
1 assessment and corrective action process.
]                                  4.2.3.B              Conduct an ' urgency review' of previously identified problem areas and arpadha                                    i j                                                        close out of those relating to accreditation and / or regulatory issues.
s j                                  4.2.3.C              Ersure corrective actions are developed and tracked for all idenufied trammg issues.
4.2.3.D              Review effectiveness of past corrective actions. Incorporate this review into the self i                                                        assessment process.
J i                                  4.2.3.E              Develop, communicate and reinforce clear y=6ons regardmg the Self.
Assessment and Corrective Action process for all levels of management, supenision
!                                                        and staff.
i E
  ^
1 i
e--                <e 2
3 W
I
!                                                                                                                                                                          i
;                                                                                                                                                                          i I
'                                                                                                                                                                          i e
l Fam ths3 . Il Fehnery.1997 1996 biP. I LOUT NRC Emmmunen FeAmu RT                                                      13
                                                                                                                                                  , , .  ,-.    .-g, .
w.
 
                                                                                                                                                                    --        . . . m  .
i i*
s                  +
i i
Previous Inspections, Audits and Assessmeats Containing Issues identified in the LOUT IRT Investigation Management      Self Assessment / Systematic    Experience and Selection Criteria Training Records    Quality of --    -NRC Exam, . -
Document Title Oversight -'  Corrective Action Approach to s    . StafHng                                      Training Materials    Review and Trmining                                                                                Validation          ,
ACR 11037 Root Cause Analysis                    X                                X                X                                                  X MP-1 Simulator Evaluations (Oct.1996) 1995M996 LOIT Class Company Fina'                X                  X            X                X Examination Root Cause Evolustion Surveillence A-324' Verification of LORT                                          K                                                  y Attendence' (March 1992)                                                                                                                                                                  l NRC Inspection 93-80 (sil units)                  y                                X                X                                                  y Audit A60516' Operations Activities for          y                                y                                                                                                      +
MP-3* (March 1993)
Audit A21062 *MP-1 Unit Staff Training" X                                X                                                  X (April 1993)
NRC Root Cause Analysis of MP-1 LORT              X                                X                                                                    X 1991 Unsatisfactory Program Determination NRC Inspection 96-05 NU Assessment of          ly                                y                                                                    y NRC issues                                      ,
ACR M2-96-0657 Failure to apply SAT at            X                  X            X MP-2 (Nov.1996)
Self Assessment of MP-1 LOIT                      y                                X                X                                                  y                                  l (Jan.1996)
' Sett Assessment of MP-1 LORT                      y                                y          4 y                                                  y                                  ,
(Jan.1996)                                                                                                                                                                                !
Team Assessment of 1995 NTD ActMiles              X                                X                X                                                  X                                  ,
1996 NTD Program Coordinators                    y                  y            y                y          I ly                y                y Assessment of Training Programs 1996 INPO E8A Vist                                X                  X            X                X                                                  X
                                                            ~
t                                                                                                    !
l t
i i
Feet Dr A - Il February,1997                                                        gj' 1995 MP-t IEUT NRC Eneminadon Faihees IRT t
 
4.3 SYSTEMATIC APPROACH TO TRAINING (SAT) 1 4.3.1    A lack of the effective utilization and understanding of the SAT contribmed to this event. Evident        I throughout the course of the program are errors in judgment and deviations from procedures with regard to the      !
Systematic Approach to trammg.
2 Federal regulation and industry guidelines' require that licensees utilize a systems approach to trammg that incorporates the following 5 elements:
      . Analysis of thejobs to be performed
* Design of the training program
      . Development of trammg materials
      . Implementation of trammg
      . Evaluation and revision based on student performance The Nuclear Trauung Department should implement the SAT through effective mstructor trammg, policies and trammg procedures. The team identified the following occurrences where SAT concepts, policies or procedures were not used in the design and conduct of the 1996 LOUT program.
      . The modification of the LOUT program was driven by available time in which to conduct the trammg. NTD staff revised and reduced trammg components in order to present the program within the window established by the plant. Under the SAT, trauung time should pnmarily be driven by the amount of time required for the candidates to obtain competency, not an arbitrary time-line.
i
* NTD procedures provide detailed guidance to staff, supervision and management in the process and steps required to satisfactorily modify trammg programs in compliance with the SAT. (Attachment V) Interviews with the trammg coordinator and supervisor associated with the revision indicated that these procedures were not used.
      . Part of the reduction in course time included shortenmg the on-shift component from 13 weeks to 4 weeks.
This action was taken even though the National Academy of Nuclear Trammg accredited trammg program required 13 weeks (see Attachment X). When questioned about this revision, the LOUT coordinator stated that he was unaware that the accredited program had a 13 week requirement. Additionally there is no evidence that a systematic approach was used to justify this change. While elements of an accredited tramme program should be revised based on changing circumstances, careful consideration should be given in ensuring that the expected outcomes of the revision meet the intent of the original program.
* The LOUT program was originally designed to include a four month self-study component. The intent was for candidates to self-study selected systems and to receive periodic tests on the topics studied. This self-study component was included to compensate for the relatively short time available for formal trammg. However, due to delays in the selection process, some candidates were not announced until May of 1996 resulting in the self-study component not being implemented as planned. Instead, the self-study materials were given to candidates to study concurrently with the scheduled formal trammg This represented a significant change to the revised course design. Adhering to SAT principles would have resulted in a re-analysis of the trammg program design to ensure that appropriate time and focus were given to the objectives and material affected.
* In addition to the self-study matenal mentioned above, two candidates were expected to study for the NRC admnustered General Fundammtals Exam (GFE) concurrent with fonnal trammg phase. This resulted in candidates missing LOUf trammg to attend GFE trammg.
2 10 CFR 55.2 8
ACAD # 91015 IEiSrrT i -                                              15
 
l
        .                                                                                                                                  )
j 1-l                . Some trauung matenals used in the self-study program did not reflect actual plant changes. (LNP module,                  ;
j                    Instrumentation) l                . He program re-design was based on candidates who currently held RO licenses and were pursuing an                          ,
!                    upgrade to SRO. In March,1996, after the program was revised, Trauung was notified that additional                    I candidates would be added Dese candidates were from the Unit 1 STA position and had different entry level              I
:                    skills and experience than the RO's. No significant modification to the program content was made to
!                    accommodate these new candidates. The following candidate comments were offered at the end of the program j                            " Training incorrectly assumed that the applicants had a solid systems knowledge base "
A "The Instants cannot be expected to know " expected operator actions" that are notprocedurally governed.
                            "The assumption that the candidates had a LOITlevel ofknowledge at the beginning ofthe course was a poor one" (Ifelt I was playing catch upfor several reasons) "First the presumption that my level of knowledge was at the licensed operator level. While my knowledge was good. Iwas not at a Unit
                                                                                                              ~
1 operator levelat the start ofthe class "
Evidence of weaknesses in the effective utihntion and undere=aAiag of the SAT is evident elsewhere in operator trammg
                  . Interviews with trammg staff and supervision indicate a weak underet*nding of the SAT.
                  . He Unit I licensed operator task list was last reviewed in 1993. NTD procedures require these task lists be reviewed everytwo years.
                  . De process in place (T==hn=*er) by which plant design changes and procedure revisions are tracked and reviewed for trnmmg implication for Unit's 1 and 2 is not being used. A Unit 1 operator instructor stated he has no method to ensure that all applicable plant and procedure changes have been reviewed and are included in trauung.
                  . ACR M2-96-0657 was wntten in 11/13/96 for " Failure to apply the SAT at Unit 2".
                  . Operations Tramme Manager mtervention (third level barrier) was required to prevent violation of traimng procedures at Connecticut Yankee in the fall of 1996.
                  . The 1996 self assessment of compliance with National Academy of Nuclear Trammg Accreditation criteria by            i program coordmatnrs identified that many elements of the SAT are not in comph=nce.                                  l
                  . A review of past mspections, audits and assessments indicates that numerous issues regardmg the implementation of the SAT have been previously identified. (Attachment I).
i NOTE : Durmg the IRT investigation, Unit 1 Operations Trammg supervision subnutted ACR M1-97 0109.
This ACR <bcuW a breakdown in the SAT for Unit I trammg activities.
4.3.2 
 
== Conclusions:==
 
4.3.2.A          De original accredited program was revised without sufficient =ttention placed on accreditation or regulatory comnutments.
4.3.2.B          NTD procedures were not used to ensure a systematic process was followed in redesignmg the program IOiSOLr.a- mr                                                            16
 
i j
1 1
o                                                                                                                          =
4.3.2.C            The analysis and needs assessment for program redesign was not &-:      ==M and              '
therefore cannot be verified.
4.3.2.D            The NTD staffinvolved with design and revision were unaware of accredstation commitments.
4.3.2.E              Evidence of weakness in underending and applying the SAT is evident elsewhere in the NTD.
 
====4.3.3 Recommendations====
4.3.3.A            Evaluate existing NTD procedures and instructions (OTBI's) and ensure these procedures meet NTD erra dons n    for the trauung process.
4.3.3.B              Revise NTD procedures and instructions'(OTBI's) as required.
4.3.3.C            Management and supervision should ensure compliance wit [existmg procedures.
4.3.3.D              Evaluate the as taught LOUT program agamst accreditation standards and ensure that all accreditation commitments were met.
4.3.3.E              Develop and implement corrective actions for the issues ident:fied in the recently performed internal assessment of accreditation entena 4.3.3.F              Implement the previously existing plan for reviewing the Systems Approach to                  j Trammg as part of the 1997 Continuing Instructor Trammg program                              ;
m 4 88 % ar
                                                . - = =
Tsuit DmR . I'. Fahnery. IM7                                                                                                  '
im w w m w - ,.m                      at                          17
 
Previous Inspections, Audits and Evaluations Which Identified Issues With the Systems Approach to Training Document Titic                    Analysis      Design          Development      imp!cmentation            Evaluation i
ACR I1037 Root Cause Analysis                                                                                    X                      X MP-l Simulator Evaluations (Oct.1996) 1995/1996 LOIT Class Company Final Examination Root Cause              .                                                                  X Evaluation NRC Inspection 93-80 (all units)                                              X                    X              X                      X
{
Surveillance A-374 " Verification of LORT Attendance"                                                              X (March 1992)
Audit A60516 " Operations Activitics for MP-3" (March 1993)    X                                  X                1 I
Audit A21062 "MPI Unit StaffTraining" (April 1993)                                                                X                      X NRC Root Cause Analysis of MP-1 LORT 1991 Unsatisfactory                                          X                                      X Program Detennination NRC Inspection 96-05 NU Assessment ofNRC Issues                                                    X                                      X ACR M2-96-0657 Failure to apply SAT at Unit 2 (Nov.1996)      X                                                                          X Self Assessment of MP-1 LOIT (Jan.1996)                                      X                                                            X Scif Assessment of MP-l LORT (Jan.1996)                                      X                                                            X Team Assessment of1995 NTD Activitics                                        X                  .                X ii 1996 NTD Program Coordinators Assessment ofTraining            X            X                    X              X                      X Programs 1996 INPO E&A Visit                                            X                                                  X                      X Im" $ M Nhi'*' r        mr                                      18
 
i
                                                                                                                                    ~
l 4.4 EXPERIENCE AND STAFFING.                                                                                                    :
1 The experience level of the key individuals associated with the LOUT program played a signi5 cant role in this                    l event. As a result of frequent personnel changes at all levels during the conduct of the program, staff experience                '
was diluted and important historical contexts were lost. Many of the decisions made and actions taken can be directly related to this lack of experience. In addition to experience, overall staffing levels of Unit 1 Operator Trauung was also a factor in this event.
4.4.1.1 Organintional Change _s Organintional changes prior to and during the LOUT program Dese                                  Desenpesee of Change                            Outgotrig      Replacessent imeurnbent 1095                ASOT accepts possuon in operauons. No replammern is made due to re.orraruzation Nant I/96                New LOLTI program coordmator assigned                            None 3/96                One astructor land off.                                          N/A
                                                                                                                ~
396                  New Operauens Manager far Unrt I appoimed                              . .
496                  Trauurr Director transferred to Oversight Group                  t.
6.96                Ops Trammr 1; anon suns LOUT No replammern made                    ,              _
696                  Operations Mentor appotmsd (Collate al Shtft Mgr. dutv) 496                  Opertuors Trammg Supervisor takes on sne positsort .              t
                                                                                                            ~
7/96                LOUT Coordmator acospis dEareas pasaaort 896                ASOT posnson filled but Assigned to temporurv protect 896                  1 instrumor assigned to plant procedures project 9.96                2 Contramors bared                                                N/A 9 96                Operauons Tramme Manater removed no reptamment made 1096                One instructor termmated                                          N/A i196                Director of Unit I replaced                                                                    .
Key individuals with programmatic responsibilities to the LOUT program were removed, transferred or promoted during the term of the program.
Staffing Changes in Nuclear Training During LOUT Position ~                l    Held by:          Replaced bv.-        Date:
V.P. Eng & Trng                          .                                      10/96 Director Nuclear Trauung                ;                                      4/96
                                                                                                        #                              I Manager Ops Trauung                                        Not Filled          10/96 Super. Ops Trauung                                                              4/96 Asst. Super. Ops Trauung                Vacant                                  8/96                                            i LOUT Coordmator                                                                    u96 Paul DnB . Il Fotnery,19r7 19s6 MP.I LDLfr NRC Ee Fehmus DLT y9
 
1  .                                                                                                                                      )
!e i                                  Staman Channes in Unit I Durina LOUT Panaars:              Held by:          Replacedby:    Date:
.            Recovery OfLcer                                            10/96 1
Unit Ops Drector                                            11/96
                                                            ~
ops Manager                                                3/96 i            Asa. Ops Manager                                            4/96 j              Ops Liaison                '
Not Filled      6/96 l
i 4.4.1.2 Unit 1 OTB Experience Summary.
Nine instructors started in Unit I trammg over the period from December 1992 through April 1994. This staf5ng                !
increase was driven by the poor performance seen in Unit 1 Operator trainmg in l'971 and 1992. At the time these            {
individuals were hired, an experienced cadre of managers, supervisors and instructors was in place. In the recent            !
past, the majority of these experienced individuals have been promoted or transferred out of Unit 1 OTB. As                  ;
Me*A below, while overall trammg experience of the astructors is adequate, there is little actual Unit 1 or                  '
commercial operating experience. Additionally, both supervisors had less than 1 year on thejob during the LOUT program.
1 Unit 1 Operations Trammg Branch (OTB) Experience Summary (12/31/96)
              # ofSupervisors                                                          2                                                  1 Average # years onjob as Ops Trng supervisor                              .75
              # of Supervisors with Unit 1 operations expenence                        0                                                  i
              # of Supervisors with commercial operations experience elsewhere          0
              # ofInstructors                                                          9 l
Average # years onjob as Ops instructor                                  3.3
              # ofInstructors with Unit I operations expenence                          1
              # ofInstructors with commercial operations experience _cisewhere          1
              # of Contractors temporarily filling approved vacancies                  3
              # of approved vacancies not filled                                        2 m.
Emm1Duma . Il paksusy.39r7 y
                                                                          ~V 39H MP414UrMLC sammmmann7 stems RT W
 
    ~
4.4.1.3 Stahg Levels and History
        . As of Dec. 31,1996, Unit 1 OTB Authorized Stn%g totaled 17; 2 supervisory positions and 15 instructors.
        . As of Dec. 31,1996, Unit 1 OTB Actual Staffing totaled 15; 2 supervisory positions,9 utility instructors and 3 contractor instructors (one instructor is temporarily assigned to an on-site procedures project). As of 1/1/97 there were 5 vacancies posted.
The chart below displays Unit 1 OTB authorized staffing, qualified instructors and actual complement through the period of the LOUT class.
Unit 1 OTB Staffing History Sw
__                                                              Jan..Dec.1996                                                    l
                                                                                                                  + Total Staff 18
                                      # Leaving    M# Hired    * *
* Authonzed        ^ Qualified instructors r~
16 ,                                                                            ,'
a n ...        ..................a........g
                                                .    =    =        -
                                                                                                          =    -
l 12 10 8
6 4
r.
Jan        Feb.      Mar    Apr  May      Jun      Jul  Aug    Sep          Oct  Nov      Dec      , _
  ~
Unit 1 OTB staff was in flux immed:ately prior to and throughout the LOUT program. The net result of these changes was to weaken the depth and breadth of knowledge of the entire staff. Some of the significant facts that can be related to inexperience are listed below:
          . He LOUT coordmator stated that he was unaware that the accredited program had a 13 week OJT requirement.
* The LOUT Coordmator did not utilize procedures effectively to drive program changes.
* The newly appointed Operations Trammg supervisor did not recognize that the SAT was not being followed.                ~
during program design and revision.
Faul Draft - 11 Fetrusy,1997 1996 MP l LOL T NRC Imanananon Fahsu 13tT y
 
  .                                                                                                                      ~.
l
                  .      The Op- a---- Manager and Operations Tsmuung Supervisor did not rec ^=nire that the program revision was 4                        not in compliance with the SAT and tramag precobres
                  .      The Shift Manager assigned as class menterhadsopeevious experience in this role.
J
                  .      The Operations Trannng Supvm. failed to respond to warmng signals given to him by staff and candidates l                          regarding problems associated with the program
                  .      Key indmhals mvolved with the NRC exani teview had only muumal experience with the process.
                  .      The importance of accurate record keeping was act understood by the staff. This resulted in the inability to document completion ofall%.4 tramag by the LOUT mnAiA=tae.
4.4.1.4 Changesin Management Philosophy Frequent changes in Nuclear Orgamration Man =.nent have had a disruptive affect on the Nuclear Orgamratinn as a whole and trammg in particular. Associated with these personnel Ehanges were changes ranging from corporate visions to actual classroom practices. This constantly changing perspective has resulted in confusion at the lower levels of the organization about exactly what is and is not important.                              --
The impact of these changes is two fold. First, it has appeared to confuse the staffin assessing and establishing priorities. Secondly, these changes drew staff resources and attention away from the funA=mant=1s basics of training; particularly the trammg and qualificaten process standards set by the industry and the regulator.
l Following are examples ofinitiatives that were recently suspended,
                  .      Confidence Weighted Testmg                                        .      Inter Leafing
                  .      Standardized trammg maternal development                          .      MentalVisualization andImagery
                  .      Self Directed Work Teams The number and timing of personnel changes coupled with inadequate turnover and inadequate oversight led to a breakdown in program contmuity and quahty. Cd="--- with the decline in experience was the decrease in the number of available qualified instructors. While astruction was mamtained on schedule, it was at the expense of other important tramnag activities. These actmties included mamtenance of tramme materials, upgrade of exam banks and tunely review and revision of task lists.
Other aspects relating to avaariaara and stafEng are ==nifactaA in actual classroom and simulator trammg The staff and mstructors involved with the conduct of the LOUT program had no actual operatmg plant experience.
This was re --              t by both ==d=A=ta= and staff. 'Ibe following comments'come from LOUT andidata feedback and coordinator w.%-- '- =E "Having current Unit Supervisors coadingin the simulator would be ofbenefit" "The LOUT class requestedand strongly urgod the Operations Department to provide training support with experiemnd SROs" "The level oftraining experience has been compromised The manning issue out in training waspart oftheproblem. This will continue to be aproblem until they are adequately stafed" "The current resources dedicated to the LOUTprogram are not sufficient to meet the needs ofthe students" "On several occasions we would be tom toperform a step one way and then the next day a drferent instructor would aall us ur were performing this step wrong. "
INwiSYir.n mr                                                        U
 
i l
l l
1 Staffing and experience issues have been recogmzed in previous assessments both at Unit I and NTD as a whole.
Source                                                        Comment Self Assessment of Unit 1 LOIT
* Instmetors voth more operations experience are needed.            l (January 1996)                                                                                                            l 1
Self Assessment of Unit 1 LORT
* Broadening the experience base of all instructors is needed.
(January 1996)
Team Assessment of1995 Nuclear
* Instruction is skewed to the instructor's expertise, resulting in Training Department Activities                        incomplete learnmg.                      _
I (April 1996)
* Tranung focuses on passing tests not on operating the plant.
1996 Program Coordinators
* Many staffIr.ck recent plant experience                            ,
Assessment of Trammg programs                                                                                            1 (November 1996) 4.4.2     
 
== Conclusions:==
 
4.4.2.A              The frequency of change for personnel associated with Unit 1 Operations Trammg and the difficulty encountered in retmmng qualified and experienced staffis clearly affecting the ability of NTD to deliver quality trammg.
4.4.2.B              The succession plannmg process is ineffective.
 
====4.4.3 Recommendations====
4.4.3.A              Management should consider the use ofINPO Imnees or industry experts to function in the Unit 1 OTB chain of command until experience level of the incumbents can be raised to an appropriate level.
4.4.3.B              Management should review current conditions and practices v,tich result in difficulty of NTD to retam qualified and experienced staff.
4.4.3.C              Management should develop and implement an aggressive succession plan for all key positions in trammg (Tramme Coordinator and above).
4.4.3.D              Trammg management should take a proactive stance in ensuring that transition and turnover for individuals being promoted or transferred is completed.
rm . u ,,,n.y. im                                                    9 199e MP-1 LOLTNRC Exarmnanan FaGurus IRT                            ~
 
i      .
l
                                                                                                                      =
1 4.5 SELECTION PROCESS and CRITERIA FOR PROGRAM EhTRY AND NRC EXAM.                                                l 4.5.1.1 Candidate Selection Process Nine candidates were originally selected by MP-1 line management to participate in trammg to gain their SRO j          licenses in order to strengthen the unit's work control activities. Prerequisites for entering the program are J
described in Section 3.0 of the LOUT Program Description (TPD), Section 6.2.4 of the LOUT Program Implementing Procedure (TPIP), ANSI 18.1-1971 and NUREG 1021. The TPD and TPIP also specify l          responsibilities for candidate nommation and selection. There was, however, no formahzed process for makmg the I          candidate selections. The original candidates consisted of five reactor operators and four STA's (" instant"
;          candidates). The reactor operators participated in Assessment Centers to evaluate their leadership and supenisory skills. The STA's were not evaluated. Shift Managers provided input to the Unit Operations Manager regarding l        the work history, perfonnance and qualifications of reactor operators. The Trammg Department was not involved in screenmg the candidates.
The IRT found indications that some of the candidates selected for the program may have been marginally motivated to obtain an NRC license:
                    . One candidate withdrew from the 1996 LOUT program during the firstEcek.
                    . One candidate refused remediation after failing the comparry exanunation. This candidate has since termmated employment with the company.
                    . Feedback which the IRT received from interviews with management, supenision and staff was not consistent regarding the quality of candidates selected.
On Jan.1,1997, shortly after the NRC license exammation, the MP-1 Operations Manager issued a new departmental instruction detailing the screening process for the selection of RO trauung program candidates. The manager also intends to issue a similar instruction for the selection of SRO candidates.
4.5.1.2 Selection for NRC Exam The IRT found several weaknesses in assessing the readiness of candidates to take the NRC exam:
1
                    . Verbal and written concerns from the LOUT Program Coordinator regarding emndid* readiness and      I the recommendation for an academic review board was not addressed by trammg department and unit management.
                      . The Unit Operations Manager and Assistant Manager assessed the candidates' perfonnance on their final simulator exam. The Program Coordmator recommended that an independent contractor be used to assess the readiness of the candidates to take the NRC exam.
                      . Section 5.4 of NTM 7.079 TPIP indicates that the Manager, Operator Trammg is responsible for approving license candidates for NRC exammation. This position was sacant during the last few months of the program
                      . Records could not substantiate that candidates had completed all phases of the program training requirements.
I N NE E "" -r m ar                                        24                                                -
 
i                                                                                                                                        t j
i      -
'j      ,
i j          4.5.2      Conclusions
;                      4.5.2.A            L.ack of a procedurahzed selection process allowed some margmally motivated i
candidates to enter the 1996 LOUT program                                                      ,
;                    4.5.2.B            Candidates' readme=s to take the NRC exammation could have been more thoroughly and objectively assessed.
4.5.2.C            'Ihe same criteria was not applied to select STA emndid= tac as upgrade eindid='a=
~
to assume the same SRO role in the plant.
a 4
!          4.5.3      Recommendations:
i
:                      4.5.3.A            Operations Management sha.!!d forms.lize the SRO candidate selection process                    1 l'
similar to its recent mstruction for RO e=adid='a selection. The process should                I include involvement and input from NTD.                                                        I J                                                                                                                                        !
;                      4.5.3.B            Renehmark industry "best performers" and develop a formal process to assess the                :
                                                                                                    ~
4                                        candidates' readiness to take the simulator exam.
4.5.3.C            Senior Management should reinforce its wMons of Trauung Department and j                                        Unit personnel in fulfilling their responsibilities for thoroughly assessing and
~
remaAi=tino the performance oflicense candidates.
i l
4 I
1                                                                                                                                        :
j i
1 l
l 1
i
}
4 i
d a
i f
r 5
1 1
i j
d 8
1 j          r o a . ii n    ,v '"'
19W MP.3 LOUT NRC Eammmansm 7ahses lit?                        25 3
I
 
4.6          MAINTENANCE OF PROGRAM AND STUDENT RECORDS
* 4.6.1        During the investigation of this event the LOUT program personnel records were found to be inadequate.
This represents a challenge to 10 CFR 50.120, Trammg and Qualification of Nuclear Power Plant Personnel and 10 CFR 55, Operators' Licenses.
        . Documentation could not be easily produced to verify that student prerequisites and prior trammg requirements were met.
        . Documentation of student attendance and completion of LOUT program requirements could not be verified during the course of this investigation.
* Previous trauung (LOIT and STA) documentation of student attendance and completion of program requirements could not be verified during the course of this investigation.
        . Inconsistencies were noted on the tracking of student attendance and OJT time tracking.
        . The existing computer program - Taskmaster, was not used to track student qualifications.                        .
1
        . Excel spreadsheets are used to track attendance. The lesson indentfiers on, tee spread sheets are not clearly tied to trauung program description identifiers.
        . No "As Taught" schedule existed and had to be created for this investigation.
        . Here are no Trammg Program Control Committee Meeting Minutes describing or approving any changes or deviations from the accredited program.
        . Here is no requirement for an intemal or extemal audit to be conducted at the completion of the program.
        . OTBI-3, Operator Trammg Attendance and Performance and OTBI 5, Mninumnce of Trauung Records and Files have not been utilized.                        -
        . ACR Ml-97-0117 was initiated on 1/17/97 to document and further investigate student and program records for the 1996 LOUT Class.
4.6.2       
 
== Conclusions:==
 
4.6.2.A            Personnel and program records cannot be verified as being complete and accurate.
This presents a challenge to 10 CFR 50.120,10 CFR 55 and QA requirements.
4.6.2.B            Currently two computer programs exist to track personnel and program records -
neither is being used at this time.
4.6.2.C            There was inadequate oversight of personnel and program records by trammg and unit management as well as the oversight orgamzation.
4.6.2.D            Operator Traimng Branch staff, supervision and management are not following
                                                                                  ~~~
established procedures to maintain records.
 
====4.6.3 Recommendations====
4.6.3.A          Ensure that the investigation associated with ACR Ml-97-0117 verifies that all licensed operator trammg records are in compliance with 10 CFR requirements.
4.6.3.B          Implement the approved records management system.
4.6.3.C            Implement external program auditing at a frequency appropriate to ensure record retention requirements are being met.
Tsul Daft .11 Fehnaary,1997                                        sy 1996 MP ! LCXIT NRC Imamsmaan Fahns RT                              -
 
1 1
t
!                  4.7            TRAINING MATERIALS.                                                                                              ;
:                                                                                                                                                  l 4.7.1          he IRT identified the followmg issues relatmg to trauung matenals:
e    Many system texts are out ofdate.
g l                              e    PDCR (plant design changes) and procedure changes are not communicated to Trammg                              1 j                              e    NTM 1.04, Comnutment Trackmg is not utilized. No means of verifying that trauung material is current j                                    other than an instructor re-researching material prior to *=rhing lesson or distributing text.
t e    Lesson plan orgamration does not follow text orgamzation. This interferes with effective leanung d
!                              . Exams and lesson materials are not updated concurrently. Sndents have been tested on material not j                                    addressed in the lesson or in class.
j                              e    Exam bank questions are not in the industry standard four part multiple choice format.
a l
                                . He cognitive level of many questions is not representative of current industry norm.
                                . Picture boards of control panels are out of date, lending to negative tramng.
f
                                . No material is provided for shift managers to effectively implement the on-shift time of License candidates.
                                                                                                                                                  )
4.7.2          Conclusions                                                                                                    j 4.7.2.A                    He condition of trauung material does not support a program that heavily depends    i on self-study of systems and administive topics.
4.7.2.B                    Communications between the line orgam7= tion and trauung have broken down in regard to plant design changes and procedure revisions.
4.7.2.C                    Lack of procedural compliance for commitment tracking on MP1 has lead to            !
dagr=dation of trauung material quality.                                            i 4.7.3          Recommendations                                                                                                ,
4.7.3.A                    Review and revise all trammg material to include PDCR and procedure changes.
4.7.3.B                    Evaluate the issue of trammg material quality for applicability to all trauung programs                                    --
1 4.7.3.C                    Man =& should ensure the requirements of NTM 1.04, Comnutment Tracking are being met.
4.7.3.D                    Ensure that instructional matenals are adequately upgraded and maint=inai before being used in tranung 4.7.3.E                    Develop and implement a process that ensures that applicable PDCRs and procedures changes are communicated to NTD in a timely manner.
r t o.a . inenw.im
:=we.:tourwac1                    nr.in rr                                              9.7
 
      .                                                                                                                                                            j i
i i
i                                                                                                                                                                  i r                                                                                                                                                                    i 4.8        NRC EXAM, REVIEW AND VALIDATION.                                                                ~
]-
4.8.1      The IRT idennfied several issues in the review, validation and adnunistration of the NRC exam.
i
                                  .                                                                                                                                  l The exam bank for Unit I contains numerous three distracter questions - Confidence Weighted Testmg                          '
i                                        (CWT) format rather than the four part multiple choice used by the NRC.
i'                                .
The exam questions supplied to the NRC did not include appropriate references nor do they include l                                        knowledge / ability (K/A) values.
t e
l                                        Exam question information in the " Taskmaster" system that was reviewed was incomplete.
e
!                                        Nuclear Traming Manual (NTM) 4.01 - Evaluate / Develop Test Items, was not consistently implemented.
l                                        ' Ibis procedure requires the documentation of K/A's, objectives, and references.
                                                                                ~
{                '            -
There is no formal method for review of the NRC wntten exam. The process was dependent on staff l                                        experience to provide an ad~a>=*a exam review. Overall experience of the reviewers was inad~a>=*a j                                        Only one of four reviewers had previous experience in reviewing an NRC exam.
f                                .
There is no formalized method of selecting validators to ensure that a rhesentative sample of the target i                                        audience validates the exam.
e j                                        The final validation was conducted by two experienced operators who expressed concem over the length 4
of time it took to complete the exam Both noted the difficulty of the ma-. .44ive questions, but                          i
!                                        assumed that the candidates would be more prepared for them. Both concluded that the exam was                              i lengthy, but acceptable.
;                                e      Questions about the length, scope, and difficulty of the exam ra==iaad, but the staff was uncertam of what                I j                                        actions could be taken to address the situation.
i                                                                                                              .
e 4
During the exam, an extra half hour was given to the candidates to complete the exam: Candidates still
!                                        expressed that they felt rushed during the exam. Only one raad d='a completed the exam at the four hour i                                        mark.
o e      After the exam,10 exam questions (10% of exam) were challenged by the unit.
{
l                                      ' At the request of the IRT, an iadapandant assessment of the wntten exam was conducted by Nuclear l                                        Trammg and Educational Services, Inc. (see #=rh= ant III). In his evaluation the marh=>* concluded -
:                                        "It is my conclusion that while there is reason to question the validity of the licensing exam due to its l                                        length, the information available suggests that the exam, however d@ cult, was valid."
                                  .      A review by 3 of the IRT members (all industry peers and previous SROs) cocIcYu'ded that the wntten j                                        exam, while lengthy, was within the NRC testmg guidelines.
                                                                                                                                                      ,-o'-  .
                                                                                          'o  , ,
* 1                4.8.2     
 
== Conclusions:==
 
4.8.2.A            The exam bank provided to the NRC did not meet the guidelines of NUREG/BR-0122, "Ey==inars Handh~+ for Developing Operator Licensing Written E--.. =^ ions" and could not be used by the NRC.
4.8.2.B            Lack of a formali=d exam review and validation process contributed to an ineffective review and validation of the NRC exam.
FaalDumit . Il Fahrusy, test                                      my o 19H MP.110UT NRC Imammunnen Faiheus IRT                              D
 
1 7
4 4.8.2          Conclu.sions: (cont) j                                            4.8.2.C                          lick oflicchsing exam review experience by reviewers contributed to an ineffective
!                                                                              review and validation of the NRC exam.
4.8.2.D                          Line ownership of the exam review and ulidation process needs improvement.
;                                            4.8.2.E                          The review and validation process took place simultancously during the same week, i                                                                              thereby reducing the effectiveness of the review and validation process.
1
 
====4.8.3 Recommendations====
1 7
!                                            4.8.3. A                          Upgrade exam bank to industry standards? Implement NTM 4.01 to ensure exam questions l                                                                              are properly constructed and documented.
i i                                            4.8.3.B                          Benchmark industry "best performers" and adopt practicesisto a formalized process for the revim and validation of NRC exams. Provide expectaiEns and trauung to individuals 4
scheduled for exam review and niidation.
1                                            4.8.3.C                          Line Management should incorporate exam functions into their resource scheduhg to i                                                                              ensure that appropriate personnel who are representative of the target audience are i                                                                              anilable for NRC exam review and validation.
j                                            4.8.3.D                          Separate the review and validation of NRC exams processes so that they do not occur concurrently.
1 I
i
                                                                      ~
l i
.                                                                                          .                                    ._ ..      . .i 1
I h
1 j
e E
4 l
i 7 mal Diaft . Il February.1997 1996 MP.! LCXJTWkC IJanusman Fatmus RT y
l
 
4
,      5.0          EVENT CONSEQUENCES                                                                                                  l The consequences of the LOUT failures as listed below are the result of past, present and expected outcomes of the i      factors discussed in the previous sections. These consequences entail regulatory as well as direct and indirect 4
financialimpacts.
1 t
)      Regulatory Impact                        . Increased scrutiny from the NRC in the form of trammg inspections is likely.
* Action by the National Academy of Nuclear Trammg is possible.
1 d                                                                                                                                          l Direct Costs:                            .
IRT investigation - NU and contractor IRT member salaries; lost productisity of      I interviewees and contributors.
                                                                                                              ~
l
,                                                    NTD staff salaries for NTD review ofIRT issues in other programs                    l 1
1
* NTD salaries for trammg material and process upgrade.
* Student and instructor salaries required for remediation.
l i
Indirect Costs                            . He event required the attention of high level managers affecting productisity and oversight in other areas.                                                            !
3 4
4
* The event had significant morale impact on the staff and emndid='~ involved.
1                                                +  The event will erode the credibility of the NTD with INPO and the NRC.
i
,                                                e  he event caused additional negative publicity toward Millstone in the midst of I
already strong public opinion.
                                                  .  "Get Well" actions taken as a result of this event will affect the ability of NTD to i
                                                  . meet the plant's trammg needs.
l e  Potential impact on recovery and restart due to delays in licensing additional SROs.
I a
i 4
Fmal Deaft .11 February.1997 n
v 1996 MP 1 LO!T NitC Emanenssum Facaris 13tT
_c
 
                                                                                                                            =
4 1
ROOT CAUSE, LESSONS LEARNED AND CONCLUSIONS
!      6.0 l
6.1        Root Cause A lack of management oversight and accountability by management, supervision, and staff coupled with I
the ineffectiveness of the corrective action program to remedy previously identified,long standing issues
,          led to the degradation of training program quality.
6.2        Lessons Learned
            'Ibe following are the principal lessons leamed of broad organnational significance:                              _
l Afanagement oversight shouldfocus more on specific activities within theirfunction.                                \
l All levels of management must be more focused on specific activities within the function. Managers should          j carefully review decisions and plans until they have solid assurance that the sub_ordmate functions are inanaged and supervised to their expectations.                                                    .._
Strict adherence to established procedures and policies provides a barrier to program quality                      j deterioration.                                                                                                    i l
Compliance with NTD procedures and mstmetions ensures adherence to the Systems Approach to Trauung                ]
Trauung and line management should ensure that suborrhnates understand and are held accountable for
                                    ~
implementation of policies and procedures.                                                                        i Ineffective Corredive Adion Program.
Many of the basic causal factors contributing to'the un==tiefnernry performance on this exammation were found to be similar to those that precipitated previous failures across the three Millstone units. These unresolved weaknesses in operator trammg had been identified in numerous self-assessments, QA audits, INPO enluations        j and NRC inspections and should have been viewed as precursors to this event. Had these issues been remedied in a timely manner the negative outcome of the LOUT class might have been prevented i
Cross discipline and cross plant evaluation and corredion ofidentifiedproblems should beperformed                l Problems and issues identified in one area result from many casual factors. In some cases these factors exist in  i other areas, a=paA=Uy where the function is similar, or common procedures are used. Self assessment              l effectiveness will be increased when identified problems are evaluated and corrected cross discipline or cross unit.
Effeds ofExperience Dilution Personnel changes can have a disruptive effect on organnanonal contmuity. The change management process for personnel important to safety (Operator Trammg for example) should warrant similar. evaluation and consideration given plant hardware changes. This does not have to be taken to the extremes of a 50.59 evaluation, but management attention to the effects of personal change should be commensurate with the importance of thejob. In the 1996 LOUT event ALL personnelinvolved with the design and impt=- 4i;on of the program were changed at least once. The results of this experience dilution were evident in decisions made and actions taken. Ensuring the contmuity oflong term programs should be a priority for management.
TsdlDesR . H Febnary.1997                                                                                            ~
j-1996 MP 3 LOUTNRC Emmmenen Fatams DLT
 
l l
1 i
l 6.3          Summary of Conclusions                                                                                                                              I 4.1.2 Management Oversight Conclusions.
4.2.2 SelfAssessment and Corrective Action Conclusions.
,                                                  4.3.2 Systematic Approach to Training Conclusions.
4.4.2 Expenence andStaffing Conclusions.                                                                        . -
4.5.2 Candidate Selection Criteria Conclusions.                                                                    ,
4.6.2 Training Records Conclusions.                                                                                l 4.7.2 TrainingMaterials Conclusions.                                                                                l 4.8.2 NRl' Fmm, Review, and Validation Conclusions.
The following list Sununanzes conclusions made during the IRT investigation:
4.b.A                  Management ovmight of the 1996 LOITT program was ineffective in idenufying and correcting challenges and i                                          deficiencies in the presenting of quahty trauung Management rmssed numerous opporturuties to ensure that training delivery was of the highest quality.                                                _
l                  4.1.2.B                M=n=-maar failed to consider previously idenufied ovmight issues (across all units) as *' warning flags" and adjust d
its actions appropriately.
4.2.2.A                The Self Aennent and Corrective Action process has not been used effectivelyto identify and correct weaknesses in traininf,                                                                                                                i 4.2.2.B                The Self Auenment and Corrective Action process does not systematically assess past problems to ensure the                  j
,                                          adequacy and sustamabihty of past corrective actions.
4.2.2.C                The Self Asserament and Corrective Action process in many cases focuses on a single plant and does not consider if
_ _ problems identified in one unit are present elsewhere.                                                                          ,
4.2.2.D                Some saeaments in the past, including the accrednation self evaluation report, did not include line managemern ownership.                                                                                                                  ,
l 4.3.2.A                The ongmal accredned program was revised without sufficiera auention placed on accred4tation , -          - =;
4.3.2.B                hTD pr=Awas were not used to ensure a systematic process was followed 4.3.2.C                %e analvsis and needs ==eument for program redesign was not '-- - 4 and therefore cannot be verifiect 4.3.2.D                The hTD staffinvolved with design and revision were unaware of accrednation -            - - = =
4.3.2.E                Evidence of weakness in aaA--aadag and applying the SAT is evident elsewhere in the b7D.
4.4.2.A                The freqJency of change for personnel associated with Unit 1 Operations Training and the difficuhy          e-d in retaining qualified and experienced staffis clearly affecting the ability ofNTD to deliver quahty training.
4.4.2.B                The succession planning process is ineffective.
4.5.2.A                1.ack cf a proceduralized selection process resuhed in some margmally sneuvated and quahfied candidates entering the 1996 IDITT prograza 4.5.2.B                Candidates' readmess to take the NRC ernmmation could have been more thoroughly and objectively assessed.
4.5.2.C                The same criteria was not applied to select STA condidates as upgrade candidates to assume the same SRO role in the plant.
4.6.2.A                Personnel and prograrn records cannot be currently verified as being complets an.1 accurate. This presents a challenge to 10 CFR 50.120,10 CFR 55 and QA requirements.
4.6.2.B                Currently two computer programs exist to track perscanel and program records . neither is being used at this time 4.6.2.C                There was inadequate oversight ofpersonnel and program records by trammg and unit managemera as well as the oversig!n organnatiart 4.6.2.D                Operatcr Traming Branch staff, + a and managemern are not following estabhsbed procedures to mamtam records.
4.7.2.A                Tbc condition of training matenal does not support a program that heavily depends on self-study ofsystems and admaustrative topics.
4.7.2.B                  Wumentions between the line orgamzation and trammg have broken down in regard to plant design changes and procedure revision.
FsnalDran II Fetuuary.1997                                                        q ay 1996 MP-1 lot.TT NRC f.amnsuaan Faibsus IKT                                        -
 
I i
j 6.3          Su==ary of Conclusions (cont.)
4.7.2.C                                                                                                                              l Lack of procedural comphance for commenant tradung on MP 1 has lead to degradsbon of trauung maasnal                  i quality.
4.8.2.A        The exam bank prowded to the NRC dul not meet the guidelines ofNUREGSR4122, "F.wamenerg }4annewde for
                                                                                                                                                        ]
Developing Operator licensmg Wntien F.mammations" and could not be used by the NRC.
1 4.8.2.B        Lack of a formahzad exam review and vahdation process conenbuted to an ineffective review and validation of the      ;
NRC exam.
l 4.8.2.C        Lack oflicensing exam review expenance by revwwers contributed to an I,-#rew review and validation of the NRC exam.                                                                                                                l 4.8 2.D        Line ownerdup of the exam review and validation process needs 4 4.8.2.E        The review and validation proces: took place sunuhansously & ring the same week, thereby reducag the effecuveness of the review and validation process.
Ge6 b      e l
I l
1 i
I 1
4i:                                                    l Faal Draa . Il 7sarumry. tw?                                                a 1996 MP-1 t&*r NRC s- patues RT                                            O3
 
      . .        - . - . - . .                - - - -- - - .                                          -    .-      .~ . .- - . ..                          - . . - . -
    .                                                                                                                                                                                  )
I i
a                                                                                                                  .
3 l                7.0
 
==SUMMARY==
OF RECOMMENDATIONS 1
:                                                  ES            Recommendationsfrom the Executive Summary
^
4.1.3 Management OversightRecommendations.                                                                                        ;
;                                                  4.2.3 SelfAssessment and Correctu Action Recommendations.
j                                                  4.3.3 Systematic Approach to Training Recommendations.
l                                                  4.4.3 Experience andSta[fing Recommendations.
l                                                  4.5.3 Candidate Selection Criteria Recommendations.
l                                                  4.6.3 Training Records Recommendations.
:.                                                  4.7.3 TrainingMaterials Recommendations.
4.8.3 NRC Exam, Review, and Validation Recommendations.
j                The following list SummanzeS the IRT's recommendations for corrective action:
.                        ES-1              Draw upon INPO loanees or industry experts to temporardy funcuan in the MP-1 Operator Training Brandi urail the experience level of the incumbents can be raised to an appropnate level.
j
!                        ES-2              Conduct m depth reviews for applicabihty ofissues from this investagation to other 6ining prograns for all Mdistone Units. Consider temporary "stop-work" orders for those prograrns deternuned to be na==pe=hle                                              )
4 j                        ES-3              Evaluate conirm a- of the lic-ad RO to SRO cancbdales based on ======tian resuhs. Evaluate how identaEed                                  ;
w.=ka==== of the candidates pertain to the m=*e of the hannaad stafE                                                                      i 5                        ES-4              conduct reviews ofpreviously id re#=d problem areas and expedne close.out of thoes relatag to accreditation, l                                          regulations, and trainmg quality. niin should iach* re-evaluating pnanum and resource ---a=            against other
;                                          Nuclear Traimes Deparanent excellence inniauves.
I                        ES-5              Impismata an effective correcove astion program which includes = ~ ~j for closure in a timely and effective
                                                                        '          ~
j                                          manner.                                                                                                                                    l
(                        W                  $ggior manag=nent dioidd Clarify aInd romfWoe espectaticEE ofTW and Opsatiom manag===re regardnig their                                  l r== pan =M*== for oversight of tranung. t=p1===* methods and man ==es to ensure that overngla is effective
{
j                        4.13.A              Review inchnery pd=ae* sad INPO id==Aad "best perfonners" wish nespect to =-g===* oversight Adopt best                                    ;
prachoss into overught process.
i.
4.13.B .            Reinforce with Trainhas and Opershons ==aag===a their . , '~              regarding overeda of tramang.
4.13.C            Fnahl=h high gwp=me=* regardag overught and i=pis==W teethods to ensure that overught is daran== mad j                                            frequent, and useful.
j                        4.13.D            Ensure abat overngla f==a=+ is --namad and addrenad aa==='=ely acmas all immes.
l                        4.23.A            Review industry smdance and INFO identdied '1:sst performers" with respect to self            and corrective j                                            amicsL Adopt best pracuces in the NTD self amesmans and correcuve acuan process.
I                        4.23.B            Conduct an ' urgency review' of previously id==#=1 probiern areas and expedne close out of those relateg to j                                            aceradam= and / or regulatory issues.
l                        4.23.C            Ensure correctrve actions are developed and wacked for allid-mined tranung issues.
l                        4.23.D            Review ofIsaivensu of past correctrve accons. Incorporate this review into the self ======am process.
5 4.23.E            Develop, commuucate and reinforce clear =*r==hons regenhng the Self a=a-naar and Corrective Amion process for all levels of ===g===nr supervmon and aaft f                        433.A              Evaluate eximing NTD procochires and instructions (OTBrs) and ensure these procedures meet NTD evp-emians for
:                    -                      the trammg process.
a
!                        4JJ.B              Revise NTD procedures and instrucuans (OTBrs) as regerest
;                        433.C              +;                and supervmon should ensure ar==ph==re with exuung procedures.
S                        433.D              Evaluese the as taught LJDUT propam against accrednauen standards and ensure that all accrahtahon j                                            were met.
l                        433.E              Develop and implement correcuve actions for the issues idme#wt in the recently perfonned internal asseurnent of 4                                            accredsstion creena.
I j                        433.F              hnplement the previously exatmg plan for revieweg the Symems Approach to Training as part of the 1997
;                            _                r'a===nns Instructor Training prognun i
FmalDm8 . Il Fatmary.1991 19N MP.I LOL7TNRC bummunen Fatues ELT q
s 1-                                                        _-          -      _      . , - .
 
e i
1 7.0
 
==SUMMARY==
OF RECOMMENDATIONS (cont.)                                                                                                              I 4.43.A        Management should canwier the use ofINPO Lannees or in&utry experts to funcuan in the Unit 1 OTB chain of command until expenance level of the incumbents can be raised to an appropnate level.
4.4.3.B        Management should review current conditions and practices which result in difficuhy ofNID to retain qualified and expenenad staff.
4.43.C        Management should develop and implement an aggressive succession plan for all key positions in training (Training Coordmator and above).
4.4.3.D        Training management should take a proactive stance in ensunng that transition and turnover for individuals being promoted or transferred is congleuwt 4.53.A        Opersuons Managanent abuld forrnahn the SRO can&date selection process similar to its recent instruction for RO candidate selectiort The process should include involvement and irgut from hTD.
4.53.B        Ben
* nark =Amr) "best performers" and develop a formal process to assess the candidates' rea& ness to take the simulator exant
        %              4.53.C        Senior Management shouMforce its al-w;ons of Training Departraent and Unit per.onnel in infallmt their responsibilities for thoroughly assssamg and ramaAatmr the performance oflicense candidates.
4.63.A        Ensure that the investigation assocuted with ACR M1 97-Oll7 verifies that all bcensed operator trammg records are in comphance with 10 CFR requirements.
4.63.B        Irnplement the approved records manarement systent 4.63.C        Implement external program auditmg at a frequency appropnate to ensure record retention requrements are being met.
4.73.A        Review and revise all trammg matenal to include PDCR and procedure changes.
4.73.B        Evaluate tins issue of tranung matenal quahty for applicabihty to all trammg programs.
4.73.C        M1 should ensure the requirements ofbTM 1.04, Consanment Tracking are being met.
4.7.3.D        Ensure that instrucuonal matenals are adequately upgraded and === amad before being used in tranung.
4.73.E        Develop and unplement a process that ensures that applicable PDCRs and , - *- changes are --anad to NTDin atunely manner                                                                                _
4 83.A        Upgrade exam bank to industry standards. Implement NTM 4.01 to ensure exam quesuons are property constructed            i and docur==nad                                                                                                          l 4.83.B        Benchmark industry "best performers" and adopt practices into a formalized process for the review and validation of NRC exams. Provide evpaerah and training to individuals ach*Adad for exam review and vahdatiert                        1 1
4.83.C        Line Monarement should incorporate exam functions into their resource erheader to ensure that appropnate
                                                                                                                                                              ]
personnel who are ..,      i Gve of the target ausence are available for NRC exam review and validatient              ;
4.83.D        Separate the review and validation ofNRC exams processes so that they do not occur concurrently.
an. .
Final than . t1 Fdruary,1997                                                ec 1996 MP.t LOLTNILC 8- Fahes IILT                                            JJ
 
l 2
:m -
l 8.0          ATTACHMENTS                                                                                          l l
:      ~      -
l l
1                                I                Summary of Previously Identified Issues II                References j
III                NETS Review of NRC Examination
?
IV                Evaluation of LOUT Issues Against NRC & INPO Requirements 1
V                  Analysis of NTD Procedures VI                Barrier and Failure Mode Analysis            -
VII                Conclusion and Causal Factor to Recommendat on Matrices
]                              VIII              Previous Pass Rates MP-1 LOIT & 1996 Exam Results IX                IRT Charter X                  accredited Program Sequence                                          1 l
:                                                                                                                      1 i
e a
_m.
* s i
i i
j Fmal Draft . II Februsy,1991                                      3g 1996 MP 11.DUTWILC f.nammenon Faksus IRT                          JV
 
  - . =      ,              -        .        . . _ .    -.              ._.  .      .      -.
J Attachment I Summan of Previously Identified Training Issues Listed below are summaries and paraphrases of previously identified trammg issues. Some of the issues listed below originated in MP-2, MP-3, or CY - and include LORT issues - but are included due to the similarity of conditions found in 1996 MP-1 LOUT program.
ACR 11037 Adverse Trend In Weekly Operations Simulator Evaluations for MP-1 Root Cause Analysis
,          (October,1996) e  Reduced training contact time contributed to crew failures.
Standards for performance and evaluation ofindisiduals and crews had degraded.
Lack of ownership by both the Shift Managers and Trammg Mentors contributed to degraded crew performance.
1995 /1996 LOIT Class Company Final Examination Root Cause Evaluation (September,1996)
* Ineffective strategic planmng (in trammg).
* Lack of Self Assessment and Corrective actions.
* Human e: Tor or misjudgment on the part of supervision.
          . InsufHeient oversight by Operations manager.
* InsufEcient oversight by trammg manager.
* InsufBeient oversight by supervision.
Inadequate verbal or written communication between trammg staff and management and plant management.
NRC Inspection 93-80 (June 1993)
Control measures do not address how trammg for important of significant procedure changes and license amendments is given.
Control measures do not address when a root cause analysis for major trammg deficiencies is to be conducted.
* Some licensed operators failed to attend required LORT.
Audits of operator trammg required by technical specifications were not done (Unit 1 &3).
Limited distribution of TPCC minutes and lack of detail in minutes make it difBeult for others to monitor activities of the TPCC e
TPCC committee size limits the committees ability to provide checks and balances among all aspects of the trammg program.
* A TPCC member approved his own trammg exemption.
The problems . ..will continue to be an issue until the experience level of the new staffis adequate . .
* Exam banks were not kept up to date.
          . 1990 assessments of LORT failed to identify problems contributing to LORT failures in 1991 and 1992.
E.:$vTiP-~ w. ar                                            - .-
 
                                                                                                  =
Attachment I Summary of Previously Identified Training Issues (continued)
Surveillance A-324 " Verification of....LORT Training Attendance (March 1992)                      l e
Attendance records were found to be of poor quality. (NRC initiated concern)
Audit A60516 " Operations Activities for MP 3 (March 1993) e  Management did not observe simulator performance.
Topics not covered in LORT were not included in sample plan (NUREG-1021).
Exams were not time validated (WREG-1021).
Task lists did not differentiate those tasks that require continuing trammg (NUREG-1220).
Learnmg objectives did not have conditions, actions and standards (NUREG-1220).
* Lesson plans did not include the characteristics described in NUREG 1220.
e  Test items did not cover all the learnmg objectives.        ~-
                                                                                      .2 Audit A21062 "MP1 Unit Staff Training" (April 1993)
At least 38 individuals did not attend all sessions required.
  . Training effectiveness reports were not issued in 1990 and 1991.
* Management Simulator observations were not made.
* Monthly trammg reports were not issued.
  . Instructor observations were not completed annually.
NRC Root Cause Analysis of Unit 1 LORT 1991 Unsatisfactory Program Determination ( June 1993)
  . Inadequate learmng objectives with respect to high standards.
* Poor practices not corrected by TPCC.
  . Inndequate evaluation criteria forjob setting.
Inadequate audit and evaluation and inadequate external experience review e  Inadequate review of external experience.
* Inndequate review ofinternal experience.
NRC Inspection 96-05 NU Assessment of NRC Issues (December,1996)
  . Evaluation process lacked accountability.
  . Lack of Shift Manager ownership.
* Lack of critical evaluations and crew feedback.
  . Weak higher order examination skills.
  . Lesson plans and texts require anh--mant.
  . Exam bank needs enhancement                                                              ..
ACR M2-96-0657 Failure to apply SAT at Unit 2 (November,1996) e  Plant evaluation feedback is not being consistently evaluated for trammg impact.
  . Task lists have not been routinely utihzed to keep progruns current.
Identified plant and industry events have not been consistently incorporated in trammg
* Trammg for OTT instructors has not been conducted for some OJT instructors.
E .h srT l i r e at                                                                              '
 
e Attachment I Summary of Previously Identified Training Issues (continued)                      I l
Self Assessment of Unit 1 Licensed Operator Initial Training ( January 1996)
        . Need to upgrade adnunistrative training identified.                                                                l
;      e    Candidates need more meetings with and observations by OPS management.                                    " ' ~
Task list needs to be reviewed and irrelesunt tasks deleted.
4      . Instructors with more operations experience are needed.
        . Some OJT and OJT cards were not effective.
        . A wider variety of simulator scenanos are needed.
Self Assessment of Unit 1 LORT (January 1996)
        . The pTant needs to become more aware of their crew's remediation.
Improvements are needed in identifying candidates who are at risk in failing.
        . Broadening the experience base of all instructors is needed.                                  .r
        . The JPM process needs to be made more interesting.
4 Team Assessment of1995 Nuclear Training Department Activities (April 1996)                                              l More emphasis needs to be placed on industry events and trends in training.                                        '
        . Communication between customers and instructors is poor.
        . Confidence Weighted testing is not used as designed.
        . Instructors need more plant experience.
        . Trammg focuses on passing tests not on operating the plant.
Instruction is skewed to the instructor's expertise, resulting in incomplete leammg.                              j
        . The value of the Self Assessment Process is unclear to many NTD staff.
1996 Program Coordinators Assessment of Training programs (November 1996)
The report states that approxunately 50% of NTD programs are not being maintained. Either thejob                    l analysis or the task analysis is out of date.
Post Assessment feedback processes (I.ong Term) are not drising program content changes.
l
        . Industry events beyond SOERs are not routinely incorporated into the trammg programs                              i
        . Work observations are not factored into the programs Prerequisite knowledge requirements are not either identified or considered by the line prior to enrolling candidates in trammg programs Line Management and Trammg Supervisor Management observations of trammg are not adequate.
        . Attendance problem at training has been tolerated by both Trammg and Line Management.
        . Program Maiemance has not been done in accordance with the NTM e    Expectations set forth by management implied that podium contact time uss more important than material maintenance.
        . Lack of staff resources impacted material maimenance process. Budgets were cut reducing staffing without changmg work scope, streamlining processes, or incorporating technology to improve effectiveness. In fact a majority of groups were tasked to do more with less by simply being directed to work harder and longer.
        . Implementation of new practices such as CWT, focus groups, and self directed teams impacted the effectiveness of the Nuclear Trammg Department. Additionally this also sent a message that this was all that mattered, especially when compared to pro 6 ram maintenance.
        -:tw . n m.y.im 1996 MP 1 LCUT NR.C I.nammanan Faihres RT
 
Attachment I Summary of Previously Identified Training Issues (continued) 19% Program Coordinators Assessment of Training Programs (November 1996)
Many TPIPs are out ofdate. The staffdoes not view this as a customer service contract that they have with the plant, the NRC, and INPO.
  . Inadequate staffing exist to effectively mentor the candidates.
Weak planning skills and changing customer needs greatly effect the efficiency and the effectiveness of the Nuclear Trammg Department.
  . Taskmaster must be replaced. The units have too many open commitments to insure that trammg is provided with the most recent materials.
  . Many stafflack recent plant experience
  . Supervisors do not spend the appropriate amount of time coaching and developing their instructional staff.
  . Certified Instructor Trammg does not meet the staff needs.                        -
  . Individual development plans were not effectively implemented at all levels of management.
  . In several instances task lists and initial and continuing trauung programs are not current or are in the process of being updated.
  . Exam validation process is weak due to resource limitations. Several program coordinators believe that their exams do not discrimmate between competent and non-competent personnel.
  . Development and preparation times are inadequate and greatly impact the quality of service we are providing.
  . Labs and In Plant trauung are not observed by trammg or line management.
  . In several programs OJT cards are out of date or are in the process of being updated.
  . OJT and OJE are not properly monitored by line management orjob incumbents.
  . Job Incumbents are not provided refresher trammg on management's expectations regarding OJT and OJE.
  . Except for Requalification exammations, simulator trnmme is not observed by line and tramme management in accordance with expectations in the approved TPIPs.
  . Many simulator instructors need facilitator trnmmg and continuing refresher trammg in simulator mstructional skills.
  . Several aspects of simulator trammg appear disconnected from the plant (Reahsm).
  . Instructors do not have time to properly validate, curriculum test, or prepare for simulator tramme
  . Core models need to be upgraded and enhanced on all simulators to meet current industry erpartations regarding startup trammg
  . DR backlog impact quality of trainmg on MP 1 simulator.
  . Quality of critiques and use of freeze, backtrack, and replay need improvement.
  . NTD customer feedback process is weak and feedback indicates that in some programs feedback is not promptly acted upon.
  . NTD long term feedback process is not effective.
  . NTD self assessment processes such as monitoring trnmmg effectiveness and instructor assessments are weak.
  . Significant Events Evaluation Cnmmittee is not being used.
  . NUTIMS needs to be implemented to better track trauung wmminnents.
  . NTD does not benchmark programs ngnmq the industry me!DraA Il Fstuusry,199*
1996 MP-3 LOL%'RC        Fahses IKT
 
                                                                                                                                              -      .-    +
Attachment II Independent Review Team t
1996 MP-1 LOUT NRC Examination Failures REFERENCE DOCUMENT INDEX                                                                            >
i IDENTIFIED
  ~          '
DOCUMENT        -BY          DATE1 DOCUMENT DESCRIPTION
: NUMBER:      TEAM      . ENTERED.
            <    MEMBER -                      '
01        Muccino        U3d7            IRT Charter                                                                                            ,
02        Muccino        1/3/97          Resumes -IRT Members 03        Muccino        I/3/97          Organization Charts - Nuclear Training Group 04        Muccino        1/3/97              1996 INPO Millstonc Audit Report -Training Section 05        Muccino        1/3/97              19% LOUT Schedule 06        Muccino        1/3/97          MP-1 LOUT Program Description (Rev.0) 07        Muccino        1/3/97          NU LOUT Program Implementing 140cedure (NTM-7.079 Rev. 1) 08        Muccino        1/3/97          Examination development & Administration (NTM-5-02 Rev. 3) 09        Muccino        1/3/97        >
Evaluate / Test items (NTM-4.01 Rev. 0) 10        Muccino        1/3/97          ACR Mi-96-1024 LOUT Written Exam Performance (12/8/96)                                          ,,
1I        Muccino        1/3/97          ACR Mi-%-1025 LOUT Simulator & JPM Exam Performance (12/20/96)                                ,
12        Muccino        I/3/97                                                                                                        '
ACR M3-%-1154 MP-3 Operator Exam Performance (1I/20/96) 13        Muccino        1/3/97          News Article on Exam Failures *The Day'(1/1/97) 14        Muccino        1/3/97          MP-1 Weekly Simulator Failures - Root Cause Evaluation (10/28/96) 15        Muccino        I/3/97          MP-2 LOIT Company Examination Failure - Root Cause Evaluation (9/27/96) 16        Muccino      1/3/97            NRC Inspection report (EA 93-130) Millstone LORT Program (6/7/93) 17        Muccino      1/3/97            NRC NOV and Civil Penalty - Millstone LORT (9/2Q/93)
Muccino                                                                                      ' '
18                      1/3/97            Reply to NOV (EA 93-130)(10/19/93) 19        Muccino      1/3/97            Letter G. Meyer (USNRC) to B. Kenyon - Meeting on Training (12/6/96) 20        Muccino      1/3/97            Company LOUT Written Examinations & Scores for MP-1 Candidates 2i        Muccino      1/3/97            Previous Exam Results for LOUT,19%, Students 22        Muccino      I/3/97            NRC Written Senior Operator License Exam (12/2/96) 23        Muccino      1/3/97            Comments on NRC Exam - P. Hir.rs.. Lor..p to H. Miller (12/13/96) 24        Muccino        1/3/97            Student Feedback on,1996, LOUT Program 25        Muccino        1/3/97            Preliminary Written Exam Score Analysis 26        Muccino        I/3/97            Brief Biographics of LOUT Candidates NA"[nYle'i""    r.am m
 
. - - - . . -    . . _ - .  -        - - - ~ = - . . - -                        . - -. . - -        - . .-            . - .- .      - . - . _ . - - - . _ . .  -    -
i IDENTIFIED                                                                                              l DOCUMENT          BY                        DATE                                              DOCUMENT DESCRIPTION                                            ,
NUMBER          TEAM                    ENTERED MEMBER 27          Muccino                      1/3/97    DPUC Audit of CL&P Nuc. Operations - RC. Brown & Assoc. (12/13/96) 28          Muccino                      1/3/97    1991 LORT Failure - Corr. Action Plan - E. Mroccka to T. Martin (10/7/91) 29          Muccino                      1/3/97    1992 LORT Failure - Root Cause Investigation 30          Muccino                      I/3/97    1992 NRC LORT Evaluation 50-245/92-23 (I1/2/92) 31          Muccino                      1/3/97    1992 NRC LORT Inspection Report 50-245 (12/3 I/92) 32          Muccino                      1/3/97  NRC MP-1,2 & 3 Training Inspection Exit Report (5/12/93) 33            Gilbert                    1/13/97  Memo Dan Daigler(VY) to :                  / - Sim. Obs.                                                  '
34            Gilbert                    1/13/97  PECo SRO Exam 35            Gilbert                    1/13/97  PECo SRO Exam 36            Gilbert                    I/13/97  Susquehanna SRO Exam 37            Gilbert                    1/13/97  Susquehanna RO Exam 38          Muccino                      1/13/97  Catawba Operator Training Procedurc 3.0 Design and Development 39          Muccino                      1/13/97  Catawba Operator Training Procedurc 6.0 Training Programs                                                i 40          Muccino                      1/16/97  Candidates 1995 LOIT Attendance and Grades 41          Muccino                      I/16/97    1996 LOtfr Attendance Reports 42          Muccino                      I/16/97  Individual Training Program Descriptions for Instants 43          Muccino                      1/16/97  Candidates License Applications & Waivers (Form 398) 44          Muccino                      1/16/97  1996 LOIR Company Final Simulator Exam Results 45          Muccino                      1/16/97  9/30/96 Memo                  , , - Resources for LOUT 46          Muccino                      1/16/97              _ s Timeline on Events Impacting LOUT 47          Muccino                      1/16/97    1996 LOIR Diagnostic Exam and Results 48          Muccino                      1/16/97  Assessment of Quality of LORT Examinations (12/19/96) 49          Muccino                      1/16/97  GFS Scores for LOlJT Candidates (1/14/97)                    I, 50          Muccino                      1/16/97  NTD past Initiatives Status 51          Muccino                      1/16/97  1996 Continuing Instructor Training Attendance Matrix 52          Muccino                      1/16/97 , NETS Assessment of NU & Susquehanna Exams 53          Muccino                      1/16/97 " Unit I instructor Longevity P
                                                              \
7$hE[nTbhr.s m e
I
 
                                                                          -- . . . . - . . - - - . - . . ~ . _ . . . --. . - - -
IDENTIFIED DOCUMENT      BY      DATE                                                              DOCUMENT DESCRIPTION NUMBER      TEAM    ENTERED MEMBER 54      Muccino    1/16/97    ACR Mi-97-0109 MP-1 Operator Training Weaknesses (1/15/97) 55      Muccino    I/16/97 . Misc.1996, LOUT Correspondence from Staff 56      Muccino  1/16/97    MP-1,2 & 3 Organization Charts 57      Muccino    1/16/97    Exam Bank -Taskmaster Printout                                                                                                      ,,
58      Muccino  1/16/97    Task List to Determine OJT. (3/8/95)                                                                                                >
59      Muccino  1/16/97    Taskmaster Student Qualification Records 60      Muccino  1/16/97    NTD ExecIlence Plan 61      Muccino  1/16/97    MP-1 Ops Training Resource Schedule 62      Muccino  1/16/97    ACR Mi-%-0339 Corrective Action of QA Audit (7/30/96)                                                                                          ,
63      Muccino  1/16/97    ACR M2-%-0657 MP-2 NTD Procedure Compliance (I1/13/96) 64      Muccino  1/16/97    Pilot SAT Training Module (5/2/96) 65      Muccino  1/16/97    MP-1 Scif-Assessments (1/16/96 & 1/18/96) 66      Muccino  1/16/97    Operators K&A's 67      Muccino    1/16/97    MP-I LOIT/ LOUT Class IIistory 1992 ,1996, 68      Muccino    1/16/97    1995 NTD Assessment Team Report (4/12/96) 69      Muccino    1/16/97    Accreditation Renewal Self-Evaluation Report MP-1,2 & 3 (7/94) 70      Muccino    1/16/97    1/17/97 Briefing Document IRT to J. Hayer 71      Muccino    1/16/97    Attendance Roster for Simulator Sessions 72      Muccino    1/1667    Instinctor Training Program lini>'Anwriting Procedurc NTM 7.208 73      Muccino    1/16M7    T. Gilbert Memo to                        .
Info. Request (1/16/97) 74      Muccino    I/16/97    1996 LOUT Student feedback Correspondence 75      Muccino    1/16/97    1996 NTD Organizational Changes                                                                          .I 76      Muccino    I/16/97    August 1996 Simulator Schedule                                                                                                                i 77      Muccino    1/16/97    NTD Coordinator Scif Review (1/2/97)                                                                                                    f 78      Muccino    1/16/97    MP-1 Department Instruction I-OPS-8.06 Licensing School Selection Process (1/1/97) 79      Muccino    1/21/97    QAS Audit Report MP-1,2,3 & CY Licensed Operator Training 95024845 (4/26/95) 80        Gilbert  1/21/97    ACR Mi-97-Oll7 NTD Incomplete Training Records (1/17/97) 81        Dooley  1/21/97    Notes from 12/6/96 NRC Exit Meeting on SRO Exam 82        Dooley  1/31/97    OTBI-3 Operator Training Attendance and Performance Records                                                                                  i 83        Dooley  1/31/97    OTHI-5 Maintenance ofTraining Records and Files 84        Dooley  1/31/97    Memot-                            . y)- (Candidates) LOUT Program Performance iI/I3/%
NMYnD""    r    m                                                                                                                                                            i
 
n .
I IDENTIFIED DOCUMENT                                              .
BY. ~                                        DATE                                                                                                        DOCUMENT DESCRIPTION.                                                                                                                                                                                              ,
                -NUMBER                                          TEAM ~                          ENTERED.                                                                                                                    -f                                                                                                                                                                                                                f MEMBER                                                                                                                                                                                                                                                                                                                                                                            F 85          Dooley                                        1/31/97                                                                  NRC Letter to J. Thayer (LOUT Exam Results) 86          Dooley                                        1/31/97                                                                  Compilation of policies regarding Management Observation of Licensed Operators 87          Dooley                                        U31/97                                                                    Root Cause Evaluation: Management Observation of Licensed Operators t
i
[
i i                                                                                                                                                                                                                                                                                                                                                                                                                                                !
I b
i 1.
i I
L bulDraft - Il Fehery,199' 1996 Mf-l LOUT NRC Examureton Fednes IRT
 
4
                                                                                                                                          =
A'ATACHMENTIII NUCLEAR EDUCATION &
TRAINING SERVICES, INC.
1
!                                                                                                  FAX: (718)398 1659                          !
i
:                                                                                              VOICE: (718)398 1658                              i 2
i DATE        1/22/97      l                  .THIS IS PAGE 1 OF                          6 i
i i
i FAX TO Ken Muccino                                                                        i i                                                  FROM Mike Woram i
i l SUBJECT Licensing Exam Review
\-
i 1
i January 22,1997 Mr. Ken Muccino Northeast Utilities Service Company Millstone Station i
Waterford.CT
 
==Dear Mr. Muccino:==
 
At the request ofthe Independent Review Team, I have rr.ies ed the re Reactor Operator Licensing Exarnination administered at MillsDne i ;                          ,
8,1996. For an additional basis for comparison SRO licensing exam administered during the same quarter.
I
{                                              In reviewing these examinations, I considered two major criteria:
3                                  (1) i    '
Were the questions well writtec, addressing topi:s appropriate to thejc j                                  (2) i i      .
the examination in the allotted time?Could a candidate with a                  ,
2,
 
_ _ _ . _ _ _ _ . . _ _ _ _ -                                      . _ ~._..__-_ _ _ _ . _ .. _ _ _._. ._ _. _ _ _
1 l
$                                                                                    .                                                                        l 4
With regard to item (1), the appropdateness andacotistruction reservations asfollows:                                                              .
onlyminor of i                                        =
lt seems unlikely that the task behind Question 004 is one p personnel.
m .-
4 4
* j.
i However, it is unlikely that this minor stru candidate's ability to address the question.
4 i
                                            ._        In general the questions were wordier than they needed                                            to b j                                                    to share the widely held opinion that for a question                                  seems to be operatio ented, some i
i operations story must be told as part of the question. This pro    _
other      factors to make the examination unreasonably long.
in this examination.
\
,                                      Concerning item Q), the length of the exam, I found several causes foi The first. andleast significant was the wordiness of the quest            ,
!                                                  probably not be much cause for concern. Particularly becaus i^                                                of these questions is that they are slightly wordier than th
!                                                  certainly not outside the normal range for this industry.                          .
:                                                                                                                                                            i
!                                                  The second concem had, in rny opinion, a very large impaci noted that twenty three questions most probably required the u i
Generally, the use ofattached references results in a questioi                                              i
,                                                licensing exam because they tend to be ve                                                                    '
!                                                the Requal Exam section ofthe Exarniner's Standards                      .
re were (NUREG i.
actually twenty nine questions for which a reference would hav However, six of these were Admin type questions for whic
{
This is roughly the equivalent ofputting a two hour, twenty th
[                                                examination in the middle of the licensing exam. This leaves on i                                                candidates to address the remaining sewaty seven questions.
i*
* To assess this I performed the following analysis:T                                            .
i
,                                              a
{                                                        Operators and that the examination is be
\                                                        possesses the requisite knowledge 1o answer all questions correctly.
i-
* 4
!                                                      Since this candidate can be expected to score 100%, the lev assessed by deterinining how long this candidate would take to com l                                                    -
 
t examination.
l
                                    =
  "                                    To this end each question is given a score of I or 2. A score of 1 is knowledgeable candidate could be expected to answer the questi time than it took to carefully read the question. In nearly all cas are at the Recalllevel of knowledge.
A score of2 is given if the candidate will need to spend addition knowledge from several areas to figure out the answer. Th always Higher knowledge level questions.
The actual scores have debatable meanings in demselves and are m comparing examinations.
Using this scoring system 1 found that the Millstone licensing exam h The audit cram had a score of 129 and the other (non KEllstone) lice score of 142. As such, it is my opinion that based on content alone, the Mills licensing examination was significantly more time consuming that the oth examinations cited. The question by question details ofthis evaluation examinations is shown in Tables I and 2 below.                                                  i i
As a result ofthe three factors affectiiig thebgtli ofthe Millstone licensing It would be reasonable to question the validity of this examination. The                      ,
exam may have been too difficuh to discriminate between a competent a operator. It is possible that a competent SRO might fail this examination du                                  3 It is my understanding that two ofyour best qualified senior operators di in question prior to it being administered. It is my funher understand                        es were 82% and 84% and that they both finished the exam in less than four houI incumbem SRO to pass this exam. complete a ti:ne validation as w i
On the basis of the information that is presently available, i nust conclude th                      l examination was at the highest end of acceptable difficulty but at this time t that supports a conclusion that the examination was invalid. Imust poi your best qualified personnel score in the low eighties does not, in my opm' io j  definitive time validation. Ifyou wished to pursue the validity issue fde four result. mainstream incumbent (i.e. competent but not superstar) SROs wou
              ' due to its length, the information available suggest                        ,
Any conclusion to the contrary would have to be supported by a much more d validation that currently exists
                                                                                                      , _ ,          .,,-e- -
 
    ~                                              j l
l Yours tmly Michael G. Woram l
1 4
GW I
f                                            -
e O
                            ==
Pm 9
e 9
 
1 1                        .
Tatde 1. Millstone I SRO Licensing Exam - Evaluation Summary j                                                        Q    H    Pt  R        Q    H      Pt    R 1                                                                                                            Q    H    Pt          R      Q    H      Pt      R I                                                          1        1            26    /    2            51        1                    76  /      2 2    /    1            27    /    2            52        1                    77  /      2 3    /    1    /*      28    /    2            53          1                    78  /      2
:                                                        4        1            29    /    2    /      54    /    2                    79          1                              !
5        1    /*      30 i
1            55    /    2                    80    /      2 i                                                        6                /*
1            31    /      2            56    /    2                                                                :
i                                                                            -
81    /      2                              !
}                            . . . _                    7          1    /*      32          1            57  /    2                    82    /                                      i
!                                                                            -                                                                          2                              '
!                                                      8                        33 1                          1    /      58  /    2                    -83    /      2 9    /    2              34    /    2            59  /
:                                                                                                                    2          /        84  /      2        /
i 10              /*      35    /
1                          2      /    60    /    2                    85  /      1 il        1    /*      36    /    2                  /
61        2          /
I
;                                                                                                                                          86    /              /
;                                                      12 1            37    /    2            62    /    2-                  87    /
l                                                                                                                                                      2 l                                                      13        1              38    /    2      /      63    /  2                    88          1
        .                                              14        1              39    /    2            64    /  2          /        89    /    i        /
3 15    /    2              40    /    2            65    /  2
!                                                                                                                                          90          1 j                                                      16    /    2              41    /    2            66    /
:                                                                                                                    2                    91          1 17    /  2      /      42    /    2      /      67        1                    92    /    2        /
!                                                    18    /  2              43    /    2
:                                                                                                        68    /    2                    93    /    2      /
I                                                    19    /  2              44          1            69    /    2                            -
1                                                                                                                                        94          1 1                                                    20      /  2              45          1            70 i                                                                                                                    1                    95          1                            -
j                                                  21      /  2      /      46    /    2            71' ' /  2 I
                                                                                                                              /          96    /    2        /
22      /  2      /      47    /    2            72    /  2                    97    /    2        /
23      /  2      /      48    /    2            73        1                    98    /    2        /
i                                                  24      /  2 i
i 49 1 (/          74    /  2                    99          1      /
1                                                  25 /        2              50    /    2    I 75                              1m i                                                                                                                  1                          /    2
!,                                                Z        14 37 '10          E      19 44      6      E    18  43        4          I    18    42      9                        -
i                                                  H= Higher Level Question                                                              mummummumm-ummm R= Reference Required                                    69    166 i                                                  "=ReferenceNot Supplied            Pt= Time /Dimculty Score E                  29 1
 
!                                                                          Table 2. MiBstone 1 SRO Audit Exam - Evaluation Summary Q        H          Pt        R                  Q            H I                                                                                                                      Pt        R              Q      H        Pt                                  I
      '                                                                                                                                                                  R            Q        H          Pt      R
{                                            I        /          2          :                26            /          2          /              51                1
!                                                                                                                                                                                      76                  1
:      .                                  2        /          2                            27          /          2 52                1                77
!                                                                                                                                                                                                          1 3        /          2                              28                      i                        53                1      /          78                  1
;                                          4        /          2                            29
:                                                                                                                      1                          54                1                  79                  1
{                                        5          /          2                            30                      1                          55                1    /
6 80        /      2
                                                    /          2                              31                      1                          56              1
{
81        /        2 7                    1                            32 s                                                                                                                    1                          57    -
1                  82                I
;                                        8                    1                            33          /          1                          58                1      /    _    83        /      2          /
l                                      9          /          2                              34                                                                              ''
:                                                                                                                    1                          59      /      2        /
i 84        /        1        /
10                    1                            35 i                                                                                                                    1                          60      /        2                  85                1 i                                      11 i                                                            1                            36            ,
1                          61 1                  86        /      2 12                    1                            37          /          2                          62 i                                                                                                                                                                1                  87                1 i*                                    13                    1                            38                                /
1                          63      /        2        /          88                1          /
i k                                      14 i
1                            39                      1                          64                1                  89                1
!                                      15                    1                            40 1                                                                                                                  1                          65      /      2        /          90                1 i                                    16                    1                            41
;                                                                                                                  1                          66                3                  91 i
1          /
'                                    17        /          2                            42
.                                                                                                                1                          67      /      2
}
92          /    2 IS                    1                            43                      1                          63      /      2                    93                1 19                    1                            44
                                                                                                      /          1                          69                1                  94
{
1          /
20                    1                            45
;                                                                                                                1        /                70              1                    95
;                                                                                                                                                                                                    1
                          ,        21                    1                            46 i                                                                                                                1                          71      /      2                    96          /    2
{                                  22                    1                            47                      1                          72 1                                                                                                                                                    /      2          /        97                1 j                                  23                    1          /                  48                      1        /                73        /
!                                                                                                                                                            2                    98                1 24 /                  2                              49 1                          74        /      2                    99                1
}i                                25                    1                              50                      1                          75                                    IN i
1
                                                                                                                                                                                            /    2 E          9                                      2
  !                                                      34 l 1                                    5          28 4                        E          10    35 7          E          8    32          5 H= Higher Level Question                                                                                                                      -mumm mumm mumi R-Reference Required i
                                  "= Reference Not Supplied                                      Pi= Time /Diiliculty Score                                                      3        32    u9 17 i
j                                                                                                                                                                                                              .        .
                                                                                                                                                                                                                      , ~
 
l ATTACIIMENT IV Evaluation of 1996 MP-1 LOUT Issues Against NRC / INPO Requirements 10 CFR Part 50: Attachment B - Ouality Assurance Criteria for Nuclear Power Plants This area of regulation applies to those structures, systems and components that prevent or mitigate the consequences of postulated accidents that could cause undue risk to the health and safety of the public. In the course of the IRT review, the following challenges to the good business practices addressed by 10 CFR Part 50:
Appendix B were identified.
Criterion I Organization.
(3) The authorities and duties ofpersons and individuals performing activities affecting the safety-related functions ofstructures, systems and components shall be clearly established and delineated in writmg.
      'lle expectations for the following LOUT program principals was not clear - thereby allowing performance that did not result in success: LOUT Coordmator; Operations Manager; LOUT class mentor; Shift Managers.
Criterion II. Quality Assurance Program.
(4) The quahty assurance program shallprovide control over activities affecting the quality ofthe idennfies structures systems and components consistent with their importance to safety.
(10) Management ofother organizationsparticipating in the quahty assurance program shall regularly review the status and adequacy ofthatpart ofthe quality assurance program they are executing.
The LOUT program is of crucial importance to plant safety. Contrary to the above, Nuclear Oversight, NTD and Unit I management failed to provide effective control.
Criterion V. Instructions, Procedures, and drawings.
: 1) Activities afecting quahty shall be prescribed by documented instructions, Procedures, and drawings ofa type appropriate to the circumstances and shall be accomplished in accordance with these instructions, Procedures, anddrawings.
Contrary to the above the following deficiencies were observed with regard to the design and implementation of the LOUT program. NTD procedures were not followed in the design and development of the LOUT program, including:
            . Training Program Analysis and Design.
* Trammg Records.
* Curriculum Control.
Criterion VI. Document Control
: 2) These measures shall assure that documents including changes are reviewedfor adequacy andare distributed to and used at the location where the prescribed activity is performed.
Contrary to the above, Unit 1 OTB failed to routinely incorporate plant design and procedure changes into taining materials.
IN%tSTiLra-at                                              . _ . .
 
    . _ . _ . _ . _ _ . _ _ _ . _ _ _ _ . . _ . _                          . . . _ . _ . . _ . _ _ _ = _ . _ . _ _ . _ . _ _ . _ _ _ _ _ _ . . _
1  .
i  -
                                                                                                                                                    =
l
:l ATTACHMENT IV Evaluation of 1996 MP-1 LOUT Issues Against NRC / INPO Requirements (cont.)
;            Criterion XVI. Corrective Action.
: 2) Measures shall be established to assure that conditions adverse to quality, ss ch asfailures, malfunctions, defciencies, deviations, defective materialandequipment, andnon-conformance arepromptlyidenn)edand corrected.
Contrary to the above, numerous cases were identified where the Self Assessment / Corrective Action process has not been used effectively to identify and correct weaknesses in trammg Criterion XVII. Quality Assurance Records.                                                                                                '
: 1) Suffcient records shall be maintained tofurnish evidence ofactivities afecting quality.
: 3) The records shallalso include closely relateddata such as quali) cations ofpersonnel, procedures and equipment.                                                                      -.
Contrary to the above, Unit 1 OTB was unable to produce records supporting that candidates had completed all-phases of the LOUT program                                                                                        -
10 CFR 50.120 Trainine and Oualification of Nuclear Power Plant Personnel.                                                                '
This regulation directs the licensee to establish, implement and maintain a trauung program derived from a system's                        j approach to trammg The following challenges to 10 CFR 50.120 were observed                                                                '
10 CFR 50.120 (2).The training program must be periodically revised to reflect industry experience as well as changes to thefacility, procedures, regulations and quality assurance requirements.
Contrary to the above, Unit 1 OTB failed to routinely incorporate plant design and procedure changes into trammg materials.
10 CFR 50.120 (2). Suffcient records must be maintained by the licensee to maintain program integrity and kept availablefor NRC inspection to venfy the adequacy oftheprogram.
Contrary to the above, Unit 1 OTB was unable to produce records supporting that emndid*c had completed all phases of the LOUT program. It also failed to document the analysis lendmg to the LOUT program revision.
mal thaft s 11 Fatsuary.1997                                                                                                            4 1996 MP-1 LOUT NRC f- 7mikses DLT
 
1 i      .
i ATTACHMENT IV i
!                                      Evaluation of1996 MP-1 LOUT Issues Against NRC /INPO Requirements (cont.)
ACAD 91-015 The Obiectives and Criteria for Accreditation of Trainine in the Nuclear Power Industry.
'                This document establishes the standards that must be followed to achieve and mamtam accreditation by the National Academy of Nuclear Trammg During the course of the IRT investigation several cases were identified      + -
l                  where these objectives and criteria were challenged. The purpose of this summary is to provide representative examples of these challenges.' It is not an all encompassing accred;tation evaluation. Management should conduct j                  a self evaluation of these items to ensure high standards of trammg are being met.
l                  Training Program Content                        '~
e Cnteria 1.1 Industry guidance and associated training materials are usedin conjunction with a systematic l                  developmentprocess to establish and maintain trainingprogram content.
i                  e Feedback from trammg program coordinators suggests that trammg is more focused on passing the NRC exam a
than on effective operation of the plant.
;                  Organization and Management of Training                                                        l Cnteria 2.10 Personnel entering initial training programs possess expected entry-level knowledge and skills,            I and experience.
Feedback from several madiA*= indiW that their systems knowledge was not at the level assumed by the Program i
Development and Qualification of Staff                                                                                  :
i Criteria 3.1 Training managers, instructors, andprogram developmentpersonnelpossess and maintain the                      j educational, technical and experience qualgications requiredfor theirpositions.
The inexperience of the Unit 1 OTB staff and supervision were idennfied as a casual factor to the event.
Analysis, Design and Development.
Objective 4 A systematicprocess is used to determinejob performance requirements, spectfy trainingprogram content, prepare supporting materials and maintain the training program.
NTD procedures were not used to modify trammg programs in compliance with the SAT.
Conduct of Classroom and Individualized Instruction and Trainee Evaluation.
Criteria 3.1 Individualized instruction when usedprovides trainees with sufficient guidance and supporting materi.21sfor achieving the learning objectives.
Candidates commented that the individuahzed remediation actions taken were ineffective.
Conduct of Simulator Training and Trainee Evaluation.
Cnteria 7.2 Instructors are prepared to deliver efective and consistent (simulator) training.
Some nadid*= commented that simulator trammg was inconsistent from day to day regardmg instructor Eq + M ons.
Systematic Evaluation of Training Effectiveness.
Criteria 8.4 Change actions (e.g. regulatory changes, changes injob scope [e, results ofevaluations and inspections procedure changes plant system changes) are evaluatedfor applicability to initial and continuing training.
Contrary to the above, Unit 1 OTB failed to routinely incorporate plant design and procedure changes into        ,
trammg materials.                                                                                                '
malDraft . H Fahnaary.1997 19s6 MP.1 LCUT NRC Ezasemanan Fakses IRT
 
9 i        .
i
;                                                                                                                                                i l                                                                                                                                                I ATTACHMENT V                                              i 1
1 REVIEW OF NTM TRAINING PROCEDURES                                                  j 1
4 4
Modification to the accredited licensed operator trauung program was based on the proficiency of the candidates            !
i                      and time available before the NRC exam Changes to accredited programs must be made in a systematic manner.                I 1
A review of NTM trauung procedures was conducted to detemune the adequacy of these procedures to support and guide trammg program resision.
i j                      Based on NTD guidance the following procedures should have been followed.
NTM1.14 Validation of                    This procedure provides a systematic method of evaluatmg the i                    Equivalent Training                      candidates previous knowledge and skills. This analysis would 3                                                              then be used to determine the candidates' experience and i                                                              knowledge with the intent of determmmg what trauung is needed.
j                                                              his procedure was not implemented for this program 3
1 ATM1.15 Individualized                  This procedure provides a systematic process to develop Training Curriculum                      indisiduahzed traming programs based on the analysis conducted
;                                                              in NTM 1.14 Validation of Equivalent Training. This plan is then i                                                              followed when pursuing qualification. While these plans were j                                                              developed, they were not completed in all cases.
i ATM201 Evaluare/                        This procedure provides detailed guidance on the steps to be DevelopJob Performance                  followed when changing the design of a tramme program This
,                    Data                                    procedure was not implemented for this program J
i                    NTM101 Evaluare/                        Procedure NTD 2.01 directs the instructor to this procedure when j                    Develop Learning Objectives ch=ging the design of trammg programs. His procedure discusses the method of evaluating learnmg objectives and details j                                                              what products have to be developed in this process. His j                                                              procedure was not implemented for this program t
E
 
== Conclusion:==
 
i                    Established NTD procedures provide sufficient guidance and oversight to modify tramme programs in a systematic j                    manner. These procedures were not used in the 1996 LOUT program 4
M 1
  ]                                                                                       .
J 1
^
j                    sul Draa .11 Fatsvary,1997 19e6 MP-1 LOUT MLC Enssamanon Faihsts RT l
 
        - - _ - - - ~ _ _ .              . -          -      -. -              -.-              -            -- .- _ - - .                      _ - -        . . - - . . .
[    .                                                                              ATTACHMENT VI
;                                                                                    19% LOUT Program i
Barrier and Failure Mode Analysis Bartter Failure orn-      -                1st Ice! -                        2nd Level                    3rdlevel Trns sup          Trna Mgmt    Ops Mgmt -
Sps+-4 to t                  breaks down                  ID            ID. CA            ID,CA,Cl      ID,CA,Cl    Cl i
LOUT modification was dnven by available tune m utuch        MJ, M3        MJ,M3              M3            P1 to conduct the traming Procedures were not used to gusde revision                  ID            CA                Cl                        Cl
'i On-Shdt tune cut from 13 wks to 4 wks w/o reference to      IDI            MJ                IDI          SK accredited c -      =
Self-Study -Ws not unplemented per plan.                                                                                                        )
MJ            MJ                IDI          Cl                                I Some candadates armounad only daw before class start        SK            SK                MI            MJ Target Audience changed without program modificatson                                                                                            i
,                                                                                          SK            hu                IDI          SK                                l j                              inexw.w of ongmal 14UT Coordmator not                                      IDI                Cl            ID
;                              recognized bv management 1^
Ops Trauung management was not cognazant of the                                                IDI magrutude of program changes k *ud
+
Two candidates required to attend GFE sunultaneously        SK              SK                Cl            M3 with LOUT.
Requaed a' re of trauung by Ops and Ops Trng                                CA                CA, C1 - -    P1, C1      C1
,                            memt were not conducted                                                                              .-
i                            The Ops Dept Mentor (Shaft Manager) sonducted only                          C1                Cl            P1,M1.C1 one observation of traming "Ihe Ops Liaison posruon was open throughout the class.                      C1                Cl            P1, C1 fanda7 =w exam bank sent to NRC.                            SK              SK                C1 4
,                            Sr. mgan faded to recogmze the efIcct of personnel                                              C1            Cl j                            changes.
Feedback from candidates and Coordanator was not acted _                    MJ                Cl            P1,C1 on
}
l                            Academic Review board requened by Coordmator was not                        MJ                C1            Cl held Issues identzfied at one unit are not always reviewed for                    ID                C1,C3        C1,C3        Cl
.                            applicabihty at other units.
a The existing computer program as not used to track student  CA, ID,        ID                CA, Cl qualifications.
Candidates mformed of prelumnary fadures pnor to            SK              SK                IDI 4                            conspleting operatxws exant 2
                            ~.xam E      review and vahdation process was meffective.          SK              SK                IDI l ~
k-- - - *es were noted on the tradang of student            ID              ID                C1                        C1 auendance and OJT time.
1 1here are no TPCC Meetmg Musules approvmg changes            ID            CA,ID              ID,Cl        ID,C1        Cl to the LOUT Program Many systern texts are out of date. Lasson plans do not      CA. ID          ID                C1            C1          Cl follow text areama6on The followmg partial hst of FPI Failure Codes was used to group the major failures of the 199614UT progran These groupmss were 1                            performed to identify common causes and themes of the event.
1 1
Frequenev of Occurrence
,                            FPI Code        Toual      1st Barrier    2nd Bamer      3rd Barrier  Dee of faDure anode C1                  36                            29            7      laaha" performaa~              enne and trending ID                  17          6              11                    Inattenuon to detail
;                            SK                  12          6              6                      f awe skills or knomiedge CA                  10          2                8                    Cornnutted =a= not carned out l                            MJ                  9            2              7                      Mart ==ad 4
1D1                  7            1              6                      Unewareness P1                  5                            5                    1=-h=" aa"on to trm==*
1 M3                  4            1              3                      cwa
* expectations C3'                  2                            2                      Inadequate root and - cause analysis 2                            2                      Vague or unclear expectauons
{
i l                            alblthas
* II E9hruary, Afr7                                                                                                _
i J
 
i t
ATTACIIMENT VII Matrix of Conclusions and Casual Factors to Corresponding Recoml mendations TABLE VIIA: Team Conclusions, referenced to Recommendations IRT Conclusion                                                                                                  Recommendation 4.1.2. A    Management oversight of the 1996 LOUf program was ineffective in              4.13.A- Review industry guidance and INPO identified "best performers" with respect to management oversight. Adopt best identifying and correcting challenges and deficiencies in the presenting of  practices into oversight process.
quality training. Management missed numerous opportunities to ensure                                .      ..              .
that training delivery was of the highest quality.                            4.13D- Reinforce with Traimng and Operations n anagement their responsibih. . ties regardm.g oversight of tram.mg.
: 4. I3.C- Establish high expectations regarding oversight and implement methods to ensure that oversight is documented, frequent, and useful, 4.1.2.B    Mana ;..e: failed to consider previously identified oversight issues          4.13.D- Ensure that oversight feedback it communicated and addressed consistently across all units (across all units) as "waming flags"and adjust its actions appropriately.
4.2.2.A    De Self Asacssment and Correctrve Action process has not been used            4.23.A- Review industry guidance and INN) identified "best performers" with respect to self- assessment and corrective action.
effectively to identify and correct weaknesses in training.                  Adopt best practices in the NTD scif assessment and corrective action process.
4.23.B- Conduct an ' urgency review' of previously identified problem areas and expedite close out of those relating to accreditation and / or regulatory issues.                                        e 4.23.C. Ensure corrective actions are developed and tracked for all identified training issues.
4.2.3.D Review effectiveness of past corrective actions. Incorporate this review into the self assessment process.
4.23.E- Develop, comrmmicate and reinforce clear expectations regarding the Self Assessment and Corrective Action process for all levels ofmanagement, supervision and staff.
                                                                  $                                      ES Conduct reviews of previously identified problem areas and expedite close- out of those relating to accreditation, regulations, i
and training quality. His should include re- evaluating priorities and resource commitments against other Nuclear Training Department excellence initiatives.
ES Implement an effective corrective action program which inc!udes accountability for closure in a timely and effective manner.
4.2.2.D    The Self As_ ._; and Corrective Action process does not                      4.23.D- Review effectiveness of past corrective actions. Incorporate this review mto the sedassessment process.
systematically assess past problems to ensure the adequacy and sustainability ofpast corrective actions.                                    4.23.E. Develop, communicate and rem. force clean expectations regarding the Scif Assessment and Corrective Action process for all levels of management, supervision and staff.
ES Conduct reviews ofpreviously identified problem areas and expedite close- out of those relating to accreditation, regulations, and training quality. His should include re- evaluating priorities and resource commitments against other Nuclear Training Depastment excellence initiatives.
lg ES Implement an effective corrective action program which includes accountability for closure in a timely and effective manner.      l
                                                                              }                                                                                                                                                                    !
                                                                      \
inel oi.a - it rebn=ry.tss7                                                                                                                                                                                                                        l 1996 MP.t t.OUr NitC Emembenan Fedwes IRT I
4
 
TABIE Vila: Team Conclusions, referenced to Recommendations IRT Conclusion -                                                                                                                                    Recommendafien 4 2.2.C      De Self Assessment and Corrective Action process in many cases focuses        4.23.D. Review effectiveness of pasi ewective actions. Incorporate this review into the scif assessment process.
on a single plant and does not consider if problems identified in one unit are present elsewherc.                                                        4.23.E- Develop, communicate and rem. force clear expectations regarding the Self Assessment and Corrective Action process for all levels of management, supervision and staff.
ES Conduct reviews of previously identified problem areas and expedite close- out of those relating to accreditation, regulations, and training quality. His should include re- evaluating priorities and resource commitments against other Nuc8 ear Training Department excellence initiatives.
ES 5- Implement an effective corrective act:en program which includes accountability for closure in a timely and effective manner.
4.2.2.D      Some assessments in the past, including the accreditation self evaluation    4.23 E- Develop, communicate and reinforce clear expectations regarding the Self Assessment and Correctin Action process for all report, did not include line management ownership.                            levels ofmanagement, supervision and stalT.
43.2.A        De original accredited program was revised without sufficient attention      433.C- Management and supervision should ensure compliance with existing procedures, placed on accreditation commitments.
4 33.D- Evabate the as taught LOITT program against accreditation standards and ensure that all accreditation commitments were met.
i 43.2.8        NTD procedures were not used to ensure a systematic process was              433.A- Evaluate existing NTD procedures and mstructions (OTHI*s) and ensure these procedures meet N~I D expectations for the followed                                                                    training process.
433.H- Revise NTD procedures and imtructions (OTHl's) as required, 433.C- Management and supervision should ensure compliance with existing procederres 43.2 C        De analysis and needs assessment for program redesign was not                433.F- Implement the previously existing plan for reviewing the Systems Approach to Training as part of the 1997 Continuing documented and therefore cannot be verified.                                  Instructor Training program 43.2.D      De NTD staffinvolved with design and revision were unaware of                433.C. Management and supervision should ensure compliance with existing procedures.
accreditation commitments.
433 E- Develop and implement corrective actions for the issues identified in the recently performed internal assessment of accreditation criteria.
43.2.E      Evidence in weakness in understandmg and applying the S AT is evident        433.A- Evaluate existing NTD procedures and instructions (OTUI's) and ensure these procedures meet NTD expectations for the elsewhere in the NTD.                                                        trmining process.
l7
                                                                          ,                            433.B. Revise NTD procedures and instructions (OTill's) as required.
1 433.E. Develop and implement corrective actions for the issues identified in the recently performed internal assessment of accreditation criteria.
4.4.2. A      De T.e.q of change for personnel associated with Unit I Operations          4.43.A- Management should consider the use o If4p0 Imances or industry experts to function in the Unit i OTB chain of command Training and the difficulty encountered in retaining qualified and          until experience level of the incumbents can be raised to an appropriate level.
experienced stafTis c!carly afecting the ability ofNTD to deliver quality                                                                                                                                                f training.                                                                    4 43 D. Management should review current conditions and practices which result in difficulty of NTD to retain qualified and j experienced staff.                                                                                                                            t 4.43.C- Management should develop and implement an aggressive succession plan for all key positions in training (Training Coordinator and above) 4.43.D- Training management should take a proactive stance in ensuring that transition and tumover for individuals being promoted or transferred is completed.
we rum -    rebre-r. toot 1996 MP.I LOUR NRC Examine 6cn Fedurer IRT
 
e e TABM VHa: Team Conclusions, referenced to Recommendations IRT Conclusion                                                                                                                Recommendation 4.4.2 B    ne succession planning process is ineffective.                            4.43.B- Management should review current conditions and practices which result in difDeulty of NTD to retain qualified and expenenced stafE        ,
4.43.C- Management should develop and implement an aggressive succession plan for all key positions in trainir:g (Training Coordinator and above).
4.43 D Training management should take a proactive stance in ensuring that transition and turnover for individuals being promoted or transferred is completed.
4.5.2.A    Lack of a proceduralized selection process resutted in some inarginally  433.A. Operations Management should formatire the SRO candidate selection process similar to its recent instruction for RO motivated and qualified candidates entering the 199f LOUT program.      candidate selection ne process shoui&Shvolvement and input from NTD.
4.53.B- Benchmark industry "best performers" and develop a formal process to assess the candidates' readiness to take the simulator exam.
43.2.B      Candidates
* readiness to take the NRC examinatien could have been more  4.53.C. Senior Management should reinforce its expectations of Training Department and Unit personnel in fulfillmg their thoroughly and objectively assessed.                                    respensibilities for thoroughly assessing and remediating the performance oflicense candidates.
4.6.2.A    Personnel and program records cannot be currently verified as being      4.63.A- Ensure that the investigation associated with ACR MI 0117 venties that all licensed operator training records are m complete and accurate. His presents a challenge to 10 CFR 50.120,10      compliance with 10 CFR requirements.
CFR 55 and QA requirentents.
4.63.B- Implement the approved records management system.
4.63.C. Implement extemal program auditing at a frequency appropriate ensure record retention requirements are being met.
4.6.2 11  Currently two computer programs exist to track personnel and program      4.63.B. Implement the approved records management system.
records - neither is being used at this time.
4.6.2.C    nere was inadequate oversight of personnel and program records by        4 63.A- Ensure that the investigation associated with ACR Mi-97-0117 verifies that all licensed operator trainmg records are in training and unit management as well as the oversight organization.      compliance with 10 CFR requirements.
                        '                                                                                                      4.6 3.C- Implement external program auditing at a frequency appropriate ensure record retention requirements are being met.
4.6.2.D    Operator Training Branch sta!T, supervision and management are not        433.C- Management and supervision should ensure complisnce with existing procedures following established procedures to maintmin records.
4 fi.2. A  De condition of trmining materiel does not support a program that heavily 4.73.A- Review and revise all training material to include PDCR and procedure changes.
depends on self-study of systems and administrative topics.
4.73.B- Evaluate this issue of training material quality for app;icability to all training programs.
4.73 C. Management should ensure the require,ments of NTM l.04, Commitment Tracking are being met.
4.73.D. Ensure that instructional materials are equately upgraded and maintained before being used in training.
4.7.2.B    Communications between the line organization and training have broken    4.73.B- Evaluate this issue of training material quality for applicabihty to all training programs.
down in regard to plant design changes and procedure revision.
4.73.C- Management should e isure the requirements of NTM l.04, Commitment Tracking are being met.
4.73.D- Ensure that instructional materials are adequately upgraded and maintained before being used in training.
4.73.E- Develop and implement a process that ensures that applicable PIX'Rs and procedures changes are communicated to NTD in a timely manner.
bialDran - 11 Febnmy,1997 1996 MP 1 LOUT NRC Emanunsann Failwes IRT
 
e-    , <
TABLE VHa: Team Conclusions, referenced to Recommendations IRT Conclusten                                                                                                                                                                                                Recomrnendation                                                l 4.7.2.C      lack of procedural comphance for _            - tracking on MPI has lead      4.3 3.C- Management snel supervision sht,uld ensure compliance with existing procedures.
to degradation of training material quality.
4 8.2.A      The exam bank provided to the NRC did not meet the guidelines of                4.8.3.A- Upgrade exam bank to industry stalidards. Implement NTM 4.01 to ensure exam questions are properly constructed and NUREG/BR-0122          "Exammers llandbook for Ikveloping(Water                documented.
Licensing Written Examinations" and could not be used by the NRC.
4.8.2 B      Lack of a fornu % zed exam review and wahdation process contnbuted to an        4.8.3.11- !!cnchmark industry "best performers" and adopt practices into a formalized process for the review and validation of NRC ineffective revkm and vali<tation of the NRC exam.                              exams. Provide expectations and training to individuals scheduled for exam review and validation.
4.5.2.C      Lack of heensing exam review experience by reviewers contnbuted to an          4.8.3.C- line Management should incorporate exam functions into their resource scheduling to ensure that apprornate personnel ineffective review and validation of the NRC exam.                              who are representative of the target audience are available for NRC exam review and validation.
4.8.2.D      Line o-..:.y of the exam review and validation process needs                    4.8.3.C- Line Management should incorporate exam functions into their resource scheduhng to ensure that appropriate personnel improvement.                                                                    who are representative of the target audience are available fer NRC exam review and validation.                                                                                                                        i 4.8.2.E      The review and validation process took place simuttaneously during the          4 8.3.D- Separate the review and vahdation of NRC exams processes so that they do not occur concurrently.
same week, thereby reducing the effectiveness of the review and validation                                                                                                                                                                                                                            ,
i process.
P i
e i I t
4 7
a b
l
                                                                                                                                                                                                                                                                                                                            \\
I inal Dran - a e regruary.1997 1996 MP.I LOUT NRC E -              Feihees IRT k
 
k c    . 1 ATTACIIMENT VII Matrix of Conclusions and Casual Factors to Corresponding Recomniendations 1
Attachment VH.B Casualfactors matrixed to applicable Recommendations                                                                                                                                                                                    t                                                                            :
Causal Factor ;                                                                                                                                                                                      . Recommendellan(s)                                                                          ;;        f Systems approach to training breaks down.                ES Conduct m-depth reviews for applicatmlity ofissues from this investigation to other training propams for sti Millstone Units. Consn8er temporary"stop-work"                                                                                                          I orJers for those proyams deternuned to be unneceptahle. !                                                                                                                                                                                                                  i k
ES- 4. Conduct reviews of previously identified problem areas and expedite close- out of those relating to accreditation, regulations, and training quality. This should                                                                                                    [
include re- evaluating priorities and resource comnutments against other Nuclear Training Department excellence initiatives.                                                                                                                                    j            ,
ES. 3- Implement an effective corrective action program which includes accountability for closure in a timely and effective manner.                                                                                                                              l          ,
l Es Senior management should clarify and reinforce expectations ofTraining and Operations ..                                                                                                  ,      ^regardmg their responsibilities for oversight of training.
Implement snethods and Inessures to ensure that overssght is effective.
4.IJ.B. Reinforce with Training and Operations _-                                                                                          .        ^ their responsibilities regarding oversight of training.                                                              t 4.13.C- Establish high expectations regarding oversight and implement methods to ensure that oversight is docu mented, frequent, and useful                                                                                                                                  I L
4.13.D- Ensure that oversight feedback is communicated and addressed consistently across all unita.                                                                                                                                                                        !
4.23.B- Conduct an
* urgency review' ofpreviously identified problem areas and expedite close out of those relating to accreditation and / or regulatory issues.
F 4.23.C. Ensure corrective actions are developed and tracked for all identified training issues.
l' 4.23.D. Review effectiveness ofpast corrective actions. Incorporate this review into the self assessment process.                                                                                                                                                          i 4.23.E. Develop, communicate and reinforce clear expectations regarding the Self Assessment and Corrective Action process for all levels ofmanagement, supervnuon and staff.
[
433.A- Evaluate existing NTD procedures and instructions (OTDl's) and ensure these procedures meet NTD expectations for the training process.
433.B. Revise NTD procedures and instructions (OTBI's) as required.
l                                                                                                                                                                                                                                                                                                                                          [
433.C. Management and supervision should ensure -                                                                                                        _ with existing procedures.
l
                                                                                                                                                                                                                                                                                                                                            ~
LIMTF rnodefication was driven by available time in which 4.13.B. Reinforce with Training and Operations .. .                                                                                                    their responsibilities regardmg oversight of training.                                                              [
4.13.C- Establish high expectations regarding oversight and implement methods to ensure that oversight is documented, freepsent.
433.A- Evahinte existing NTD procedures and instructions (OTBI's) and ensure these procedures meet NTD expectations for the training process.
433.B- Revise NTD procedures and instructions (OTBI's) as required.                                                                                                              jj 433.C. Management and supervision should ensure compliance with existing procedures.
433.D- Evaluate the as taught Lotfr program assinut accreditation standards and ensure that all accreditation -                                                                                                          were met.
Prcredures were not used to guide revision.              43.3.C- Management and supervision should ensure -                                                                                              , . - - with existing procedures On-Shift tiene cut frorn 13 wks to 4 wks w/o reference to  433.D- Evaluate the as taught LOUT prograni against accreditation standards and ensure that all accreditation -                                                                                                          were enet.                                          f cacredited comunements.
i                                                                                                                                                                                                                                                                                                                                          }
Self-Study component not :                  ' per plan. 433.D- Evaluate the as taught IJOUT program against accreditation standards and ensure that all accreditation -                                                                                                    -- - were met.                                          i I
f 4                                                ,
inal Dran - ti February,'I997                                                                                                                                                                                                                                                                                                          l 1996 MP.IIDtir NnC E            Failures IRT                                                                                                                                                                                                                                            i                                              i f                                                                                                                                                                                                                                                                                                                                        i h
_ - _ . _ _ _ _ _ - _ _ _ _ _ _ . - _ _ _ _ _ _ _ _ _ _ - _ - _ _ _ _ _ - _ - _ _ _ _ _ _ _ _ _ _ _ _ - - - - _                                                                                        _ - _ - _ _            _      _ - _~
 
Attachment VH.B Casualfactors matrixed to aM di                            cable Recommendation) (continued)
Causal Factor Recommendation (s)
Some candidates announced only days before class start ES. 6- Senior management should etarify and reinforce expectations of Training and Operations management regardmg their restensibdities for oversight of trainin Implement methnds and measures to ensure that oversight is effective.
4.1.3.H- Reinforce with Training and Operations management their responsihitities regarding oversight of training.
4.1.3.C- Establish high expectations regarding oversight and implement methods to ensure that ovenight is documented, frequent, and useful.
4.1.3.D- Ensure that oversight feedback is comrnunicated and addressed consistently across all units.
Target Audience changed without program modification.
ES Senior management stiould clarify and reinforce expectations of'I raining and Operations management regarding their responsibihties for oversight of trainin Implement methods and measures to msure that oversight is elTective.
i 4.1.3.II- Reinforce with Training and Operations management their responsibilities regarding oversight of training.
4.1.3.C- Establish high expectations regarding oversight and implement methods to ensure that oversight is documented, frequent, and u;eful.
4.I.3.D. Ensure that oversight feedback is communicated and addressed consistently across all units.
Inc Am.se level of the original I.Olff program Coord.
4.4.3.D. Training management should take a proactive stance in ensuring that transition and turnover for individuals being promoted or transferred is completed.
not recognised by management.
Ops. Traimng management was not cognirant of the          4.1.3.II. Reinforce with Training ar.d Operations management their responsibilities regarding oversight of training.
magnitude ofchanges incorporaled into program.
4.1.3.C. Establish high expectations regarding oversight and implement methods to ensuee that oversight is documented, frequent, and useful.
Two candidates required to attend GFE simultaneously      4.1.3.B- Reinforce with Training and Operations management their responsibihties regarding oversight of training.
with LOUT.          .
4.1.3.C. Establish high expectations regarding ove.rsight and implement methods to ensure that oversight is documented, frequent, and usefbl.
Required assessments of training by Ops and Ops.          4.I.3.B- Reinforce with Traimng and Operations management their responsibilities regarding oversight of training Trrining management were not conducted.
4.1.3.C- Establish high expectations regarding oversight and implement methods to ensure that oversight is documented, frequent, and useful.
De Ort Dept. Mentor (Shift Manneer) conducted only        4.1.3.B- Remforce with Training and Operations management their respomibilities regarding oversight of training.
one observation of training.
p        4.1.3.C. Establish high expectations regarding oversight and implement methods to ensure that oversight is documented, frequent, and usefid.
The Ops. Liaison position was open throughout the time of  4.1.3.D- Reinforce with Training and Operations management their responsibilities regardmg oversight of training.
the class 4.1.3.C. Establish high expectations regarding oversight and implement methods to emure that oversight is documented, frequent, and useful.
Poor quahty exam bank sent to NRC.
4.8.3.A- Upgrade exam bank to industry standards. ImpL.m.a NIM 4.01 to ensure exam questions are properly constructed and documented.
Sr. management failed to recognize the effect of personnel change.
4.4.3.D- Training management should take a proactive stance in ensuring that transitio ar d turnover for individuals being promoted or transferred is completed.
Feedback from candidates and Coord. was not acted on.      4. l.3.U- Reinforce with Training and Operations management their responsibilities regarding oversight of training.
4.1.3.C. Establish high expectations regarding oversight and implement rnethods to ensure that oversight is documented, frequent, and useful.
Requested Academic Review board was not held.
4.1.3.D- Reinforce with Training and Operations management their responsibilities regarding oversight of training.
4.1.3.C- Establish high expectations regarding oversight and implement methods to ensure that oversight is documented, frequent, and useful.
Issues identified at one umt are not always reviewed for  4.1.3.B- Reinforce with Traimng and Operat ons management their responsibilities regarding oversight of training.
applicability at other units.
4.1.3.C- Establish high expectations regarding oversight and implement methods to ensure that oversight is documented, frequent, and useful.
ne existing co. 3.eer program is not used to track student 4.6.3.0- ImpL ma the approved records management system.
qu-lificationt Inaltian .1I February.1997 1996 MP-t LOUT NRC Enemmenion rashes tRf
 
Attachment VH.B Casualfactorsmatrixedtoapplicable Recommendations (continued)
CausalFacter '                                                                                                                                                                                                                                                                                        Recommendatsen(s) .
Candidetes ink.. J of pre::.. wy failures prior to            4.1.3.Il- Remforce with Training and Operations management their responsibilitics regarding oversight of training.
completing operations exam.
4.1.3.C- Establish high expectations regarding oversight and implement methods to ensure that oversight is documented, frequent, and useful.
Eram review and validation y.w w was ineffective.            4.8.3.11- Ilenchmark industry "best performers" and adopt practices into a formalized process for the review and validation of NRC exams. Provide expectations and training to individuals scheduled for exam review and validation.
I      ..L. ies were noted on the tracking of student        4.6.3.I3- Implement the approved recerdu management system.
attendance and OTT time There are no TPCC Meeting Mmutes approving changes            4.8.3.11 Reinforce with Training and Operations management their responsibilities regardmg oversight of training                                                                                                                                                                                                                          ;
from the LOtTf Program.
4.1.3.C- Establish high expectations regarding oversight and implement methods to ensure that oversight is documented, frequent, and useful.                                                                                                                                                                                              j Many s,. , .. texts ere out of date. lesson plan does not    4.7.3.D- Ensure that instructional materials are adequately upgraded and maintained before being used in training.
follow text organiration.
I
                                                                                                                                                                                                                                                                                                                            .\
I 4
f i,
* I i
4 i
e II i
bieltheft Il February.1997 1996 MP.1 t.OUT HRC En        Fedures rRT
 
  ~
-i ATTACHMENT VIII j
PREVIOUS PASS RATES for UNIT I INITIAL OPERATOR TRAINING and 1996 NRC EXAM RESULTS J
}                                                              -
Millstone Unit 1 LOIT/ LOUT Statistics Year            # of candidates              # of Licenses obtained                  Pass rate on the first i
sitting for exam                          RO and SRO~.                  ._ attempt :
!                                                                                                                                                l l                    1993                          6                                  6                              100 %
i                                                                                                                                                !
1994                          3                                  2                                67 %                      i 1994                          6                                  6                              100 %
!                    1996        ,                7                                  1                                14 %
j 1996 NRC EXAM RESULTS -
Candidate -                        Status                      ' Written Exam              Operations Exam Candidate 'A'                        RO                              Pass                        Fail
!              Candidate 'B'                        RO                              Fail                        Fail
.              Candidate 'C'                        RO                              Fail                        Fail 1
Candidate 'D'                        Instant                        Fail                        Pass    <*-
!              candidate 'E' i                '
Previous SRO                    Fail                        Fail                    s Candidata 'F'                        Instant                        Pass                        Pass I          Candidate 'G'                        Instant                        Fail                        Fail A
i l
a sulDenA 11 February 1997 j              19M MP 1 Lot 3 NItC Enammatan Failurus IRT
 
I IRT PROJECT APPROVAL 1
MP1 LOIT NRC EXAMINATION FAILURES Charter (Rev. 0) i 1
i                      . Mission, Background, Scope 1
                      . Deliverable                                                                i
                      . Resource Allocation                                  2 4                      . Schedule
?                Prepared by John H.Ferguson AnYw N                  hnMN Date \11IQ\%
Accepted by Jay K. Thayer                ##                  &    Date N/f/96
                                                            /              /            //
4
          ,                                                        2_  _ . . -
l 1
i I
 
_.    ._        _ _ . . _ - -        - . _ _ . ~ . . _ . _ _ _._._ . . _ _ . _ . _ _ _ _ _ _ _ . - . _ __ .-_
!                                                        Independent Review Team Charter 1
MP1 LOUT NRC Examination Failures Revision 0 t
i              Minien 1
j    -
An IRT has been established to investigate the circumstances related to the failure of 6 of 7 MP1
)              LOUT candidates on the week of December 2. The team will independently assess the LOUT j              process, NRC exammation, candidate readmess self assessment process, and root cause(s) of those failures.
The mission of this IRT is to produce a report as described below to be used for corrective action
,              including orgamzational learmng. If, in the conduct of the investigation, your team discovers
;              signiScant conditions adverse to quality that could contribute to the initiation or exacerbation of a consequential event, you are to recommend immediate intenm compensatory measures to neutrahze j              such threats while pennanent corrective action is being formulated and deployed.
,              The team is requested to produce a report with the following content:
i              e Executive summary that includes the most important messages to my level and above, specific j                    actions that need to be taken at those levels, and any brief details and elaboration that you j                    believe to be vital to our understandmg of the message and action.
                +
                  ' A descriptive of the event, including, for every condition and action that was not right, proper or l[                  expected, what in your view would have been the appropriate action or condition.
I l                    Principal lessons to be learned by the orgamzation from the event (condition) you are j                    investigatmg i
The factors that controlled the consequences of the event, including 1) the trb-g events or
;                    conditions that consumm*A he              t problem,2) the pre-existmg causal factors that set the stage for j                    the problem and made the situation vulnerable to the eve.it, 3) the factors that exacerbated the event or made the consequences worse and 4) the mitigating factors that kept the event from j
having more severe consequences. This section should discuss the underlying causal factors,
;                    including all missed opportumties to have either detected, corrected or avoided the factors contributing to vulnerability, consummation or exacerbation.
;              e Generic implications, including more of the same and items downstream tainted by each j                    important condition adverse to quality.
I
* Extraneous conditions adverse to quality; those things found in the course of the event or its
:-                  investigation that were not right, yet did not contribute to the occurrence or severity of the matter j                    bemg investigated.
* Previous sirmlar events and conditions; those items from experience records and n*=W
}                    memories that indicate sinulanties in consequences or causation that might reflect on the
}                    station's performance in problem analysis or corrective action effectiveness.
;              e Quahty and safety impact, including separate and distinct discussions of consequences and j                    significance 1
1        *
                                                                                                  ,mm.
4                                                                                                                      _
 
                                      - - ~            .    -_- . -- - - - - - - . .-. . - -- .- - . .
e                                                                                                                                  l l
l 1
l
                . Proposed corrective actions including:
;                      . unenm e- r .=ory measures
;                      e  corrective action for problem effects
                        . corrective actions for causes
]
j                      .
corrective action for the generic implications o' oath the problem and its causes,
                        .                                                                                                              {
corrective actions for the self-assessment deheiencies and independent assessment                            '
deSciencies tru ! allowed the causal factou of their underlying causal factors to lie i
: i.                        unaddressed by the orgamration, and
                        . a follow-up plan to menitor:                                                                                l 1
                          . The deployment of corrective actions e
the effectiveness of the corrective actions in accomplishing their intent, and
                          . any anticipated or unanticipated adverse side effects of the corrective action.                          I The team is encouraged to use the Phoenix manuaTguidelines. If those methods are not suitable,                    i i
use methods with which the team is familiar. and which you believe to be suitable. Advise me of    ~
l the methods you intend to use and the problems with the other methods prior to initiation of team trauung. Please include this as an attachment to your report.
;                Backeround e
i                        The potennal 6 of 7 LOUT candidates failing the NRC license exammation indicates either a
}                signi6 cant LOUT program deficiency or an unanticipated increase in NRC liemed operator
;                exammaton standards or both. If the unacceptable results are due to an LOUT program failure it could indicate the potential for generic weaknesses in the licensed operator trauung program j    .          Et9De i
j The team shall consider but not limit its resiew to the followmg issues:
i e  Any cause(s) of this event that are common to the causes of previous LOUT failures at NU. If j                    so, why the corrective actions to date did not prevent this event.
3 i                . Whether there was adequate recognition of potential LOUT failures prior to the NRC exammation and what actions were taken or should have been taken Adequacy of utility review process to measure and assess LOUT candidates readmess to pass
!                    NRC exanunation and assume NRC licensed operator duties.
Whether unit management's expectations and training program standards are clearly sta                  l 1                    enforced and consistent with industry and regulatory standards.
                . Specific aspects of trauung program which broke down.
}      ,
  !              . Level of difficuhy of NRC exammation compared NRC Region 1 norms, t
                . The adequacy of Oversight actions and involvement regarding this incident and earlier indicators j                  or precursors.
er m    v,,              .                      , - . _
 
l'                                                                                                                                                      1 i
i The adequacy of trauung, Maa==ne't, self assessment process to develop and implement and
;                                  nunntain LOUT performance standards i.
!
* Any 10CFR App. B violations.
i                              .
Selection process and qualification process for LOUT candidate personnel and criteria utilized to                  '
{                                  detennine pr*antial for candidates to pass NRC exam and perform.                                              - -
: i.                                                                                                                                                      <
                              .                                                                                                                        )
Adequacy of training staff (capabilities and resource allocation).                                                    '
Deliverable                        _
t l                              The team will conduct its review and prepare its report in accordance with NGP 3.15, Root Cause
{                                                                                                                                                      l Evaluation Program Any necessary or desirable deviations from company-provided guidance for                              i 4
doing investigations should be documented, since a possible reason for the current event could 2
involve shortfalls of the guidance given to previous investigation teams The Phoemx manual may be utihzed for evaluation process. All supporting data will be catalogued and.nuuntameri in IRT j                              referencelibrary.
4 The team will present its findings and reen. =A=tions in a draft (Rev.0) report to J. Thayer, J.
,                              Ferguson and H. Haynes awd-g to the following schedule. C.- # :Jcorrections to factual
}                              content will be ino.gus d in a fmal version (Rev.1) for approval.
Resource Allocation
;                            . Resources                                                Team Members I                                      ~
Team Leader                                      Ken Muccino (NU)
One (1)NU r ie                                  Thamme Gilbert - Nucl. Training Dept.
Four (4) external r it's                        Steven Five (Duke Power) j Wendell Barron (Duke Power) l                                                                              Charles Sawver Jr. (Duke Power) 1.arry Dooley - Consultant (previously Asst. Train.
;                                                                                                      Mgr. Pilgrim Station)
Event Evaluation Training & Support              William Corcoran - Consultant -NSRC k
Team Work 1 *= tion                                                      Team Skills /Knowledne Tramma Building - Conf. Rm. 5                    BWR basics
;                                                                              NRC exammation
!                                                                              Personnel evaluation Process methods
{,                                                                              Resource utilization Training effectiveness measurement and methods
}                                                                              Training standards setting and mamtenance 7                                                                              Training oversight Team leadership
;                                                                              Phoenix or FPl process controller 4        6 4
                                              ,,.w,    _. - , ..-,          -    ,                  ,.                        n.  -_ .n--
 
l l
1 Ernectations / Resnoneihilities of Team Leader                                                      .
* Perform admmistrative and managerial duties necessary for an efficient and effective review.
Ensure team is comprised of quali6ed subject-matter experts to accomplish mission.
Develop charter, agenda, schedules, interview lists and other rid =w e
Ensure that appropriate background documentation, trammg and resources are prosided to achieve a successful outcome
  . Ensure evaluation and report are conducted in accordance with NGP 3.15 and related procedures and methodologies.
Ensure that evaluation includes only pertinent, factual, sub=timiatad and venfiable findmgs, conclusions and recww =-Mons.
Work with line management to ensure corrective actions are developed which:
            - address all aspects of the problem (s)
              - can be reasonably implemented
            - have measurable results e
Furnish periodic feedback throughout review period to sponsor (s), appropriate line management and communications representatives.                                    --
Schedule lash                                                        Responsible              Schedule        .l Lndividual              Target 1
  . Develop and approve draft charter            -    '
John Ferguson            12/19/96          i e    Allocate resources and team members                    John Ferguson            12/19/ %
* Teamleader Team orientation / tramme                                                        1/12/97 Process (W. Cocoran 2 hours)
        . Team building
* Incident specifics
  .      IRT Planning                                        Team                      1/13/97
* Develop plan / assignments
* Fimbu contact / interviewee lin e    Data documents required
  . Interviews / data collection                              Team
* 1/13-1/17/97
* Analysis and Assessments                              Team                      1/14-1/17/97
* Report preparation (Rev. "0")                          Team                      1/20-1/24/97
    . Draft report to J. K. Thayer, John                    Teamleader                1/24/97 Ferguson and H. Haynes for review - - - - -
* Final report presentation Rev.1                        John Ferguson            1/29/97 (approved byIRT manager)
 
ATTACHMENT X                                                                                                                                                                                                                                        ,
MILLSTONE ONE OPERATOR TRAINING RND QUALIFICATION SEQUENCE i
[Nul TRADEEl                                                                                                    A IEHCTG4                                                                                                                                                            DETR(T SENIOR NON-LICENSED OPERATOR MITlRL TRANWS UCIT)                                                              CN G ERTE                                                                                                                                                                                  CNem A                                                                                                      I                                                                                                                                                                    I NLIT FUPERMENTR S                                                                                                                                      RERCTOR OPEMTOR                                                                                                                                                -
CLRS5HuuM/st1F S      UDY                                                                                    MITIAL LK'ENSi TRRNING (LOIT) 1 WEEK                                                                                                                          CLASSROOP - 15 WEEKS                                                                                                                                                                ,
-          FUNDAMENTALS EXHH                                          . . .                                                                          ON -SHFT - 13 WEEKS                                                                                                                                                                t a                                                                                                                            SIMULRTOR - 15 WEEKS                                                                                                                                                              i SUPERVISORY SKILLS - 1 WEEK (SRO ONLY)                                                                                                                                                                                    gy DN-THE inu TMMNG                                                                                                                              g g y d FED M                                                                                                                                          I 98STf98T WWI '.
2 _ G WEEKS a
NON-LICPM 3                                                                                                                                                  i                                                                                  I CONTINUING T.8 MING                                                                                                                                                                                                                            LICENSED OPEMTOR NLif SYSTEMS TRRINING                                              (NLCT)                                                                                          SEN W RE M                                                                                                                                                t CLRSSHOOM/PLRNT                                    RVG.; 7 WEEKS /YEM                                                                                                                    MNN                                                                                                  CONTNUNG TMNNG            !
6 WEEKS                                                                                                                                                                                                          A (LORT)            ,
RV6. 7 WEEKS / YEAR      l 3                                                                                                              SENIOR REACTOR OPERATOR                                                                                                                                                                          ;
SYSTEMS                                                                                          NITIRL LICINSE TMNIE (LOUT)                                                                                                                                                                              i          ,
QUALF CRTION                                                                                                                        CLF SS W)OM - 7 W1KS                                                                                                                                                    I l
ON-THE-JO B TMNING                                                                                                                                                                                            T 13 WE KS g                                                                                _ ag                                                        g                                      ,
p    pE.                                                                                                  SUPERVISORY SKILLS - 1 WEEK                                                                                                                                                                                    !
a                                                                                                                                                                                                                                                                                                                !
                                                                                                                                                                                                                          .a                                                                                                          4 lFIA.I.Y ERNLFED Dull                                                                                                                    lFIA.A.Y ER. FED SRD!
A
                                                                                                                                                                                                                          -a RADMSTE PEFRTOR TRF i(NE                                                                                                                                                                                              ss                                                                  t  ,
l OJT - ; 2 WEEKS (RV0.'
I TMNWG V0GRM CLRSSKON                                                                                                                                  i DN-SHFT                                                                                                                            i MANAGEMENT SEMINRRS                                                                                                                                                                t I                                                                                                      i m
: g.                                                                                                                                                                                                                                                                                      j 4
                                                                                      - - . . _ _ - - . . - - . . - - . . _ . _ _ _ . - - _ - - _ _ _ . _ . - _ _ _ ~ _ - . _ . . - _ _ . - - _ _ . - - _ _ _ . _ - _ _ - _ - - _ _ _ _ _ _ _ - _ . _ - _ - _ - _ _ _ _ _ _ - _ _ . . - _ - - _
 
              -                                                                                                                                Docket No. 50-213 50-245 50-336 50-423 B16253 Attachment 3 Haddam Neck Plant -
Millstone Nuclear Power Station, Units 1,2, and 3 Summary of Expanded Reviews
_ s. e.
March 1997      -
 
, . - - . - -        - - - . - . - - . - - - - . -                                        . - - ~ . _ _ . - . . -
i  ,
U.S. Nucisar Regulatory Commission                                      _                          l B16253\ Attachment 3\Page 1 l                                                                  Millstone Unit 1 Review Summarv i
              ' Summarv of LOIT/ LOUT Findinas
;              LOIT/ LOUT program requirements, such as on-shift watch standing, OJT, reactivity
.              manipulations, and other supporting training records, have been reviewed for trainirig l              programs leading to new RO/SRO licenses. For Millstone Unit 1, the classes reviewed j              were the 1993 -1994 LOIT and LOUT programs; the 1995 LOIT program; and the 1996 j              LOUT program. Sixteen current Millstone Unit 1 license holders have been reviewed i
through this process. The preliminary review has determined that discrepancies exist j              or records cannot be found to support completion of prerequisites for licensed operator l              training, or completion of NNECO's requirements to be licensed as an Operator / Senior j              Operator. A summary of the discrepancies applicable to current Millstone Unit 1 l              operator licensees is provided below:
i l                1. Some candidates did not complete the required number of hours of under-j                    instruction watch standing.
i l              2. One candidate counted reactivity manipulations in a manner which is inconsistent l                  with NNECO or industry standards. Specifically, the deficiency involved taking .
!                  multiple credits for evolutions which by current standards would be counted as a 4
single manipulation. After the re-count of reactivity manipulations by current standards, the candidate still had greater than the minimum required five manipulations.
l              3. The OJT records of several candidates are either incomplete or portions are lost.                l i
,              4. Some candidates completed required watch standing and/or OJT items after submittal of the NRC Forms 398.
~
Actions Taken for LOIT/ LOUT Findinas
)                                                                                    =- .=
;              Millstone Unit 1 management evaluated discrepancies in the initial training of each
;              licensed operator reviewed, in one case, an inactive, staff SRO was administratively l-              restricted from performing licensed duties pending completion of data verification.
l-              Subsequently, Millstone Unit 1 determined that there is no longer a need for this i              individual and another staff SRO license holder to maintain their licenses. Therefore, i              these licenses have been terminated pursuant to 10 CFR 55.55(a). In all other cases, it j              was determined that the nature of the discrepancies was such that the individuals' j              qualifications to stand licensed watches was not called into question. Actions to correct l              the program deficiencies identified in the foregoing " Summary of LOIT/ LOUT Findings" will be completed prior to restarting LOIT for Millstone Unit 1.
i i
i
 
U.S. Nucl:ar Regulatory Commission B16253\ Attachment 3\Page 2
.          Summarv of Licensed Operator Reoualification Trainino (LORT) Findinas i
i A review of LORT program activities against the Systems Approach to Training (SAT) based, accredited training program (used to meet the requirements of 10 CFR 55.59) was conducted between February 12 and February 21,1997. This review evaluated
;          program administration, content, attendance, evaluations (examinations), and effectiveness reviews, against the requirements of the NU LORT Program implementing Procedure (TPIP) and the Unit-Specific Training Program Description.
l          The review was conducted by a supervisor and an instructor from an Operator Training group other than the one being reviewed.
Approximately thirty discrepancies were documented through an Adverse Condition j          Report. The deficiencies were categorized by the nature of the required corrective actions, as follows:                                                                      1 l          1. Several discrepancies were administrative in nature rather than departures from program requirements. For example, examination approval signatures were not dated.
i
,          2. Several discrepancies wcm historical in nature and either did not require corrective j            ~ action or are resolved by following program requirements on a going forward basis.    ,
;              For example, validation of equivalent training completed outside the 14-week          ;
attendance grace period.
: 3. Several discrepancies require corrective action prior to the start of LORT Cycle 97-02 and each future cycle. For example, learning objectives not approved by
;              Curriculum Advisory Committee (CAC) as required by the Nuclear Training Manual (NTM).
;          4. Several discrepancies require corrective actions in accordance with an approved l            Corrective Action Plan. For example, current process permits CAC members to l
approve their own training validations without higher level review.
Based on the discrepancies identified and the required corrective action, the start of    l Cycle 97-02 LORT instruction has been delayed from February 24,1997 to March 3, 1997. Having completed specified short term actions, Millstone Unit 1 management considers it appropriate to resume implementation of the LORT program, while longer term corrective actions and further enhancements are identified and implemented.
Because of known inaccuracies, NNECO is hereby requesting that the license applications for the seven 1996 LOUT class SRO applicants for Millstone Unit 1 be withdrawn.
 
U.S. Nucirr Regul tory Commission                                                      _
B16253\ Attachment 3\Page 3                                                          -
Millstone Unit 2 Review Summary Summary of LOIT/ LOUT Findinas LOIT/ LOUT program requirements, such as prerequisites, on-shift watch standing, OJT, reactivity manipulations, and other supporting training records, have been reviewed for training programs leading to new RO/SRO licenses. For Millstone Unit 2, the classes reviewed were the 1995-1996 LOIT and the 1995-1996 LOUT programs. Fifteen current Millstone Unit 2 license holders have been reviewed through this process. The preliminary review has determined that discrepancies either exist or records cannot be found to support completion of licensed operator training progr;m prerequisites or elements. A summary of the discrepancies applicable to current Millstone Unit 2
                                                        ~
operators or SROs is provided below:
: 1. Some candidates did not complete the required number of hours of under-instruction watch standing. Documented program requirements were 13 weeks on shift with stated expectations to the candidates of approximately 10 weeks on shift.
Emphasis was placed on OJT card completion and reactivity manipulations versus hours on shift
: 2. Some candidates counted reactMt/ manipulations in a manner which calls into question the proper completica of the requirement to perform five control manipulations which affect reactivity. Examples of this practice included taking credit for manipulations supervised by SRO candidates, and taking multiple credit for evolutions that may only allow for the credit of a single manipulation. In some cases, the preliminary re-count of reactivity manipulations by NNECO's review team may result in the conclusion that certain trainees performed fewer than the minimum    i required five. No active licensed watch stander has been identified who lacks the      I minimum five required reactivity manipulations. NNECO continues to assess the completion of reactivity manipulations.
l
: 3. The OJT records of several candidates were either incomplete or portions were lost.      1
.      4. Some candidates for instant SRO and one RO candidate did not meet all the prerequisites specified ln the NU training program. For example, one previously-licensed candidate did not have the minimum six months at Millstone Unit 2 prior to
?
the start of LOIT. Other instant SRO candidates did not complete the specified non-licensed operator qualifications. One staff candidate for RO did not have six-months experience standing watch as a non-licensed operator, as required by the NU program. The appropriate rWews and documentation to permit such deviations from the approved training program were not completed.
 
i' U.S. Nuclear Regulatory Commission B16253\ Attachment.3\Page 4 Actions Taken for LOIT Findinas
!        Active watch standers found to have incomplete training were removed from licensed i        activities. Subsequently, documentation packages were prepared for each operator j
who was removed from licensed duties. These documentation packages provided a j        basis for reinstating operators to licensed duties. For watch standers who failed to      4 -
l        complete the required hours of under-instruction watch standing while enrolled in LOIT, l        credit was taken for under-instruction watch standing completed after NRC examination j        (a Unit-specific requirement for watch stander qualification), and for actual watch hours as a licenced operator under-the supervision of a senior licensed operator. For
)        operator licensees who were missing one or more OJT Cnatures or cards, the training j        was completed.        Additional program prerequisite or completion discrepancies i        subs ~equently were identified for several watch standers. Accordingly, these individuals l        were removed from licensed duties and will be retumed only after proper disposition of        !
the deficiency.                                                    -
t l        An administrative control is in place to prevent inactive licensees with training            '
l        discrepancies from being returned to active status prior to the proper disposition of the training discrepancies.
l j        Management _ had previously deferred the start of the next Millstone Unit 2 LOIT i
program, in order to upgrade training materials and processes to industry standards.
Actions to correct the program def!:iencies identified in the foregoing " Summary of LOIT Findings" will be completed priorib mstarting LOIT for Millstone Unit 2.
l        Actions Taken for LOUT Findinas l'
Active watch standers found to have incomplete training were allowed to remain on-shift. Post-NRC examination under instruction watch standing combined with actuai watch hours as a licensed senior operator resulted in on-shift experience in excess of training program requirements. OJT completion discrepancies were completed, or j      determined to have been completed during the training program.
I(
Summary of Licensed Operator Reaualification Trainina (LORT) Findinas l        A review of LORT program activities against the SAT based, accredited training                )
j        program (used to meet the requirements of 10 CFR 55.59) was conducted between                )
;        February 12 and February 20,1997. This review evaluated program administration, 6
content, attendance, evaluations (examinations), and effectiveness reviews, against the requirements of the NU LORT Program implementing Procedure (TPIP) and the Unit-Specific Training Program Description. The review was conducted by a supervisor and an instructor from an Operator Training group other than the one being reviewed.
I i
 
U.S. Nucl:ar R:gulatory Commission B16253\ Attachment 3\Page 5 Approximately twenty-five discrepancies were documented through an Adverse Condition Report. The deficiencies were categorized by the nature of the required corrective actions, as follows:                                                            ,
i
: 1. Several discrepancies were administrative in nature rather than departures from program requirements. For example, simulator training crews are limited to 5 personnel, except for intact operating shifts, which are not limited and may be large based on current plans for 1997 training.
: 2. Several discrepancies were historical in nature and either did not require corrective  l action or are rers. sed by following program requirements on a going forward basis.
l For example, the TPIP requires that recommendations for removal from licensed        '
activities be formally documented. However, such recommendations had been routinely made by telephone notification of line management.
: 3. Several discrepancies require corrective action prior to the start of LORT Cycle        i 97-02. For example tne approved Training Program Description was specific to the      i two-year program for 1995-1996.
: 4. Several discrepanc'r;, require corrective actions in accordance with an approved Corrective Action P a t For example, the memo reporting completion of the biennial review of the task list does not contain details as to how the review was conducted.                            .    .
Based on the discrepancies identified and the required corrective action, the start of Cycle 97-02 LCRT instruction was delayed from February 24 to March 3,1997. Having completed specified short term actions, Millstone Unit 2 management finds it appropnate to resume implementation of the LORT program, while longer term
;    corrective actions and further enhancements are identified and implemented.
l i
O w
i k
1
 
s U.S. Nuclear Regul; tory Commission B16253\ Attachment 3\Page 6                        _ . .
Millstone Unit 3 Review Summary Summarv of LOIT/ LOUT Findinas LOIT/ LOUT program requirements, such as prerequisites, on-shift watch standing, OJT, reactivity manipulations, and other supporting training records, have been reviewed for training programs leading to new RO/SRO licenses. For Millstone Unit 3, the classes reviewed were the 1994 and 1995 LOIT program, and the 1995 LOUT program. Nine current Millstone Unit 3 license holders have been reviewed through this process. The preliminary review has determined that discrepancies exist or records cannot be found to support completion of prerequisites for licensed operator training, or completion of NNECO's requirements to be licensed as an Operator / Senior Operator. A summary of the discrepancies applicable to current Millstone Unit 3 operator licensees is provided below:
: 1. One candidate did not complete the required number of hours of under-instruction watch standing.
: 2. One candidate completed required OJT items after the submittal of the NRC Form 398, but prior to the NRC license 1xamination.
: 3. Two candidates for instant SRO did not have an appropriately documented individualized training program specified in lieu of the standard NU program. These two candidates were either previously licensed or were trained for NRC licensing, and-as such, were given an abbreviated training program in 1994 - 1995. No documentation of the specific program expectations can be located.
Actions Taken for LOIT/ LOUT Findinas Millstone Unit 3 management evaluated discrepancies in the initial training of each licensed operator reviewed. In all cases, it was determined that the nature of the discrepancies was such that the individuals' qualifications to conduct licensed activities was not called into question.      For the one licensed operator who has incomplete training (under-instruction watch standing), Milistone Unit 3 management haa approved NTD's recommended basis for the continued acceptabikty of the licensee's qualifications.
Millstone Unit 3 LOIT/ LOUT programs are in progress at this time. NTD management has reviewed a sample of the training processes and record keeping to ensure that discrepancies or results similar to those found elsewhere are not being duplicated in the current class. Actions to urrect the program deficiencies identified in the foregoing
  " Summary of LOIT/ LOUT Findings" will be completed prior to NRC examination of the current LOIT/ LOUT class in July,1997.
 
U.S. Nucl:ar Regulatory Comr%sion B16253\ Attachment Fye 7 Summary of Licensed Operator Reaualification Trainino (LORT) Findinas A review of LORT program activities against the SAT based, accredited training program (used to meet the requirements of 10 CFR 55.59) was conducted between February 12 and February 26,1997. This review evaluated program administration, content, attendance, evaluations (examinations), and effectiveness reviews, against the requirements of the NU LORT Program implementing Procedure (TPIP) and the Unit-Specific Training Program Description. The review was conducted by a supervisor and an instructor from an Operator Training group other than the one being reviewed.
Approximately thirty discrepancies werc documented through an Adverse Condition
        ~~~
Report. The deficiencies were categorized by the nature of the required corrective actions, as follows:
: 1. Several discrepancies were administrative in nature rather7han departures from program requirements. For example, an unlicensed Shift Technical Advisor failed the 1996 Annual Examination and has not yet completed remediation.
: 2. Several discrepancies were historical in nature and either did not require corrective action or are resolved by following program requirements on a going forward basis.          l For example, missed attendance had not been made up or resolved within the specified time frame.
l
: 3. Several discrepancies require corrective actions in accordance with an approved Corrective Action Plan. For example, documentation of the completion of makeup training is inconsistent from crew to crew.
Based on the discrepancies identified and the required corrective action, Millstone Unit 3 management considers it appropriate to continue implementation of the LORT program, while longer term corrective actions and further enhancements are identified and implemented.
: s.                      \
                                                                                                      -    _    l e
l J
 
U.S. Nucl:ar R::gulatory Commission                                          _
B16253\ Attachment 3\Page 8 Haddam Neck Plant Review Summary Summarv of LOIT/ LOUT Findinas LOIT/ LOUT program requirements, such as prerequisites, on-shift watch standing, OJT, reactivity manipulations, and other supporting training records have been reviewed for training programs leading to new RO/SRO licenses. For the Haddam Neck Plant (HNP),
the classes reviewed were the 1995-1996 LOIT program, and the 1996 LOUT program.
The preliminary review has determined that discrepancies either exist or records cannot be located to support completion of prerequisites for licensed operator training, or completion of CYAPCO's requirements to be licensed as an Operator / Senior Operator.
A summary of the discrepancies applicaMe to Be three current HNP operator licenses from the LOIT/ LOUT program is provided oelow:
: 1. Reactivity manipulations were counted in a manner which is inconsistent with industry standards. An example of this practice is taking multiple credits for evolutions which by current standards would be counted as a single manipulation.
4        In one case, the re-count of reactivity manipulations by the review team left the trainee with fewer than the minimum required five.
: 2. One candidate completed one O.lT card after the submittal of the NRC Form 398, but prior to the NRC license examination.
Actions Taken for LOIT/ LOUT Findinas CYAPCO management immediately took action to administratively remove from licensed duties all members of the 1995-1996 LOIT/ LOUT program. Subsequently, nine of the twelve licenses were no longer needed and their licenses have been terminated p.ursuant to 10CFR55.55(a). The three remaining licenses continue to be restricted from licensed duties pending completion of the data review. The results of
                                                                          ~
this data review will be submitted to the NRC by April 2lT997.
Summary of Licensed Operator Reaualification (LORT) Findinas The HNP LORT program continues as the unit prepares for decommissioning and the attendant changes in license status. Training topics are being selected from the task lists consistent with systems and activities (procedures) currently in use at the facility.
Based on having appropriate management oversight of the HNP LORT program, the reduced number of relevant training topics, and the decreased size of the training population, CYAPCO has determined that the LORT program should continue.
CYAPCO has directed that a specific review of the HNP LORT program, similar to that performed for the Millstone units, be completed by April 4,1997.}}

Latest revision as of 19:41, 12 December 2021

Redirect to:

Retrieved from "https://kanterella.com/w/index.php?title=ML20140C357&oldid=3367839"