ML20237B727
| ML20237B727 | |
| Person / Time | |
|---|---|
| Issue date: | 10/20/1987 |
| From: | Stello V NRC OFFICE OF THE EXECUTIVE DIRECTOR FOR OPERATIONS (EDO) |
| To: | |
| References | |
| TASK-PII, TASK-SE SECY-87-262, NUDOCS 8712170097 | |
| Download: ML20237B727 (11) | |
Text
--
,.u...
g N bEO
'o
\\
()
-t a
t i
s..... f POLICY ISSUE (Information)
October 20, 1987 SECY-87-262 For:
The Commissioners From:
Victor Stello, Jr.
Executive Director for Operations z
Subject:
NRC ADMINISTRATION OF REQUALIFICATION EXAMINATIONS--10 CFR 55.57 q
Purpose:
To inform the Commission of the staff's implementation of the provisions of 10 CFR 55.57 requiring that each licensed operator and senior operator pass an NRC-administered requalification written examination and operating test at least once during the term of the individual's six year (l
license as a condition of license renewal.
Current implementation practices for requalification reflect staff experiences with initia? licensing.
I BackgrIund:
In 1980, in response to SECY 79-330 (series), the Commission directed the staff to conduct requalification examinations for all Reactor Operators (R0s) and Senior Reactor Operators 1
(SR0s).
The purpose of these examinations was to evaluate the effectiveness of the facility licensee's continuing training program and pinpoint deficient areas.
The focus was not on the individual, but rather on the requalification program itself.
Both a written examination and a simulator examination were to be administered.
This requirement was later changed by a Staff Requirements Memorandum (SRM) dated August 3, 1982, to (1) a written examination, and (2) an operational examination consisting of either a simulator examination for plants with simulators or an oral examination in the facility control room.
By memorandum dated June 8, 1983, Regional Administrators were advised to schedule requalification examinations to coincide with the facility's normal requalification examination schedule.
Draft Examiner Standard ES-601, dated May 24, 1983, was enclosed for guidance on the " Administration of NRC Requalification Program Evaluation." ES-601 included the p
following guidance:
D 8712170097 871OPO CONTACT:
PDR SECY 87--ne Jack W. Roe, NRR ppg 492-4803
a j
2-The NRC written requalification exam should be primarily based upon the learning objective stated in the materials used by the facility in its requalification program.
Content and format of this exam should be the same as currently required by the licensee approved requalification program.
Revision 1 to ES-601 was issued on June 14, 1984, and contained the same basic guidance.
Revision 2 to ES-601
)
dated April 15, 1986 eliminated the specific requirement to
)
base the NRC written examination on the facility learning objectives for requalification training.
Instead, the guidance included the following statement:
l The NRC administered requalification examination i
shall be comprehensive to test the overall J
requalification training program.
The exam shall i
not be limited to the particular cycle or module just completed at the time of the exam.
(
At this point the NRC administered requalification
(
]
examination began to take on the complexion of an NRC l
administered initial examination.
This was considered l
appropriate because of program improvements that had been j
. 'r implemented to enhance the creation of content valid, J
operationally oriented, performance based examinations.
The
(
NRC Exam Question Bank, knowledge and abilities catalogs and l
the Examiner Handbook (DRAFT) were intended to improve the l
r quality of NRC examinations.
Subsequent revisions to the Examiner Standards (Rev. 3 dated September 1, 1986, and Rev. 4 dated May 26,1987) further reduced the distinction between an NRC requalification examination and an l
initial / replacement examination.
)
1 A number of programmatic weaknesses have been identified by
{
the NRC during the conduct of requalification exams.
l Significant licensee efforts have been expended to upgrade l
their programs as a result of these NRC requalification evaluations.
The staff believes that a net positive benefit to safety has resulted from its participation in licensee requalification evaluations, and intends to continue the program with some modification.
On March 25, 1987, a complete revision to 10 CFR 55,
" Operators' Licenses" was published in the Federal Register q
(52 FR 9453) to be effective on May 26, 1987.
This revision V
included a new provision for the renewal of operator l
I licenses.
Whereas the previous focus had been on the facility requalification training program, this. provision resulted in a dual focus where both the program and the i
l
r i
l l.
o individual were being evaluated.
In addition, the term of the license was extended from two to six years.
Specifically, the final rule states, in 10 CFR 55.57(b)(2)(iv), " Renewal of Licenses," that the licensee must pass "a comprehensive requalification written examination and operating test' 3
administered by the Commission during the term of a six year license.". This requirement was based on an SRM dated February 27, 1987, wliich required that the "NRC examine all i
i licensed operators on a random basis so that no operator j
will go longer than six years without being examined by the NRC...."
To implement this provision, NRC examiners followed the guidance in ES-601, Revision 4 of NUREG-1021 " Examiners' Standards," which required a comprehensive written requalification examination (approximately 60% of an initial / replacement NRC licensing examination) and an operating test (similar to an initial replacement operating test).
However, through informal contact with individual i-operators and facility training staffs, the NRC staff has become concerned that the implementation of this provision may be having an adverse impact on licensed operators at nuclear power plants along with a potential adverse impact O
on plant safety.
.T In order to obtain direct industry feedback on the issues involved, a public meeting was f. eld on September 10, 1987 with industry representatives organized through the Nuclear l
Utility Management and Resources Committee (NUMARC).
Two significant issues concerning implementation of this provision were addressed at the meeting.
First, the stress on individual operators is heightened because the individual operator's license is now at risk.
The industry representatives indicated that a primary cause of stress'is that the operators did not believe the NRC examination reflected their training or the way they operated the plant.
l The operators also were concerned that even if they passed l
one NRC examination, the random-selection process would expose them to repeated re-examinations by the NRC.
A third i
I factor contributing to operator stress was the 10 days notification that an operator would be examined by the NRC.
The industry representatives indicated that as a result of I
this short notice, personal planning was disrupted in order to cram for the NRC examination.
In addition, one utility representative stated that the requalification program was reoriented toward assisting the ' perators to pass' the NRC o
O exams in lieu of planned requalification and refresher training.
The second major area of concern raised by the industry representatives was that the NRC examination did not reflect all of the industry's training objectives,
- _ _ _ _ _ _ - _ _ _ - _ - _ _ ~
l
. consequently, industry had to provide impromptu training to.
those operators selected for the NRC examination.
Because operators were chosen randomly from throughout the ranks of licensed operators, these specia1 training needs resulted in disruptions to the composition of their normal operating crews and to the shift schedules of their licensed operators.
The industry representatives noted that this situation leads to unusually high levels of overtime for i
l t
licensed personnel and the loss of highly experienced workers who transfer or resign rather than take the required NRC examinations.
The potential significance of the issues raised by the industry representatives and the staff's concern for the effect on safety of plant operations led the staff to suspend all NRC administered requalification written examinations and operating tests for power and non power reactors on September 11, 1987, pending an evaluation of the concerns raised and the implementation of any changes to the o
program.
Discussion:
To minimize the potential for an adverse impact on the safe operations of facilities, the staff is implementing, on a O,
trial basis, revisions to the NRC administration of requalification evaluations of individual operators r.equired
.1 by 10 CFR 55. This revised evaluation process is intended to provide the staff with an improved assessment of the effectiveness of facility requalification training programs.
One of the difficulties in NRC administration of requalification examinations is that many utilities have not completed the conversion of their continuing training programs to a systems approach as was done for accreditation of their initial training programs.
INP0 has established licensed operator requalification training as a separate functional area for their second round of accreditation.
This separate focus by INP0 on requalification as a part.
of the accreditation program should ensure that a shift toward systems based training in this area will be promptly implemented, and thereby significantly improve the quality of the training and result in better preparation for the NRC requalification examinations.
Also, as part of their plant evaluation program, INPO reviews operator training, including enhanced simulator crew evaluations.
Requalification training programs are designed to provide a continuing process to maintain the proficiency of licensed O
personnel beyond what may be available through on-the-job experience and to maintain their currency on plant chsnges and events that affect plant operations. The objectives of the NRC requalification examination are to ensure that this
l 9 (3
%.)
l proficiency and currency is being maintained with an emphasis on the knowledge and ability pertinent to safety; (that is, low frequency evolutions such as startups and shutdowns, abnormal and emergency procedures, and plant changes and events that affect safe operations).
The following discussion addresses present Commission practice, the basis of industry concerns, and proposed NRC program j
revisions to address industry and staff concerns while l
l maintaining the integrity of the NRC examination process.
- 1. Written Requalification Examination a.
Current Practice As indicated in the " Background" section of this paper, the current NRC requalification written examination is essentially identical to an NRC replacement (or initial licensing) examination, the only difference is that the requalification exam is shorter.
The examination I
questions are prepared either by a contractor or an examiner from an NRC question bank that has been evolving over several years.
This bank was developed to assist in p
the preparation of initial licensing examinations.
wJ b.
Concerns Questions on the written exam that are not related to facility learning objectives, emphasize theoretical concepts, and require inappropriate memorization.
c.
Program Revision The written requalification will not be patterned after initial licensing examinations.
The written exam will be administered after completion of the operating tests.
A new format to test the operators' understanding of technical specifications and procedures will be employed.
Programmatic changes will be incorporated into the written requalification examination that will further enhance its operational orientation and ensure that it is based on proficiency and currency issues at the facility.
The facility will provide input to the examination, including proposed questions, based on the facility's requalification program learning objectives derived from a systematic analysis of job performance requirements.
NRC examiners will prepare the examination, utilizing supplied reference material, learning objectiv'es and O
proposed a"estioas.
Tne "ac ~411 expect eech asceasee to develop and maintain a question bank that reflects the learning objectives developed for the plant's requalification programs.
The examination will be
t O
i subjected to a revised review process before it is used to ensure that it is comprehensive and that it adequately addresses the issues of proficiency, currency, and operational safety.
The examination will be administered by the NRC, as is done at present for initial licensing tests.
The NRC examiners will grade the written examination in parallel with facility staff.
Although j
facility comments will be reviewed, NRC will make the final determination of an individual's grade.
This approach has been successful in providing insights into the licensee's training staff, as well as testing the individual (SECY 87-142).
The written requalification examination will be administered by the NRC in an open book format.
That is, information that is readily available to the-operator in the control room will be made available to the operator _during the test.
Criteria for NRC j
preparation and review of open book written examinations will be developed by the NRC to emphasize the need for an j
operationally oriented examination.
Test takers will 1
have the same reference material available during the test as is available to them on-the-job. In this way, the O
operators' understanding and use of technical
)
specifications and procedures will be tested in lieu of 1
reliance on memorization.
Test takers will be required
{
to sign a statement that they have not received prior information about the specific content of the examination.
l
- 2. Simulator Evaluation a.
Current Practice l
Currently, the NRC prepares the scenarios and administers simulator tests to evaluate the ability of individual I
operators to operate in a control room environment.
Based on past staff practice with initial licensing, these exams are administered to three-man teams (SRO, 2 R0s), regardless of the normal configuration of control room crew configurations or training configurations.
The basis for this testing technique was to evaluate the operator in the minimum routine control room staffing configuration required by technical specifications.
For each candidate in the simulator, there is a counterpart examiner who shadows that candidate for the dyration of the test.
O C
. b,~.
b.
Concerns The NRC practice of randomly selecting crew members for evaluation counters the licensee's efforts to promote team training and places the operators in an artificial control room environment.
The selection of individuals is disruptive in that individuals are identified irrespective of their status on the shift in training.
That is, shift crews are assembled by NRC selection and do not necessarily reflect working or training relationships.
Many NRC prepared simulator scenarios appear to be unrealistic and overly complex.
Furthermore, there is more subjective judgment exercised as to the individual supervisory skills of the senior reactor operators.
This emphasis on supervisory skills has created friction between the operators and the examiners.
The industry believes that these problems are a result of the lack of operational experience possessed by many NRC examiners, e
c.
Program Revision Programmatic changes will be incorporated into the
(])
simulation facility portion of the operating test that will further enhance its operational orientation and
7 assure that it is based on proficiency and currency issues at the facility.
The simulation facility portion of the operating test will be modified and expanded.
At least two crews will typically be selected.
One will consist of an operating crew as trained and staffed by the facility.
The other will be selected from staff engineers who are also licensed.
Where no simulation facility exists, the walk-through will be expanded to include an evaluation of team-dependent and time-critical behavior at the facility.
The scenarios used will be reviewed and selected by NRC and will be based on those developed and used in the facility's training program, and to the extent practical, will reflect actual operating experience.
Evaluation of crew performance will be done by examiners observing the crews in the simulation facility on a not-to-interfere basis and evaluating individual performance with emphasis on team-dependent and time-critical behavior.
Individual weaknesses that may be observed will be further evaluated during the subsequent plant walk-thropgh.
The crew evaluation portion of the operating test will be used
/~'
in the evaluation of the facility requalification training
(-
program, to complement the individual pass-fail results.
j AV
- 3. Walk-through Evaluation
- a. Current Practice Currently, the NRC administers a time-limited plant walk-through which is essentially identical to that administered during an initial licensing examination.
1 The operator is evaluated on his/her knowledge of various I
auxiliary and emergency systems, electrical systems, radiation and nuclear monitoring systems, various administrative requirements and operationally orientated theory topics.
b.
Concerns The primary concern is with the lack of operational experience of the examiners and their lack of in-depth knowledge of the specific facility for which they were assigned to administer examinations.
The industry felt that in many instances the examiner lacked sufficient I
familiarity with plant systems and procedures to adequately evaluate their operators.
This was of particular concern (J
to the industry because of the subjectiveness of the 5
N operating test evaluations and the lack of any method of industry feedback to the NRC on the conduct of the operating test.
c.
Program Revision Minor revisions will be made to the walk-through examination.
Each candidate will be evaluated by an NRC examiner.
Programmatic changes that will be incorporated into the walk-through portion of the operating test will further enhance the test's operational orientation and ensure that it is based on proficiency and currency issues at the facility.
Furthermore, additional attention will be given to providing the NRC examiners with plant and simulator familiarization. This will rely primarily on plant tours and hands-on experience in lieu of reliance on review of training materials.
An individual's in-depth system knowledge will be evaluated by an NRC examiner during an oral plant walk-through.
The thrust of the examination will be to test the individual's knowledge and understanding of the plant systems, components, and integrated plant operation in the actual plant environment.
The systems' selected Q
for walk-through will emphasize those systems determined to be significant contributors to core-melt and/or off-site public risk for the plant.
In addit. ion, the operators' knowledge of emergency procedure immediate
V'O actions will be evaluated, as well as his knowledge of recent plant operating events.
Weaknesses observed during the simulator crew evaluation will be individually evaluated in detail and will. assist in developing the walk-through agenda.
A mechanism to allow feedback concerning the conduct of the operating test will be developed.
The mqchanism developed will be non-obtrusive to the conduct of the examination.
The approach will l
be to have an exam critique form at the end of the written exam completed by the examinee and returned to the NRC prior to exam grading.
The candidate would be asked to identify any problems with the exam administration.
Such problems would be taken into account by NRC management during the evaluation of the examination.
j Examiners will be expected to have a significant, in-depth knowledge of the selected systems and related plant I
operations in order to conduct the walk-throughs.
j Adequate training will be provided to the examiners, i
including detailed system walk-downs at the plant, if l
necessary, as well as future training in the underlying l
principles of the symptom-based emergency procedures.
- 4. Administrative Issues
- 'i a.
Current Practice i
The NRC formally notifies facility training program managers 90 days in advance of the examination date that i
their facility has been selected for the administration f
of NRC requalification examinations.
The requalification
)
I examinations are normally scheduled in conjunction with replacement examinations (i.e., initial licensing of new candidates).
The NRC randomly selects individual j
operators for the examinations, although the sampling will be constrained to ensure that reactor operator, senior reactor operator, and off-shift (staff) licensed personnel are examined.
Those operators selected are notified 10 working days prior to examination administration of their selection.
This short notice was designed to reduce the time that an individual was exposed to the anxiety of taking an exam, and to promote a more representative sample so that the "as found" J
condition of the licensee's continuing training program
)
would be evaluated.
Operators who had previoqsly taken and passed an NRC requalification examination could be O
selected ia this se n14a9-
i l
fm O b.
Concerns Of greatest concern to individuals is the stress and anxiety placed on the operators as they are taken off shift for training out of their normal rotation sequence.
The operators remaining on shift carry an additional t
burden to compensate for the removal of the subject j
candidates selected.
The approach has not fostered a professional approach to requalification training.
Many of the concerns with the administration of the NRC requalification program are actually based on concerns with the written examination and operating test.
- However, the primary concerns in this area are that the 10-day notification of individual operators does not allow sufficient time to adjust crew shift schedules in an orderly manner to accommodate the NRC crew selection.
In addition, since operators can be selected even though I
they may have previously passed an NRC requalification examination, the operators feel that their license is continually threatened.
O c.
erostam aevisioa Facility training program managers will be notified 90
.,h days in advance of NRC requalification evaluations.
Site visits will be scheduled to coincide with the requalification training cycle of the utility.
The facility will know 30 days in advance which crews will be evaluated.
NRC will request greater involvement by utility plant management and training management during preparation and conduct of requalification examinations.
Normally, the crew currently in the facility requalification training cycle will be selected.
A random sample without replacement will be used to preclude an operator from being subject to re-examination after successfully passing an NRC requalification examination during the term of his license.
Recognizing j
that there are also staff engineers and managers who possess NRC licenses, the sample will typically include a second shift made up of licensed personnel who are not l
routinely performing shift duties.
This approach will broaden the base for making a judgment on the effectiveness of the facility requalification training program.
(]
- 5. Program Implementation The NRC requalification program will be implemented as I
soon as all necessary guidance and criteria are de-
)
veloped.
These are anticipated to be completed by the l
)
' end of November 1987.
NRC-administered requalification examinations will be resumed using a phased approach.
The intent would be to select facilities that have converted to a systems approach to training in accordance with INP0 Guideline 86-025, Guidance for Continuing Training of Licensed Personnel. With the recent emphasis by INPO on requalification in the accreditation process, j
and the improved input from facility licensees in the NRC examination process, program effectiveness should be improved as well as the overall effectiveness of licensed operators.
To foster a more uniform and well informed approach, the regional managers responsible for implementing the program will be more involved through participation on an oversight group to assist Headquarters staff in revising the examiner standards.
Initial implementation will use the " lead plant concept."
Feedback from the initial requalification examinations I
will be evaluated and appropriate changes made prior tp final codification of the program in the examiner standards.
Impact on staff resources will be carefully evaluated and adjustments made to accommodate increased
)
workloads as appropriate.
The reliance on feedback from regional experiences to develop an implementation program for requalification
.:'i that is distinct, but complementary to initial licensing, will foster improved safety through a more technically rigorous and operationally or* nted appt
/
'~
n
~
ctor 5 e o,
Executive Dire r
for Operations i
6 O
___-___-_____a