ML14176A364
| ML14176A364 | |
| Person / Time | |
|---|---|
| Site: | Robinson  |
| Issue date: | 10/30/1987 |
| From: | Hannon J Office of Nuclear Reactor Regulation |
| To: | Roe J Office of Nuclear Reactor Regulation |
| References | |
| NUDOCS 8711030470 | |
| Download: ML14176A364 (28) | |
Text
oval S il DISTRIBUTION:
Central FiTes OLB RF NRC PDR REaton LWiens MEMORANDUM FOR:
Jack W. Roe, Director JHannon Division of Licensee Performance TSzymanski and Quality Evaluation Bob Lata CJulian 7 FROM:
John N. Hannon, Chief EUftey Operator Licensing Branch RAllen Division of Licensee Performance and Quality Evaluation
SUBJECT:
MINUTES FROM MEETING WITH CAROLINA POWER AND LIGHT COMPANY (CP&L) OCTOBER 28, 1987 RE: OPERATOR REQUALIFICATION EXAMINATIONS AT H.B. ROBINSON UNIT 2 A meeting was held at the H. B. Robinson site on October 28, 1987 to discuss the participation of CP&L in a pilot requalification examination program.
Attached as enclosure (1) is a discussion paper, Requirements and Procedures for Requalification Examinations, which forms the basis of the pilot program approach. A list of attendees is enclosed as enclosure (2).
The pilot approach was well received by CP&L and viewed as having the potential to significantly enhance the operational quality of the requalification examination process.
An examination team from Region II Operator Licensing Branch will administer the pilot examination during the week of December 14, 1987. Points of contact were established for CP&L, Region II and Headquarters to coordinate this pilot effort. Additionally, two members of Region II staff remained on site to establish milestones and dates acceptable to the NRC and the licensee to complete this initial pilot examination.
John N. Hannon, Chief Operator Licensing Branch Division of Licensee Performance and Quality Evaluation
Enclosure:
As stated 87110304708710 PDR ADOCK.'05000261 PDR__
DW/MINUTES/EATON2
Contact:
John Hannon 49-24868 OL OB :DLPQ OLB LPQ:BC R
n.rh LWiens JHannon 10/94/87 10/OI/87 10ht/87
REQUIREMENTS AND PROCEDURES FOR REQUALIFICATION EXAMINATIONS The NRC requalification examinations administered under the provisions of 10 CFR 55.59(a)(2)(iii) have been revised to minimize the potential for an adverse impact on the safe operations of facilities, and to provide the staff with an improved assessment of the effectiveness of facility requalification training programs. To assist with the implementation of the revised examination, a Requalification Examination Team was assembled consisting of Section Chief's from all of the Regions and led by the Operator Licensing Branch.
The Team was tasked to design guidance and criteria for the implementation of the revised NRC Requalification Examination Program. The Team, in its deliberations, recognized the effort expended by the industry in developing a systems approach to training in accordance with INPO Guideline 86-025, and relied upon existing requalification program standards for guiding the NRC examination development and implementation. This approach will allow the NRC to administer requalification examinations that are consistent with existing facility developed programs, thereby reducing the impact on the facilities and improving the reliability of the NRC assessment of requalification training programs.
Each requalification examination will be developed by NRC examiners and a facility representative and will be reviewed by the Resident Inspector. The examiners will meet with the crews to be examined approximately one month in advance to discuss the examination.process. The examination will be based upon the facility requalification program and its objectives. This approach will result in a more technically sound and operationally oriented examination. In addition, co-evaluation of operator performance by the NRC
examiners and the facility for the written and simulator examinations will enhance the ability of the examiners to assess individual and program performance.
The operating test will consist of a simulator evaluation which will emphasize time critical and team dependent behavior with a flagging system to enable examiner follow-up of individual weakness on the plant walk-through evaluation. The walk-through evaluation will include topics identified by the NRC and the facility that are significant contributors to core-melt and/or offsite public risk. The written examination will consist of a two section open book examination: Section A will be a "Plant Proficiency" section and is administered on a static simulator; Section B will be a "Limits and Controls" section and is administered in a classroom setting.
The examination criteria and standards are provided herein to establish guidance for the administration of the first trial examination during November/December, 1987. Enhancements to the implementation of the requal ification examination process will be considered after the first trial prior to conducting further requalification examinations.
REQUIREMENTS AND PROCEDURES 1.1 Administrative Controls The following criteria outline the requirements for the administration of the NRC Requalification Evaluation Program.
A. Procedure for Facility Notification The attached corporate notification letter will be used to inform facility program managers of the impending requalification program evaluation. This notification will occur at least 90 days in advance. Site visits will be scheduled to coincide with the requalification training cycle of the facility. The facility will know 60 days in advance which operators will be evaluated.
B. Procedure for Candidate Selection Normally the crew currently in the facility requalification training cycle will be selected. A random sample without replacement will be used to preclude an operator from being subject to re-examination after successfully passing an NRC requalification examination during the term of the license.
The sample will include up to two additional shifts made up of licensed personnel who are not routinely performing shift duties.
C. Request for Facility Involvement The facility will be requested to provide an employee (licensed SRO) from the operations staff to serve as a member of the examination team. The function of this individual is to technically assist in the development of the written examination items, plant walk-through topics, and simulator scenarios. To ensure examination security, the individual must not be in any way involved in requalification program activities between the time of examination review and administration. Also, the individual must be instructed and agree not to communicate in any fashion the content or scope of the examination to unauthorized persons. The individual will be requested to sign a statement after examination administration that he/she has not given information about the specific content of the examination to any unauthorized person.
For the simulation facility examinations, a facility management representative is required to be present during admininstration of these examinations. In addition, the facility will be required to grade, in parallel with the NRC, both the written and simulator evaluations.
D. Timetable for Administration of Requalification Examinations 90 days in advance
-The facility is notified.
60 days in advance
-The facility provides proposed examinations and proposed crew composition.
30 days in advance
-A facility visit by NRC to meet with representative and operators to discuss the examination process.
14 days in advance
-NRC examination team visits the facility to prepare for the examinations.
7 days in advance
-NRC prepares for examination in the regional office.
Examination Week
-Administer Operating Test and Written Test to selected crews.
E. Procedure for Examination Administration The simulator examination Is to be administered first, followed by the plant walk-through evaluation. The written examination will be administered after all simulator and plant walk-through evaluations have been completed. This order of administration is intended to provide for an operational atmosphere prior to the written examination. Changes which may be made to this or any other portion of the requalification program evaluation must be approved by HQ OLB.
The operating test will consist of a simulator evaluation which will emphasize time critical and team dependent behavior with a flagging system to enable examiner follow-up of individual weakness on the plant walk-through
evaluation. The walk-through evaluation will include topics identified by the NRC and the facility that are significant contributors to core-melt and/or offsite public risk. During the walk-through examination operators will perform Job Performance Measures which assess the individual's ability to perform in-plant tasks.
The written examination will consist of a two section open book examination:
Section A will be a "Plant Proficiency" section and is administered on a static simulator; Section B will be a "Limits and Controls" section and is administered in a classroom setting. Section A is designed to evaluate the operator's knowledge of plant systems, integrated plant operations and instruments and controls. In addition, recognition of Technical Specification LCOs and the operator's ability to diagnose postulated events will be evaluated. Section B of the written examination is designed to evaluate the ability of the operator to analyze a given set of conditions and determine the proper procedural and administrative guidance.
Section "A" "Plant Proficiency" of the written examination is designed to utilize the simulator as a reference tool in answering question. The questions should be related to plant systems and controls (much the same as the initial examination sections 2, 3, and 6).
Section "B" "Limits and Controls" of the written examination is designed to utilize plant procedurs (including adom, operation, abnormal) and administrative controls (including TS, Eplan, Admin procedures) in an open book format. Questions should be in an open book format.
F. Criteria for Satisfactory Requalification Programs A satisfactory requalification program must meet each of the following:
(1) 90% pass/fail decisions agreement between the NRC and facility grading of the written examination, and (2) NRC and facility examination score grading must be within 5% on 80%
of the written exams, and (3) at least 80% of all operators pass the examination, and (4) no major discrepancies exist with facility evaluators for the simulator examinations.
G. Criteria for Satisfactory Requalification Performance for an Individual Individuals must achieve at least 80% overall as graded by the NRC, to pass the examinations.
REQUIREMENTS AND PROCEDURES 1.2 Simulator Examinations The purpose of this portion of the operating test is to assess the crew's time-critical and team-dependent behavior during simulated events.
A. Procedure for Developing Scenario Scope and Content Proposed examination scenarios will be provided by the designated facility.
The test team will verify the scenarios against the facility requalification program learning objectives and the K/A catalog with importance factors of 3.0 and above. The scenarios may be modified by the test team with the technical advisement of the facility representative. The scenarios should sample areas such as LERs, emergency and abnormal procedures, and design and procedural changes that exercise the crew's ability to use facility procedures in accident prevention and mitigation. The scenarios should evaluate each crew member as appropriate to his/her license, and shall exercise their abilities in the use of Emergency Operating Procedures, Technical Specifications, Emergency Plan.
Sample Size - The scenarios provided by the facility must be comprehensive and realistic. A minimum of 15 scenarios should be provided by the facility.
B. Procedure for the Review of Scenario Complexity Review scenarios to ensure against (1) too complex or (2) too simple:
(1) too complex - beyond the scope of lessons covered (unrealistic)
- beyond the ability of procedures
- beyond the simulation facility capability (2) too simple -
should be integrated events
- team dependent behavior
- time critical behavior The test team will review scenarios to ensure expected responses are provided and can be performed in allotted time.
C. Guideline for Overall Test Length A scenario's net time (not including briefings, setup or simulation facility problem related times) should average 50 minutes, based upon real time performance.
Position rotation should be dependent upon facility rotation practices.
The number of scenarios shall be sufficient to ensure that each licensee is tested to the extent that he/she may operate during both normal and emergency operating conditions.
D. Procedure for the Flagging System Any weakness observed on competencies 1 - 6 on NRC Form 157, page 4, will be perspective items for follow-up on the plant walk-through.
Passive observations will be made. Questioning of operators during the scenario should not be necessary. The Chief Examiner may question the plant management for clarification if necessary.
(1) Individual observations-record any deficiency noted for input into walk-through examination. Failure for an individual, on the simulator portion only, must be based upon gross actions such as:
(a) an inability to manipulate controls (b) failure to~actuate a Rx trip when required (c) failure to ensure ESAS systems inject when required (d) failure to emergency borate or initiate ADS when required (e) violation of Technical Specifications (f) failure to protect health & public safety that challenge the safety status of the facility.
(2) Team observations - evaluation of performance in the areas of supervision, team performance and communications for program evaluations.
E. Grading Criteria for Performance of Team Dependent/Time Critical Behavior:
-Team Performance Completion of expected actions understanding system/plant response diagnosis of events/conditions compliance/use of procedures
-Communication Clear crew informed prioritize ensures receipt
-Supervision Timely, decisive action thought out directions prioritize directives oversi ght directives are carried out radiation concern The scales developed to evaluate operator crew performance will be used to evaluate the competency areas already identified on NRC Form 157, page 4.
F. Guideline for the Number of Examiners Assigned The minimum number of examiners required is 1 for every 2 operators plus 1 examiner as coordinator and the Chief Examiner, who may act as the coordinator.
Additionally, the facility will be requested to provide a member of the plant operations staff to observe the process. The Chief Examiner is the principal point of contact between the facility management and the NRC.
G. Procedure for Crew-Evaluation Immediately following each scenario the crew performance will be evaluated by the facility graders. This evaluation will be conducted after each scenario at the simulation facilily. The evaluation is to be observed by the NRC examiners. The critiques will be video taped if the scenarios are video taped. If the NRC observes unsatisfactory overall team performance that is not identified by the facility evaluators, the program would be judged unsatisfactory.
H. Guidelines for Video Taping The following basic criteria apply:
(1) If available (2) image clarity and degree of coverage must be adequate
(3) After initial set-up of the camera by the licensee's personnel under observation of the Chief Examiner, the recording will be made with unattended camera(s). The only intervention will be to change the tape.
(4) agreed to by all operators and evaluators Guidelines for pilot test:
The video tape will be used primarily to evaluate areas of difference between facility and NRC examiners parallel evaluations of the candidates. However, there is no intention to restrict the use of the video tape.
REQUIREMENTS AND PROCEDURES 1.3 Plant Walk-Throuah Purpose of this portion of operating test is to assess the individual's thorough knowledge of systems determined to be significant contributors to core-melt and/or off-site public risk for the plant.
A. Procedure for identifying appropriate systems Facility identifies systems and the rationale for their selection of the systems most applicable to maintenance of public health and safety in the mitigation of the consequences of an event and those systems that can direct ly initiate an event. The test team will make a comparison between these systems and the systems identified in the "Examiner's Handbook" in Categories I and II. The facility management will be informed of any changes made in the topics identified.
B. Procedure for system selection Sample content will be a minimum of two systems not to exceed three. At least 50% of the systems will be from Category 1 of the "Examiner's Hand book".
C. Procedure for test time allocation The system walk-through will be planned for approximately 2 hours2.314815e-5 days <br />5.555556e-4 hours <br />3.306878e-6 weeks <br />7.61e-7 months <br /> in length.
Simulator follow-up topics should be allotted no more than an additional hour.
A minimum of 4 Job Performance Measures will be evaluated outside the control room.
D. Procedure for including topics flagged during simulator test Weaknesses from the simulator should be evaluated to the extent necessary to determine the need for remedial training.
E. Procedure for grading pass/fail All topics will be of an importance of 3.5 minimum. The standard of perfor mance will require that the job incumbent satisfies the learning objectives on each job performance measure. Topics will be reviewed in advance by facility reviewer to verify congruence with facility requalification learning objectives.
REQUIREMENTS AND PROCEDURES 1.4 Written Examination The purpose of the written examination is to assess the individuals thorough knowledge of systems and procedures.
A. Procedure for Item Development Proposed examination items will be provided by the designated facility. The test team, which consists of NRC examiners and a designated facility representative, will verify the examination items against the facility requalification program learning objectives. Examination items will be reviewed by the test team for appropriateness, clarity, and importance to safety and may be modified, deleted or replaced if necessary. Up to 20% of the exam items may sample from subjects not emphasized during the requali fication cycle as specified by 10 CFR 55.59(a)(2)(i).
Since the written examination is completely open reference, examination items must meet the following:
(a) Items that require only memorization or recall are not permitted; (b) Items should require that the examinee comprehend, inter pret, integrate, or apply available information; (c) Items may contain situations, aspects, or conditions that do not duplicate lesson plans or references;
(d) Items should require examinees to locate and use references.
NRC substituted items shall be reviewed against facility learning objectives, if available, to assure that the item depth or level of knowledge is consistent with facility learning objectives and the type of license held.
The written examination will be composed of two sections, each designed to be completed in 1 to li hours. Each section will be separate.
Any examination items developed by the NRC shall be reviewed by the des ignated qualified facility representative, who holds an SRO license, and the NRC RI if available, prior to administration.
B. Procedure for Parallel Grading Using the examination and key the facility and NRC will independently grade each examination. Facility grading will be reviewed by the NRC. A satisfactory facility requalification program must meet each of the following:
(1) 90% pass/fail decisions agreement between NRC and Facility grading and (2) NRC and facility examination score grading must be within 5% on 80% of the exams and (3) at least 80% of all operators pass the examination.
Individuals must achieve at least 80% overall as graded by the NRC, to pass the examinations.
C. Procedure for examination proctoring Each section of the examination shall be proctored by the NRC in accordance with NUREG 1021.
D. Guidelines for Reference Material The facility shall be responsible for providing the following reference materials:
During the "Plant Proficiency" section of the examination (simulation facility), one copy of all controlled material available in the control room shall be available to examinees. Examination reference material will NOT include material that is intended for training only.
During the "Limits and Operations" section of the examination each examinee shall have the following material (complete, current issue)
(1) TS (2) EOP/AOP/OP (3) EPIP (as available in the control room)
(4) Administrative procedures applicable to operations (5) Other plant reference material available in the control room (e.g.,
curves and data book, forms, flow charts...)
E. Procedure for Examination Process Critique The operators will be provided with an examination process critique sheet and encouraged to complete the form at the conclusion of the written examination.
The facility will be responsible for collating the critiques and providing them to the regional Branch Chief after the administration of the examination.
(Insert corporate addressee)
SUBJECT:
REQUALIFICATION PROGRAM EVALUATION In a telephone conversation between M (title) and M
(section chief), arrangements were made for an evaluation of the requalification program and licensed personnel at the (facility name).
The evaluation visit is scheduled for the week of (date).
For this visit, the NRC examiners will administer NRC prepared operating test and written examinations.
The NRC examiners will visit the site approximately 30 days in advance to meet with the appropriate facility personnel and operators to review the schedule and the process for these examinations.
For the examiners to adequately prepare for this visit, it will be necessary for the facility to furnish the approved items listed in Enclosure 1 "Reference Material Requirements".
M has been advised of our reference material requirements and where they are to be sent.
Additionally, it is requested that a Licensed SRO from the facility's Operations Department be designated as the facility representative for these examinations.
This individual must not be scheduled for an NRC administered examination during this visit, or participate as an instructor once selected. Also, the reviewer will be required to certify that, as a result of the review, no portion of the examination will, or subsequently, has been compromised.
The facility management is responsible for providing adequate space and accommodations to properly develop and conduct the examinations., "Administration of Requalification Examinations," describes our requirements for developing and conducting the examinations.
M.
has also been informed of these requirements.
Also, a facility operations management representative should observe the simulation facility examination process at the site., contains the "NRC Rules and Guidance for Examinees" that will be in effect during the administration of the written examination. The facility management is responsible for ensuring that all operators are aware of these rules.
This request for information was approved by the Office of Management and Budget under Clearance Number 3150-0101, which expires May 31, 1989.
Comments on burden and duplication may be directed to the Office of Management and Budget, Reports Management Room 3208, New Executive Office Building, Washington, D.C. 20503.
Thank you for your consideration in this matter. If you have any questions on the evaluation process, please contact (regional section chief and telephone number).
Sincerely, (Appropriate Regional Title)
DISTRIBUTION:
Project Manager Branch Chief, OLB Resident Inspector Regional Section Leader Examiner(s)
Facility Training Coordinator Facility Operations Manager
2 ENCLOSURE 1 REFERENCE MATERIAL REQUIREMENTS For the written examination, the following items must be provided to the NRC 60 days prior to the examination date:
Since the written examination is completely open reference, examination items must meet the following:
(a) Items that require only memorization or recall are not permitted; (b) Items should require that the examinee comprehend, interpret, integrate, or apply available information; (c) Items may contain situations, aspects, or conditions that do not duplicate lesson plans or references; (d) Items should require examinees to locate and use references.
The written examination will be composed of two sections, each designed to be completed in 1 to 1i hours. Each section will be seperate.
Section A will be administered on a static simulator; Section B will be administered in a classroom setting.
Section A is designed to evaluate the operator's knowledge of plant systems, integrated plant operations, and instrumentation and controls.
In addition, recognition of Technical Specification LCOs and the operator's ability to diagnose postulated events should be evaluated.
Section B of the written examination is designed to evaluate the ability of the operator to analyze a given set of conditions and determine the proper procedural and/or administrative guidance.
- 2. All reference material and objectives for the proposed test items.
For the simulation facility, the following items must be provided to the NRC 60 days prior to the examination date:
- 1. A minimum of 15 scenarios.
The scenarios should sample areas such as LERs, emergency and abnormal procedures, and design and procedural changes that exercise the crew's ability to use facility procedures in accident prevention and mitigation.
The scenarios should evaluate each crew member as appropriate to his/her license, and shall exercise their abilities in the use of Emergency Operating Procedures, Technical Specifications, Emergency Plan.
The scenario's net time (not including briefings, setup or simulation facility problem related times) should average 50 minutes, based upon real time performance.
For the plant walk-through examination, the following items must be provided to the NRC 60 days prior to the examination date.
- 1. A list of topics appropriate to the plant walkthrough examination.
Systems which are applicable to the examination should be identified to the NRC, in addition, the rationale for this selection will be stated.
The topics selected should be based upon systems which are most applicable to maintenance of public health and safety in the mitigation of the consequences of an event and those systems that can directly initiate an event.
All reference material required to support an examination on these topics.
- 2. 15 Job Performance Measures.
These performance mearsures should be in-plant operator functions, that are required for the safe opertion of the facility. They shall include acceptable performance criteria.
- 3. If additional reference material is required for examiner preparation, it will be requested during the pre-exam site visit.
ADMINISTRATION OF REQUALIFICATION EXAMINATIONS
- 1. Up to three crews shall be selected for evaluation.
Normally the crew currently in the requalification cycle will be selected. A random sample without replacement will be used to preclude an operator from being subject to re-examination during the term of the license. The sample will include up to two other shifts made up of licensed personnel who are not routinely performing shift duties.
- 2. The simulator and a simulator operator(s) will be provided for examination development. The date(s) and duration of time needed t.o develop the examinations will be agreed upon by the Chief Examiner and the facility.
- 3. The reference material used in the simulator will be reviewed by the Chief Examiner, no material will be made available that is solely used for training.
- 4. A single room shall be provided for completing Section B of the written examination. The location of this room and supporting restroom facilities shall be such as to prevent contact with all other facility and/or contractor personnel during the duration of the examination.
- 5. Minimum spacing is required to ensure examination integrity as determined by the chief examiner.
Minimum spacing should be one candidate per table, with a 3-ft space between tables.
No wall charts, models, and/or other training materials shall be present in the examination room.
- 6. Copies of reference material for Section B of the written examination will be provided for each candidate.
The reference material will be reviewed by the chief examiner and will consist of Technical Specifications, operating/abnormal procedures, administrative procedures, Emergency Plans as available to the plant operators.
- 7. Video taping capabilities can be utlized. The facility should contact the Chief Examiner for restrictions related to its usage.
NRC RULES AND GUIDANCE FOR EXAMINEES
- 1. Restroom trips are to be limited and only one examinee at a time may leave.
You must avoid all contacts with anyone outside the examination room to avoid even the appearance or possibility of cheating.
- 2. Use black ink or dark pencil only to facilitate legible reproductions.
- 3. Print your name in the blank provided on the cover sheet of the examination.
- 4. Fill in the date on the cover sheet of the examination (if necessary).
- 5. Use only the paper provided for answers.
- 6. Answer each question on the examination.
If additional answer sheets are needed use only the lined paper provided, number each answer as to category and number, for example, A.1, B.2.
Answer one question per page.
- 7. Use abbreviations only if they are commonly used i.n facility literature.
- 8. The point value for each question is indicated in parentheses after the question and can be used as a guide for the depth of answer required.
- 9. Show all calculations, methods, or assumptions used to obtain an answer to mathematical problems whether indicated in the question or not.
- 10.
Unless solicited, the location of references for answers need not be stated.
- 11.
Partial credit may be given. Therefore, ANSWER ALL PARTS OF THE QUESTION AND DO NOT LEAVE ANY ANSWER BLANK.
- 12.
If parts of the examination are not clear as to intent, ask questions of the examiner only.
- 13.
You must sign the statement on the cover sheet that indicates that the work is your own and you have not received or been given assistance in completing the examination. This must be done after the examination has been completed.
- 14.
Cheating on the examination means an automatic denial of your renewal application and could result in more severe penalties.
- 15.
When you are finished and have turned in your completed examination, leave the examination area, if you are found in this area while the examination is still in progress, your license may be revoked.
AGENDA FOR MEETING WITH CAROLINA POWER LIGHT COMPANY CONCERNING REQUALIFICATION EXAM October 28, 1987 1:00 p.m. -
Introductions, present agenda
- Provide program description to CP&L for review 1:15-1:45 - CP&L review program 1:15-1:45 - OLB meet with RII representatives 1:45-2:30 -
Review program with CP&L 2:30-4:00 - Discuss details of program A. Administrative B. Simulator Evaluation C. Plant walk through D. Written E. Operator Feedback 4:30 p.m. - Discuss logistical details points of contact -
NRC and CP&L Crew identification
- Crew identification Milestones for development
- Future meeting requirements 5:00 p.m. - Adjourn MEETING ATTENDEES Licensee C e ACharles A. Bethea Charles A. Casto
- j. M. Curley Caudle A. Julian F. L. Lowery Ken Brockman R. S. Allen John Munro Edward E. Utley R. M. Latta Guy Beatty John N. Hannon Terrence R. Byron Len Wiens J. B. Allen Ronald Eaton B. J. Furr