NRC Generic Letter 82-13, Reactor Operator and Senior Reactor Operator Examinations
text
WASHINGTON, D. C. 20555
June 17, 1982
TO ALL POWER REACTOR LICENSEES, APPLICANTS FOR AN OPERATING LICENSE AND HOLDERS OF A CONSTRUCTION PERMIT
Gentlemen:
Subject: Reactor Operator and Senior Reactor Operator Examinations (Generic Letter 82-13)
Enclosed are the results of a meeting held in Bethesda, MD, on January 6, 1982, to discuss changes to the examination used to license Reactor and Senior Reactor Operators. The questions and comments raised during that meeting are discussed in the enclosed summary.
This letter is for your information only and requires no response or action.
Sincerely,
Darrell G. Eisenhut, Director Division of Licensing Office of Nuclear Reactor Regulation
Enclosure:
- 1. Response to Questions/
Comments
- 2. List of AttendeesGeneral Topic: Written Examination Questions
Questions/Suggestions:
Questions should not require lengthy calculations that are not done on the job.
Questions should not be taken from the FSAR since it is worst case and may not be up to date.
Technical Specification questions should not require knowing actions after one hour since the SRO can look these up.
More short answer or multiple choice questions should be used.
Health Physics questions should concentrate on operating information not time-distance-shielding type questions.
Mitigating core damage should be included in exams.
RO Exams should not include Tech Specs or Emergency Plan since these are not her responsibility.
If new topics are to be included in the exams, the utilities should be notified. In addition to submitting good questions, utilities should be able to identify questions they feel are bad.
Response:
As indicated in the January 6 meeting, emphasis in the exam will be on operational information. To ensure that exam questions are in line with the guidelines given to you, we are reviewing the questions and deleting or modifying ones that do not comply. As discussed in the exam consistency topic, we are developing a computerized bank of exam questions. When fully developed and reviewed, we will make this bank available to the public. We encourage submission of questions for the bank and comments on questions that you feel are not valid. In this effort we are eliminating the Health Physics questions that are not generally the responsibility of the RO or SRO, and arranging that Tech Spec questions concentrate on understanding of the bases, general knowledge of what actions are required immediately (within one hour) and why, and what systems have tech spec limits and why. In the operational exam the candidate's ability to find and use the tech specs will be examined. Question content may still include calculations to determine the candidate's understanding of the principles involved in nuclear plant operation, but calculations for the sake of calculations will be avoided.
We are investigating short answer and multiple choice question formats and have used both in exams that have been given. We will not, however, shift entirely to this format until we have assured that the questions present a fair opportunity for the candidate to show his or her knowledge and we have notified the industry that we will be changing exam style.
We are continually developing new questions from the training material provided. We will continue to use the FSAR as a source of information, but we recognize its limitations. As always, out of date information should be pointed out in the exam critique so that questions and answers can be updated. We hope in the near future to develop a regular information letter to inform the public of new areas of interest in the exam.
.
General Topic: Examination Administration
Questions/Suggestions:
What responsibilities/authority do proctors, especially when it is the Resident Inspector, have?
Will questions be made available to allow trainers to know what areas should be covered?
The exam should be open book, allowing access to the materials the operator would have in the control room (i.e., Tech Specs).
Will NRC guidance be updated to reflect the new exam format?
What options are open for taking the written exam? AM/PM sessions? One sitting? And what options for splitting sections are available if a split period used?
Reducing the number of questions (time) increases the importance of each question.
Formula/Equation sheet should be standardized.
Can the utilities get copies of questions submitted by other utilities?
NRC should recommend a list of standard texts.
A firm date for implementing the new format should be set.
Reporting results should be standardized. Some people get summary sheets, some don't know results until licenses or denials arrive. Results should be in faster.
Why not leave a copy of the exam after it has been taken?
Can the utilities get a copy of the exam in advance to allow for a thorough review.
Why establish a time limit? The exam should find out what you know, not how fast you can write.
Why not use qualified industry people to administer exams at other facilities?
Better guidance is needed on what will be covered in the exam and better information on grading criteria and granting waivers should be available. There is no way to check on the results of oral exams.
Are machine prepared, machine graded exams possible? Probable?
Better clarification of the March 28, 1980 letter on qualifications is needed.
.
Response:
Your concerns fall into several areas. First, the exam may be taken either at one sitting or broken into two parts. The utility has the option, but all candidates must take the exam the same way. The exam will be given, as indicated, section 1, 2, and 3 or 6, 7, and 8 in the first three hours and 4, 5, or 9, 10 in the second three hours if a split sitting is elected. The examiners or proctors have the authority to allow individual breaks during the exam, but the exam must still be finished within the allowed time. The examiner or proctor will keep track of elapsed time.
Second, only examiners are authorized to modify an exam. Proctors, even resident inspectors, are not authorized to make changes. Proposed changes should immediately be called to the attention of the assigned chief examiner.
Third, as discussed in the exam questions topic, we are reviewing the questions, investigating multiple choice and short answer questions to allow more areas to be covered in the limited amount of time, however, a time limit will remain. We are investigating machine generated/machine graded formats but have no immediate plans for implementing such a system. Before a change like that will be adopted, the approach will be validated and your comments solicited prior to any action being taken. Once fully developed, we will consider making the questions contained in the exam question bank available to assist you in your training program and in keeping the questions current.
Fourth, we do not intend to go to an open book exam. We are working to ensure that the written exam tests for information that the candidate should know without aids and the operational exam tests his ability to use aids such as procedures and Tech Specs.
Fifth, we are in the process of developing updated guidance on exam content, objective and subjective grading criteria, exam administration and application content. We will not endorse a set of reference texts. This guidance, coupled with greater accessibility of exam questions, should improve information available on what the exam will cover. Any texts that suitably cover the material are acceptable.
Sixth, we are implementing a system to allow automatic, computer-aided tracking of applications and exam results. Our goal is to have all results reported to the individuals and utilities within two months of completion of the examination. As more examiners are certified this goal will be reviewed to see if we can reduce it even further. You can assist us by refraining from calling for results or submitting FOIA requests until after the two months have passed. Exam results are not final until all portions of the exam have been completed and internal audits for consistency and fairness are done. At that point, exam summary sheets will be sent to the utility, and licenses or denials and a copy of the written exam will be sent to the individual.
Finally, almost all old format reexaminations have been completed and sufficient experience has been gained in preparing new format exams. Therefore, only new format exams will be given for examinations scheduled after July 1, 1982.
.
General Topic: Examination Consistency
Questions/Suggestions:
Some examiners lack the knowledge and training necessary to give oral exams. They use poor methods and intimidate the candidates.
Some examiners are too academically oriented. There are large inconsistencies between examiners. You almost have to know who is preparing your exam so you can prepare the candidates in the proper areas.
Some examiners will not accept an operating method that differs from a "cookbook" method even though the alternate method may be equally acceptable. Therefore, correct answers are marked wrong.
There are no standards for required level of knowledge for oral exams. Therefore, the depth required varies between examiners.
Response:
As discussed in other sections, we are working on guidelines for exam content and developing a question bank of valid questions. Since these will include plant specific, as well as generic questions, the operating philosophy of each facility can be reflected in their exam questions.
For oral exams we are preparing guidelines for the examiners. Appropriate portions will be made publicly available.
We have also established a training and certification program for contract, examiners. This will ensure that a minimum competence has been obtained prior to conducting exams. To ensure competence and consistency, all examiners, NRC and contractors, will be audited periodically by the OLB Section Leader. We are sensitive to the issue of competency and consistency because we are actively transferring examiner functions to the Regional Offices. Any specific feedback from an examination is welcome and will be kept confidential.
.
General Topic: Simulator Exams
Questions/Suggestions:
Plant specific simulators should be mandated by NRC.
What are NRC's future plans for simulator exams? More guidance is needed on what will be covered in simulator exams, especially for non-plant-specific simulators.
Response:
A Commission Paper has been prepared recommending that the requirement for non-plant-specific simulator exams be removed. We have studies underway to develop valid operational exams, including simulator exams. At this time we feel that dynamic transient operation is an important aspect of the exam but that the limitations imposed by non-plant-specific simulators and the scheduling problem encountered reduce the effectiveness of the exam and do not justify the resources required. Therefore, we are considering returning to the old exam method of performing power transients, startups and shutdowns on those facilities that do not have a plant specific simulator available. Any change in the exam will be discussed before it is implemented. Until the need for operating tests on the facility or other testing methods being studied by the staff are validated, the operating examination will continue to follow the existing guidelines in NUREG 0094. Therefore, there is a continued need for simulator training to comply with the guidance in NUREG 0094. The Commission has not acted on the staff recommendation at this time. The Commission paper does not include any requirements for actual plant operations at this time.
As discussed under the exam administration topic, we are developing guidelines for simulator exams. We are also modifying the operational exam summary sheet to facilitate the simulator exam and to ensure that areas examined at, the simulator are not duplicated in the plant walk-through.
Miscellaneous Question:
Where are guidelines for medical applications?
Response:
Basic requirements are in Part 55. Amplifying guidance in Regulatory Guide 1.134 which endorses ANSI/ANS 546.
Suggestion:
Fallback to RO on instant SRO failure isn't worth, much since engineers or supervisors aren't in the union and cannot perform RO duties.
Response:
We agree that fallback has little value. When a candidate is certified to need an SRO license to perform his or her duties, we don't see how having an RO will help. Under the new format exam, this problem should disappear.
.
Suggestion:
NRC should put certification of training instructors on high priority since this could have a real impact on training.
Response:
We agree entirely. Our highest priority, as always, is licensing new operators. As more examiners are more examiners are trained and certified, we will be pursuing the area of instructor certification more vigorously. All Operating License applicants are required to have certified instructors prior to fuel load and we have certified instructors at several operating plants. We are monitoring the INPO work on training program accreditation and are investigating means to certify vendor and consultant instructors that teach systems and operations courses. More information will be available on this subject in the near future.
Question:
What are NRC's plans for requalification exams?
Response:
In the Commission paper on non-plant-specific simulators we discuss the subject of requalification at length. To summarize, as directed by the Commission, we will start giving requalification examinations in conjunction with scheduled visits for replacement exams. We are targeting at least one site visit to each facility this year and expect to administer requalification examinations to 20% of the licensed operators. Our initial plans are to conduct the requalification exams during replacement exams; however, we will work out availability and schedules with each utility. In the Commission paper, the staff proposed that for utilities with a plant specific simulator, the requalification exam will be given only on the simulator. Otherwise, a written exam and practical test will be given. Failure of the exam will require accelerated retraining in weak areas, as is required now, and NRC reexamination. If significant weaknesses in the utility requalification program are revealed by the exams, NRC administered requalification exams for all license renewal applicants will be likely until the requalification program has been sufficiently upgraded. Details of the program are in the Commission paper. No Commission action has been taken at this time.
Suggestion:
NRC should periodically issue a listing of generic weak points.
Response:
When our automated system is operational, we intend to issue quarterly information reports that will include observed weak areas. It will also include areas of concern at NRC that might affect the content of future exams and general information on the exams. We hope to be producing these reports in the very near future.
.
Suggestion:
Cold plants need amplification of H. Denton's March 28, 1980 letter regarding experience requirements.
Response:
It is difficult to provide better generic guidance than exists in the letter Task Action Plan Item I.A.2.1 NUREG-0737 and NUREG 0094. Specific cases can be discussed with the appropriate OLB Section Leader. The OLB policy is to grant exemptions or waivers only in specific cases, not on a generic basis. Therefore, the requirements should be discussed with the Section Leaders.