ML20070D260

Operator Licensing Examiner Standards

ML20070D260
Person / Time
Issue date:	06/30/1994
From:	Office of Nuclear Reactor Regulation
To:
References
NUREG-1021, NUREG-1021-R07-S01, NUREG-1021-R7-S1, NUREG-1021-S01, NUREG-1021-S1, NUDOCS 9407070304
Download: ML20070D260 (120)

Text

_

4 I

NUREG-1021 Rev.7,Supp.1 Oaerator Licensing Examiner Standards (V 1) ate l'uhlished: June 1994 Manuscript Completed: June 1994 Division ofIteactor Controls and lluman Factors Office of Nuclear lleactor llegulation U.S. Nuclear Itegulatory Commission Washingion, DC 20555-000i f..a== aw,,

%..... E d

9407070304 940630 PDR NUREO 1021 R PDR t

AVAILABILITY NOTICE Availability of Reference Materials Cited in NRC Publications Most documer;ts cited in NRC publications will be available from one of the following sources:

1.

The NRC Public Document Room, 2120 L Street, NW., Lower Level, Washington, DC 20555-0001 2.

The Superintendent of Documents, U.S. Government Printing Office, Mail Stop SSOP, Washington, DC 20402-9328 3.

The National Technical Information Service, Springfield, VA 22161 Although the listing that follows represents the majority of documents cited in NRC publica-tions, it is not intended to be exhaustive.

Referenced documents available for inspection and copying for a fee from the NRC Public Document Room include NRC correspondence and internal NRC memoranda; NRC bulletins, circulars, information notices, inspection and investigation notices; licensee event reports:

vendor reports and correspondence; Commission papers; and applicant and licensee docu-ments and correspondence.

The following documents in the NUREG series are available for purchase from the GPO Sales Program: formal NRC staff and contractor reports, NRC-sponsored conference proceedings, international agreement reports, grant publications, and NRC booklets and brochures. Also available are regulatory guides, NRC regulations in the Code of Federal Regulations, and Nu-clear Regulatory Commission Issuances.

Documents available from the National Technical Information Service include NUREG-series reports and technical reports prepared by other Federal agencies and reports prepared by the Atomic Energy Commission, forerunner agency to the Nuclear Regulatory Commission.

Documents available from public and special technical libraries include all open literature items, such as books, journal articles, and transactions. Federal Reg / ster notices, Federal and State legislation, and congressional reports can usually be obtained from these libraries.

Documents such as theses, dissertations, foreign reports and translations, and non-NRC con-forence proceedings are available for purchase from ine organization sponsoring the publica-tion cited.

Single copies of NRC draft reports are available free, to the extent of supply, upon written request to the Office of Administration, Distribution and Ma;l Services Section, U.S. Nuclear Regulatory Commission, Washington, DC 20555-0001.

Copies of industry codes and standards used in a substantive manner in the NRC regulatory process are maintained at the NRC Library,7920 Norfolk Avenue, Bethesda, Maryland, for use by the public. Codes and standards are usually copyrighted and may be purchased from the originating organization or, if they are American National Standards, from the American Na-tional Standards )nstitute,1430 Broadway, New York, NY 10018.

O

l l

I i

4

.-- )

j NUREG-1021 OPERATOR LICENSING EXAMINER STANDARDS JUNE 1994

(

Rev. 7, Supp.1

ABSTRACT The Operator Licensing Examiner Standards provide policy and guidance to NRC examiners and establish the procedures and practices for examining licensees and applicants for reactor operator and senior reactor operator licenses at power reactor facilities pursuant to Part 55 of Title 10 of the fade of Federal Reculations (10 CFR 55). The Examiner Standards are intended to assist NRC examiners and facility licensees to better understand the initial and requalification examination processes and to ensure the equitable and consistent administration of examinations to all applicants.

These standards are not a substitute for the operator licensing regulations and are subject to revision or other internal operator licensing policy changes.

Revision 7 was published in January 1993 and became effective in August 1993.

Supplement 1 is being issued primarily to implement administrative changes to the requalification examination program resulting from the amendment to 10 CFR 55 that eliminated the requirement for every licensed operator to pass an NRC-conducted requalification examination as a condition for license renewal. The supplement does not substantially alter either the initial or requalification examination processes and will become effective 30 days after its publication is noticed in the Federal Reaister.

The corporate notification letters issued after the effective date will provide facility licensees with at least 90 days notice that the examinations will be administered in accordance with the p) revised procedures.

(v A vertical line in the right margin indicates that material has been added or changed; a vertical line in the left margin indicates that material has been deleted.

(

Examiner Standards iii Rev. 7, Sup. 1, June 1994

l 0

Supplement I to Revision 7 of NUREG-1021,

" Operator Licensing Examiner Standards" Instructions Remove existing Revision 7 pages and insert replacement pages as noted below:

Section / Standard Remove Page(s)

Insert Page(s)

Abstract iii 111 Supplement Instructions blank iv ES-201 3 - 8, 13, 14 3 - 8, 13, 14 ES-202 3-8 3-8 ES-205 1, 2, 5, 6, 13 1, 2, 5, 6, 13 ES-301 5 - 8, 13, 14 5 - 8, 13, 14 ES-302 9, 10 9, 10 ES-303 9 - 12 9 - 12 ES-401 1, 2 1, 2 ES-501 3-8 3-8 ES-601 1 - 40 1 - 41 ES-602 1 - 10, 31, 32 1 - 10, 31, 32 ES-604 5-8 5-8 ES-605 1 - 14 1 - 13 Supplement 1 Inserted by:

_ Date:

i O

Examiner Standards iv Rev. 7, Sup. 1, June 1994 t

ES-201 section chief's approval provided the assigned examiners are available and other examinations on the national examination schedule are not affected.

If examinations involving contract examiners must be rescheduled or if there is a need to modify the task order, the chief examiner shall notify the project manager in the Division of Reactor Controls and Human Factors (DRCH) before l

the work modification (refer to Section D).

The chief examiner should schedule the operating tests to maximize the use of examiners and optimize the mix of reactor operator and senior reactor operator applicants in the crews assembled for the simulator examinations. Normally, for purposes of test integration and continuity, the chief exaniner should schedule the same examiner to administer all three operating test categories to an applicant.

However, under certain circumstances, such as might occur if a licensee's simulation facility is not located near the plant or if a licensee requests examinations for an unusually large group of applicants, the responsible regional section chief may authorize the chief examiner to divide the operating test among different examiners. Chief examiners shall not assign an examiner who failed an applicant on an operating test to administer any part of that applicant's retake operating test.

Operating tests normally will be administered on regular work days.

If weekend or shift work is required to administer the operating tests, the chief examiner will coordinate the arrangements with the assigned examiners and the facility licensee.

(

D.

ASSIGNMENT OF EXAMINERS Regional section chiefs will assign the required number of examiners to prepare and administer the requested number of examinations on the dates arranged with the facility licensee.

Normally, the section chief will assign a sufficient number of examiners so that no examiner will have to administer more than 4 operating tests per week.

Under no circumstances will an examiner be required to administer more than 5 operating tests in any one week.

The section chief should consider each examiner's certification status, other examination commitments, and general availability when making assignments.

Section chiefs should determine the need for contract examiner support at least 6 months before each scheduled examination date and request the appropriate level of support from DRCH as described below.

l Section chiefs will assign a chief examiner for every examination.

If a chief examiner is not available in the region, the section chief will request one from another region or headquarters.

Contract examiners will not be tasked to act as chief examiner.

If an examiner is assigned to an examination that might appear to present a conflict of interest, the examiner shall inform his or her immediate supervisor of the potential conflict.

The examiner shall discuss the following areas, as applicable, with his or her supervisor:

1.

the length of time the examiner worked at the facility

\\s Examiner Standards 3 of 15 Rev. 7, Sup. 1, June 1994

/

ES-201 2.

the time that has elapsed since the examiner left the facility 3.

the reasons the examiner terminated employment at the facility 4.

the nature and extent of previous relationships with former associates being examined or employed at the facility 5.

anything that could affect the administration, performance, evaluation, or results of the examination or create the acoearance of a conflict of interest Contract managers will not accept an examination assignment that appears to create a conflict of interest.

If they can not reassign their examiners to eliminate the conflict, they shall notify the project manager, DRCH, and the l

OLB technical monitor as soon as possible so that the problem can be evaluated and resolved.

Section chiefs will not assign an examiner who failed an applicant on an operating test to administer that applicant's retake operating test, nor will they assign an examiner who was previously employed by a facility licensee (or one of its contractors) any direct responsibilities for developing or administering written examinations or operating tests at that licensee's facilities for at least 2 years after the examiner terminated employment with the facility licensee.

Regional management will control other in-office examination activities concerning the facility, such as technical consultation and quality assurance reviews of examinations.

Criteria that identify every possible conflict-of-interest issue cannot be anticipated.

Supervisors must apply sound judgment to the facts of each case.

If there is any doubt regarding a particular case, the supervisor should consult with OLB to resolve the issue.

Section chiefs or their designees should use the computer-based scheduling system to request contractor support from DRCH.

DRCH will assign contract l

examiners at least 4 months in advance to allow for necessary adjustments to facility training or examination schedules.

If a region requires a new contract examiner assignment or a change in an existing assignment less than 4 months before the date of an examination, the request should be made as soon as possible (and preferably before work starts) by telephone and follow up with an entry or change on the computerized scheduling system.

The OLB technical monitor and regional section chiefs will normally resolve conflicts in scheduling contract examiners.

If they cannot agree, the Branch Chief, OLB, and regional branch chiefs shall resolve the conflict.

Section chiefs will ensure that an examination assignment sheet in the format of Attachment 3 is prepared as far in advance as possible, but at least 2 weeks before the scheduled examination date.

The assignment sheet should be completed after the preliminary license applications are reviewed and approved and all waiver requests are resolved (refer to Section E and ES-204).

The assignment sheet will identify the chief and other examiners by name and list Examiner Standards 4 of 15 Rev. 7, Sup. 1, June 1994

/]

ES-201 O

the applicants by name, docket number, and the type of examination (e.g., SR0 Upgrade, R0 written only) they are to be administered.

All applicants listed on the assignment sheet should be administered complete examinations (written and operating) as indicated under " Examination Type" unless waivers have been granted in accordance with ES-204. A copy of the assignment sheet will be distributed to all examiners assigned, the NRC resident inspector at the facility, the NRR project manager, and regional distribution as established by the regional administrator. A copy of the assignment sheet will be updated (pen-and-ink) to reflect the examinations as they were administered and placed in the examination master file for the facility.

E.

REVIEWING LICENSE APPLICATIONS When the preliminary license applications are received approximately 30 days before the examination date, the regional office shall review the applications in accordance with ES-202 to determine if the applicants meet the eligibility requirements specified in 10 CFR 55.31.

Those applicants who satisfy the eligibility requirements will be identified on the examination assignment sheet as described in Section D above.

F.

PREEXAMINATION SITE VISITS In some instances, it may be advantageous for some of the assigned examiners (mv) f to make a preparatory trip to the facility before the scheduled examination date.

This will allow the examiners to familiarize themselves with the plant and its simulation facility and provide an opportunity for the examiners to review and validate the examination materials with the facility licensee as discussed in Section G.

The responsible regional section chief will evaluate each initial examination assignment to determine if a separate preparatory site visit should be conducted. The section chief should carefully weigh the costs and benefits associated with each additional trip to the facility and consider factors such as the experience of the assigned examiners, the number of written examinations and operating tests to be validated, and the status of the simulation facility (e.g., Is it new or recently upgraded?) when making a decision.

If the operating tests will not be ready for validation in time for a preparatory site visit, the section chief should consider the alternative of reviewing the written examination (s) with the facility licensee in the regional office and validating the operating tests at the beginning of the examination week.

The section chief shall determine the need for a preparatory site visit as soon as possible after confirming the examination date with the facility licensee. The section chief shall also ensure that any contract examiners who are involved with the examination assignment are informed of the region's intent to conduct the site visit and that the visit and any associated contractor support that might be necessary are recorded on the computer-based

[_}/

scheduling system. This will enable the contract examiners to expedite the s

Examiner Standards 5 of 15 Rev. 7, January 1993

ES-201 delivery date of the examination materials, if possible, and allow DRCH to l

1ssue the appropriate task orders.

G.

FACILITY PREEXAMINATION REVIEWS The chief examiner should make arrangements for the facility licensee to review the initial licensing examinations before they are administered.

This review will normally be conducted 2 weeks before the week in which the examinations are scheduled and should include all sections of the licensing examination (i.e., the written examination (s), the simulator scenarios, and the walk-through outlines) that have been completed and approved for facility review by the responsible section chief.

If the examination materials are being developed by a contractor and circumstances necessitate an earlier prereview by the facility licensee, the chief examiner must notify the contractor of the advanced delivery date as soon as possible so that the contractor's resources can be scheduled accordingly.

If the contractor is unable to support the early delivery date, the responsible section chief will make a decision whether to delay or waive the review or delay the examination.

The regional office will conduct the examination prereview in accordance with the guidelines and instructions contained in Attachment 4.

ATTACHMENTS / FORMS:,

" Sample Corporate Notification Letter",

" Reference Material Guidelines For Initial Operator Licensing Examinations",

" Sample Examination Assignment Sheet",

" Guidelines for Facility Prereview of Initial Licensing Examinations" Form ES-201-1,

" Preexamination Check Sheet" Form ES-201-2,

" Sample Examination Security Agreement" O

Examiner Standards 6 of 15 Rev. 7, Sup. 1, June 1994 w

O ES-201 Sample Corporate Notification Letter NRC Letterhead (date)

(Name. title)

D{ame of facility)

(Addresi).

(City. State. Zio code)

Dear RLad:

In a telephone conversation on (date) between Mr. (Name. title) and Mr. (Name.

title), arrangements were made for the administration of licensing examinations at the (facility name).

The written examinations are scheduled for idalgl, and the operating tests are scheduled for (date).

Your staff will be given an opportunity to review the licensing examinations in accordance with the guidelines in Revision 7 of NUREG-1021, " Operator Licensing Examiner Standards," (ES-201, Attachment 4) during the week of (date).

()

To prepare the examinations and meet the above schedule, it will be necessary for your staff to furnish the reference materials identified in Attachment 2 of ES-201 by (date). Any delay in receiving approved and indexed reference material or the submittal of inadequate or incomplete reference material may l

result in the examinations being rescheduled.

The NRC will prepare and adm'nister the written examinations in accordance with ES-401 and ES-402 of NUREG-1021.

In order to conduct the requested examinations, it will be necessary for your staff to provide adequate space and accommodations on the date noted above. The NRC's guidelines for administering the written examinations are described in ES-402, Attachment 1.

The NRC will prepare and administer the operating tests in accordance with ES-301 and ES-302 of NUREG-1021.

In order to conduct the requested operating tests, it will be necessary for your staff to make the simulation facility available on the dates noted above.

Your staff should retain the original simulator performance data (e.g., system pressures, temperatures, and levels) generated during the dynamic operating tests until the examination results are final.

(

ES-402, Attachment 2, and ES-302, Attachment 1, contain a number of NRC i

policies and guidelines that will be in effect while the written examination and operating tests are being administered.

.Q k_)

Examiner Standards 7 of 15 Rev. 7, January 1993 j

i l

O ES-201 2

Your staff should submit preliminary reactor operator and senior reactor operator license applications and waiver requests at least 30 days before the first examination date so that the NRC will be able to review the applications and the medical certifications and evaluate any requested waivers.

If the applications are not received at least 30 days before the examination date, a postponement may be necessary.

Final, signed applications certifying that all training has been completed should be submitted at least 14 days before the first examination date.

This request is covered by Office of Management and Budget (0MB) Clearance Number 3150-0101, which expires October 31, 1995.

The estimated average burden is 7.7 hours8.101852e-5 days <br />0.00194 hours <br />1.157407e-5 weeks <br />2.6635e-6 months <br /> per response, including gathering, xeroxing and mailing the required material.

Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to the Information and Records Management Branch, Mail Stop T-6 F30, Office of Information Resources Management, U.S. Nuclear Regulatory Commission, Washington, D.C. 20555; and to the Paperwork Reduction Project (3150-0101), Office of Information and Regulatory Affairs, NE08-3019, Office of Management and Budget, Washington, D.C. 20503.

Thank you for your cooperation in this matter.

(Name) has been advised of the policies and guidelines referenced in this letter.

If you have any questions regarding the NRC's examination procedures and guidelines, please contact (Name of reaional contact) at (telechone number), or (Name of branch chief or lection chief) at (teleohone number).

Sincerely, (Appropriate regional representative)

Distribution:

Public Document Room NRC Document Control System Regional Distribution O

Examiner Standards 8 of 15 Rev 7, Sup. 1, June 1994

V ES-201 Guidelines for Facility Prereview of Initial Licensing Examinations 1.

The prereview may be conducted at the site or in the NRC regional office as discussed in Sections F and G of this standard. Only those sections of the licensing examination that have been reviewed and approved by the appropriate NRC authority may be submitted to the facility for review.

2.

The facility reviewers will be given access to the examination materials after they sign a security statement (form ES-201-2).

3.

The prereview should normally take one day for each written examination and another day for the operating tests; more time may be necessary for I

large classes when more material must be reviewed.

If the review takes more than one day, the chief examiner should brief the section chief on the status of the review (i.e., scope of changes and unresolved issues) at the end of each day.

Regional management is expected to monitor the review process for possible indications of inadequate facility reference material or insufficient quality assurance by the NRC.

4.

The facility reviewers should make their recommendations and comments on the copy of the written examination (s) and answer key (s) provided to them by the NRC examiner.

Simple editorial changes that do not change the intent of the question require no justification, but every substantive change (e.g., deleting a question, replacing a distractor, n

or revising an answer) must be supported by approved facility reference V)

(

material.

If the original reference material submittal does not support the change, the reviewers are expected to provide the necessary documentation before the NRC will change the examination.

5.

The operating tests (simulator scenarios and job performance measures) may be prereviewed and evaluated in the office or validated at the facility if the regional section chief determines that a site visit is necessary.

If the operating tests are not validated during the prereview or if they undergo significant alteration as a result of the areliminary validation, they should receive final validation at the aeginning of the examination week (refer to ES-302).

6.

An NRC examiner will be available throughout the review and validation process to discuss the examination materials and the reviewers' concerns.

The facility reviewers should return the marked-u, examination materials to the chief examiner and ensure that be or she l

understands their comments and recommendations before they conclude the prereview meeting.

If the facility reviewers have significant concerns regarding the quality or difficulty of the examination materials, it is their responsibility to convey such concerns to NRC management.

7.

The facility reviewers will not be allowed to retain a copy of any of the examination materials (written, simulator, or walk-through) or written notes taken during the review arocess.

A copy of their prereview comments will be made availaale to them for reference during n

(

the post-examination review (refer to ES-402).

Examiner Standards 13 of 15 Rev. 7, Sup. 1, June 1994

.[

M O'

ES-201 Preexamination Check Sheet Form ES-201-1 Facility Date of Examination Due Task Description Initials Date Date

-180 1.

Examination schedule agreement reached

-180 2.

Contract examiners requested if required

-120 3.

Examiners assigned

-120 4.

Contract examiners assigned by DRCH l

-120 5.

Corporate notification letter sent

-90 6.

Reference material received

-60 7.

Proctoring arranged, if needed

-30 8.

Preliminary applications received

-30 9.

Travel arrangements made

-21

10. Written examinations prepared (ES-401)

-14

11. Assignment sheet prepared

-14

12. Final applications received

-14*

13. Category B walk-through outlines received

-14

14. Written examinations approved by section chief for facility review

-10*

15. Simulator scenarios prepared (received by region 7 working days before exam date)

-7

16. Written examinations reviewed by facility

-7

17. Final applications reviewed; assignment sheet updated; waiver letters sent

-5

18. Approved scenarios job performance measures, and questions received by all examiners

-5

19. Final written examinations approved by section chief Dates may be adjusted to permit facility prereview and validation.

Examinor Standards 14 of 15 Rev. 7, Sup. 1, June 1994

O ES-202 b

1.

All Acolications a.

To apply for an R0 or SR0 license, an applicant must submit an NRC Form 398, " Personal Qualifications Statement-Licensee," and an NRC Form 396, " Certification of Medical Examination by Facility Licensee" (computer-generated duplicates are acceptable).

The application is not complete until both forms are filled out, signed by the appropriate personnel, and received by the NRC.

Detailed instructions for completing NRC Form 398 are provided with the form.

Refer to Sections D.2 for additional information regarding retake applications.

b.

As noted in ES-201, the facility licensee should submit preliminary, uncertified license applications and medical certifications to the region for review at least 30 days before the examination date. The region shall review the applications as soon as possible, determine license eligibility, process the medical certifications and any waiver requests (refer to ES-204),

and request any additional information that might be necessary.

The medical data in support of NRC Form 396 are normally good for 6 months from the date of the medical examination.

However, if the applicant is reapplying after withdrawing a previous

/]

application or accepting a final license denial, he or she may V

request a waiver by checking Item 4.f.4 on NRC Form 398 and either providing an explanation in Item 17, " Comments," of the form or submitting a separate letter with the application.

When the applicant has completed all the required training for the desired license level, the facility licensee's senior management representative on site must certify that fact by placing a check in Item 19.b of NRC Form 398, signing the form, and submitting it to the region at least 14 days before the examination date.

c.

Each applicant must satisfactorily complete the NRC's generic fundamentals examination (GFE) section of the written operator licensing examination for the applicable vendor or a site-specific theory section in an NRC-administered licensing examination that was given between February 1982 and November 1989 and resulted in the issuance of a license, d.

If the facility licensee has checked "yes" in Items 12.a and 12.b of NRC Form 398, indicating that the applicant has successfully completed a Commission-approved, SAT-based training program that uses a simulation facility acceptable to the Commission under 55.45(b), the licensee need not complete Items 13, " Training," 14,

" Experience," and 15, " Experience Details," on the form.

New applications must still include the number of significant control p

manipulations in Item 13.3.

b Examiner Standards 3 of 10 Rev. 7, January 1993 l

a

ES-202 If the application is complete and properly certified and Items e.

12.a and 12.b are checked "yes," the region may accept the l

application without further review unless there is reason to request further information on the applicant's qualifications.

The reviewer shall check the " meets requirements" block at the bottom of Form 398, sign and date the form, and enter the applicant's name and other data on the examination assignment sheet in accordance with ES-201.

f.

If Items 12.a and 12.b on Form 398 are checked "no," the region l

shall review the application against the specific R0 or SR0 eligibility guidelines described in Section E.

If the facility licensee completed Items 13, 14, and 15 even l

though they were not required, the region may review the information provided against the eligibility guidelines for the requested license level and resolve any deviations with the facility licensee.

g.

If the region determines that the application is incomplete or that the applicant does not meet the requirements in 10 CFR 55.31, it will note the deficiencies and request the facility licensee to supply the additional information to complete the application.

If the region determines that the applicant still does not meet the eligibility requirements, the regional licensing authority will notify the applicant in writing that the application is being denied and inform him or her of the deficiencies on which the denial is based (Attachment 1).

The section chief, or designee, shall check the "does not meet requirements" block at the bottom of Form 398 and sign and date the form.

The applicant shall not be permitted to take the license examination until he or she is determined to meet the eligibility requirements.

If the applicant does not accept the proposed denial, he or she may request the Director, Division of Reactor Controls and Human Factors (DRCH), Office of Nuclear Reactor Regulation (NRR), to review the application denial or request a hearing in accordance with 10 CFR 2.103(b)(2).

Further action will be taken in accordance with ES-502.

2.

Retake Acolications If the applicant is reapplying after a license denial, 10 CFR 55.35 applies and the applicant must submit new complete Forms 398 and 396.

The NRC will not process a retake application if the applicant's request for reconsideration or a hearing on the previous licenso denial is still outstanding (refer to ES-502). The region shall review the application as described in Section D.1, subject to the following additional conditions:

Examiner Standards 4 of 10 Rev. 7, Sup. 1, June 1994 i

h ES-202 Y

If the first application was denied because the applicant failed a.

the written examination or operating test or both, the applicant may file a new application 2 months after the date of the final deni al. The applicant may file a third application 6 months after the date of the second denial and successive applications 2 years after the date of each subsequent denial, b.

If the applicant previously passed either the written examination or the operating test, he or she may request a waiver of that portion of the licensing examination (refer to ES-204).

Waivers are limited to the first reapplication and must be requested within 1 year of the date of the failed examination.

The new Form 398 shall describe the extent of the applicant's c.

~

additional training since the denial and certify that the applicant is ready for reexamination.

E.

LICENSE ELIGIBILITY GUIDELINES Each facility licensee has committed, through its final safety analysis report, technical specifications, or quality assurance program, to specific 18.1-1971 guidelines for nuclear power plant worker qualifications (e.g., ANSI

/"]

or ANSI /ANS 3.1-1981).

If an examiner reviews an applicant's eligibility, he or she must ensure that the proper facility licensee commitment is used. The V

regional office should refer all questions regarding license eligibility to the Chief, OLB, for resolution.

Revision 2 of Regulatory Guide 1.8, " Qualification and Training of Personnel for Nuclear Power Plants," describes a method acceptable to the NRC staff for l

complying with the Commission's regulations with regard to the training and i

qualifications of nuclear power plant personnel.

For the positions of shift supervisor, senior operator, and licensed operator, Regulatory Guide 1.8, Revision 2, generally endorses the guidelines contained in ANSI /ANS-3.1-1981; specific clarifications, additions, and exceptions are noted in Section C,

" Regulatory Position," of the Regulatory Guide. The license eligibility guidelines in Regulatory Guide 1.8, Revision 2, and ANSI /ANS-3.1-1981 are summarized below.

1.

Reactor Operator a.

Experience j

(1)

The applicant should have a minimum of 3 years of power plant experience, at least 1 of which is spent at the nuclear power plant for which the license is sought (preferably in the performance of nonlicensed operator duties),

n v

Examiner Standards 5 of 10 Rev. 7, January 1993 f

[

ES-202 1

l (2)

The applicant should spend at least 6 months performing plant operational duties as a nonlicensed operator at the nuclear power plant for which the license is sought, b.

Training (1)

The applicant should complete at least 13 weeks as an extra person on shift in training for the R0 position.

This training should include all phases of day-to-day operations and be conducted under the supervision of licensed personnel.

(2)

The applicant should be trained in nuclear power plant fundamentals, plant systems, use of those systems to control or mitigate an accident during which the core is severely damaged, and operating practice.

(

(3)

The applicant should complete at least 500 hours0.00579 days <br />0.139 hours <br />8.267196e-4 weeks <br />1.9025e-4 months <br /> of lectures t

on the principles of reactor operation, design features and general operating characteristics of the nuclear power plant involved, instrumentation and control systems, safety and emergency systems, standard and emergency operating procedures, and radiation control and safety procedures.

(4)

The applicant should satisfactorily complete an NRC-approved train!ng program of at least 1-week duration at a nuclear power piort simui nor.

The simulator training center should certify the applicant's ability during a reactor startup to manipulate the controls, keep the reactor under control, predict instrument responses, use instrumentation, follow procedures, and explain annunciator alarms that occur during operation.

(5)

The applicant must manipulate the controls of the reactor (not simulator) during 5 significant changes in reactivity or power level (refer to 10 CFR 55.31(a)(5)).

Every effort should be made to diversify the reactivity and power changes for each applicant.

Startups, shutdowns, large load changes, and changes in rod programming are some examples and could be accomplished by manually using such systems as rod control, chemical shim control, and recirculation flow control.

c.

Education The applicant should have a high school diploma or equivalent.

O Examiner Standards 6 of 10 Rev. 7, Sup. 1, June 1994

i ES-202 2.

Senior Reactor Ooerator a.

Experience (1)

The applicant should have a minimum of 4 years of responsible power plant experience as defined in Section C.3.

At least 2 of those 4 years should be nuclear power plant experience.

(2)

The applicant should have actively performed licensed R0 duties for at least 1 year at the facility for which the SRO license is sought. The NRC may accept the following education or experience qualifications to satisfy this requirement if the applicant supplies sufficient details in the license application for the staff to make a judgement regarding its equivalence.

A 4-year degree in engineering or the equivalent (e.g., a degree in engineering technology or the physical sciences that includes course work in physics, mathematics, or engineering; a professional f

engineer's (PE) license obtained by passing the PE examination).

q At least 1 year as an active licensed R0 at a comparable facility (same vendor, similar vintage) or 18 months as an R0 at a noncomparable commercial power reactor.

At least 2 years in a position equivalent to a licensed R0 at a military reactor (refer to Section C.3).

Experience obtained in licensed positions (or their equivalent) on other large-scale reactors will be evaluated on a case-by-case basis. Applicants must also submit a waiver request in accordance with ES-204 if they want this experience to apply toward the requirement.

(3)

At least 6 months of the applicant's nuclear power plant experience should be at the site for which the license is sought.

O Examiner Standards 7 of 10 Rev. 7, January 1993

ES-202 (4)

During the 2 years nuclear power plant experience, the applicant should participate in reactor operator activities at power levels greater than 20 percent for at least 6 weeks.

b.

Training (1)

The applicant should complete at least 13 weeks as an extra person on shift in training for the SRO position.

This training should include all phases of day-to-day operations and be conducted under the supervision of licensed personnel. Any portion of the 13 weeks that is spent at or above 20 percent power may also be used to satisfy the experience guideline in Section E.2.a(4).

(2)

If the applicant has not held an R0 license at the facility and one of the qualifications specified in Section E.2.a(2) is substituted for that experience, the training guidelines of Sections E.1.b(4) and E.1.b(5) should be met.

The applicant should satisfactorily complete a training program that is comprehensive in its coverage of both R0 and SR0 knowledge, skills, and abilities and must take an SRO-instant license examination.

(3)

The applicant should be trained in nuclear power plant fundamentals, plant systems, use of those systems to control or mitigate an accident during which the core is severely damaged, and operating practice.

(4)

The applicant should also complete the additional instruction specified in Section 5.2.1.6 of ANSI /ANS-3.1-1981 in subjects related to the duties of an SRO.

c.

Education The applicant should have a high school diploma or equivalent.

F.

ELIGIBILITY CONSIDERATIONS FOR COLD EXAMINATIONS Cold examinations are those administered before the unit completes preoperational testing and the initial startup test program as described in its final safety analysis report.

1.

Each applicant must satisfactorily complete the training programs that are described in Section 13.2 of the final safety analysis report and approved by the NRC. The NRC's review and approval are based on O

Examiner Standards 8 of 10 Rev. 7, Sup. 1, June 1994

l i

4 1

1 b.c ES-205 PROCEDURE FOR ADMINISTERING THE l

GENERIC FUNDAMENTALS EXAMINATION PROGRAM A.

PURPOSE j

This standard describes the procedures and policies pertaining to the administration of the generic fundamentals examination (GFE) section of the written operator licensing examination at power reactor facilities.

It describes how the examinations are scheduled and constructed, how to solicit facility licensees for applicants to take the examinations, and how to promulgate the examination results.

B.

BACKGROUND 1.

Sections 55.41 and 55.43 of 10 CFR Part 55 require that the written operator licensing examinations for reactor operators (RO) and senior reactor operators (SR0) include questions on various mechanical com)onents, principles of heat transfer, thermodynamics, and fluid mec1anics, and fundamentals of reactor theory, including the fission process, neutron multiplication, source effects, control rod effects, criticality indications, reactivity coefficients, and poison effects.

2.

The fundamental knowledge ind abilities (K/As) required of an operator

[m do not vary significantly among facilities of the same vendor type, so V) the NRC implemented the GFE program to standardize the fundamental examination coverage for all applicants at pressurized and boiling water reactors (PWRs and BWRs).

The GFE does not ap)1y to operators at gas-cooled or nonpower reactors, nor is it applica)le to applicants who were previously issued an R0 or SR0 license based on a site-specific written examination that was administered after January 31, 1982, and included j

the material covered by the GFE.

3.

The Operator Licensing Branch (0LB) has assigned a GFE coordinator to serve as the point of contact for the GFE contractor and the regional offices concerning the examination.

The OLB GFE coordinator is res)onsible for coordinating and reviewing all activities associated j

wit 1 the generation, reproduction, and distribution of the GFE to the facility licensees and the dissemination of the final score reports to the regions.

The regional operator licensing section chiefs should assign an individual to coordinate the administration of the GFE in the region.

C.

EXAMINATION SCHEDULE Until the end of 1994, the BWR and PWR GFE examinations will be administered three times a year, on the Wednesday following the first Sunday in February, June, and October.

Beginning in 1995, the examinations will be administered p

twice a year, on the Wednesday following the first Sunday in April and October.

Examiner Standards 1 of 13 Rev. 7, Sup. 1, June 1994 h

i/

ES-205 D.

EXAMINATION SCOPE AND STRUCTURE The GFE examination shall contain 100 questions covering the " Components" and

" Theory" (including reactor theory and thermodynamics) sections of NUREG-1022,

" Knowledge and Abilities Catalog for Nuclear Power Plant Operators:

Pressurized Water Reactors," or NUREG-1023, " Knowledge and Abilities Catalog for Nuclear Power Plant Operators:

Boiling Water Reactors."

Passing grade for the GFE examination is 80 percent.

The knowledge and ability (K/A) topics applicable to the PWR and BWR GFE examinations have been categorized into various component, reactor theory, and thermodynamics groups as shown in Attachment 1.

The number of test questions required to evaluate each topic are identified on the attachment.

The NRC will use Institute of Nuclear Power Operations (INPO) Academy Documents89-003, " Generic Fundamentals Test Item Catalog - PWR Operator," and 89-004, " Generic Fundamentals Test Item Catalog - BWR Operator," as the primary sources of test questions when developing the GFE.

The ratio of previously used (i.e., " validated") test questions to new or unvalidated test questions will be adjusted as the size of the validated question bank increases.

The questions used on the GFE examination shall conform with the applicable construction, style, and scoring guidelines in NUREG/BR-0122, " Examiners' Handbook for Developing Operator Licensing Written Examinations."

The examiner assigned responsibility for developing the GFE shall submit the examinations to the OLB GFE coordinator and any other designated reviewers at least 20 calendar days before the scheduled administration date.

OLB will provide comments and recommended changes to the examination author as soon as possible.

The final examinations should be ready at least 14 days before the GFE administration date.

E.

EXAMINATION ADMINISTRATION The OLB GFE coordinator shall coordinate the following examination activities with the regional offices, the GFE contractor, and the facility licensees.

1.

Preexamination Activities a.

The NRC will send a letter to each facility licensee 60 days before the GFE administration date.

The letter will notify the facility licensee of the date of the examination and request the facility licensee to submit a list of licensed operator trainees to whom they plan to administer the examination. A sample notification letter and a proposed registration format for the licensees' use are provided in Attachment 2.

O Examiner Standards 2 of 13 Rev. 7, January 1993

ES-205 c.

The regional offices shall distribute the GFE examinations to their respective facility licensees.

Sample cover letters for facility licensees that did and did not participate in the examination are provided in Attachment 4.

The regional OLA shall update the applicants' status (pass or fail) in the operator licensing tracking system (0LTS) and ensure that a hard copy of the GFE results is placed in each applicant's Part 55 docket file.

d.

The OLB licensing assistant will ensure that copies of the final master BWR and PWR examinations are placed in the NRC's Public Document Room.

ATTACHMENTS / FORMS:

4,

"GFE Test Item Distribution",

" Sample Notification Letter",

" Sample Examination Cover Letter",

" Sample Results Letters" l

O t

l l

l OV Examiner Standards 5 of 13 Rev. 7, Sup. 1, June 1994

O ES-205 GFE Test Item Distribution Pressurized Water Reactors No. of K/A Topic Items Group I Components 191001 Valves 4

191002 Sensors and Detectors 10 191003 Controllers and Positioners 5

191004 Pumps 7

191006 Heat Exchangers and Condensers 3

191008 Breakers, Relays, and Disconnects 7

Group 11 Comoonenta 191005 Motors and Generators 5

191007 Demineralizers and Ion Exchangers 3

Group I Reactor Theory 192004 Reactivity Coefficients 4

192005 Control Rods 4

192008 Reactor Operational Physics 8

Group II Reactor Theory 192003 Reactor Kinetics and Neutron Sources 2

192006 Fission Product Poisons 6

Group III Reactor Theory 192001 Neutrons 1

192002 Neutron Life Cycle 2

192007 Fuel Depletion and Burnable Poisons 1

Group I Thermodynamics 193009 Core Thermal Limits 2

193010 Brittle Fracture and Vessel Thermal Stress 5

Group II Thermodynamics 193003 Steam 2

193007 Heat Transfer 2

193008 Thermal Hydraulics 8

Group III Thermodynamics 193001 Thermodynamic Units and Properties 1

193004 Thermodynamic Processes 2

193005 Thermodynamic Cycles 1

193006 Fluid Statics and Dynamics 5

Total Items 100 9

Examiner Standards 6 of 13 Rev. 7, January 1993

\\

- - ~ ~. -

ES-205 Sample Results Letters NRC Letterhead Docket No. 50-(Number)

(Date)

IName and title)

LEacil tv oame)

(Street adc ress' (C-tv. State. Z' o Code)

Dear (Name):

On (date), the NRC administered a generic fundamentals examination (GFE) section of the written operator licensing examination to employees of your facility.

Enclosed with this letter are copies of both forms of the examination including answer keys, the grading results for your facility, and copies of the individual answer sheets for each of your employees.

Please. forward the results to the individuals along with the copies of their answer sheets. A "P" in the column labeled RESULTS indicates that the individual achieved a passing grade of 80 percent or better on the GFE.

Those individuals having an "F" in the RESULTS column failed the examination.

On (date), the NRC administered a generic fundamentals examination (GFE) section of the written operator licensing examination.

Your facility did not participate in this examination. However, a copy of the master (BWR or PWR) examination, with the answer key, is enclosed for your information.

If you have any questions concerning this examination, please contact Mr.

(Name of the OLB GFE coordinator) at (ohone number).

Sincerely, (Appropriate regional representative)

Enclosures:

1.

Examination Form "A" and "B" with answers 2.

Examination Results Summary for Facility Name 3.

Individual Answer Sheets

Enclosure:

As stated

[ Sections marked (*) apply only to those facility licensees that participated

'in.the examination, and the sections marked (**

apply only to those facility licensees that did not participate in the examin)ation.]

' q 9

V l

Examiner Standards 13 of 13 Rev. 7, Sup. 1, June 1994 L

a a a#_.

a._ma, s--p

.a_.._

a.

.aw.__mau...Am_

,1

-aaa

,. -..,.A A.

___m

-J.

,i M.4 x

m<

.ma_

O f

J O

1 l

I l

l 4

O I

m

'"M m-i w

,y

Tv ___

w-

ES-301 3.

Cateaory C. "Intearated Plant Ooerations" This category of the operating test im)lements items (1? through (8),

(11), (12) and (13) identified in 10 CFR 55.45(a).

It as the most performance-based category of the operating test and is used to evaluate the applicant's ability to safely operate the plant's systems under dynamic, integrated conditions.

The simulator test is administered in a team format with up to three applicants (or surrogates) filling the R0 and SR0 license positions, as appropriate, on an operating crew This format enables the examiner to evaluate th(refer to ES-302)ilP.y to function e applicant's ab within the control room team as appropriate to the aa tgned position, in such a way that the facility licensee's procedures are adhered to and that the limitations in its license and amendments are not violated (refer to 10 C'R 55.45(a)(13)).

t Each team or crew of applit. ants is administered a set of scenarios designed so that tie examiners can individually evaluate each applicant on a range of ccrpetencies applicable to the applicant's license level.

Attach.nent I briefly describas those competencies, and Forms ES-303-3 and ES-303-4, the " Integrated Plant Operations Competency Grading Worksheets" for R0s and SR0s break down each competency into a number of specific rating factors to be considered during the grading process (see ES-303).

Each applicant must demonstrate proficiency on every competency applicable to his or her license level. The only exception is that SR0 Competency Number 5, " Control Board Operations," is optional for SRO-upgrade applicants.

D.

OPERATING TEST DEVELOPMENT Examiners shall prepare for each category of the operating test in accordance with the following general guidelines and specific instructions:

1.

General Guidelines a.

To minimize predictability and enhance test idegrity, examiners should evaluate varied subjects, systems, and operations with applicants that are not being examined at the same time.

b.

Examiners may select and modify testing materials (i.e., JPMs, questions, and simulator scenarios) from the facility's examination banks or develop them using the facility's reference material.

However, in order to protect the integrity and security of the examination process, examiners should use no more than 10 percent of the facility's testing materials without significant modification, unless authorized by the Chief, Regional Support and Oversight Section, OLB.

Every selected test item must satisfy the qualitative criteria specified herein or be modified accordingly, New and modified materials should be evenly distributed among the eD operating tests being prepared.

(b Examiner Standards 5 of 26 Rev. 7, Sup. 1, June 1994 i

ES-301 c.

Examiners should review the K/As associated with normal, abnormal, and emergency tasks and evolutions as a source of topics for use i

in evaluating applicant competency in each category of the operating test.

The knowledge and abilities associated with the tasks and questions planned for the operating test should have importance factors of at least 2.5.

Tasks with importance factors of less than 2.5 may be used if there is a substantive reason for including them (e.g., a recent licensee event or a significant system modification).

The K/As should be appropriate to the plant-specific requirements for the applicant's license level.

The examiner should reference the facility's job and task analysis (if available), learning objectives, and other reference material to confirm that the operating test is correctly oriented to the facility and the applicant's license level.

d.

SR0 applicants, whether upgrade or instant, will be examined for the highest onshift position for which the SR0's license is applicable (e.g., shift supervisor), regardless of the position to be assigned when licensed.

SR0 applicants should demonstrate their supervisory abilities and an attitude of responsibility for safe operation, and are expected to assume a management role during plant transients and upset conditions while taking Category C of the operating test. The operating test briefing, discussed in ES-302, ensures that the applicants are advised of this policy.

Differences in administrative controls and facility design will affect the SR0's responsibilities, but, in general, the following guidelines should be used to differentiate the SR0 operating test from that of an R0.

The SRO, in directing licensed activities, must evaluate plant performance and make operational judgments accordingly. SR0 applicants should, therefore, be more knowledgeable in areas such as operating characteristics, reactor behavior, and instrument interpretation.

The SR0, in directing licensed activities, must have a wider and more thorough knowledge of facility administrative controls and methods, including limitations imposed by the regulatier,s and the facility's technical specifications and their bases.

The SR0 may be assigned responsibilities for auxiliary systems that are outside the control room (e.g., waste disposal and handling systems) and are not normally operated by licensed operators.

Because the SR0 may have additional Examiner Standards 6 of 26 Rev. 7, January 1993

(O ES-301 responsibilities for those systems, the SR0 applicant should demonstrate knowledge of the system designs as they relate to maximum permissible concentrations, effluent release rates, and other radiological considerations, as appropriate.

e.

Examiners should incorporate facility-specific and industry-generic operating experience into the operating test whenever possible.

Documentation such as licensee event reports, sionificant event reports, and service information letters are readily available sources of operationally oriented plant anomalies.

f.

If the applicants at a facility qualify for dual or multiunit licenses, the operating tests should evaluate their knowledge of the design, procedural, and operational differences between the units.

The examiners should divide their operating test coverage among the units and not become predictable by conducting their walk-through tests on only one unit.

Different applicants may be examined on different units, or each applicant may be asked to explain or demonstrate his or her understanding of variations in control board layouts, systems, instrumentation and procedural actions between the units at the facility.

/m\\

Most dual or multiunit stations have a simulator that is modeled U

after only one of the units.

Therefore, the examiner should ensure that the a>plicants are properly tested on the different systems, control aoard layouts, and any other differences between the units during the walk-through portion of the operating test.

For example, after administering Category C of the operating test on Browns Ferry Unit 1, the examiner should focus the control room systems portion of Category B of the operating test on Unit 2 or Unit 3 or both.

g.

The operating test should examine a broad range of knowledge and abilities, systems and components operations and events.

The three categories of the test should not be redundant, nor should they duplicate material that is covered on the written examination.

h.

Every facet of the operating test, including the questions and answers, JPHs, and simulator scenarios, should be planned, researched, validated, and documented to the maximum extent possible before the test is administered.

i.

Examiners are encouraged to share operating test outlines and materials in an effort to reduce examination preparation time.

However examiners are expected to individually research and study thetodcsandsystemstobeexaminedontheoperatingtestso that t1ey are prepared to ask whatever follow-up questions might n

be necessary to determine if the applicant is competent in those areas.

Examiner Standards 7 of 26 Rev. 7, January 1993

ES-301 j.

Examiners should comply with the guidelines in ES-603 when developing new JPMs.

Examiners may adapt facMity procedures for use as JPMs by identifying critical steps and entering comments on how to execute particular steps.

In either case, the JPM should include the elements identified in Form ES-603-1 (e.g., initiating and terminating cues, critical ster, and performance criteria).

k.

The prescripted questions for Categories A and 8 may include a combination of open-and closed-reference items.

Open-reference items that require applicants to apply their knowledge of the plant to postulated normal, abnormal, and emergency situations are preferred.

Closed-reference items may be used to evaluate the immediate actions of emergency and other procedures, certain automatic actions, operating characteristics, interlocks, and set points, as appropriate to the facility.

Examiners should minimize the use of closed-reference questions that rely solely on memory.

Examiners should refer to Section B.2 and Tables 1 through 3 of ES-602, Attachment 2, " Guidelines for the Development and Review of Open-Reference Examinations," for more guidance. Although the l

guidelines were developed specifically for written requalification examinations they are also generally applicable for the open-referencewalk-throughformat.

Examiners may also use the applicable portions of Form ES-602-1, "NRC Checklist for Open-Reference Test Items," to verify the quality of their follow-up questions.

2.

Specific Instructions for Cateaory A. " Administrative Tooics" Although the administrative topics may be examined separately, examiners are encouraged to link, associate, or integrate them with tasks and events conducted during Categories B and C whenever possible.

Although this practice can improve the continuity of their operating test and shorten the time required for its administration, examiners should keep in mind that the applicant's proficiency in the administrative topics should be deliberately evaluated and not inferred solely from observations made during the simulator portion of the operating test.

a.

For s_gh of the administrative topics listed below, select the required number of subjects to be evaluated during the operating test.

Section C.1 provides examples of the types of subjects that should be evaluated under each topic; the lists are not all-

)

inclusive.

TODic Number of Sub.iects A.1, " Conduct of Operations" 2

A.2, " Equipment Control" 1

A.3, " Radiation Control" 1

A.4, " Emergency Pl an" I

b.

For each administrative subject determine the best method for evaluating the applicant's knowledge or ability in that area.

The examiner may use either one of the following methods to conduct the evaluation for each subject:

Examiner Standards 8 of 26 Rev. 7, Sup. 1, June 1994

I ES-301 If the region intends to prereview the operating tests with the facility licensee, the chief examiner should request the assigned examiners to submit the completed walk-through test outlines to the region 7 days before the scheduled prereview, if possible.

The chief examiner, or designee, shall review the test outline within 2 working days after receipt to determine if other categories of the test are du)11cated and to ensure that test integrity is not compromised )y excessive overlap between consecutive tests.

The reviewer will forward any comments to the originating examiner for resolution.

e.

Forward the completed JPMS and follow-up questions to the chief examiner so that they are received by all examiners at least 5 days before the start of the week in which they will be administered.

4.

Specific Instructions for Cateaorv C. "Intearated Plant Operations" a.

Based on the anticipated crew compositions, determine the number of scenarios and scenario sets necessary to rotate each R0 and SRO-I applicant into the lead reactor operator position so that he or she can perform a direct reactivity manipulation.

For example, a crew consisting of two R0s and one SR0-I will normally require O

three scenarios to evaluate each applicant's performance on the reactor controls.

The chief examiner can significantly reduce the number of simulator scenario sets required to examine a large group of applicants by administering the same set of scenarios to two different crews of applicants.

However, the examiners must ensure that the crews remain out of contact until both crews have completed the set of scenarios (refer to ES-302).

The chief examiner should ensure that additional or replacement scenarios are prepared and available while administering the operating tests in accordance with Section C of ES-302.

b.

For each scenario, select a set of initial conditions from the simulator reference manual provided by the facility licensee.

The initial conditions should be varied among the scenarios and should include startup, low power, and full power situations.

Enter the desired initial conditions on Form ES-301-3, " Scenario Events."

c.

For each scenario, develop a logical sequence of operations, malfunctions, and transients so each applicant can demonstrate his or her skill in the competencies a)plicable to the applicant's license level.

Refer to step (d) 3elow for additional guidelines.

Each scenario set must, at a minimum, exercise each applicant on the following types of evolutions, failures, and transients in the fm quantities identified for the ap)licant's license level on Form V)

I.

ES-301-5, " Transient and Event C1ecklist."

Examiner Standards 13 of 26 Rev. 7, Sup. 1, June 1994

- -D

ES-301 normal evolutions:

activities such as feed pump startup, turbine loading, and generator synchronization reactivity manipulations: normal evolutions such as a reactor startup or chi.nging power with boron concentration, control rods, or cort flow instrument failures: malfunctions such as nuclear, control, or process instrumentation failures component failures:

a failure that involves a significant system response and requires operator action to correct major plant transients:

significant transients, such as a loss-of-coolant accident or a loss of electrical power, that would lead to an automatic protective action such as a reactor trip or scram or an engineered safety feature actuation Qng of the normal evolutions for R0 applicants shall involve a significant change in reactivity, during which the applicant must be positioned as the lead or primary reactor operator.

The R0 applicant may be placed in either the lead or balance-of-plant (B0P) position during all the other evolutions identified on Form ES-301-5.

Reactivity manipulations are significant if they produce a clearly observable plant resoonse, such as bringing the reactor critical from a substantially subcritical state, raising power to the point that reactivity feedback from nuclear heat addition is noticeable and a heatup rate is established, or changing reactor power manually with control rods or recirculation flow.

Since the SRO-I applicant's manipulative skills on the control console have never been examined by the NRC, both their operational and supervisory knowledge and abilities must be evaluated.

SRO-I applicants must fill the lead or primary R0 position during some events and the SR0 supervisory position during others, as shown on Form ES-301-5. The one normal evolution required while in the primary R0 position muit involve a significant reactivity change.

SRO-U applicants are given credit for their previous R0 license evaluation and experience and are normally not required to manipulate the controls.

However, if the examiner detects weaknesses during the test, he or she may use additional O

Examiner Standards 14 of 26 Rev. 7, Sup. 1, June 1994

O ES-302 Operating Test Briefing The examiners shall brief each applicant on the policies and guidelines applicable to the ooerating test category being administered.

The applicants may be briefed individually or as a group, at the chief examiner's discretion.

Part A - SIMULATOR AND WALK-THROUGH TESTS (CATEGORIES A, B, AND C) 1.

SR0 applicants will be tested at the level of responsibility of the senior licensed shift position (i.e., shift supervisor, senior shift supervisor, or whatever the title of the position may be).

2.

The examiner is a visitor at this facility.

You are expected to escort the examiner and ensere that he or she complies with safety, security, and radiation protection procedures.

3.

You should not operate plant equipment without appropriate permission from the operating crew.

Nothing the examiner says or asks will be intended to violate this principle.

4.

If the examiner poses a question or asks you to perform a task that is unclear, you should not hesitate to ask him or her to reword or clarify the question or task.

{AU)

The examiner will take notes throughout the test to document your 5.

performance, and sometimes the examiner may take a short break for this reason.

The amount of note-taking does not reflect your level of performance. The examiner is required to document satisfactory as well as less than satisfactory performance.

6.

The operating test is considered "open reference." The reference material that is normally available to operators in the facility and control room (including calibration curves, previous log entries, piping and instrumentation diagrams, calculation sheets, and procedures) is also available to you during the operating test.

However, you should know from memory certain automatic actions. set points, interlocks, operating characteristics, and the immediate a.tions of emergency and other procedures, as appropriate to the facility.

If you desire to use a reference, you should ask the examiner if it is acceptable to do so for the task or question under consideration.

You may NOT solicit technical information from other operators, engineers, or technical advisors.

7.

The examiner is not allowed to reveal the results of the operating test when it is complete.

8.

If you have any questions concerning the administration of any category of the operating test, the examiner will answer those questions before starting that category of the test.

Examiner Standards 9 of 11 Rev. 7, January 1993 i

Part B - WALK-THROUGH TEST (CATEGORIES A AND B) 1.

The walk-through test covers control room systems, local system operations, and administrative requirements.

The examiner will evaluate each of these areas using a combination of job performance measures (JPMs) and specific questions.

2.

Before beginning each JPM, the examiner will describe the initial conditions, explain the task that is to be completed, indicate whether the task is time-critical, and explain which steps are to be simulated or discussed.

You should perform or simulate the required actions as if direc od by plant procedures or shift supervision.

Do not assume that the examiner will accept an oral description of the required action unless the examiner indicates otherwise.

3.

Time-critical JPMs must be completed within the predetermined time interval in order to obtain a satisfactory grade for that JPM. - You will be permitted to take whatever time is necessary to complete those JPMs that are not time-critical, provided you are making reasonable progress toward achieving the task standard.

4.

When performing JPMs, you are expected to make decisions and take actions based on the facility's procedural guidance and the indications available.

Some of the tasks that the examiner asks you to perform will require the implementation of an alternative method directed by plant procedures.

5.

The examiner will ask follow-up questions at the end of each JPM to investigate your knowledge of the applicable system or task.

Many of the questions will require you to use plant reference material, while others should be answered without the use of references.

If you need to consult a reference to answer a question, ask the examiner if it is acceptable to do so. There is no specific time limit for any question, however, you may be evaluated as unsatisfactory on a question if you are unfamiliar with the subject or reference material and are unable to answer the question in a reasonable period of time.

You will not be permitted to conduct unlimited searches of the plant reference material during the examination.

6.

To facilitate the examination, please inform the examiner when you consider your performance of each JPM and your answer to each question to be complete.

7.

If you need a break during the test, you should ask the examiner.

8.

Do you have any questions before we begin the walk-through test?

O Examiner Standards 10 of 11 Rev. 7, Sup. 1, June 1994

O ES-303 Operator License Examination Report Form ES-303-1 U.S. NUCLEAR REGULATORY COMMISSION OPERATOR LICENSE EXAMINATION REPORT APPLICANT'S NAME DOCKET NUMBER 55-I R

EXAMINATION TYPE (INITIAL OR RETAKE)

FACILITY NAME REACTOR OPERATOR HOT SENIOR REACTOR OPERATOR (SRO) INSTANT COLD FACILITY SRO UPGRADE BWR DESCRIPTION SR0 LIMITED TO FUEL HANDLING PWR WRITTEN EXAMINATION

SUMMARY

WRITTEN BY TOTAL EXAMINATION POINTS GRADED BY TOTAL APPLICANT POINTS

(}

DATE ADMINISTERED APPLICANT GRADE OPERATING TEST

SUMMARY

ADMINISTERED BY DATE ADMINISTERED A.

ADMINISTRATIVE TOPICS 8.

CONTROL ROOM SYSTEMS AND FACILITY WALK-THROUGH C.

INTEGRATED PLANT OPERATIONS (SIMULATOR TEST)

EXAMINER RECOMMENDATIONS CHECK BLOCKS PASS iAIL WAIVE SIGNATURE DATE WRITTEN EXAMINATION OPERATING TEST FINAL RECOMMENDATION LICENSE RECOMMENDATION ISSUE LICENSE SIGNATURE - SECTION CHIEF DATE DENY LICENSE Examiner Standards 9 of 27 Rev. 7, January 1993 l

ES-303 2

Form ES-303-1 l

APPLICANT DOCKET NUMBER:

55-PAGE OF A.

ADMINISTRATIVE TOPICS EVALUATION COMMENT PAGE i

(S OR U)

NUMBER 1.

CONDUCT OF OPERATIONS 2.

EQUIPMENT CONTROL 3.

RADIATION CONTROL 4.

EMERGENCY PLAN B.1 CONTROL ROOM SYSTEMS SAFETY JPM GRADE FUNCTION (S OR U)

QUESTION GRADE (S OR V)

SYSTEM COMMENT r.MDE PAGE SYSTEM /JPM TITLE

,, OR U)

NUMBER l

1.

2.

3.

4.

5.

6, 7.

B.2 FACILITY WALK-THROUGH 1.

2.

3.

Examiner Standards 10 of 27 Rev. 7, Sup. 1, June 1994 i

ES-303 3.a Form ES-303-1 APPLICANT DOCKET NUhtER:

55 PAGE OF C.

REACTOR OPERATOR INTEGRATED PLANT OPERATIONS (SIMULATOR TEST) GRADING

SUMMARY

COMPETENCIES /

SCENARIOS COMMENT RATING FACTORS WEIGHT 3.0 2.0 1.0 TOTAL OGSERVED PAGE NO.

1.

ALARMS / ANNUNCIATORS 1

2 3

A.

NOTICE / ACKNOWLEDGE 0.30 0.90 0.60 0.30 B.

INTERPRET / VERIFY 0.40 1.20 0.80 0.40 C.

PRIORITIZE 0.30 0.90 0.60 0.30

(

)

2.

DIAGNOS!$

1 2

3 A.

RECOGNIZE 0.40 1.20 0.80 0.40 B.

USE OF REFERENCE MATERIAL 0.20 0.60 0.40 0.20 C.

DIAGNOSE 0.40 1.20 0.80 0.40

(

)

3.

SYSTEM RESPONSE 1

2 3

A.

LOCATE / INTERPRET 0.33 1.00 0.67 0.33 j

B.

SYSTEM OPERATION KNOWLEDGE 0.33 1.00 0.67 0.33 1

C.

EFFECT OF ACTIONS 0.33 1.00 0.67 0.33

(

)

4.

PROCEDURES / TECH SPECS 1

2 3

l A.

REFERENCE 0.20 0.60 0.40 0.20 B.

E0P ENTRY /IMMEDIATE ACTIONS 0.40 1.20 0.80 0.40 C.

PROCEDURE COMPLIANCE 0.20 0.60 0.40 0.20 D.

TECH SPEC ENTRY 0.20 0.60 0.40 0.20

(

)

5.

CONTROL BOARD OPERATIONS 1

2 3

A.

LOCATE 0.25 0.75 0.50 0.25 B.

MANIPULATE 0.25 0.75 0.50 0.25 C.

RESPONSE

0.25 0.75 0.50 0.25 D.

MANUAL CONTROL 0.25 0.75 0.50 0.25

(

)

6.

COMMUNICAfl0NS 1

2 3

A.

PROVIDE INFORMATION 0.33 1.00 0.67 0.33 B.

RECEIVE INFORMATION 0.33 1.00 0.67 0.33 C.

CARRY OUT INSTRUCTIONS 0.33 1.00 0.67 0.33

(

)

l t-Examiner Standards 11 of 27 Rev. 7, January 1993

ES-303 3.b Form ES-303-1 APPLICANT DOCKET NUMBER:

55-PAGE OF C.

SENIOR REACTOR OPERATOR INTEGRATED PLANT OPERATIONS (SIMULATOR TEST) GRADING

SUMMARY

C04PETENCIES/

SCENARIOS COMMENT RAi!NG FACTORS WElGHT 3.0 2.0 1.0 TOTAL OBSERVED PAGE NO.

1.

ALARMS / ANNUNCIATORS 1

2 3

A.

PRIORITIZE 0.30 0.90 0.60 0.30 B.

INTERPRET 0.35 1.05 0.70 0.35 C.

VERIFT 0.35 1.05 0.70 0.35

(

)

2.

DIAGNOSIS 1

2 3

A.

RECOGNIZE 0.25 0.75 0.50 0.25 B.

ACCURACT 0.25 0.75 0.50 0.25 C.

DIAGNOSE 0.25 0.75 0.50 0.25 D.

CREW RESPONSE 0.25 0.75 0.50 0.25

(

)

3.

STSTEM RESPONSE 1

2 3

A.

INTERPRET 0.35 1.05 0.70 0.35 B.

ATTENTIVE 0.20 0.60 0.40 0.20 C.

PLANT EFFECTS 0.45 1.35 0.90 0.45

(

)

4.

PROCEDURES 1

2 3

A.

REFERENCE 0.25

0. 75 0.50 0.25 B.

CORRECT USE 0.50 1.50 1.00 0.50 C.

CREW IMPLEMENTATION 0.25 0.75 0.50 0.25

(

)

5.

CONTROL BOARD OPERAi!ONS 1

2 3

A.

LOCATE 0.25 0.75 0.50 0.25 B.

MANIPULATE 0.25 0.75 0.50 0.25 C.

RESPONSE

0.25 0.75 0.50 0.25 D.

MANUAL CONTROL 0.25 0.75 0.50 0.25

(

)

6.

COMMUNICAil0NS 1

2 3

A.

CLARITY 0.45 1.35 0.90 0.45 i

B.

CREW INFORMED 0.35 1.05 0.70 0.35 1

C.

RECElVE INFORMATION 0.20 0.60 0.40 0.20

(

)

7.

O!RECilNG OPERAil0NS 1

2 3

A.

TIMELY ACTION 0.20 0.60 0.40 0.20 B.

SAFE DIRECil0NS 0.40 1.20 0.80 0.40 C.

OVER$1GHT 0.20 0.60 0.40 0.20 D.

CREW FEEDBAct 0.20 0.60 0.40 0.20

(

)

8.

TECHNICAL SPECIFICAfl0Ns 1

2 3

A.

RECOGNiiE 0.40 1.20 0.80 0.40 B.

LOCATE 0.20 0.60 0.40 0.20 C.

COMPLIANCE 0.40 1.20 0.80 0.40

(

)

Examiner Standards 12 of 27 Rev. 7, Sup. 1, June 1994

bv ES-401 PREPARATION OF SITE-SPECIFIC WRITTEN EXAMINATIONS l

FOR POWER REACTORS I

A.

PURPOSE This standard specifies the requirements, procedures, and guidelines for the preparation of site-specific written examinations for the initial licensing of 1

reactor operator (RO) and senior reactor operator (SRO) applicants at power reactor facilities.

B.

BACKGROUND The content of the written licensing examinations for R0s and SR0s is dictated by Sections 55.41 and 55.43, respectively, of Title 10 of the Code of Federal Regulations (10 CFR).

Each examination shall contain a representative selection of questions on the knowledge and abilities (K/As) needed to perform duties at the desired license level.

The written operator licensing examination is administered in two sections, a generic fundamentals examination (GFE) section and a site-specific examination. The GFE covers those K/As that do not vary significantly among reactors of the same type (i.e., pressurized or boiling water) and is n

generally administered early in the license training process; refer to ES-205

(

for a description of the program.

C.

EXAMINATION PREPARATION Examiners shall prepare site-specific written operator licensing examinations in accordance with NUREG/BR-0122, " Examiners' Handbook for Developing Operator Licensing Written Examinations," and the guidelines and instructions summarized below.

The Examiners' Handbook provides specific instructions for preparing content-valid examinations and guidance on writing and formatting various types of written examination questions.

The guidelines and instructions contained herein take precedence over conflicting guidelines in the Examiners' Handbook.

1.

EqngrJ1 Guidelines a.

The examiner shall prepare the examination so that an applicant who is capable of safely operating the plant will be able to complete and review the examination within 4 hours4.62963e-5 days <br />0.00111 hours <br />6.613757e-6 weeks <br />1.522e-6 months <br /> and achieve a grade of 80 percent or greater.

b.

When both R0 and SR0 examinations are to be given at approximately the same time, the examiner shall duplicate no more than 75 percent of the R0 examination questions on the SR0 examination.

The remaining SR0 questions shall evaluate the additional knowledge and abilities required for the higher license level.

The Examiners' Handbook and 10 CFR 55.43(b) provide guidance on Examiner Standards 1 of 7 Rev. 7, January 1993 j

ES-401 the additional topics to be considered for the SR0 examination, c.

To avoid compromising the integrity and security of the examination, no more than 25 percent of the questions on the examination may be repeated from the past two NRC-administered examinations at the same facility. The examiner shall also use no more than 10 percent of the questions in the facility licensee's written examination question bank without significant modification, d.

All test questions shall be in the multiple choice (preferred) or matching format. Matching questions should be limited to a total value of two points per question.

e.

It is appropriate and desirable for the examiner to hypothesize events or circumstances leading to events to examine the applicants' analytical abilities and knowledge of corrective actions.

2.

Use of Referea.ce Material The examiner is expected to use the following reference materials when preparing the site-specific, written operator licensing examinations:

a.

Questions from the NRC's examination question bank (EQB), subject to the restrictions in Section C.I.c above, b.

Facility reference material requested in accordance with ES-201, including the learning objectives used in the applicants' training program and the facility's examination question bank, if provided, c.

Personal files of questions (those of the examiner preparing the examination as well as those of other examiners),

d.

Licensee event reports, safety evaluation reports, information notices, current industry and facility problems, and validated facility question banks.

e.

The examiner may use reference material such as diagrams, sketches, and portions of facility procedures as attachments to the written examinations and ask applicants to identify components and other items on the attachments.

The examiner shall ensure that any reference material used in the examinations is easy to read, clearly marked, and provide an effective and objective way for the applicant to demonstrate knowledge of the topic or concept.

Examiner Standards 2 of 7 Rev. 7, Sup. 1, June 1994

ES-501 The applicant must have a grade of 80 percent or greater on the written examination and a grade of " satisfactory" on all three categories of the operating test to qualify for a license.

3.

If the management reviewers (i.e., the section chief or licensing official) do not agree with the final recommendation, the examiner of record and chief examiner shall be consulted to discuss and resolve any disagreements.

Such disagreements usually arise because of inadequate documentation of a denial recommendation.

It is, therefore, very i

important for examiners to be complete and accurate in their grading and comments.

4.

If a recommendation is overturned during the regional management review, the affected summary evaluations will be lined out and initialed by the licensing official. The licensing official will enter the new summary evaluation in the appropriate block and explain the change on Form ES-303-2, " Operating Test Comments," and attachment it to the applicant's Form 303-1.

5.

If an ap)11 cant did not complete the SR0 upgrade training program or failed tie upgrade examination, regional management should ensure that the R0 licensee complies with the requirements of 10 CFR 55.53(e), (f),

and (h) and 10 CFR 55.59(a) before resuming active duties as an R0.

The regional office should also conduct a case-specific review of the SR0 O

upgrade examination to determine if the applicant failed as a result of significant deficiencies in R0 knowledge or abilities.

Pursuant to 10 CFR 55.7, the NRC may, by rule, regulation, or order, impose upon any licensee such additional requirements it deems appropriate or necessary to protect the health and to minimize danger to life and property.

If the SR0 upgrade applicant's deficiencies pose such a threat, the NRC may require the facility licensee to provide remedial training and reevaluation and submit evidence of its completion to the NRC.

6.

Regional management should also review the overall examination results and any generic findings, deficiencies, or issues identified in the examination report to determine if any follow-up action is required.

E.

NOTIFICATION OF RESULTS The regions will notify the facility licensees and applicants of the examination results only after they are reviewed and approved by the licensing official.

1.

The region should normally notify a representative of the facility licensee by telephone and confirm the results by mailing, under a separate cover letter, a copy of Form ES-501-2 and, for each applicant that failed or had significant deficiencies that warrant further evaluation and retraining by the facility licensee, a copy of the applicant's Form ES-303-1 and written examination answer sheet.

Examiner Standards 3 of 24 Rev. 7, Sup. 1, June 1994

ES-501 If the written examinations were administered much before the operating tests and management has approved the results of those examinations, the region may notify the facility licensee of those results rather than wait until the operating tests are completed.

A copy Form ES-501-2 shall also be sent to the Management Assistant, Operator Licensing Branch (0LB).

The form (s) shall nol be placed in the public document room or distributed with the final examination report.

If any of the examinations are later regraded in response to an applicant's request for review (refer to ES-502), the original Form ES-501-2 on file in the regional office shall be corrected by lining out the old grade, entering the new grade, and initia111ng the change.

Whenever a change is made, the region shall mail a copy of the revised form to 0LB.

2.

After the licensing official has signed the license and denial letters, the region shall send each applicant the following material with his or her letter:

l a copy of Forms ES-303-1 and ES-303-2 and a copy of Forms ES-301-3 (and Forms ES-301-4 if the applicant failed Category C of the operating test) that reflect the "as run" scenario conditions but do ng1 contain any rough examiner notes regarding the applicant's performance (pen-and-ink markups of the original, approved scenarios are acceptable);

and a copy of the applicant's written examination cover and answer sheets (a copy of the master written examination and answer key will also be included if the applicant failed the written examination).

F.

RETURNING FACILITY REFERENCE MATERIAL If desired by the facility licensee, the chief examiner shall ensure that the reference material provided for examiners to prepare for the examinations is returned as soon as possible.

If none of the applicants failed the examination, the material should be returned as soon as the licenses are issued.

If any applicant was denied a license based on an examination failure, the reference material should be retained until the 20-day period during which the applicant may request a regrade has expired.

If an applicant submits a request for regrade in accordance with L5-502, the chief examiner shall determine what reference material needs to be retained and return all unnecessary material. All the reference material should be returned to the facility licensee within 30 days following the resolution of any appeals.

The chief examiner shall inform all the examiners involved with the examination when the reference material should be returned.

Examiner Standards 4 of 24 Rev. 7, Sup. 1, June 1994 l

ES-501 G.

EXAMINATION REPORT The chief examiner shall prepare the final examination report when all portions of the examination have been graded and documented. The sample examination report that is included as Attachment I should be used as a guide.

1.

The final examination report shall document the results of the examination, any generic strengths and weaknesses noted while administering the operating tests and grading the written examinations, any significant deficiencies in the quality of the facility's reference material (including its examination banks, if provided) and issues discussed at the exit meeting.

2.

The report shall include a copy of the facility licensee's comments on the final written examination (s), a specific NRC resolution for each comment, and a specific justification for every additional item deletion or change.

Generic comments submitted by the facility licensee about the examinations or the administration process should also be included in the report, but they do not necessarily require a regional response or resolution. A copy of the master written examination (s) and answer key (s) and a simulation facility report shall also be included (refer to Section H).

O 3.

The applicants' names and specific grades (i.e., Form ES-501-2) shall V

ng1 be published in the examination report.

4.

The region shall send the final examination report to the facility licensee and copies to the public document rooms.

H.

SIMULATION FACILITY REPORT The region shall include a simulation facility report (see Attachment 1, ) in the final examination report if operating tests were conducted on a simulation facility.

The purpose of this report is to document the examiners' concerns about the performance or fidelity of the simulation facility.

If no problems were observed, this should be stated in the report.

If the examiners observe performance or fidelity problems during the preparation or conduct of the operating tests, they should briefly but specifically describe those problems in the simulation facility report. All deficiencies should be described in sufficient detail to allow screening and classification during a simulation facility inspection.

The examiners may include concerns about physical fidelity (hardware or equipment discrepancies) or functional fidelity (performance of the simulation facility during normal, surveillance, abnormal, or emergency events) in the simulation facility report.

Each deficiency should include a description of the operation, event, or transient that was in progress, and how the simulation facility failed to faithfully model the expected performance of the reference plant.

Examiner Standards 5 of 24 Rev. 7, Sup 1, June 1994 J

I.

EVALUATION OF CONTRACT EXAMINER SUPPORT The chief examiner shall complete Form ES-501-3, " Contract Examiner Evaluation," for examinations administered in whole or in part by contract examiners and send a copy of the completed form to the Chief, Regional Support and Oversight Section, OLB.

Written examinations and operating tests prepared by contract examiners should be evaluated for content validity, adherence to the Examiner Standards, and general quality.

The reasons for requested changes and problems encountered in making the changes should be documented.

If the chief examiner audited any contract examiners during the administration of operating tests, he or she shall document the results of the audit (s) by completing the appropriate sections of an " Operating Test Audit Form" (OLB Manual Chapter 170, Attachment 2) and forwarding the completed form (s) to the Chief, Regional Support and Oversight Section, OLB.

J.

RECORD RETENTION 1.

The original or a copy of the following items should be placed in the facility's master examination file and retained for two initial examination cycles (i.e., the file should contain the last two examinations administered at the facility):

a.

ES-201, Attachment 1, " Corporate Notification Letter."

b.

ES-201, Attachment 3, " Examination Assignment Sheet," with pen-and-ink changes to identify the applicants that were actually examined.

The final written examination and answer key (enclosure to c.

examination report) with all changes incorporated, and the NRC-approved examination that was prereviewed by the facility licensee (including comments made by the facility licensee),

d.

Form ES-401-3, " Written Examination Quality Assurance Checkoff Sheet."

l e.

Forms ES-301-3, " Scenario Events," and ES-301-4, " Operator Actions," for each scenario set administered and Forms ES-301-1,

" Administrative Topics Outline," and ES-301-2, " Individual Walk-through Test Outline," for each walk-through test.

All record copies should have the required signatures and reflect the "as run" test conditions (pen-and-ink markups of the original, approved forms are acceptable),

f.

Form ES-403-1, " Examination Grading Quality Assurance Checkoff Sheet."

Examiner Standards 6 of 24 Rev 7, Sup. 1. June 1994

ES-501 Form ES-501-2, " power Plant Examination Results Summary Sheet" g.

h.

Form ES-501-3, " Contract Examiner Evaluation" (if applicable),

i.

ES-501, Attachment 1, examination report, with all enclosures.

j.

Form ES-201-2, " Examination Security Agreements" l

2.

The following items will be placed in each applicant's docket file:

l Forms ES-303-1, " Operator License Examination Report," ES-303-2, a.

" Operating Test Comments," (original copies, all pages) and Form (s) ES-301-3, " Scenario Events."

If the applicant failed Category C of the operating test, form (s) ES-301-4, " Operator Actions," will also be retained. All record copies should have the required signatures and reflect the "as run" test conditions (pen-and-ink markups of the original, approved forms are acceptable),

b.

All correspondence with the applicant.

The applicant's written examination cover and answer sheets.

c.

O ATTACHMENTS / FORMS:,

" Sample Examination Report",

" Sample License Letters",

" Proposed Denial Letter" Form ES-501-1, "Postexamination Check Sheet" Form ES-501-2,

" Power Plant Examination Results Summary" Form ES-501-3,

" Contract Examiner Evaluation" bG Examiner-Standards 7 of 24 Rev. 7, Sup. 1, June 1994

O ES-501 Sample Examination Report NRC Letterhead Date Docket No.

(Name)

(Name of facility)

(Street address)

(City. State Zio codel Dear (Name)

Subject:

Examination Report On (date), the NRC administered examinations to employees of your company who had applied for licenses to operate (Name of facility). At the conclusion of the examination, the examiners discussed the examination questions and preliminary findings with those members of your staff identified in the enclosed report.

In accordance with 10 CFR 2.790 of the Commission's regulations, a copy of this letter and the enclosure (s) will be placed in the NRC Public Document Room.

Should you have any questions concerning this examination, please contact (Name) at (telechone number).

Sincerely, (Name)

(Title)

Enclosures:

1.

Report Details 2.

Examination (s) and Answer Key (s) (SR0/RO) 3.

Facility Comments (Enclosures 3 and 4 may be combined) 4.

NRC Resolution of Comments (Enclosures 3 and 4 may be combined) 5.

Simulation Facility Report cc:

Facility Training Manager (w/ enclosures)

Public Document Room (w/ enclosures)

NRC Document Control System (w/ enclosures)

Regional Distribution O

Examiner Standards 8 of 24 Rev. 7, Sup. 1, June 1994

I ES-601 l

CONDUCT OF HRC REQUALIFICATION EXAMINATIONS l

A.

PURPOSE Section 55.59(a) of Title 10 of the Code of Federal Reaulations (10 CFR) requires licensed operators and senior operators (hereafter " licensees") to complete a requalification program developed by the facility licensee and to pass a comprehensive requalification written examination and an annual operating test.

In lieu of accepting a certification by the facility licensee that the licensee has passed the required examinations and tests administered by the facility licensee within its Commission-approved program, the NRC may administer a comprehensive recualification written examination and an annual operating test.

This standarc provides general guidance and requirements for conducting those NRC requalification examinations and for evaluating the facility licensee's licensed operator training program to ensure that it is effectively maintaining the competency of its licensees.

Specific requirements and guidance for conducting the comprehensive requalification written examinations and the annual operating tests, including both the plant walk-through and dynamic simulator sections, are provided in ES-602 through ES-604.

These standards are not a substitute for the operator licensing regulations and are subject to revision and other changes to the internal l

operator licensing program policy.

B.

BACKGROUND i

Section 306 of the Nuclear Waste Policy Act (NWPA) of 1982 authorized and i

directed the NRC to promulgate regulations, or other appropriate guidance, for the training and qualification of nuclear power plant operators.

Those l

regulations were to include requirements governing NRC administration of requalification examinations and requirements for operating tests at nuclear power plant simulators.

The NRC's Office of the General Counsel has concluded that a requalification evaluation program consisting primarily of periodic, onsite requalification inspections supplemented with NRC examinations at facilities where the NRC believes that ineffective training is causing licensees to commit errors satisfies the statutory requirements in Section 306 of the NWPA.

The revised oversight program will require the NRC to actively oversee the facility licensees' requalification training programs, and the l

Commission's regulations will continue to contain legally binding requirements l

that apply to the conduct of operator requalification examinations by facility licensees.

Regional managers will consider overall facility performance (e.g., systematic assessment of licensee performance (SALP) ratings), the results of the NRC's inspection programs (e.g., requalification, emergency operating procedure, and resident), the results of routine initial and requalification examinations, and other factors when determining the scope of the requalification inspection and examination activities at a facility.

Generally, only the inspection requirements of Inspection Procedure (IP) 71001, " Licensed Operator Requalification Program Evaluation," will need to be met; however, augmented Examiner Standards 1 of 41 Rev. 7, Sup. 1, June 1994

ES-601 activities can be initiated in accordance with program office guidance when necessary to ensure safe plant operation.

Those activities could include a training program inspection in accordance with IP 41500, " Training and Qualification Effectiveness," operational evaluations of onshift crews, or NRC examinations conducted in accordance with this series of Examiner Standards.

The NRC will conduct requalification examinations only when it has lost confidence in the facility licensee's ability to conduct examinations or when the staff believes that the inspection process will not provide the needed insight.

Regional management should consider conducting requalification examinations or operational evaluations when any of the following conditions exist:

requalification inspection results indicate an ineffective licensee requalification program, operational problems for which licensee error is a major contributor, a SALP Category 3 rating in plant operations is attributed to licensee performance, or allegations regarding significant training program deficiencies.

The decision to conduct NRC examinations should be implemented through the normal resource planning system, such as the plant performance review (PPR) and master inspection planning (MIP) processes, since an inspection activity will be replaced with examinations that are more resource-intensive.

Using the existing inspection planning process will ensure that the regional office and the Office of Nuclear Reactor Regulation (NRR) consider the need for conducting examinations, as well as the alternative, expanded inspection tools, when allocating the required resources.

Operational evaluations should be considered as a reactive effort based on immediate safety concerns.

C.

SCOPE The NRC-conducted requalification examinations will measure the effectiveness of a requalification program by evaluating the ability of the facility licensee to adequately prepare written examination questions, job performance measures (JPMs), and simulator scenarios and its ability to properly evaluate its licensees' performance.

The examination procedures are based on a systems approach to training (SAT) program, as defined in 10 CFR 55.4, and, to the extent possible, rely on existing requalification program standards for developing and implementing the NRC examination.

The SAT approach allows the NRC to conduct requalification examinations that are fundamentally consistent l

with existing facility-developed programs, thereby reducing the impact on the facilities and improving the reliability of the NRC assessment of requalification training programs.

The NRC-conducted requalification examination will normally be composed of l

^

three parts:

a two-section, open-reference written examination, a walk-Examiner Standards 2 of 41 Rev. 7, Sup. 1, June 1994

(3 ES-601 through evaluation, and a dynamic simulator evaluation. The three examination parts are further described in ES-602, 603, and 604, respectively, if, under the facility licensee's requalification program, an examination structure or methodology that is different from that described herein is used, the regional office should consult with the Operator Licensing Branch (0LB) to determine the appropriate examination procedure.

To the extent practical, the examination will be based on the facility licensee's requalification program and its learning objectives.

The facility licensee is expected to use the plant-specific job and task analyses (JTA) as the basis for developing the examination materials and for substantiating the importance rating factors for each task.

The facility licensee may use NUREG-1122 or -1123, " Knowledge and Abilities Catalog for Nuclear Power Plant Operators:

Pressurized Water Reactors [or Boiling Water Reactors]," for additional guidance in identifying job-specific importance rating factors.

The use of a JTA will result in more technically sound and operationally oriented examinations.

An examination team, composed of NRC examiners and facility representatives, will develop, review, and conduct each requalification examination.

Parallel evaluation of licensee performance by NRC examiners and facility evaluators will enhance the ability of the NRC to assess both individual and program performance.

The administrative guidelines and procedures for conducting an NRC requalification examination are outlined in Attachment 1, " Examination Timetable."

D.

EXAMINATION PREPARATIONS 1.

Communication 1 a.

When the NRC determines that it is necessary to conduct requalification examinations, the regional office will notify the facility licensee to be evaluated at least 90 but preferably 120 days before the examination start date, using the corporate notification letter shown in Attachment 2.

If possible, the NRC will schedule the site visits to coincide with the requalification training cycle of the facility.

Depending on the number of licensees and crews at the facility, it may be necessary to conduct the examinations over a period of two or more weeks to attain the required sample size.

The requalification training cycle, referenced here and throughout NUREG-1021, is that continuous period of time (not to exceed 24 months) within which the facility licensee conducts its operator requalification training program.

If the purpose of the examination is to retest licensees who previously failed an NRC-conducted requalification examination, the regional office should modify the corporate notification O'

letter, as appropriate.

Examiner Standards 3 of 41 Rev. 7, Sup. 1, June 1994 i

ES-601 b.

The facility licensee is expected to respond to the corporate notification letter at least 60 days before the evaluation by submitting the materials and information requested in the letter, l

l The facility licensee may request that the NRC chief examiner or another NRC representative meet with the licensees to be examined and with appropriate facility licensee managers.

Such a meeting should be scheduled during the examination preparation week as l

discussed in Section D.5.

l l

c.

The NRC will confirm with the facility licensee at least 30 days in advance which licensees have been selected to participate in the evaluation.

2.

Selection of Licensees a.

The NRC expects facility licensees to train and examine their licensees in the same crew configurations with which they normally operate the plant.

Generally, the NRC expects the crew to include no more than five operators, but it will consider larger crew sizes on a case-by-case basis.

At times, the examination team may configure crews that do not routinely perform shift duties together to ensure an adequate sample size for the examination. Mixed crews of shift and non-shift licensees should not be configured unless the facility licensee routinely evaluates mixed crews in its requalification training program or the facility licensee's normal crew size is so large that separating a normal crew is required for examination purposes.

b.

All crew members for requalification dynamic simulator examinations must be currently licensed on the facility and up-to-date in the facility licensee's requalification program, c.

The selections will be made to minimize perturbation on the facility licensee's schedules and plant operations.

Operating crew (s) in training will be given first priority during the examination week (s).

If the facility's program is being reevaluated after an unsatisfactory evaluation, the selection process should favor licensees who either failed their previous NRC-conducted examination or were not previously examined, d.

During retake examinations, licensees who have passed an NRC l

l requalification examination may be included in the dynamic simulator crew evaluation.

These licensees will not be required to take the written or walk-through portions of that examination.

l The licensees' performance on the simulator examination will be evaluated in accordance with the guidance of ES-604.

Examiner Standards 4 of 41 Rev. 7, Sup. 1, June 1994 w_____-__

ES-601 J

e.

A shift technical advisor (STA) may be added to the crew if the facility normally uses an STA during requalification training.

The NRC expects the STA's duties and responsibilities to be the same as those assigned during requalification training and plant operations.

f.

The NRC will review the list of crews and licensees the facility licensee submits and recommend changes as necessary.

3.

Reference Material a.

The facility licensee is expected to supply the reference materials requested in the corporate notification letter (see to Attachment 2).

The NRC will evaluate the facility reference materials for adequacy in advance of the scheduled preparation week, using the " Evaluation Checklist for Facility Reference Material," Fonn ES-601-2.

b.

The NRC reserves the right to prepare the requalification examinations using the NRC's examination question banks and facility background reference materials if the facility licensee's test items are inadequate for examination preparation.

If the NRC prepares the examination, the NRC may require additional reference

,Q material comparable to that listed in ES-201, Attachment 2,

" Reference Material Guidelines for Initial Operator Licensing Q

Examinations."

The facility is expected to provide a sample plan that meets the c.

guidelines of Attachment 3, " Examination Sample Plan," for the NRC's use in developing the examination.

4.

Examination Team Selection l

a.

The NRC will contribute no fewer than two examiners to the examination team.

The regional office should consider assigning additional examiners if the operating crews for the dynamic simulator examinations contain five or more licensees. To promote consistency in requalification program administration, regional office management should try to assign an examiner who participated in a prior requalification inspection or examination at the facility to be part of the NRC examination team, In most cases, it is expected that OLB will send a representative to observe the examination process or an examiner to participate as an additional member of the examination team.

The Chief of the Regional Support and Oversight Section, OLB, will work with the responsible regional section chief to make the necessary arrangements.

OO Examiner Standards 5 of 41 Rev. 7, Sup. 1, June 1994

ES-601 b.

The facility licensee is expected to provide an employee to work with the NRC as part of the requalification examination team.

The employee must be from the operations staff and an active senior reactor operator (SRO) as defined in 10 CFR 55.53(e) or The NRC encourages the facility licensee to designate another(f).

employee from the training staff to be a member of the examination team.

This employee should also be a licensed SR0 but may be a certified instructor.

If the facility licensee desires and the chief examiner agrees, additional employees from the operations or training staffs with qualifications comparable to the other facility examination team members may be included on the examination team.

The function of these examination team members is to provide facility-specific technical assistance to the NRC in developing and reviewing the written examination items, plant walk-through topics, and dynamic simulator scenarios.

If necessary, the facility representatives may participate in the actual conduct of the operating test or written examination as facility evaluators.

However, the facility representatives should only be used as evaluators if they routinely perform that function during the administration of the facility licensee's requalification program.

5.

Examination Develooment The facility licensee may develop prososed written examinations and operating tests and forward them to tie NRC as part of its reference material submittal (see Attachment 2).

In accordance with 10 CFR 55.59(a)(2)(li), the facility licensee must ensure that the operating tests require the operators to demonstrate an understanding of and the ability to aerform the actions necessary to accomplish a comprehensive sample of tie items specified in 10 CFR 55.45(a)(2) through (13),

inclusive, to the extent applicable to the facility.

Approximately two weeks before the scheduled examinations, the NRC examiners will visit the facility to make final preparations for the examination.

The written, walk-through, and dynamic simulator examinations will be developed in accordance with ES-602, ES-603, and ES-604, respectively. The examination should distinguish between reactor operator (RO) and SR0 knowledge and abilities to the extent that the facility training materials allow tbc examiners to make these distinctions.

The NRC examiners will rely upon the facility licensee's l

examination team members for site-specific technical assistance in developing, reviewing, and validating the written examination static scenarios and items, plant walk-through topics (JPMs), and dynamic simulator scenarios.

The chief examiner and the regional section chief will determine the length of time on site and the number of examiners required based on the l

experience of the examiners, the quality of the facility licensee's testing material, and the level of effort required to develop new test items.

Examiner Standards 6 of 41 Rev. 7, Sup. 1, June 1994 l

ES-601 If requested by the facility licensee, the chief examiner will brief the licensees and managers about the requalification examination process.

The examiner will use this time to explain the examination and grading processes and to respond to any questions that licensees may ask about the NRC's examination procedures.

If the schedule does not allow them to meet during the preparation week, they may meet at any mutually agreeable time.

6.

Examination Security l

t To ensure examination security, each facility representative who acquires knowledge of the content of the NRC requalification examination before it is administered will be subject to the security restrictions f

described below from the time he or she first acquires the specific knowledge until the examination exit meeting.

)

To the maximum extent possible, only the examination team members and a simulator operator should be given specific knowledge about the content of the examination. The facility evaluators should be given the package of simulator scenarios and JPMs the week before the examination to allow them to prepare for their evaluation, including coordinating the use of the simulator to perform JPMs and scenarios, if the facility licensee submits a proposed examination, those who participate in developing the p

examination become subject to the security restrictions when their involvement begins.

Also, if facility representatives other than the examination team members are used to time validate the written examination, they too are subject to the security restrictions as soon J

as they are exposed to the examination questions.

Facility representatives who acquire specific knowledge of the NRC examinations will sign Form ES-601-1, " Examination Security Agreement,"

or reasonable facsimile before their examination involvement begins and again after the examination process is complete (i.e., the exit meeting).

E.

PROGRAM AND LICENSEE EVALVATION PROCEDURES I

1.

Examination Administration For each selected licensee, conduct a requalification examination l

a.

using ES-602, ES-603, and ES-604 for the written, walk-through, and simulator portions of the requalification examination, respectively.

Examiners will document licensee performance on Form ES-601-5.

b.

The number of persons present during an operating test should be limited to ensure the integrity of the test and to minimize distractions to the licensees.

Under na circumstances will another licensee be allowed to witness an operating test.

Operating tests are not to be used by the facility licensee as p) training vehicles for future requalification examinations.

y Examiner Standards 7 of 41 Rev. 7, Sup. 1, June 1994 l

ES-601 Other examiners may observe an operating test as part of their training or to audit the performance of the examiner administering the operating test.

The chief examiner may permit other observers, such as resident inspectors, regional personnel, researchers, or NRC supervisors, to observe an operating test if the applicant does not object to the observer's presence.

Deviations from this policy must be approved, in advance, by the Chief, OLB.

Other non-NRC personnel (e.g., representatives from the Institute of Nuclear Power Operations (INP0) or the Nuclear Energy Institute (NEI)) may observe the operating tests with prior approval from the Chief, OLB. The chief examiner will control the observers' activities in accordance with guidance provided by OLB.

2.

Examination Gradina l

a.

The facility licensee is expected to grade the written examinations and the operating tests in parallel with the NRC examiners.

b.

The facility evaluators are expected to provide preliminary pass / fail results for the simulator and walk-through portions of the examination by the end of each day and the final results before the exit briefing, or at the end of each examination week for multiple week examinations.

c.

The NRC will notify the facility licensee immediately if any licensee's performance on the examination is sufficiently poor to require immediate removal from licensed activity.

The NRC will also notify the facility licensee of the results of the l

examination in accordance with ES-605, d.

The facility licensee will provide the NRC with the final results of the written examinations and an overall summary of the examination results within two weeks after the exit meeting.

3.

Evaluation of Reaualification Proarams A requalification program evaluation requires a minimum sample size of 12 licensees.

The sample size is determined by counting the number of licensees taking the dynamic simulator examination, including those operators who only participate in the simulator examination for the purpose of meeting crew composition requirements, but excluding those licensees who are being reexamined after failing a previous NRC-conducted examination.

In instances where fewer than one-half of the licensees taking the dynamic simu',ator examination complete the entire examination, the regional bran e chief will determine whether a valid program evaluation can be made.

The regional branch chief will contact the Chief, OLB, in these instances.

Examiner Standards 8 of 41 Rev. 7, Sup. 1, June 1994

/'

ES-601 a.

A satisfactory requalification program meets g_qc.h of the following:

(1)

At least 75% of the licensees must pass all portions of the examination in which they participate.

The pass rate is determined by dividing the number of licensees that pass all portions of the examination in which they participate by the total number of licensees in the sample.

l In the event of a crew failure, only those licensees who receive an unsatisfactory evaluation in the individual follow-up will be counted when calculating the licensee pass rate.

When calculating the pass rates, fractions should be rounded up to the next highest number.

For example, if 15 licensees arc avaluated, 75% passing would.be 11.25, thus 11 of 15 passing would Hat meet the 75% requirement, but 12 would.

(2)

At least two-thirds (66%) of the crews must pass the simulator examination.

For requalification examinations with more than three crews O

participating, three-of-four, or four-of-five crews must d

pass to satisfy this requirement.

b.

The following areas will be considered in the werall program evaluation and may be used to identify facility weaknesses that wal be documented in the examination report.

(1)

The facility evaluators do not concur with the NRC examiners on all UNSATISFACTORY crew evaluations.

(2)

More than ont facility evaluator is determined to be unsatisfactory.

ES-603, Attachment 2, " Evaluation of Facility Evaluators," is a checklist of items to be used to determine evaluator competence.

(3)

The facility licensee failed to train and evaluate a licensee in all the positions permitted by the individual's license.

(For instance, the facility is required to train and evaluate an SR0 in the R0 position as well as in directing operators.) An SR0 will not be required to perform R0 activities during the simulator portion of the operating test, bu; his or her performance will be evaluated if the facility normally places the SRO in a shift R0 position during the dmulator examination. Otherwise, R0 skills will be evaluated during the performance of JPMs.

C

\\x Examiner Standards 9 of 41 Rev. 7, Sup. 1, June 1994

ES-601 (4)

The facility licensee has insufficient administrative controls to preclude an R0 or SR0 with an inactive license from performing licensed duties.

Licensees must meet the requirements of 10 CFR 55.53 to restore an inactive license to active statut,.

(5)

The facility licensee has insufficient quality control of its examination bank. The NRC will evaluate the facility's performance in this area if post-examination changes to facility-developed test items result in significant l

modifications or deletions of more than 10% of the questions on the written examination.

(6)

The facility licensee's failure decisions are not as conservative as the NRC's.

The NRC will review with the facility managers any case where th< facility licensee passes a licensee that the hRC fails to ensure that the rationale for the evaluation is fully understood.

In addition, the NRC will assess whether the facility licensee's evaluations are conducted in accordance with documented facility guidance and whether facility managers periodically assess their evaluation process.

The NRC also expects the facility program to explicitly link licensee examination failure with gonfg performance. In this way, all facility failures and NRC failures will agree.

If the facility program has standards of operator performance that are NOT explicitly linked to unsafe performance, and thus do not meet the threshold stated in these standa.s for licensee failure of the examination, then, for ali ' a liity failures, the facility is expected to differentiate mang those failures where the operator performed at an unsafe level, from those failures where the operator failed for other than safety reasons (i.e., not meeting higher facility-established performance standards).

In these cases, those licensees identified as failing for safety reasons would also be considered NRC failures.

4.

Ey.Alyation of Licenne Performance To pass the NRC-con 6ucted reaualification examination, the licensee must l l

pass a written carrination and an operating test consisting of a walk-through examinat t in and a dynamic simulator examination, developed and administered in acordance with ES-602, ES-603, and ES-604, respectively, unless OLB has authorized the regiont.1 office to use the facility licensee's alternative examination methodology.

The licensee must also be a member of a crew that passes the dynamic simulator examination to pass the operatina test.

~

&W Examiner Standards 10 of r Rev. 7, Sup. 1, June 1994

_=-

=

ES-601 F.

UNSATISFACTORY LICENSEE OR PROGRAM EVALUATION 1.

Actions Followina an Unsatisfactory Licensee Evaluation In all cases, a facility licensee's administrative procedures should ensure that a licensee who has failed a requalification examination is removed from licensed duties, given remedial training, and reexamined before being allowed to return to licensed duties.

This also applies to an SR0 who performs only R0 level duties at the facility when the failure is due solely to activities of SR0 responsibility.

ES-605 contains the procedure for notifying the licensee about his or her performance on the requalification examination and guidance about the actions to be performed for a licensee to return to licensed duty.

Although the regulation'(10 CFR 55.57(b)(2)(iv)) that required a licensee to pass an NRC-administered requalification examination as a prerequisite for license renewal has been deleted, it would be inappropriate to renew the license of any operator who failed to pass any NRC-conducted requalification examination, without some level of NRC involvement in the retesting process.

The amount of NRC involvement may include conducting the retest in accordance with the appropriate Examiner Standard (s); inspecting the facility licensee in accordance with IP 71001, " Licensed Operator Requalification Program Evaluation,"

as it retests the licensee; or reviewing the reexamination prepared by O

the facility licensee. The regional office, in consultation with OLB, will determine the appropriate level of involvement on a case-by-case basis. depending on the quality of the facility licensee's program. As long as the licensee submits a timely renewal application, the term of the license will continue until the renewal requirements are satisfied or the licensee fails three NRC-conducted examinations as discussed in ES-605.

If a licensee who failed a requalification examination is not prepared for a reexamination after six months of remedial training, the regional l

office will request the following information from the facility licensee:

confirmation that the facility licensee still has a need for the individual's license, the expected completion date of the licensee's remedial training and when the facility licensee will be ready to administer its retake examination, and assurance that the licensee will not be returned to licensed duties until he or she successfully retakes the examination (or portion thereof) administered by the facility licensee with a satisfactory requalification program or in accordance with the-provisions of the confirmatory action letter (CAL) if the facility O

licensee has an unsatisfactory program and the NRC has not determined it to be " provisionally satisfactory."

Examiner Standards 11 of 41 Rev. 7, Sup. 1, June 1994

..m~.,

m.-

ES-601 The NRC will inform the facility licensee that a comprehensive requalification examination may be necessary if the licensee is not ready to take a retest within one year after failing the examination.

2.

Actions Followina an Unsatisfactory Reaualification Proaram Evaluation The following actions will be taken for all requalification programs evaluated as unsatisfactory, but the Regional Administrator or designee may specify additional actions, as appropriate.

The specific sequence of actions is not critical, though this sequence of events corresponds to a typical regional response to an unsatisfactory program evaluation.

The Regional Administrator or designee should defer determining whether a plant shutdown is required until all items in item (b) of this section are reviewed, a.

The facility licensee is expected to identify program deficiencies and corrective actions to improve licensee performance.

The NRC will use a CAL to establish a formal dialogue and to document the facility licensee's corrective action commitments.

A licensee who fails the requalification examination, as determined by the NRC, will be subject to an NRC-administered l

reexamination before resuming licensed duties.

A licensee whose performance does not meet facility standards, as l

determined by the facility licensee, is expected to be remediated and reevaluated by the facility licensee in accordance with the provisions of the facility licensee's requalification program.

The NRC will review and/or monitor the reexamination to assure the adequacy of the facility licensee's requalification program.

b.

The NRC will schedule a meeting with senior facility managers to review the examination results, the identified deficiencies and their root causes, the proposed corrective actions and the schedule for their implementation and the need for follow-up inspections and examinations.

(Refer to Section F.3 for additional guidance for the conduct of augmented inspections.)

The Regional Administrator will evaluate the examination and inspection results and make a decision regarding the continued operation of the facility and possible enforcement action against the facility licensee.

At a minimum, the following factors should be considered when making this determination:

the results and corrective actions from previous program evaluations, the significance of generic performance deficiencies identified during the program evaluation, recent SALP ratings, recent facility events that relate to licensed operator performance, and Examiner Standards 12 of 41 Rev. 7, Sup. 1, June 1994

ES-601 v

recomendations by the NRC staff (including the results of any operational evaluations and inspections),

c.

If the unsatisfactory program evaluation is due to licensee performance deficiencies, an operational evaluation is required.

The operational evaluation is intended to help the Regional Administrator determine whether the facility's remaining operating crews are suitably qualified to continue to operate the facility.

In this case, the facility licensee identifies the individual licensees and shift crews it proposes to use to continue plant operations.

The regional office may choose not to evaluate those licensees who passed their most recent NRC-conducted initial or I

requalification examination wHhin the past 12 months.

However, the regional office will evaluate all other licensees in those areas noted as operational deficiencies during the requalification examination, regardless of whether they have already passed or not yet taken the facility-administered requalification examination.

The regional office will conduct the operational evaluations in accordance with the guidance in ES-603 and ES-604, as applicable.

If the facility licensee proposes to use a shift crew that is significantly different from its normal configuration, even though all the licensees may have recently passed an NRC-conducted l

examination, the~ regional office may perform an operational p) evaluation of this crew.

(v The regional office should schedule the operational evaluation as soon as possible after determining that the facility licensee's requalification program is unsatisfactory.

The evaluation should not be delayed to accommodate the facility's operating schedule, the completion of programmatic corrective actions, or the completion of remedial training for licensees who failed the requalification examination.

The operational evaluation may identify further program deficiencies that may need to be reflected in the CAL discussed in item (a) of this section or that may warrant additional inspection by the NRC.

Additional licensee weaknesses that require remediation may also be identified.

d.

The NRC will review the corrective actions the facility is to perform, the expected follow-up actions by the NRC, and the schedule for each.

As part of the follow-up activities, the NRC may conduct additional operational evaluations, requalification retake examinations, and augmented inspections, as necessary.

Before these activities, the NRC will verify that the facility licensee has completed the applicable corrective actions and will obtain a certification of crew readiness from the facility managers.

Regional managers should consider using a new chief examiner and U

Examiner Standards 13 of 41 Rev. 7, Sup. 1, June 1994

ES-601 having examiners from other regional offices or OLB participate on those operational evaluations and requalification retake examinations that have restart approval implications, e.

The Regional Administrator will incorporate into the decision on follow-up activities any extraordinary circumstances surrounding the examination activities that may have a bearing on the validity of the examination results.

f.

A requalification program that the NRC determines is unsatisfactory will remain unsatisfactory until all identified corrective actions agreed upon by the NRC for restoring the program to satisfactory status and related NRC follow-up activities are completed.

However, for purposes of allowing facility examiners to perform reexamination functions, a facility may v.tain a status of

">rovisionally satisfactory" if the facilhy has completed to the N1C's satisfaction all short-and intermedicte-term corrective actions agreed on with the NRC.

Once the NRC has determined that these corrective actions have been satisfactorily implemented, the Regional Administrator or designee will determine whether to permit the facility to reexamine all licensees who failed the NRC-conducted requalification examination for the pur)ose of returning l the licensees to licensed duties. Any licensee w1o fails the NRC-conducted examination still needs to pass a future NRC-administered (i.e., conducted, inspected, or approved, as appropriate) requalification examination to renew the license.

Long-term corrective actions are expected to be completed before the NRC's next requalification program evaluation.

To attain a satisfactory rating following an unsatisfactory evaluation, the subsequent requalification program evaluation, with a sample size of at least 12 licensees, must satisfy the passing criteria in Section E.3.

3.

Auamented Insoection Guidelines If it is determined that an augmented requalification program inspection is required, regional management shall define its scope and depth based upon the nature of the deficiencies.

The regional office should consider the following activities in addition to those specified in Section F.2.

a.

The regional office may conduct augmented inspection coverage of all shifts.

The inspection procedures for shift coverage should be used as appropriate.

Inspection activities should give particular attention to the following areas:

operator performance and attitude, operator overtime, management oversight, and shift staffing.

Examiner Standards 14 of 41 Rev. 7, Sup. 1, June 1994

ES-601 b.

The regional office may develop a long-term training program l

inspection plan based on Inspection Procedure (IP) 41500,

" Training and Qualification Effectiveness." Such an inspection plan may include the following:

ongoing status reviews of requalification training effectiveness with an emphasis on known program deficiencies and implementation of short-term corrective actions, a root cause inspection to establish the root cause(s) for the unsatisfactory requalification program evaluation and to verify that the facility licensee's proposed corrective action plan should preclude or minimize the probability of recurrence, and an inspection to evaluate the adequacy of the facility licensee's corrective actions and to determine the effectiveness of the facility licensee's SAT-based requalification program.

c.

The regional office may convene an enforcement panel to determine if action is warranted based on the requirements of 10 CFR 50.54, paragraph (1-1).

The potential exists that a requalification program rated unsatisfactory on two successive NRC evaluations O

does not meet the minimum requirements of 10 CFR 55.59(c) as required by 10 CFR 50.54(1-1). The basis for any proposed enforcement action will be the inadequate corrective action or requalification program element deficiencies (identified by the inspections of item (b) of this section) which led to the successive requalification examination failures.

G.

REQUALIFICATION PROGRAM EVALUATION REPORT The regional office will prepare a final requalification program evaluation report after the Regional Administrator or designee approves the requalification examination results.

A copy of the written examination only needs to be included in the program evaluation report if written examination problems are addressed in the report.

The regional office will issue the report within 30 days of receipt of the facility licensee's final results or the examination exit meeting, whichever is later, and will file a complete copy of the report in the facility's requalification file.

The chief examiner is responsible for completing the " Power Plant Requalification Results Summary (and Continuation) Sheet," Forms ES-601-3 and ES-601-4.

The examiner will enter each licensee's scores in the appropriate columns.

Under the column titled " Simulator," the examiner will enter the results of the licer,see's individual follow-up evaluation.

If the licensee did not receive an individual follow-up evaluation, the examiner will enter a i

passing score.

If a licensee was a member of a crew that failed the dynamic simulator examination but the licensee passed or did not receive an individual Examiner Standards 15 of 41 Rev. 7, Sup. 1, June 1994

ES-601 follow-up evaluation, enter a pass in the _imulator column for that licensee.

Crew failures will be summarized in the overall results at the top of Form ES-601-3.

The regional office will send a copy of the summary (and continuation) sheet to the operator licensing assistant (0LA), OLB.

OLB uses the results summary to verify the data in the operator licensing tracking system (0LTS) so that statistics can be kept on licensee performance.

As it contains Privacy Act information, the regional office will not include the results summary in the examination report.

If a small number of licensees are given retake examinations, the regional office may issue an addendum to the original requalification evaluation report instead of issuing a new report.

If the reexaminations are conducted concurrently with initial examinations or inspected during a requalification program evaluation in accordance with IP 71001, the results may be reported as part of the initial examination or inspection report.

H.

INDIVIDUAL REQUALIFICATION EXAMINATION REPORT After the regional office completes the requalification evaluation, it will keep a copy of each licensee's NRC-conducted written, walk-through, and simulator examination results and return the original documents to the facility licensee.

The facility licensee is required by 10 CFR 55.59 to maintain records of these examination results along with a copy of the written examination until the operator's license is renewed or for two years after the license expires.

The NRC chief examiner will ensure that Form ES-601-5, " Individual Requalification Examination Report," is completed for each licensee who takes an NRC-conducted requalification examination. The report will include the individual's:

written examination grade, crew evaluation from the dynamic simulator examination, the results of the individual follow-up (P or F) from the dynamic simulator examination, and number of JPMs performed correctly (and percentage), if conducted.

The regional office will send a copy of this report to the facility training manager and to the licensee with a letter notifying the licensee of the examination results in accordance with ES-605.

The regional office will also file a copy in the licensee's docket file.

I.

OPERATOR LICENSE RENEWAL POLICY Licensees are not required to take an NRC-conducted requalification examination in order to renew their licenses.

However, if a licensee takes but fails to pass an NRC-conducted examination, the NRC will not renew the license until the licensee passes a retake examination conducted by the NRC, Examiner Standards 16 of 41 Rev. 7, Sup. 1, June 1994

ES-601 passes a retake examination administered by the facility licensee and inspected by the NRC in accordance with IP 71001, or passes an examination approved by the NRC.

The regional office, in consultation with OLB, will determine the appropriate level of involvement on a case-by-case basis depending on the quality of the facility licensee's requalification program.

' ES-605 contains the specific procedures to follow for a licensee who fails one or more NRC-administered requalification examinations, and the procedure for processing license renewal applications.

J.

RECORD RETENTION 1.

Facility Recualification Examination File The operator licensing staff in each regional office will maintain a facility requalification file for each facility in its region and will retain those files consistent with regional office guidelines for retention of facility initial examination files.

The regional office will place the following items in the file for each requalification examination:

a.

Examiner Standards attachments and forms:

ES-201, Attachment 3, " Examination Assignment Sheet" O

Form ES-403-1, " Examination Grading Quality Assurance Checkoff Sheet" Form ES-501-4, " Contract Examiner Evaluation" (if applicable)

ES-601, Attachment 1, " Corporate Notification Letter" Form ES-601-1, " Examination Security Agreement" Form ES-601-3, " Power Plant Requalification Results Summary Sheet" and Form ES-601-4, " Power Plant Requalification Results Continuation Sheet" (if applicable)

Form ES-604-2, " Simulator Crew Evaluation" b.

A master list of all JPMs administered and the licensees to which they were administered.

c.

A master list of all scenarios conducted and licensees to which they were administered.

Facility-generated forms or Form ES-303-3, " Scenario Events," may be used to meet this requirement.

d.

A copy of the written examination and answer key.

e.

A copy of the requalification examination report.

The regional office may require that additional documents be retained in the facility requalification examination file.

t Examiner Standards 17 of 41 Rev. 7, Sup. 1, June 1994 l

ES-601 2.

Licensee Docket Files The regional office will retain the following records in the licensee docket files until the license is renewed or for two years after the license expires or is terminated:

a.

Form ES-601-5, " Individual Requalification Examination Report" b.

ES-605, Attachment 1, 2, 3, or 4, "Results Notification Letter" c.

A copy of all failed portions of the NRC-graded examination 3.

Other Files The regional office will retain reference material used to develop each examination until the NRC has resolved with the facility all failures associated with the examination and has sent a notification letter to each licensee.

K.

REQUALIFICATION FEEDBACK FORMS OLB has developed a feedback form to monitor the stress of licensees and facility personnel participating in the requalification examination.

Facility personnel will receive the feedback form and instructions as an attachment to the 90-day letter (Attachment 2 to this standard).

Licensees and facility representatives participating in the requalification examination will be requested to complete the survey at the conclusion of the examination and forward the forms to the regional branch chief.

A comprehensive review of the feedback forms for each examination will be performed by the regional branch chief or a designee (not an examiner associated with that examination trip).

The review will consider the specific content of the licensee's or facility personnel's concerns or suggestions.

The regional branch chief may use the results of the survey, if applicable, to:

forward the concerns or suggestions that may affect the administration of the operator licensing requalification program to the Director, Division of Reactor Controls and Human Factors; recommend changes to NUREG-1021 to OLB that would further alleviate licensee stress.

Recommendations should be documented and forwarded to headquarters using Report on Interaction (ROI) forms; monitor the level of stress in licensees and facility representatives.

The level of stress perceived by licensees and facility personnel can affect their overall performance on the requalification examination; and advise regional examiners about examination techniques that may be causing examination stress.

Examiner Standards 18 of 41 Rev. 7, Sup. 1, June 1994

)

ES-601 The regional office should submit copies of all completed survey forms to OLB so that stress levels can be monitored, analyzed, and reported to the Commission, and n that OLB can ensure that the appropriate corrective actions are implemented, where necessary.

The regional office should retain the completed feedback forms for two years.

ATTACHMENTS / FORMS:,

" Examination Timetable",

" Sample Corporate Notification Letter",

"Examinatic:

ample Plan" Form ES-601-1,

" Examination security Agreement" Form ES-601-2,

" Evaluation Checklist for Facility Reference Material" Form ES-601-3,

" Power Plant Requalification Results Summary Sheet" Form ES-601-4,

" Power Plant Requalification Results Continuation Sheet" Form ES-601-5,

" Individual Requalification Examination Report" 7-)

i a

[s

/

Examiner Standards 19 of 41 Rev. 7, Sup. 1, June 1994

ES-601 Examination Timetable Attachment I l

D31g*

Activity

-120/90 The NRC notifies the facility licensee.

-60 The facility licensee sends the NRC the material requested by the NRC for developing the examination (including written examination questions, simulator scenario banks, and JPMs).

The facility licensee proposes composition of the crews to be evaluated and identifies facility examination team members.

The facility licensee may request the NRC chief examiner to review the examination process with licensees and managers.

-45 The facility licensee submits its proposed requalification written examination and operating test.

-30 The NRC concurs on the operating crews to be evaluated.

-14 The NRC examiners visit the facility to review the JPMs to be administered, observe the static and dynamic simulator examinations, and validate the test items as needed.

The chief examiner and the regional section chief determine the length of time on site and the number of examiners required, based on the experience of the examiners and the quality of the facility licensee's testing material.

The facility licensee designates a simulator operator.

If requested, the chief examiner briefs the licensees and managers about the requalification examination process.

-7 The facility examination team members finalize the examinations based on preparation week activities.

Evaluators review reference material to prepare for the JPMs and simulator scenarios.

O The NRC administers the examinations to selected crews and licen-sees.

Facility licensee notifies NRC of their final results for crews and individuals at the end of each examination week.

+7 The NRC finalizes the examination results.

+14 The facility licensee transmits the written examination grades and a final summary to the NRC.

l

+30#

The NRC issues licensee results and the final requalification l

examination report.

Number of days before or after the examination, except as noted.

l Number of days after receipt of facility results.

O Examiner Standards 20 of 41 Rev. 7, Sup. 1, June 1994 b

G ES-601 Sample Corporate Notification Letter NRC Letterhead (Date)

(Name. Title' (Name of fac lity)

(Address)

(City. State. Zio code)

SUBJECT:

REQUAllFICATION PROGRAM EVALUATION

Dear (Name):

In a telephone conversation on (date), (Name. title) and (Name. title) arranged to evaluate the requalification program and licensed personnel at the (facility name).

The evaluation is scheduled for the week of (date).

NRC examiners and evaluators from your facility will conduct requalification examinations, and the NRC will evaluate the facility licensee's requalification program in accordance with Sections ES-601 through ES-604 of NUREG-1021,"OperatorLicensingExaminerStandards," Revision 7, Supplement 1.l You are encouraged to ensure that your training staff and proposed examinees are familiar with these standards.

For the NRC to adequately prepare for this evaluation, the facility licensee will need to furnish the NRC the approved items listed in Enclosure 1,

" Reference Material Guic'elines." You are also requested to submit, at your l

option, a proposed examination for use during the examination week. However, if you do submit a prop] sed examination, the personnel participating in its development will become subject to the security restrictions described in this l letter.

Please review the guidance promulgated in Revision 7, Supplement 1, of NUREG-l 1021 on the content and sco)e of simulator examination scenarios.

The scenario examination bank s aould cover the entire spectrum of emergency operating procedures (E0Ps), including alternative decision paths within the E0Ps, and it should incorporate a range of failures with various degrees of severity for the same type of event.

Each scenario should contain simultaneous events that recuire the senior reactor o)erators (SR0s) to 3rioritize their actions anc to assign other crew mem)ers particular tasks.

Each scenario should also require the SR0 to decide when to transition between E0Ps and decide which actions to take within E0Ps.

You are requested to designate at least one employee to be a member of a joint l NRC-facility examination team. The employee is expected to be an active SR0 as defined by 10 CFR 55.53(e) or (f) from the (facility name) operations department.

You are encouraged to designate a second employee from the training staff to be a member of the examination team.

This employee should also be a licensed SRO, but may be a certified instructor.

If desired and agreed to by the chief examiner, you may designate one additional employee from the training staff with appropriate qualifications to be a member of the A) f'O Examiner Standards 21 of 41 Rev. 7, Sup. 1, June 1994 J

ES-601 2

examination team.

In addition to these individuals, you will need to designate a simulator operator for scenario preview and validation during the on-site examination preparation week.

In some cases, you may need to designate a simulator operator during the test item review period. All these individuals will be subject to the examination security agreement.

The NRC restricts any facility licensee representatives under the security agreement from knowingly communicating by any means the content or scope of the examination to unauthorized persons and from participating in any facility licensee programs such as instruction, examination, or tutoring in which an identified requalification examinee will be present.

These restrictions apply from the day that the facility licensee representative signs the examination security agreement indicating that the representative understands that he or she has specialized knowledge of the examination.

The chief examiner will determine when a facility licensee representative has received specialized knowledge concerning the examination and will execute an examination security agreement.

In most cases, the examination team members will not be required to enter into an examination security agreement more than 60 days before the examination week.

The simulator operator will normally become subject to the security restrictions during the examination preparation and validation week, however, this may occur as much as 45 days before the examination week.

Sixty days before the examination administration date, please provide the NRC regional office with a 3roposed list of licensees, including crew composition, for the examination.

Tie list should include at least 12 licensees, comprising three or more crews, and the current mailing address for each proposed licensee, if different from that listed on the most recent Form 398 submitted to the NRC.

The facility licensee training staff should send this information directly to the NRC's chief examiner, ensuring that each licensee's address is sent in a manner to ensure privacy.

The facility licensee may request that the NRC chief examiner or another NRC representative meet with the licensees to be examined and the licensee managers during the examination preparation week, normally two weeks before the examination.

However, if the schedule does not allow them to meet during the preparation week, they may meet at any mutually agreeable time.

The NRC examiner will explain the examination and grading processes and will respond to any questions that licensees may have about the NRC's examination procedures.

The facility licensee training staff should schedule this meeting, if it is desired, with the NRC chief examiner.

The facility licensee is requested to distribute the "Requalification Examination Feedback Form," attached as Enclosure 3.

The NRC requests that this feedback be completed by all operators, evaluators and facility licensee representatives participating in the NRC requalification examination, including facility licensee managers.

The results from this survey will be used to measure the success of the NRC and facility licensee's efforts to reduce undue stress during the requalification examination.

O Examiner Standards 22 of 41 Rev. 7, Sup. 1, June 1994 m

ES-601 3

The facility licensee staff is responsible for providing adequate space and accommodations to properly develop and conduct the examinations.,

" Administration of Requalification Examinations," describes our requirements for developing and conducting the examinations. Also, a facility operations management representative above a shift supervisor level should observe the simulator examination process at the site.

The request for requalification examination material is covered by Office of Management and Budget (0MB) Clearance Number 3150-0101, which expires on October 31, 1995.

The estimated average burden is 7.7 hours8.101852e-5 days <br />0.00194 hours <br />1.157407e-5 weeks <br />2.6635e-6 months <br /> per response, including gathering, copying and mailing the required material.

Send comments l regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to the Information and Records Management Branch, Mail Stop T-6 F30, Office of Information Resources Management, U.S. Nuclear Regulatory Commission, Washington, D.C. 20555; and to the Paperwork Reduction Project (3150-0101),

Office of Information and Regulatory Affairs, NE08-3019, Office of Management and Budget, Washington, D.C. 20503.

The request for responses to the Requalification Feedback Form is covered by OMB Clearance Number 3150-0159, which expires on February 28, 1996. A request l for a clearance revision has been submitted to 0MB and is expected to be granted. The estimated average burden is 30 minutes per response, including copying, and mailing the completed responses. Send comments about this burden estimate or any other aspect of this collection of information, including t\\

suggestions for reducing this burden, to the Information and Records Management Branch, Mail Stop T-6 F30, Office of Information Resources Management, U.S. Nuclear Rec ulatory Commission, Washington, DC 20555; and to the Paperwork Reduction Pro;ect 3150-0159), Office of Information and Regulatory Affairs NE08-3019, O fice of Management and Budget, Washington, DC 20503.

Thank you for your cooperation in this matter. Mag l has been advised of the NRC guidelines and policies addressed in this letter.

If you have any questions on the evaluation process, please contact (Name. reaional section chief) at (telephone numbert.

Sincerely, (Appropriate Regional Title) 1.

Reference Material Guidelines l

2.

Administration of Requalification Examinations 3.

Requalification Feedback Form DISTRIBUTION:

Public Document Room NRC Document Control System Regional Office Distribution O

Examiner Standards 23 of 41 Rev. 7, Sup. 1, June 1994

=

ES-601 4

Reference Material Guidelines 1.

The facility licensee should provide test items to support all aspects l

of the requalification examination to the NRC 60 days before the examination date.

2.

The facility licensee is expected to submit the following reference materials for all NRC-conducted requalification examinations:

an examination sample plan which meets the requirements of ES-601, the facility's examination banks (written, simulator and JPM) and associated reference material. At a minimum the reference material should include Technical Specifications, abnormal and emergency operating procedures, and emergency plan procedures utilized in the requalification training.

l additional reference material as requested by the NRC chief examiner.

3.

The facility licensee is expected to attain the following test item target levels on 10/1/95, five years after the implementation of Revision 6 of NUREG-1021 (10/1/90).

A minimum of 700 test items for use in the written examination equally divided between the two sections of the written examination and which cover all safety-related elements of the facility job-task analysis (JTA).

The facility licensee is expected to maintain a dynamic bank by reviewing, revising or generating at least 150 questions a year.

New questions should cover equipment and system modifications and recent industry and licensee events and procedural changes.

JPMs to evaluate each reactor operator and senior reactor operator safety-related task identified in the facility JTA, which meet the criteria in ES-603. The JPM bank should expand at a rate of at least 10 JPMs per year until this goal is reached.

It is estimated that 125-150 JPMs will be the final result.

A bank of at least 30 simulator scenarios which reflect all abnormal and emergency situations to which a licensee is expected to respond or control.

At least five scenarios per year should be generated until all aspects of the emergency operating procedures are covered with sufficient variation in the type and scope of initiating events and level of degradation.

Emphasis should be placed on scenarios that include applicable industry events.

O Examiner Standards 24 of 41 Rev. 7, Sup. 1. June 1994

l b

ES-601 5

Administration of Requalification l

Examinations l

1.

The NRC must evaluate at least 12 licensees to perform a program evaluation.

The guidelines on crew composition in the simulator are described in ES-601, Section D.2, and ES-604.

2.

The simulator and simulator operators need to be available for examination development.

The chief examiner and the facility representatives will agree on the dates and duration of time needed to l

develop the examinations.

3.

The chief examiner will review the reference material used in the simulator.

The NRC will not authorize the use of reference material that is not normally used for plant operation in the control room to be used during the simulator test.

4.

The facility licensee will provide a single room for completing Section B of the written examination. The examination room and the supporting rest room facilities will be located to prevent the examinees from i

contacting all other facility and contractor personnel during the l

examination.

9 5.

The chief examiner will inspect the examination room to see that it meets the minimum standard that will ensure examination integrity. The minimum spacing standard consists of one examinee per table and a 3-foot space between tables.

No wall charts, models, or other training materials are allowed in the examination room.

6.

The facility licensee is expected to provide a copy of each reference document for each examinee for Section B of the written examination.

The material should include documents that are normally available to the licensees in the control room such as the technical specifications, operating and abnormal procedures, administrative procedures, and the emergency plans.

The chief examiner will review the reference material before the examination begins.

7.

The NRC requalification examination will attempt to distinguish between R0 and SR0 knowledge and abilities to the extent that the facility training materials allow the developers to make these distinctions.

Examiner Standards-25 of 41 Rev. 7, Sup. 1, June 1994

ES-601 2

8.

Prudent scheduling of examination week activities is important to help alleviate undue stress on the licensees. The facility training staff and the NRC chief examiner should attempt to formulate a schedule that will minimize delays while conducting the examination.

The following are some suggestions for structuring the examination activities to achieve this objective:

Bring in licensees in accordance with their scheduled examination times.

It is better to segregate the group'of licensees completing their examination, instead of the group of licensees that are scheduled to start their examination.

Following simulator scenarios, the facility evaluators and NRC examiners should quickly determine whether follow-up questioning is required so that the crew members may be released to talk among themselves about the scenario.

Ensure that time validation of JPMs, particularly those performed in the simulator, is accurate.

Establish a reasonable schedule to prevent licensees from waiting for simulator availability to complete their JPMs.

l 9.

The NRC no longer requires the facility licensee to videotape dynamic simulator examinations.

If the facility licensee requests to videotape l

the examination, any use of the tape must be completed before the NRC leaves the site at the end of the examination.

If a disagreement over the grading of a licensee still exists at the end of the examination week, the facility licensee may retain the tape for the purpose of submitting it to support a request for regrade by the NRC.

During the regrade, the NRC will review only the portion of the videotape under contention.

After all requalification examination grades are finalized, including the review of any regrade requests, the facility l

licensee is expected to erase all video tapes made during the l

examination.

O Examiner Standards 26 of 41 Rev. 7, January 1993

p ES-601 7

Requalification Examination Feedback Form Introduction The NRC is requesting feedback regarding the conduct of requalification examinations.

The information provided will be used to monitor, on a generic basis, the effectiveness of the NRC's and facility licensee's efforts to minimize undue stress in the examination process.

This form is not intended as a means of resolving technical or process concerns pertaining to a specific examination.

Such concerns will be resolved using the guidance in NUREG-1021, " Operator Licensing Examiner Standards."

Instructions Completion of this form is voluntary.

If you choose to provide feedback, please answer the questions in accordance with these instructions:

The questions in this form regard the examination administered by (Recion) at (facility licensee) during (exam dates); however, comparisons with previous examinations may be appropriate.

Any examinee or individual involved in the development or administration of this examination is encouraged to complete this form.

Mail completed forms to:

(NRC Reatonal Office)

(c/o Ooerator Licensina Branch Chief)

Your Backaround Please check the boxes that describe your involvement in this examination.

I was:

an examinee involved in developing the examination involved in administering the examination an examination observer other:

l Please check the boxes that describe your current position.

i (Check all that apply)

R0 SR0 operating crew member training department operations department l

other:

O Examiner Standards 27 of 41 Rev. 7, January 1993 i

J ES-601 8

2 Stress vs. Undue Stress The following questions require you to make a judgment of whether there was undue stress during the examination..

Examinations are inherently stressful events and therefore it is important that you make a distinction between stress and undue stress when making your judgments. Undue stress is unnecessary or inappropriate stress which can be practically eliminated without compromising the validity of the examination.

The distinction between stress and undue stress is not a matter of whether the stress was extreme or mild. When making your judgments you should follow these steps:

First, consider the cause of the stress.

Would it have been possible and practical to eliminate the cause of the stress without compromising the validity of the examination?

If your answer is no, then no undue stress was present.

(See point #1 on the rating scale below.)

If your answer is yes, consider the magnitude of the stress.

A source of stress may be unnecessary but also sufficiently small in magnitude to be unlikely to affect an individual's performance in the examination.

(See point #2 on the rating scale below.) The alternative is that the source may be unnecessary and also of sufficient magnitude to be likely to affect an individual's performance in the examination.

(See point #3 on the rating scale below.)

Ratina Scale:

1.

No undue stress 2.

Some undue stress Inappropriate stress was present that could have been practically avoided but would not likely affect an individual's examination performance.

3.

Significant undue stress Inappropriate stress was present that could have been practically avoided and it would likely affect an individual's examination performance.

O Examiner Standards 28 of 41 Rev. 7, January 1993

O ES-601 9

3 Examination Feedback Ratinas:

Please use the rating scale described on the preceding page to indicate your judgment of the degree of undue stress that was present in each aspect of the examination identified below. Write the number (1, 2, or 3) in the space preceding the section.

Coments:

Please comment abcut the source or cause of any undue stress, including who was affected (e.g., examinees, examiners) and suggested practical solutions. Attach additional sheets if necessary.

Preexamination Interactions with NRC C5i5iients:

Written Examination: Administrative Controls / Procedural Limits C5i5iients:

O Written Examination: Plant and Control Systems C5isi5ents:

Dynamic Simulator C555ents:

Job Performance Measures CoE5ents:

Please coment on any practices which you believed were successful in reducing undue stress.

Your cooperation in completing this form is appreciated.

O Examiner Standards 29 of 41 Rev. 7, January 1993

ES-601 Examination Sample Plan A.

INTRODUCTION An examination sample plan provides a systematic approach to selecting and developing test items to determine if a student has mastered the skills, knowledge, and abilities to be covered in a particular training program.

The sample plan should provide an explicit, documented link between the learning objectives associated with the training prograr and the test items used to perform the evaluation and to verify the relevenu to the job task analysis (JTA) associated with the licensee's position.

NUREG/BR-0122, " Examiner's Handbook for Developing Operator Licensing Written Examinations " gives explicit guidance for developi g a sam)le plan for initialexamInationsusingthePWRandBWRK/ACatao (NUlEGs-ll22 and 1123).

You may apply the same methodology to any tra ing program.

With respect to a requalification program, the scope of topics is necessarily limited because the amount of material that is covered during a requalification program is less than that covered in an initial licensing training program.

However, the NRC permits and encourages reserving a certain percentage of test items (10-20%) for topics that have high importance ratings and contain knowledge and abilities that licensees should retain because of their safety significance, but were not necessarily covered during the requalification cycle.

B.

REQUALIFICATION TEST OUTLINE The facility licensee is expected to develop a test outline for all NRC-administered requalification examinations.

At least 80% of the test outline must reflect the training curriculum of the most recent requalification cycle in a manner consistent with the distribution of emphasis in the curriculum.

The curriculum of the recualification training cycle for which the examination is being developed shoulc identify the following:

1.

Requalification lecture / classroom topics indicating the percent of the cycle devoted to each.

2.

Concentration of training exercises using the simulation facility, including the type of scenarios trained on normal) and the number of times each scenar(io was run.e.g., accident, abnormal, 3.

Special focus of the training such as plant modifications, licensee event reports (LERs), and major changes to operating practices or policy.

4.

Practical training such as operation of individual systems or components for requalification training purposes, using either the simulation facility, mockups, or actual systems and components.

Examiner Standards 30 of 41 Rev. 7, January 1993

ES-601 2

The format of the sample plan is a matter of training department preference as long as the plan results in a thorough and accurate assessment of the facility training )rogram and the intended ob;ectives.

The sample )lan is expected to contain tie following for use in developing or selecting tie test items to be used in the requalification examination:

1.

Identification of the sub;iects to be evaluated (system, component, procedure, or other training subject);

2.

The preferred testing media for evaluating each subject (written, simulator, or walk-through examination); more than one testing method may be used to evaluate a subject; 3.

The learning objectives intended to be evaluated; 4.

List of references used to develop the test items; 5.

The specific K/A catalog topic or facility JTA knowledge, skills and abilities (KSAs) that are closely linked to the learning objectives for each subject and the importance factors for each (the facility licensee may use a site-specific K/A catalog if it exists);

O All test items used in the examination should have a K/A value of 3 or 6

greater; the facility licensee may propose the use of test items with NRC K/A values less than 3 with appropriate justification; 7.

The percentage or number of points of the examination that should be devoted to the topic area (e.g., 3 points for technical specification interpretation, or 5% on reactor coolant pumps);

8.

Whether the subject is identified as safety-related in the facility JTA; 9.

Whether the subject was covered in the cycle for which the examination is being developed; 10.

The identification code or number for previously developed test items that evaluate the subject; 11.

Recent safety-related issues and events (e.g., relevant LERs).

OO Examiner Standards 31 of 41 Rev. 7, January 1993

ES-601 Examination Security Agreement Form ES-601-1 1.

Preexamination I acknowledge that I have acquired specialized knowledge about the requalification examination scheduled for the week (s) indicated in this agreement as of the date of my signature and agree that I will not knowingly divulge any information about this examination to any unauthorized persons. An unauthorized person is any individual who has not been approved by the NRC chief examiner to receive specialized knowledge of the examination.

I understand that I am not to participate in any instruction, tutoring, or examination involving those licensees scheduled to be administered this requalification examination from this date until completion of examination administration.

I further understand that violation of the conditions of this agreement may result in cancellation of the examination and/or an enforcement action against me or the facility licensee by whom I am employed or whom I represent.

2.

Post-examination I did not, to the best of my knowledge, divulge any information concerning the examination (s) administered during the week (s) indicated to any unauthorized persons.

I did not participate in instructing those licensees who were administered this requalification examination (s) from the date that I entered into this security agreement until the completion of examination administration.

Examination Period to Printed Name Pre-examination Post-examination Certification (1)

Certification (2)

Date Date Date Date Date Date Date Date'~~ ~

Date Date

~

Date Date Date Date Date Date Date Date Date Date Date Date Date Date Date Date Date Date Date Date Date Date Examiner Standards 32 of 41 Rev. 7, Sup. 1, June 1994

AsA hg j;&

q, si +: '.

A

,t 6 i<'

IMAGE EVALUATION b

[,;lff/ ' eg,

^~

\\,g

'jf#

TEST TARGET (MT-3)

• ^

////

///

/

+

s 1.0 E ""

' " e 21.2 n

1

. 7,,,

g_2p l,l l =1.8 i

l.25 1.4 t

1.6 b

==

==

4

_.. --- - 15 0 m m 4

.-----6" b

44 %A+ k A ~ 4).

tr sA

% ;j g /JKN,;p

//

.ss w g\\

eg4 4

4, o

6

, v. g.

v 6/

9 h* '

qe 1

(O b

E o

g, s..,

.s, o#

3' 4

'~

IMAGE EVALUATION A <%

o 0 t"

/ [yf 4(

),//

E TEST TARGET (MT-3)

Q+49,:*

'k

' }f "p

//q%

l.0

!! "" 0" m-22 3,1;:;-

it i-ym 2R i,i Edi 1.25 1.4 1.6

=

=

=

4

- 150mm d

6" h'A f

gg%

y y 3,,y,,,,

.q,4;.4 Azzzzz 2,

a

.p (g

5 i

e a, 9

Qr

\\1 O,3],

[(t:%f fl4, O<

IMAGE EVALUATION

//o//

/

$'7)/,gy)W N'y

[h#

9[:I9 TEST TARGET (MT-3)

+

l.0 f,"' n" n!b-n u

l..

3 s zis l}

I l.8

\\ can 1.25 1.4 1.6

.~.Z 17.

OC' _.7 4.-

150mm 4

..__..--6"

/ hf+

V Dy

/4'% /

$ y IN fgy.;f,,,%

//k o

. _o

., s _ >.

<pqw

,_ sg 6

,A "6,,3 9

f. fd'

,) 's &

0 a

t$'

i py 1

dp g

6 N. O oh

/////j

,Dta IMAGE EV Al.U ATION

~

f

'4

\\//g/

yp,@[q;g

% /

She k

TEST TARGET (MT-3)

$ >4

/g Rh*

,.o

'_ " " p@2 2 L

-=-

.n i..

l,l

,u=

le=

R 1.25 1.4 1.6 mm

- 1 5 0 m m -

4 4

6" s

ei?

,3,ll h

.g_+

. o, 9

J

n ES-601 Evaluation Checklist for Facility Form ES-601-2 Reference Material The following checklist represents the minimum content of facility-generated reference material.

Items marked " optional" should be checked if requested from the facility licensee by the chief examiner. The chief examiner or designee may use this checklist to make a quick, general evaluation of the completeness and adequacy of the' facility licensee's references.

The chief examiner may resolve any specific questions about the references with the facility staff as necessary.

I.

Quantity Actual Reference Material Reauired Minimum Submitted A.

Open-reference written 350 per section; Bank is to examination items be dynamic, revising, reviewing or generating at least 150 questions per year.

B.

Simulator scenarios 25; + 5 per year following the initial requal. exam. until at least 30 scenarios covering all I

aspects of the E0Ps are developed C.

Job Performance 95; + 10 per year following the

/7)

Maasures initial requal. exam. until the (U

JTA is fully covered D.

Technical Sp'ecifications 1 copy E.

Applicable plant procedures I set (optional)

F.

Emergency Plan 1 copy

'G.

Applicable administrative 1 copy (optional) procedures H.

Sample Plan 1 copy I.

Requalification Cycle 1 set (optional)

Training Reference Material (lesson plans, handouts, etc.)

J.

Appropriate sections of JTA 1 set (optional) or facility-specific K/A Catalogue Reviewed by:

Date:

f l,

Examiner Standards 33 of 41 Rev. 7, January 1993

ES-601 Form ES-601-2 II.

Usability Circle one A.

The Reference Material is legible yes no 8.

Reference Material is arranged and properly labeled for its function yes no C.

The Reference Material shows indication of a SAT program yes no D.

Reference Material is available to verify that test items are appropriate, job relevant, and technically accurate.

yes no E.

Reference Material is available to adequately support the examination topics yes no Comments O

Reviewed by:

Date:

O Examiner Standards 34 of 41 Rev. 7, January 1993

/

\\

ES-601 3

Form ES-601-2 III. Quality Exam Sections Reauired Standards Comments A.

Sample Plar, Subjects covered in requalification cycle are identified The test outline incorporates the following:

time spent on topic relative importance frequency of performance job level-(R0 or SRO)

K/As (or facility equivalent) of sufficient importance are identified in the Test Outline Plant-specific priorities are identified LERs, procedure changes, system modi ications, etc.)

Appropriate testing methods are indicated for each K/A (i.e., JPM, written exam, and/or simulator)

Applicable learning objectives are associated with K/As Methodology exists to tie test items to a learning objective and a K/A Sample plan includes important topict not covered in the requalification cycle Test areas appropriate to R0s and SR0s only are identified Reviewed by:

Date:

OO i

Examiner Standards 35 of 41 Rev. - January 1993

ES-601 4

Form ES-601-2 III.

Quality (cont.)

Exam Sections Reauired Standards Comments B.

Written At least 10% of all test items shall be reviewed using Form ES-602-1 Test items are important to safety Test items are clearly written Test items are appropriate to license level The criteria for open reference examinations are met Test items are associated with K/As of 3 or greater and are adequate discriminators A learning objective and applicable reference are identified for each test item The facility has identified SRO-level questions for both sections of the exam If the above criteria are not adequately met, the NRC will conduct further review of the examination bank utilizing ES-602, Attachment 2, " Guidelines for the Development and Review of Open Reference Examinations," and Form ES-602-1, "NRC Checklist for Open Reference Test Items."

Reviewed by:

Date:

O Examiner Standards 36 of 41 Rev. 7, January 1993

D ES-601 5

Form ES-601-2 l

III. Quality (cont.)

Exam Sections Baqpired Standards Comments C.

Wal k-through At least 10% of the JPM bank reviewed, using Form ES-603-1 Applicable plant systems identified by test outline:

systems covered in requalification cycle new or recently modified systems systems in recent facility LERs or vendor notices PRA-identified risk dominant systems systems in NRC generic communications Tasks / abilities for identified systems are--

applicable to the facility at the A0/R0/SR0 level have a K/A value of 3 or greater JPMs pertinent to SR0s only exist Some JPMs are performed under low power cr shutdown operating conditions Facility JPMs contain the information l

found in Form ES-603-2 Reviewed by:

Date:

O Examiner Standards 37 of 41 Rev. 7, January 1993

ES-601 6

Form ES-601-2 III.

Quality (cont.)

Exam Sections Reouired Standards Comments D.

Simulator At least 10% of the scenarios reviewed, using Form ES-604-1 Scenarios are an appropriate measure of the material covered in the sample plan Scenarios are based on--

lessons covered in requalification cycle recent industry events l Eits emergency and abnormal procedures design and procedural changes Scenarios exercise crew's ability to use facility procedures in accident prevent on and mitigation Scenario events have a K/A of 3 or greater Some scenarios are based on low powert operations Scenario identifies critical tasks that meet the criteria of ES-604, Attachment 1 Proposed examination scenarios that were used for training during the most recent training cycle have been reviewed by the NRC and replaced or modified, if appropriate, to ensure the validity of the examination and to minimize the potential for examination compromise

' Draft NUREG-1449, "NRC Staff Evaluation of Shutdown and Low Power Operation," defines low power to include the range from reactor criticality to 5% power.

Reviewed by:

Date:

O Examiner Standards 38 of 41 Rev. 7, January 1993 l

l l

L--_--_-------_--_-------------_------------------_---------------------.-------------------------

3-1 oS 06 R

/

/

/

/

/

/

/

/

/

/

/

/

/

/

E W

d %

m e

l

/

i r

a #

o F

F s

la i

.w t

in d

/

/

/

/

/

/

/

/

/

/

/

/

/

/

I r

n

/)

o I

F ta l

r u

o m

d %

P i

w e

S s

(

/

e

/

/

/

/

/

/

/

/

/

/

/

/

/

/

s s

r a #

P h

C use T

R EE ne H

tt S

i

/

/

/

/

/

/

/

/

/

/

/

/

/

/

r W

Y la R

to #

A T

M

)

n B

M e

t U)

I.

lll ir t

W A

S y

(

ln S O Te Ls U U s

M l

S i J

a P

Ei c Rf f

f O

r s

h to r

w o

Nr s

o a

o t

e OF u

r a

r r

s e

r C

e C

C C

C C

C C

C C

C C

C C

C OTn-l o

r I

e p

e d

R A

R A

R A

R A

R A

R A

R A

R O

p g

a O

n r

N F

N F

N F

N F

N F

N F

N F

Ab l

G e

r i

a t

a g

Ct a r

t o

l r

a e

c i

a e

I m v

a n

e e

o p

P t

Fr O

o R

S T

O I f Ln I lll Ay c t

Ua e

v k

Qi c

5 r

o 5

EP D

R(

TNALP R

EWO P

g g

gg

s

e t

a ar

D y

F t

mC il ic a a x R F E N C

7 5 me@

$SS N<

". E a* w Ce" l

4-1 04 R

O S

/

/

/

/

/

/

/

/

/

/

/

/

/

/

/

/

/

/

/

/

E W

mro F

s la it v

i i

n d

/

/

/

/

/

/

/

/

/

/

/

/

/

/

/

/

/

/

/

/

I r

n

/)

o I

F th r

u o

h P

(

s w

T e

/

/

/

/

/

/

/

/

/

/

/

/

/

/

/

/

/

/

/

/

E s

r h

C E

us H

e S

R N

h O

/

/

/

/

/

/

/

/

/

/

/

/

/

/

/

/

/

/

/

/

IT W

AUN I

T n

B

)

N )y t

e t

Ol ir n

CO W

A

(

e Ss T U Ll a i

s Ui M

c f

Sf P

EO J

Ror F

N-f On r

o o

e C

C C

C C

C C

C C

C C

C C

C C

C C

C C

C I

i da R

A R

A R

A R

A R

A R

A R

A R

A R

A R

A Tt 9

x N

F N

F N

F N

F N

F N

F N

F N

F N

F N

F r

Ar G

r Co e

f I n g

FI a

I y P

Lac Ai v r

t UP e

k Q(

c 5

o 5

E D

R TNALP RE WO P

eesnec iL m'

9 msEo$ m$3a$a uO o" u" xO-o$C$w W3

i Fonn ES-601-5 U.S. NUCLEA.R REGULATORY COMMISSION

/"j INDIVIDUAL REQUALIFICATION EXAMINATION REPORT y{

(Privacy Information - For Official Use Only)

Facility:

Licensee's Name:

Docket No: 55-License No:

Expiration Date:

Exam Type: RO / SRO Retake:

1st / 2nd / No Date of Last Exam:

- Written Examination Results Date(s) of Exam:

NRC Examiner (Print):

Facility Examiner (Print):

(

NRC Grading Facility Grading t

Section A (Points) of of Section B (Points) of of Overall Score (%)

Simulator Examination'Results Date(s) of Exam:

NRC Examiner (s) (~ Print):

Facility Examiner (s) (Print):

OV Crew Evaluation Pass / Fail Pass / Fail Individual Follow-up Pass / Fail / NA Pass / Fail / NA Walk-through Examination Results Date(s) of Exam:

NRC Examiner (s) (Print):

Facility Examiner (s) (Print):

No. of Successful JPMs of 5 of 5 Exam Results (%)

NRC EXAMINER RECOMMENDATIONS Category Results Signature /Date Written Pass / Fail Simulator Pass / Fail Walk-through Pass / Fail SECTION CHIEF REVIEW C(

Date:

Pass / Fail Signature:

Examiner Standards 41 of 41 Rev. 7 January 1993

O ES-602 REQUALIFICATION WRITTEN EXAMINATION A.

PURPOSE The NRC staff conducts written requalification examinations, using this standard in accordance with Title 10 of the Code of Federal Regulations (10 CFR) Part 55.59(a)(2)(iii). The NRC examiners are to follow this standard to prepare and administer all NRC-conducted written requalification examinations. l They are to use this standard in conjunction with ES-601, which tells the NRC staff how to conduct requalification program evaluations.

B.

SCOPE The written examination is useful for evaluating licensed operator (hereafter

" licensee") knowledges, skills, and abilities that are difficult to infer from behavior alone but that can be readily tested in written responses to questions that value interpretation and allow the examinee to use references.

Additionally, determining an individual's knowledge of factual information and ability to perform " paper and pencil" tasks are best evaluated by a written examination.

The written examination consists of two sections for which the examinee may n \\ refer to references (i.e., an "open-reference examination").

Section A, i

" Plant and Control Systems," is administered using a static simulator.

O Section B, " Administrative Controls / Procedural Limits," may be administered in a classroom.

Each section should be designed to last a minimum of I hour, l

including time for the licensee to review his or her work.

Combined, the two sections of the written examination will be designed to last 3 hours3.472222e-5 days <br />8.333333e-4 hours <br />4.960317e-6 weeks <br />1.1415e-6 months <br />. The exact number of questions and time allowed to complete each section will be determined by the facility licensee and will be based on the requalification sample plan and the license level (R0 or SRO) of the operators taking the examination.

Even though the examination is designed so that the examinee may use references, an examinee should not expect to have time to complete the examination by consulting references to determine each answer. A good mix of test items will contain some questions that evaluate a licensee's ability to determine a correct response without delving into reference material and others that require the use of reference material to select the correct response.

By combining test items that require using references with those that do not, the written examination can test a broader sample of licensee knowledge within a given period.

On both sections of the written examination, certain questions will test the knowledges and abilities (K/A) of a reactor operator (RO) while others will test those of a senior reactor operator (SRO).

During the development of the examination, consult the facility job task analysis (JTA) and NUREG-1122 or

-1123 to help identify the most suitable topics for an R0 and an SR0 O

Examiner Standards 1 of 33 Rev. 7, Sup. 1, June 1994

=

ES-602 respectively. Additionally,10 CFR 55.41 and 55.43 give additional guidance on item selection for R0 and SR0 written examinations, respectively.

1.

Section A. " Plant and Control Systems" (Static Simulator)

This section of the written examination is designed for using the simulater as a reference tool to provide realistic information visually and to cive the licensees an environment as close as possible to their normal control room. While administering this section, the simulator will be frozen in the middle of an evolution, transient or accident.

In developing the test items for this section, allow the use of references and relate them to plant systems and components, control room indications, instruments and controls, and recognizing technical H

specification (TS) limiting conditions for operation (LCOs).

Section A is designed to evaluate the licensee's knowledge of plant systems, integrated plant operations, and instruments and controls.

In addition, it evaluates the licensee's ability to recognize TS LCOs and to determine the effects of postulated events.

The NRC encourages facility licensees to include questions that test the ability of the licensees to use their facility curves and charts.

While administering Section A, the examination team will use one

" frozen' simulator condition or set-up.

The condition places the simulator in a " snapshot" of the plant following a major transient that resulted in an engineered safeguard feature (ESF) initiation or in a steady state situation at power.

Some equipment should be frozen in an abnormal or failed condition to provide adequate material for test items.

2.

Section B. " Administrative Controls / Procedural Limits" Section B of the written examination is designed to evaluate the ability of the licensee to analyze a given set of conditions and determine the proper procedural and administrative guidance to use.

It may include questions relating to theory, appropriate to sample the topics listed in 10 CFR 55.41 and 10 CFR 55.43, as long as they are operational in nature or test unique facility characteristics.

Section B is designed to test the licensee's knowledge and use of plant procedures and administrative controls and allows use of references.

The NRC uses administrative, operating, normal, abnormal, emergency procedures, TS, and the Emergency Plan as sources of test items for this section of the examination. The test items focus on how direction, guidance, and information found in these procedures is used or inter-preted rather than merely the process of finding the procedure in which the necessary information is located. Additionally, have the test items l

for Section B of the SR0 examination examine a licensee's understanding of the reasons and bases for procedural requirements. The use of 9

Examiner Standards 2 of 33 Rev. 7, January 1993

A ES-602 graphs, charts, tables, and drawings is appropriate.

The simulator may be made available to the examinees to make the examination more l

operationally oriented.

j C.

EXAMINATION DEVELOPMENT 1.

Facility Examination Team Members' Responsibilities a.

The facility is expected to prcvide a bank of test items developed 4

by using Attachment 2, " Guidelines for the Development and Review of Open Reference Examinations," and the appropriate sections from NUREG/BR-0122, " Examiner's Handbook tor Developing Operator Licensing Written Examinations," for developing test items and assigning point values.

The number of test items should meet the submittal guidelines of ES-601, Attachment 1, Enclosure 1, "Requalification Examination Reference Material Requirements."

Form ES-601-2, " Evaluation Checklist for Facility Reference Material," provides information that facility personnel may use to evaluate reference material sets before submitting them to the NRC.

Once a facility question bank meets the minimum number of questions identified in ES-601 (>50% of the entire bank in the objective format) and consists of questions that meet the format Ot guidance of NUREG/BR-0122, the facility may release the bank for review.

b.

The following items should be provided for each test question:

applicable K/A reference and values (R0/SRO) reference JTA (if applicable) estimated time to answer appropriate learning objectives applicable reference (e.g., lesson plan, E0P) c.

The facility is expected to provide a sample plan that meets the guidelines of ES-601, Attachment 2, " Examination Sample Plan," and may submit a proposed examination that conforms to the facility sample plan.

The proposed examination should contain a total of 30 - 40 test items, depending on the time validation (maximum time of 3 hours3.472222e-5 days <br />8.333333e-4 hours <br />4.960317e-6 weeks <br />1.1415e-6 months <br />) of the individual questions selected.

Section A should contain 15 - 20 questions and Section B should contain 15 -

20 questions.

Each section must be designed to last a minimum of one hour, with the total examination designed to last 3 hours3.472222e-5 days <br />8.333333e-4 hours <br />4.960317e-6 weeks <br />1.1415e-6 months <br />.

)

l The facility licensee will determine the number of questions in each section, basing the decision on the requalification sample plan and the license level of the operators taking the examination (R0 or SRO), and subject to the quantitative constraints of the Examiner Standards 3 of 33 Rev. 7, Sup. 1, June 1994

ES-602 previous paragraph.

Plant systems questions not directly related to the static scenario can be included in Sections A to meet the facility's sample plan and 10 CFR 55 requirements.

In addition, up to 20 percent of the test items may be from topics outside the sample plan, as long as the information stated in Section C.1.b.

of this standard is provided.

If the facility licens :e submits a proposed examination, those individuals involved in its development become subject to the security restrictions of ES-601 once examination development commences.

These restrictions remain in effect until the NRC examination is given.

If, after developing a proposed examina-tion, the facility decides not to submit it for use in the NRC-conducted examination, the developers are released from the l

security restrictions of ES-601.

d.

After the NRC has reviewed the facility's examination bank and commented on the test items selected for the examination, the facility team members are expected to prepare the examination for final NRC review and approval.

The examination may be finalized before or during the preparation week.

e.

The facility team representative will evaluate each test item that the NRC revised, for the--

appropriateness time required to answer, given the operational context of the item technical accuracy clarity K/A and objective references Following this evaluation, the facility examination team representatives and the chief examiner need to agree on the final form of the examination.

They also need to complete a time validation of the proposed examination.

A variety of methodologies have proven effective in accomplishing this task; Attachment 2, Paragraph B.2.g, provides further information in this area. Any individual involved in time validating the examination is required to sign the security agree-ment, Form ES-601-1.

The examination team may add or delete items from the examination based on the results of this time validation, and their experiences.

If any test items are added, the entire examination need not be time validated again as long as a subject matter expert (SME) has reviewed the added questions, indicating the approximate time a licensee should take to answer each question.

O Examiner Standards 4 of 33 Rev. 7, Sup. 1, June 1994

OU ES-602 f.

The facility licensee is expected to provide enough copies of each reference so that each examinee can use the references during the examination and, immediately upon completion of the examination, compile the examinations and reproduce sufficient copies for their use and use by the NRC.

g.

To help relieve the burden of 3roviding a complete set of references to each licensee, tie examination may be assembled so that a different sequence of questions appears on each licensee's examination, or, handouts of relevant information (e.g., plant curves, blank forms, etc.) may be provided with the test.

h.

The facility licensee is responsible for maintaining an updated examination question bank.

The license should review, modify, or create at least 150 questions each year to expand the bank, reflect procedure or system changes, new lesson plans, and recent licensee and industry events.

2.

NRC Examination Team Members' Resoonsibilities a.

The NRC will begin its evaluation of the sample plan, the bank of test items, and the proposed examination as soon as possible after receipt of the facility's material.

NRC will promptly evaluate

/~%

the material to allow sufficient time for the NRC or the facility to develop the test items and the facility to revise them to meet NRC standards, if required.

The NRC examiners should review the proposed test items using Form ES-602-1, "NRC Checklist for Open-Reference Test Items," to ensure appropriateness, clarity, and importance to safety, as described in Attachment 2, " Guidelines for the Development and Review of Open Reference Examinations."

If the facility licensee intends to administer both sections of the examination during a single 3-hour period as noted in Section D.I.c, the examination team members must review the examination as a whole to ensure that the items in either section do not compromise those in the other.

b.

A minimum of 80% of the test items will be chosen in accordance with the sample plan. The remaining 20% may be substituted by the examination team, using facility examination bank questions or new cuestions the exam team develops. Should it be necessary to covelop additional items to satisfy the sample plan, the NRC will request the facility to do so.

c.

If, after reviewing at least 75% of the bank, insufficient test items exist to develop an NRC examination that meets the sample plan, the NRC staff will declare the bank of test items inade-quate.

In that event, the regional managers may either cancel the scheduled examination or administer an examination, using NRC-p developed test items without consideration for the 20%

substitution constraints.

g Examiner Standards 5 of 33 Rev. 7, Sup. 1, June 1994

ES-602 d.

If the sample plan does not include testing of topics from outside of the requalification cycle, then the examination team should consider incorporating 10 to 20% non-requalification cycle specific test items.

e.

If a test item does not have a clear tie to the JTA, the examiner will discuss the applicability of the test item with the facility representatives.

D.

EXAMINATION ADMINISTRATION AND EVALUATIONS 1.

Written Examination Conduct a.

An NRC examiner or a knowledgeable facility representative who has signed the security agreement will proctor each section of the examination. As a minimum, an NRC examiner will observe the examination briefing as the licensees begin the examination to ensure that all administrative aspects of the examination are adhered to.

If an NRC examiner does not continuously proctor the examination, an examiner will periodically visit the examination room to ensure that the proctor appropriately addresses questions on the content or administration of the examination that may have arisen.

b.

Section A is administered on the facility's simulator or an approved simulation facility.

c.

Section B may be administered in the simulator or in a classroom setting as the facility staff and the chief examiner deem appropriate.

If both sections of the examination are administered in the simulator during a single 3-hour period, licensees may return to a section of the examination that they already completed or retain both sections of the examination until the time has expired.

d.

For Section A of the examination, the facility licensee is responsible for providing the group of examinees at least one copy of all controlled reference material available in the control room.

Examination reference material will NOT include material that is intended for training use only.

The licensee controls all reference material in accordance with the licensee's 10 CFR Part 50, Appendix B procedure revision control program.

This material should be authorized for use in operating the power plant, agreed upon by the facility and the chief examiner, and in effect at the time of the examination validation (i.e., the preparation week).

O Examiner Standards 6 of 33 Rev. 7, Sup. 1, June 1994

ES-602 V

e.

During the administration of Section B, each examinee will have available for use the following material (complete, current issue):

technical specifications plant procedures (E0P/A0P/0P, etc.)

emergency plan (as available in the control room) administrative procedures applicable to operations other controlled plant reference material normally available in the control room (e.g., curves and data book, forms, plant drawings, flow charts, etc.) and authorized for use in operating the plant NOTE:

"Non-controlled" reference material, such as the Emergency Procedure Owner's Group Basis Documents will nqt be provided unless these documents are authorized to be used by the control room operators during plant operations.

2.

Examination Administration Procedures The written examinations will begin only after the chief examiner has verified the adequacy of the examination facilities and made Q

arrangements for continuous proctoring of the examination as discussed in Section D.l.a of this standard. An NRC examiner may act as proctor during this examination.

However, the chief examiner is responsible for ensuring the actions of D.2.b through D.2.1 below are complete.

Each section of the written examination will be administered as follows:

a.

An NRC examiner will verify each applicant's identity and examination level against the examination assignment sheet (Form ES-201-2).

If possible, the R0 and SR0 applicants should be seated at alternate tables. Any errors or no shows will be resolved with the facility staff and the assignment sheet will be updated as required.

b.

The proctor will remind the licensees that they may use calculators to complete the examination and that no other reference material other than that provided is allowed in the examination area. The proctor will define the examination area for the applicants, c.

The proctor will pass out the examinations, answer sheets, and all handouts (e.g., steam tables and selected technical specifications) and instruct the applicants not to turn over the examination until told to do so. The applicants will be informed that pads of scrap paper are available upon request from the proctor.

NJ Examiner Standards 7 of 33 Rev. 7, January 1993

r ES-602

(

e.

During the administration of Section B, each examinee will have available for use the following material (complete, current issue):

technical specifications plant procedures (EOP/A0P/0P, etc.)

emergency plan (as available in the control room) administrative procedures applicable to operations other controlled plant reference material normally available in the control room (e.g., curves and data book, forms, plant drawings, flow charts, etc.) and authorized for use in operating the plant NOTE:

"Non-controlled" reference material, such as the Emergency Procedure Owner's Group Basis Documents will ngi be provided unless these documents are authorized to be used by the control room operators during plant operations.

2.

Examination Administration Procedures The written examinations will begin only after the chief examiner has verified the adequacy of the examination facilities and made Q

arrangements for continuous proctoring of the examination as discussed V

in Section D.1.a of this standard. An NRC examiner may act as proctor during this examination.

However, the chief examiner is responsible for ensuring the actions of D.2.b through D.2.1 below are complete.

Each section of the written examination will be administered as follows:

a.

An NRC examiner will verify each applicant's identity and examination level against the examination assignment sheet (Form ES-201-2).

If possible, the R0 and SR0 applicants should be seated at alternate tables. Any errors or no shows will be resolved with the facility staff and the assignment sheet will be updated as required.

b.

The proctor will remind the licensees that they may use calculators to complete the examination and that no other reference material other than that provided is allowed in the examination area.

The proctor will define the examination area for the applicants.

c.

The proctor will pass out the examinations, answer sheets, and all handouts (e.g., steam tables and selected technical specifications) and instruct the applicants not to turn over the examination until told to do so.

The applicants will be informed that pads of scrap paper are available upon request from the p

proctor.

Examiner Standards 7 of 33 Rev. 7 January 1993

ES-602 d.

'Following Attachment 1, " Policies and Guidelines for Taking NRC Written Examinations," the proctor will brief the applicants on i

the rules and guidelines in effect during the written examination and inform them that they may refer to the instructions directly i

beneath their examination cover sheet.

The proctor will read the first 2 policies verbatim.

The proctor will ask the applicants to verify the completeness of e.

their copies by checking each page of the examination.

The proctor should also have the applicants check to ensure that an equation sheet has been included in their examination, if required.

f.

After answering any questions that the applicants may have about the examination policies, the proctor will start the examination and record the time.

g.

The proctor will periodically advise the applicants of the time that remains to complete the examination.

Normally, a chalkboard is available for this purpose.

h.

As the applicants complete the examination, the proctor will ensure that they sign the examination cover sheet and staple it on top of their answer sheets. The

)roctor will collect the examination packages, including tie questions and answer sheets, any references used with the examination, and all scrap and unused paper.

The NRC examiner will keep the cover and answer sheets, dispose of the scrap paper, and give the packages of questions to the facility licensee for subsequent use.

1.

The proctor will remind the applicants to leave the examination area, as defined by the examination team.

3.

Written Examination Evaluations Using the examination and answer key, the facility and NRC will independently grade each section of the written examination and will complete the grading of all written examinations within 10 working days of the examination administration date.

NRC examiners will record the grades on the written examination cover sheet (Form ES-602-2) and complete Form ES-403-1, " Examination Grading Quality Assurance Checkoff Sheet," in accordance with the instructions of paragraph D of ES-403.

The contract manager signature is not applicable.

l An individual's grade will be obtained by summing the points credited to the examinee on both sections of the examination, and dividing by the total points available (i.e., compensatory grading methodology.)

To pass the written portion of the examination, licensees must achieve an overall score of 80% on the written examination.

Examiner Standards 8 of 33 Rev. 7, Sup. 1, June 1994

l 4.

Test Item E algali3D If a number of test items require significant modification during the grading of the examination, (e.g., deleted or answer changed from original key), or if greater than 10% of the test items require significant modification, the NRC will determine the root cause and reflect it in the examination report. As designated in ES-601, if significant deficiencies exist in the facility's quality control of their examination bank, the NRC will consider them as part of the program evaluation.

If technical flaws that have some degree of safety significance are found in procedures while analyzing the answers to the written examination, the facility may institute an immediate procedural change and inform all licensees of the change.

ATTACHMENTS / FORMS:,

" Policies and Guidelines for Taking NRC Written Examinations"

" Guidelines for the Development and Review of Open-Reference Examinations" Form ES-602-1, "NRC Checklist for Open Reference Test Items" O

Form ES-602-2,

" Written Examination Cover Sheet" O

Examiner Standards 9 of 33 Rev. 7, Sup. 1, June 1994 j

ES-602 Policies and Guidelines for Taking NRC Written Examinations 1.

Cheating on the examination will result in an automatic denial of your application and could result in more severe penalties.

2 '.

After you complete the examination, you must sign the statement on the cover sheet indicating that the work is your own and that you have not received or given assistance in completing the examination.

3.

To pass the examination, you must achieve a grade of 80 percent or greater.

4.

The point value for each question is indicated in parentheses after the question number.

5.

You have a total of 3 hours3.472222e-5 days <br />8.333333e-4 hours <br />4.960317e-6 weeks <br />1.1415e-6 months <br /> to complete both sections of the examination.

If both sections of the examination are administered in the simulator during a single 3-hour period, you may return to a section of the examination that was already completed or retain both sections of the examination until the allotted time has expired.

6.

Use only black ink or dark pencil to ensure legible copies.

7.

Print your name in the blank provided in the upper right-hand corner of the examination cover sheet and of each answer sheet.

8.

Mark your answers on the answer sheets provided. USE ONLY THE PAPER PROVIDED AND 00 NOT WRITE ON THE BACK SIDE OF THE PAGES.

9.

Use abbreviations only if they are commonly used in facility literature.

Avoid using symbols such "<" (less than) or ">" (greater than) to avoid a simple transposition error resulting in an incorrect answer. Write it out.

10.

Show all calculations, methods, or assumptions used to obtain an answer to any short-answer questions.

11.

You may receive partial credit on questions that are not in multiple-choice format.

Therefore, ANSWER ALL PARTS OF THE QUESTION AND DO NOT LEAVE ANY ANSWER BLANK.

12.

If the intent of a question is unclear, ask questions of the examiner only.

13.

Limit restroom trips, if possible; only one applicant at a time will be allowed to leave. Avoid all contact with anyone outside the examination room to eliminate even the appearance or possibility of cheating.

O Examiner Standards 10 of 33 Rev. 7, Sup. 1, June 1994

ES-602 20 Table 3 DEVELOPING OPEN-REFERENCE TEST ITEMS Here are the decision steps and mental model for developing analysis-level open-reference questions:

1.

Determine test purpose. Do you want to test knowledge where and knowledge tvhat?

2.

Determine the information needed to respond to the question.

Is the volume and kind of information such that you would not normally expect candidates to recall the information from memory to answer the question?

3.

If the answer is yes to both of these questions, develop an open-reference question.

4.

Construct the question as two tiers:

Tier \\ Purpose Process Criteria Outcome Evaluate preference Avoid clues Locate ref.

O

1. Knowledge where sources in stem sources

2. Knowledge Integrate multiple Info. volume Identify what\\how variables, events and detail high correct (not in memory).

answer QUESTION STEM bits, chunks of stem information (conditions, set points, components, etc.)

i 1

Mental Processes Intearative Processes

• Analyzing a.

Answer

• Integrate mental

• Sorting b.

Distractor processes with

• Eliminating c.

Distractor stem information,

• Differentiating d.

Distractor reference data,

• Evaluating and distractors Determine Answer V

Examiner Standards 31 of 33 Rev. 7, January 1993 I

O ES-602 NRC Checklist for Form ES-602-1 Open-Reference Test Items Test Item level i

1.

Does each test item have a documented link to important licensee tasks, K/As, and/or facility learning objectives?

j 2.

Is each question operationally oriented (i.e., is there a correlation between job demands and test demands)?

3.

Is the question at least at the comprehension-level of knowledge?

4.

Is the context of the questions realistic and free of window dressing and backwards logic?

5.

Does the item require an appropriate use of references (i.e., use of analysis skills or synthesis of information either to discern what procedures were applicable or to consult the procedures to obtain the answer)?

6.

Is the question a " direct look-up" question, or does one question on the examination compromise another? A " direct look-up question" is defined as a question that immediately directs an licensee to a particular reference where the answer is readily available.

7.

Does the greater) question possess a high K/A importance factor (3 or for the job position?

8.

Does the question discriminate a competent licensee from one who is not?

9.

Is the cuestion appropriate for the written examination and the selected written examination format (e.g., short answer; multiple choice)?

10.

Do questions in Section A take advantage of the simulator control room setting?

11.

Does any question have the potential of being a " double-jeopardy" question?

12.

Is the question clear, precise, and easy to read and understand?

13.

Is there only one correct answer to the question?

14.

Does the question pose situations and problems other than those presented during training?

15.

Does the question have a reasonable estimated response time?

Examiner Standards 32 of 33 Rev. 7, Sup. 1, June 1994

(D O

ES-604 simulator examinations are administered.

The NRC chief examiner is the principal point of contact between the facility manager and the NRC. The chief examiner or a designee is present during the administration of each dynamic simulator examination, b.

The examination team briefs the operating crews before the start of the simulator scenarios, using the information in the " Dynamic Simulator Briefing Checklist," ES-604, Attachment 2.

Crews should be given adequate timo to respond to all planned and c.

unplanned events. A scenario's contact time should be approximately 45 to 60 minutes.

Contact time means the actual time the licensees spend in the scenario but does not include time spent on briefings, simulator setup, or investigating simulator performance problems.

d.

Under no circumstance will sequence of events and transients be modified by any member of the examination team during the scenario.

If the scenario is not properly administered due to a simulator operator error or an unexpected simulator response, the examination team will confer immediately after the scenario set to determine if the crew has performed a sufficient number of transients and events to justify an evaluation of the required competencies.

If necessary, the examination team can run an iV additional scenario to ensure that the required competencies are covered.

Crew rotation practices shall be discussed and agreed to during e.

the preparation week and any problems resolved before the administration of the operating test.

f.

The members of the operating crew should maintain the same operating positions as during facility requalification evaluations. The crew members should rotate between positions in the manner identical to the facility's rotation practices for evaluations specified in the facility's requalification program.

g.

Senior reactor operators (SR0s) must be evaluated in at least one scenario in an SRO-licensed crew position. More than two simulator scenarios may be required to examine crews that consist of more than four SR0s.

2.

Post-Scenario Activities If the NRC examiners and facility evaluators observe actions that a.

are unclear during the simulator scenario, they should question the crew members as necessary to obtain complete documentation on the performance of events during the scenario.

Questions should be factual and should clarify performance related to observations.

n (v)

Examiner Standards 5 of 50 Rev. 7, Sup. 1, June 1994

ES-604 b.

If an examiner observes an individual who demonstrates significant deficiencies performing a critical task, the NRC exr 'Mr and the facility evaluator will discuss those deficiencies.it the i

completion of the scenario.

If they determine that the licensee's performance deficiency could not be assessed due to a lack of i

information, the examination team has the option to conduct an additional scenario or JPM to obtain this information.

During the post scenario discussion, the facility evaluator is expected to describe the licensee's deficiencies to the NRC examiner and suggest a series of follow-up cuestions designed to identify the cause of the deficiency. The bRC examiner will assess the facility evaluator's ability to diagnose the licensee's deficiency and document it in the examination report, if applicable. The NRC examiner has the option to augment the follow-up questions proposed by the facility evaluator, if necessary.

The examination team should minimize the time it takes to conduct this review of crew and individual performance to minimize the impact on the licensees. However, it is the examination team's responsibility to ensure the review is thorough and complete.

The facility evaluator will conduct an individual evaluation of the licensee in accordance with Section E.2 of this standard. The NRC examiner has the option to ask additional follow-up questions, c.

Upon completing any follow-up questioning, NRC examiners and the facility evaluators dismiss the crew to await the next scenario and inform the crew that they may discuss the completed scenario among themselves, d.

The NRC examiners and facility evaluators meet separately to compare observations and determine if any cts were omitted or incorrectly performed by the crew.

The facility evaluators discuss the crew's performance with the e.

NRC examiners after each scenario to clarify any performance deficiencies that have been noted.

The examination team determines if the as-run scenario has invalidated any predesignated cts or if any new cts should be designated for evaluating unpredicted events or actions taken by the crew during the scenario. The examination team revalidates the cts in each scenario, using the methodology contained in Attachment 1,

" Critical Task Methodology."

f.

After the crew completes the last scenario, the NRC examiners and the facility evaluators independently complete a " Simulator Crew Evaluation Form," Form ES-604-2, as discussed in Section E.

The facility evaluators also evaluate individual licensee performance Examiner Standards 6 of 50 Rev. 7, January 1993

ES-604 in accordance with their requalification program requirements.

The NRC examiners review the facility's evaluations of individual operator performance af ter completing each crew evaluation.

E.

PERFORMANCE EVALUATIONS Two separate evaluations are conducted based on the information obtained during the dynamic simulator examination.

First, a crew simulator evaluation is performed. Next, individual simulator performance is used by the examination team to determine whether follow-up questioning of the licensee is necessary. The examination team may conclude that, after observing the licensee's performance in the dynamic simulator and evaluating the response to follow-up questions, additional performance information about the licensee must be obtained to make an individual evaluation.

In this case, an additional scenario or JPH would be conducted.

The Individual Follow-up would then be documented along with the individual's crew evaluation on Form ES-601-5.

The 1;censee is subject to failure based on a competency evaluation of the licensee's performance on the dynamic simulator and the required follow-up evaluation, if deficient performance was exhibited by the licensee executing a crew critical task.

O 1.

Crew Simulator Evaluations After administering the dynamic simulator scenario set as discussed in Section D, the NRC examiners and facility evaluators inde3endently evaluate the crew's performance by completing a copy of tie " Simulator Crew Evaluation," Form ES-604-2. The facility is expected to provide its final crew evaluations to the NRC examiners before the crew members return to licensed duties or the end of the examination week, whichever is sooner.

Specific guidance for completing Form ES-604-2 is on page 1 of Form ES-604-2.

The results of the crew evaluations are factored into each individual's examination results and the facility requalification program evaluation.

Members of a crew that receive an unsatisfactory crew evaluation are expected to receive remedial training from the facility licensee and be reevaluated in accordance with the facility licensee's NRC-approved requalification program before returning to licensed duties. Although licensees are no longer required to take an NRC-administered requalification examination for purposes of license renewal, those that fail to pass (individually or as a member of a crew) an examination conducted by the NRC must be reevaluated by the NRC before their license will be renewed. The level of NRC involvement during the reevaluation i

will be determined on a case-by-case basis (refer to Section F.1 of ES-601).

NRC examiners will document the results of each licensee's crew p) performance on Form ES-601-5 in the Crew Simulator Examination section.

gv Examiner Standards 7 of 50 Rev. 7, Sup. 1, June 1994

1 O1 ES-604 2.

Individual Operatina Evaluations l

Individual operating evaluations on the dynamic simulator examination and the resulting remedial training are primarily the responsibility of the facility licensee.

Unsatisfactory licensee performance of a crew l

critical task will be followed up after the simulator scenario and documented on Form ES-601-5.

The facility evaluators are expected to document and grade individual operator performance during the dynamic simulator examination in accordance with the requirements of the facility licensee's requalification program.

The NRC expects that the facility's grading methodology will identify licensee deficiencies and that the facility evaluators will discuss those deficiencies with the NRC examiners during the meetings following the scenarios as described in Section D.

The facility evaluators are expected to document deficiencies and remediate and retest the licensees for the identified deficiencies in accordance with the facility licensee's recualification training program.

The facility evaluators are expectec to, at a minimum, identify any operator on the crew who was directly responsible for the omission or incorrect performance of validated cts.

Individual follow-up is conducted if a licensee has significant performance deficiencies linked to a CT.

The NRC examiner will assist in the development and administration of follow-up questions specific to the deficiencies displayed by the licensee performing the CT as described in Section D.2.b of this standard.

The examination team will determine the number and scose of the follow-up questions that will be asked based on a review of tie licensee's deficiencies at the completion of the scenario.

The examination team has the option to gather additional information about a licensee who displays performance deficiencies attempting critical tasks by either running an additional scenario or by using JPMs, if the dynamic simulator examination and follow-up questioning are inconclusive.

Upon completion of the individual follow-up questions the NRC examiner will complete the evaluation using the appropriate competency grading worksheet from ES-303 (ES-303-3 for reactor operators (R0s) and ES-303-4 for SR0s), or a facility's equivalent form It is only necessary to use those competency grading worksheets that apply to the knowledge or ability deficiency being evaluated.

Normally, to determine if an individual passes or fails, the examiner uses the weighted average of all the rating factors on the form.

Here, only those com)etencies that deal with the licensee's performance deficiencies should

)e filled out, and should not be weighted. If the NRC examiner gives the licensee a rating factor score of "1" in either of the following cases, the individual fails this portion of the examination --

O Examiner Standards 8 of 50 Rev. 7, January 1993

{1

\\

ES-605 l

License Maintenance, Processing License Renewal Applications, and Requests for Administrative Reviews and Hearings A.

PURPOSE The NRC staff should use the procedures herein for license maintenance activities, the processing of license renewal applications, requests for administrative reviews and hearings by licensed operators (hereafter

" licensees") in connection with failures of NRC-conducted requalification l

examinations, and dentals of applications for license renewal.

B.

BACKGROUND The renewal license application differs in some respects from the initial l

license.

To address these differences, the staff is establishing the procedures herein for processing licensee's renewal applications and requests for administrative reviews and hearings for the denial of license renewal.

I C.

PROCEDURE FOR RESPONDING TO THE RESULTS OF NRC-CONDUCTED REQUALIFICATION EXAMINATIONS 1.

Passina an NRC-Conducted Reoualification Examination

[

If a licensee passes all portions of the requalification examination that he i

or she was administered, including being a member of a crew that passes the dynamic simulator examination, the regional office will so inform the licensee using Attachment 1, "Requalification Examination Pass Letter."

2.

Failina an NRC-Conducted Reaualiffcation Examination a.

If a licensee fails any part of an NRC-conducted requalification examination, the facility licensee is expected to remove the licensee from licensed duty and take corrective action consistent with the provisions of its requalification program before returning the licensee to licensed duty.

If the facility licensee's requalification program is unsatisfactory, refer to Section F.2 of ES-601 for a list of other recommended actions to be taken, including those actions the facility licensee is expected to complete before attaining a " provisionally satisfactory" requalification program status, b.

The regional office will inform the licensee of the results of the requalification examination using Attachment 2, "Requalification Examination Failure Letter," or Attachment 3, "Requalification Examination Second failure Letter," as appropriate.

On receiving the failure notification, the licensee has 20 days in which to request an informal review of the portion of the examination that was failed. The licensee may submit the request for reconsideration to the Director, Division of Reactor Controls and V

Examiner Standards 1 of 13 Rev. 7, Sup. 1, June 1994

l ES-605 Human Factors (DRCH), U. S. Nuclear Regulatory Commission, Washington, D. C. 20555.

Note 6 provides additional information on the requirements for supporting documentation.

Although the regulation (10 CFR 55.57(b)(2)(iv)) that required c.

licensees to pass an NRC-administered requalification examination as a prerequisite for license renewal was deleted effective March l

11, 1994, the license of any operator who failed to pass any NRC-conducted requalification examination will not be renewed without some level of NRC involvement in the retesting process.

The amount of NRC involvement may include conducting the retest in l

accordance with the applicable Examiner Standard (s); inspecting the facility licensee in accordance with Inspection Procedure IP) 71001, " Licensed Operator Requalification Program Evaluation,"(as it retests the licensee; or reviewing an examination prepared by the facility licensee.

The regional office, in consultation with OLB, will determine the appropriate level of involvement on a case-by-case basis depending on the quality of the facility licensee's program. As long as the licensee submits a timely renewal application, the term of the license will continue until the renewal requirements are satisfied or the licensee fails three NRC-conducted examinations as discussed in Section C.2.e.

d.

The NRC will normally administer a second (first retake) examination approximately 6 months after issuing the first failure notification in accordance with Section C.2.b of this standard, and will concentrate on the areas in which the licensee exhibited deficiencies, The NRC will normally administer a third (second retake) e.

examination approximately 6 months after issuing the second failure notification in accordance with Section C.2.b of this standard.

The third examination will be a comorehensive requalification examination.

Regardless of the status of the facility licensee's requalification program, if a licensee fails a third requalification examination, the NRC will thoroughly review the licensee's examination performance and may conduct a complete review of the facility licensee's training program.

The third failure may be grounds for suspending or revoking the licensee's license.

If a licensee has an application pending for license renewal with the NRC at the time of a third requalification failure, that failure will provide the basis for denying the application. Notification of the licensee will be handled on a case-by-case basis and coordinated through OLB.

O Examiner Standards 2 of 13 Rev. 7, Sup. 1, June 1994

(")

ES-605 V

D.

MAINTAINING MEDICAL STANDARDS FOR LICENSEES 1.

Temocrary Disability If a licensee is temporarily unable to meet medical standards but is expected to meet those standards again in the future, the facility licensee may administratively classify that operator's license as " inactive" until the licensee is once again certified to meet all medical standards by the facility licensee.

The facility licensee need not notify the NRC nor request a conditional license for the temporary disability provided the licensee is administrative 1y prevented from performing licensed duties during the period of his or her temporary disability.

If the disability extends beyond the date of license expiration, the licensee may apply for timely license renewal in accordance with 10 CFR 55.55(a) and 10 CFR 55.57(a). The facility licensee should document the nature of the licensee's temporary disability on the medical certificate and submit a revised certificate to the NRC after the physician determines that the licensee meets the requirements of 10 CFR 55.33(a)(1).

The NRC will not renew the operator's license until it finds that all of the conditions specified in 10 CFR 55.57(b) are satisfied.

2.

Permanent Disability If the facility licensee determines that a licensee's medical condition is permanently disqualifying in accordance with Section 5.3 of ANSI /ANS 3.4-1983,

/

i

(

" Medical Certification and Monitoring of Personnel Requiring Operator Licenses for Nuclear Power Plants," the facility licensee shall notify the NRC within 30 days of learning of the diagnosis (see 10 CFR 50.74 and 55.25).

E.

PROCESSING LICENSE RENEWAL APPLICATIONS If the licensee wishes to renew the license, he or she must comply with the requirements of 10 CFR 55.57(a).

The licensee must submit NRC Forms 398 and 396 not less than 30 days before the date of expiration of the license.

In accordance with 10 CFR 55.55(b), if the licensee files a proper application for renewal at least 30 days before the date of expiration, the license shall not expire until the application for renewal has been denied or a new license has been issued.

If a licensee is waiting to be given a reexamination after failing an NRC-administered requalification examination, the licensee should still make timely application for license renewal under the provisions of 10 CFR 55.55(b).

The regional office may allow for transit time and accept a license renewal application received 25 days before the license expiration date, provided all signatures on NRC Forms 398 and 396 are dated before the 30-day timely renewal cutoff date. The submittal will not be considered timely if it is received less than 25 days before the date of license p

expiration unless positive evidence of receipt (e.g., postmark or docketing stamp) by the U.S. Postal Service or the NRC is available.

If i

Examiner Standards 3 of 13 Rev. 7, January 1993

ES-605 the application is received less than 25 days before the date of license expiration and too late for processing in the regional office, the license shall expire on the expiration date.

The regional office may issue a new license when it has completed processing the application.

The licensee will complete NRC Form 398, including the licensee's experience under the current license, the approximate number of hours the licensee spent on operating shifts, and the date and results of the applicant's most recent NRC-administered requalification examination (if taken).

The senior management representative on site shall provide evidence that the licensee has safely and competently discharged his or her license responsibilities and satisfactorily completed the facility's approved requalification program by :.igning Item 19.c on Form 398.

The facility licensee must certify on NRC Form 396 that a physician has performed a medical examination within the previous 2 years as required by 10 CFR 55.21 and submit that form along with NRC Form 398.

The regional office will review the application and issue the license renewal if it finds that the conditions in 10 CFR 55.57(b) are satisfied.

If the regional office determines that the renewal applicant does not meet the requirements of 10 CFR 55.57, it will inform the facility licensee of the deficiencies and request that supplemental information required to make a relicensing decision be forwarded by the licensee to the regional office within 20 days.

If, after evaluating the supplemental information, the regional office still concludes that the applicant does not meet the requirements for license renewal, it will issue a proposed denial letter to the licensee following the procedure in Section F of this standard.

l F.

PROCEDURE FOR DENYING AN APPLICATION FOR LICENSE RENEWAL If the licensee declines to supply additional information or the regional office concludes, after reviewing any additional information supplied by the licensee, that the application is still inadequate for license renewal, the regional office will deny the renewal application using Attachment 4, " Renewal Application Denial Letter (Region)."

Within 20 days of the date of the letter of notification of proposed denial, the licensee may exercise one of the following options:

1.

Submit a written request for the NRC to review the applicatiore.

Such a request should be sent to the Director, Division of Reactor Cantrols and Human Factors, U.S. Nuclear Regulatory Commission, Washington, D.C.

20555, and should include the reasons for the review request and supporting documentation as applicable. Note 6 provides additional information on the requirements for supporting documentation.

2.

Submit a written request for a hearing pursuant to 10 CFR 2.103(b)(2).

A hearing request is required to be submitted to the Secretary of the l

Examiner Standards 4 of 13 Rev. 7, Sup. 1, June 1994

/%

ES-605 Comission, U.S. Nuclear Regulatory Commission, Washington, D.C. 20555, with a copy to the Assistant General Counsel for Hearings, Office of the General Counsel, at the same address.

If the licensee exercises option (1) and the staff sustains its denial of the application, the Director, DRCH, will so inform the licensee using Attachment 5, " Renewal Application Denial Letter (DRCH)." The applicant has the option l

at this time to request a hearing pursuant to 10 CFR 2.103(b)(2).

G.

PROCEDURE FOR nVERTURNING REQUALIFICATION EXAMINATIONS OR RENEWAL DENIALS If, upon conducting a hearing or an informal review, the staff reverses its decision regarding the failure of a requalification examination or application denial, DRCH will take one or more of the following actions, as appropriate:

1 1.

reinstate the license; 2.

allow the licensee to renew the license pursuant to 10 CFR 55.57, if all other requirements are satisfied; or 3.

allow the licensee to perform licensed duties when he or she has successfully completed the facility's requalification program and has p

met the provisions of 10 CFR 55.53(e) or (f).

If, upon conducting a hearing or an informal review, the staff reverses its decision regarding the failure of the requalification examination, DRCH will t

inform the licensee that he or she has passed the examination using Attachment 6, " Sample Examination Notification (DRCH)."

If, upon conducting a hearing or an informal review, the staff reverses its l

denial of a licensee's renewal application and agrees that the application satisfies the license renewal requirements of 10 CFR 55.57 (assuming all other requirements which were not at issue in the hearing were satisfied), DRCH will inform the licensee using Attachment 7, " Sample Renewal Notification (DRCH)."

l H.

NOTES 1.

Letters informing a licensee of a proposed denial or examination failure must be signed by a branch chief or higher.

In the event of an appeal, a copy of DRCH's correspondence will be distributed to the appropriate regional office for tracking purposes.

2.

A copy of the correspondence sent to the licensee as a result of this process will be provided the facility licensee's representative authorized to sign the renewal application.

3.

The Director, DRCH, will keep regional office management informed of the status of the actions he or she takes during this process and will n

i acknowledge correspondence from the licensee.

(O Examiner Standards 5 of 13 Rev. 7, Sup.1, June 1994

ES-605 4.

All examination failures or denial correspondence sent to the licensee should be sent by certified mail with return recaipt requested.

5.

It is inappropriate to ask the facility licensee te reassess the need for a licensee's license while conducting an informal review or hearing.

6.

Requests for informal reviews by the NRC (a) list the items for which additional review is being requested and (b) include documentation supporting the contentions made by the licensee. The package containing the supporting documentation for the review and the review request must be mailed or delivered to the Director's office (DRCH) within 20 days of the date of the failure or denial notification.

The DRCH staff should complete the review within 45 days of receiving the package.

The staff will review requests using the guidance in ES-502.

ATTACHMENTS / FORMS:,

"Requalification Examination Pass Letter",

"Requalification Examination Failure Letter",

"Requalification Examination Second Failure Letter",

" Renewal Application Denial Letter (Region)",

" Renewal Application Denial Letter (DRCH)",

" Sample Examination Notification (DRCH)",

" Sample Renewal Notification (DRCH)"

t O

Examiner Standards 6 of 13 Rev. 7, Sup. 1, June 1994

ES-605 Requalification Examination Pass Lett,er NRC Letterhead (date)

Docket No. 55-(Number)

(Apolicant Name) (Street Address) (City. State Zio)

Dear (Namel:

I am writing you to inform you that you passed the requalification written examination and operating test administered by the U.S. Nuclear Regulatory Commission (NRC) on (date).

Enclosed is a copy of your Individual Requalification Examination Report (Form ES-601-5) summarizing the results of your examination.

Your facility training department has a copy of the master answer key.

If you have a question, please contact (Name) at (number).

Sincerely, i

(Regional Branch Chief or above) cc:

(Facility-au+horized reoresentative who sians NRC Form 398)

^~\\

(J Examiner Standards 7 of 13 Rev. 7, January 1993

ES-605 Requalification Examination Failure Letter NRC Letterhead Idate)

Docket No. 55-(Number)

(Acolicant Namstl (Street Address) (City. State Zio)

Dear (Nan,

e1:

I am writing to inform you that you did not achieve an acceptable score on the requalification (written examination and/or coeratina test) administered by the U.S. Nuclear Regulatory Commission (NRC) on (date).

Enclosed is a copy of the results indicating the area (s) in which you exhibited deficiencies.

Your facility training department has a copy of the master answer key.

This failure places you in the same status as if you had failed a facility-administered requalification examination. Therefore, you are subject to the requirements set forth in the NRC-approved requalification program for the facility for which you are licensed and must meet those requirements prior to resuming licensed duties.

The NRC will administer a second requalification examination to you in the areas in which you exhibited deficiencies.

If you believe an error was made in grading your examination, you may request within 20 days of the date of this letter that the NRC informally regrade the examination.

Requests for informal regrade should be sent to the Director, Division of Reactor Controls and Human Factors, U.S. Nuclear Regulatory Commission, Washington, D.C. 20555.

In the request, please state the items you wish to have reviewed and provide supporting documentation as applicable.

If you have any questions, please contact (name) at (number).

Sincerely, (Regional Branch Chief or above) cc:

(Facility authorized reoresentative who sians NRC Form 398)

CERTIFIED HAIL-RETURN RECEIPT REQUESTED O

Examiner Standards 8 of 13 Rev. 7, January 1993

l I

(3 ES-605 Requalification Examination Second Failure Letter NRC Letterhead (date)

Docket No. 55-(Number)

(Apolicant Name) (Street Address) (City. State Zio)

Dear (Name):

I This is to inform you that, baseit on the grading of the NRC-administered

)

requalification (written examination.and/or ooeratina test) taken on idate),

j you did not achieve an acceptable sco e.

Enclosed is a copy of the results indicating the area (s) in which you erhitited deficiencies.

Your facility training d2partment has a copy of the master answer key.

This failure places you in the same status as if you had failed a facility-administered requalification examination. Therefore, you are subject to the requirements set forth in the NRC-approved requalifit:ation program for the facility for which you are licensed and you must meet those requirements prior to resuming licensed duties. The NRC will administer a third requalification O

examination to you that will be comprehensive in scope.

V If you believe an error was made in grading your examination, you may request within 20 days of the date of this letter that the NRC informally regrade the examination.

Requests for informal regrade should be sent to the Director, Division of Reactor Controls and Human Factors, U.S. Nuclear Regulatory Commission, Washington, D.C. 20555.

In the request, please state the items you wish to have reviewed and provide supporting documentation as applicable.

If you have any questions, please contact (name) at (number).

Sincerely, (Regional Branch Chief or above) j cc:

(Facility authorized reoresentative who sians NRC Form 398)

CERTIFIED MAIL-RETURN RECEIPT REQUESTED h

G Examiner Standards 9 of 13 Rev. 7, January 1993 1

___________.--__o

ES-605 Renewal Application Denial Letter (Region)

NRC Letterhead (date)

Docket No. 55-(Number)

(Apolicant Name) (Street Address) (City. State Zio)

Dear (Name):

I am writli;g to inform you that your renewal application for a (senior.

reactor) operator license submitted in connection with the (facility name) rioes not demonstrate your eligibility for license renewal for the following ieason(s): (Reaional office to discuss deficiencies and aoolicable cart (s) of th2 Examiner Standards and 10 CFR Part 55 that acolv.)

If you do not accept this proposed denial, you may exercise one of the following options within 20 days of the date of this letter.

1.

Request reconsideration of the application denial by submitting a request to the Director, Division of Reactor Controls and Human Factors, U.S. Nuclear Regulatory Commission, Washington, D.C. 20555, stating the reasons for the request and providing supporting documentation as applicable.

If the proposed denial is upheld, you may then request a hearing pursuant to 10 CFR 2.103(b)(2).

2.

Request a hearing pursuant to 10 CFR 2.103(b)(2). To make this request, you must submit a written request for a hearing to the Secretary of the Commission, U.S. Nuclear Regulatory Commission, Washington, D.C. 20555, with a copy to the Assistant General Counsel for Hearings, Office of the General Counsel, at the same address.

If you have any questions, please contact (name) at (number).

Sincerely, (Regional Branch Chief or above) cc:

(Facility authorized reoresentative who sions NRC Form 398)

CERTIFIED MAIL-RETURN RECEIPT REQUESTED O

Examiner Standards 10 of 13 Rev. 7, January 1993

ES-605 Sample Examination Notification (DRCH)

NRC Letterhead (date)

Docket No. 55-(number)

(Licensee's name)

(Street address)

(City. State. Zio code)

Dear (Name):

In response to your letter of (date), we have reviewed the grading of the requalification (written examination. ooeratina test) administered to you on (date(s)) and reconsidered the requalification failure issued to you on (date).

In light of the additional information you supplied, we have determined that you passed the (written examination. ooeratina test) and satisfy the requirements of 10 CFR 55.57(b)(2)(iv).

[For your information, I am enclosing a copy of the staff's resolution of each of your hritten examination. operatina test) comments.]

If you have any questions, please contact (name) at (telephone number).

Sincerely, Director, Division of Reactor Controls and Human Factors Office of Nuclear Reactor Regulation

Enclosure:

As stated

[]

Include only if the regional office does not forward a discussion of the comment resolutions to the licensee.

O Examiner Standards 12 of 13 Rev. 7, January 1993

ES-605 Renewal Application Denial Letter (DRCH)

NRC Letterhead (date)

Docket No. 55-(number) feoclicant Namel (Street Address) (City. State Zio)

Dear (Name):

I am writing to inform you that the U.S. Nuclear Regulatory Commission (NRC) has reviewed the proposed denial of your renewal application for a (senior.

reactor) operator license.

Upon reviewing the information that has been submitted, the NRC has denied your renewal application.

(DRCH to discuss deficiencies and acolicable part(s) of the Examiner Standards. and 10 CFR Part 55 that acoly.) When you have met these requirements, you may submit another application to the appropriate regional office.

If you do not accept this denial, you may, within 20 days of the date of this letter, request a hearing pursuant to 10 CFR 2.103(b)(2).

To make this request, you must submit a written request for a hearing to the Secretary of O

with a copy to the Assistant General Counsel for Hearings, Office of the the Commission, U.S. Nuclear Regulatory Commission, Washington, D. C. 20555, General Counsel at the same address.

If you have any questions, please contact (name) at (number).

Sincerely, (Director, Division of Reactor Controls and Human Factors, Office of Nuclear Reactor Regulation, or above) cc:

(Facility authorized representative who sians NRC Form 398)

CERTIFIED MAIL-RETURN RECEIPT REQUESTED O

Examiner Standards 11 of 13 Rev. 7, January 1993 l

c

i 4

ES-605 Sample Renewal Notification (DRCH)

NRC Letterhead (date)_

Docket No. 55-inumber)

(Licensee's name)

(Street address)

(City. State. Zio Code) 4

Dear (Name):

j In response to your letter of (date), we have reviewed your application for license renewal and reconsidered the renewal application denial issued to you on (date).

In light of the additional information you supplied, we have determined that your application satisfies the license renewal requirements of 10 CFR 55.57(b).

[For your information, I am enclosing a copy of the staff's resolution of each of your comments.]

If you have any questions, please contact (name) at Ltelechone number).

Sincerely, Director, Division of Reactor Controls and Human Factors Office of Nuclear Reactor Regulation

Enclosure:

As stated

[]

Include only if the regional office does not forward discussion of comment resolutions to the licensee.

O Examiner Standards 13 of 13 Rev. 7, January 1993

NRC pomp 335 U.S. nucle AR REoVL AfoMY COMMISSION

1. REPO AT NUM8E R knc$ tic?.

N En am. m BIBLIOGRAPHIC DATA SHEET

<s,,numa,on.one,r." ">

NUREG-1021

2. TITLE ANo susTiTLE Rev. 7, Supp. 1 3.

DATE REPORT PUBLISHED MONTH YtAR Operator Licensing Examiner Standards June 1994

4. FIN OR GR ANT NUM8E R j

6. AUTHOR (S) 6 TYPE oF REPORT l

Staff Technical 7, PERIOD COVE R EO tins us,.e carni 8 PE R F O,d m.>.G.ORG..ANil AT ION - N AM E AN D ADDR E SS It' hnc. orovoor osviuon. onoce ar nooson. u s Nucune nesuistory comumuron. end mashne edorou: or rentra RMiN adr d Division of Reactor Controls and Human Factors Office of Nuclear Reactor Regulation U.S. Nuclear Regulatory Commission Washington, D.C. 20555-0001 9 SPONSORING OR G ANIZ ATION - N AM E AND ADDR E SS iso kac. tvoe ~Same n oaove". ir contractor. orovode Nnc onveson. orr,ce ar nossa. us Necuer noseserory commiuson, bad magiong addresd Same as above 10, SUPPLEMENTARY NOTES U

Sucolements Revision 7

11. A8STR ACT 1200.orce or uur The Operator Licensing Examiner Standards provide policy and guidance to NRC examiners and establish the procedures and practices for examining licensees and applicants for reactor operator and senior reactor operator licenses at power reactor facilities pursuant to Part 55 of Title 10 of the Code of Federal Reaulations (10 CFR 55). The Examiner Standards are intended to assist NRC examiners and facility licensees to better understand the initial and requalification examination processes and to ensure the equitable and consistent administration of examinations to all applicants. These standards are not a substitute for the operator licensing regulations and are subject to revision or other internal operator licensing policy changes.

Revision 7 was published in January 1993 and became effective in August 1993. Supplement 1 is being issued primarily to implement administrative changes to the requalification examination program resulting from the amendment to 10 CFR 55 that eliminated the requirement for every licensed operator to pass an NRC-conducted requalification examination as a condition for license renewal. The supplement does not substantially alter either the initial or requalification examination processes and will become effective 30 days after its publication is noticed in the Federal Reaistgr. The corporate notification letters issued after the effective date will provide facility licensees with at least 90 days notice that the examinations will be administered in accordance with the revised procedures, 12

• E v wORo$/oEScR:eToa s <ti,.ord. o,,ar.m ea,,

,u s.r,

.,ta.ri on i.rsne ra.,., ore. s

12. avait48,u r v stai eva s r Examiner Standards Unlimited

14. SECumil V CLAS$tf IGA rlON iThes Pett)

Unclassified im i ra

.,.o.<,,

r t

1 i f rirdoc cIfinr4 b

16.'NUmiW@ PAMS

16. PRICE NRC FORM 3M (2 8W

O i

O Printed on recycled paper O

Federal Recycling Program

..