ML21109A143

OLMC-510 Operator Licensing Examination Reports

ML21109A143
Person / Time
Issue date:	07/31/2021
From:	Travis Iskierka-Boggs NRC/NRR/DRO/IOLB
To:
Iskierka-Boggs T
References
OLMC-510
Download: ML21109A143 (9)

Text

OLMC-510 July 2021 1

OPERATOR LICENSING EXAMINATION REPORTS A.

PURPOSE The purpose of this Operator Licensing Manual Chapter (OLMC) is to establish an efficient and effective process for the creation of examination reports to support initial license examinations across all four regions. This process involves the establishment of an examination report after the initial operating tests and written examinations have been administered and graded using processes and procedures that are documented in NUREG-1021, Operator Licensing Examination Standards for Power Reactors. The intended users of this OLMC are NRC chief examiners, NRC regional management, or designees. The information contained in this manual chapter was previously included in NUREG-1021.

B.

BACKGROUND Examination reports are developed and provided to the licensee to document the examination process conducted at a facility. The examination report is to document results of the examination, and communicate significant issues identified during the examination in a consistent manner to the licensees. The examination report is to document the basis for significance determination and enforcement action, if any.

The goal of the Office of Nuclear Reactor Regulation (NRR) operator licensing program office (IOLB) is to issue the examination report within 45 days from the date of the exit meeting conducted with the facility, and to upload all required examination documentation into Agencywide Documents Access and Management System (ADAMS) within 90 days following delivery of examination report to the licensee facility.

C.

IMPLEMENTATION Prepare the Examination Report The NRC chief examiner, or designee, should prepare the final examination report when all portions of the written examination and operating tests have been graded and documented. Examination reports should be completed and issued to the facility within the 45 day requirement, and all required examination files uploaded to ADAMS, no later than 90 days following the date of issuance of examination report. If the regional office delays issuing operator licenses, it should issue the report clearly stating that, although the examinations were passed, the associated licenses will not be issued until the facility notifies the NRC in writing that all deferred requirements have been met. If this statement is used, it is not necessary to amend the examination report once the deferred licenses are issued. The report may be tailored to the specific examination that was administered (i.e., the report need not include any reference to an operating test if only a written examination

OLMC-510 July 2021 2

was administered, such as during a retake examination or an examination in which the applicant was granted an excusal from a portion of the examination).

a.

The examination report shall document the following:

The quality of the submitted written examination and operating test material in relation to the range of acceptability expected by the NRC.

NUREG-1021, Revision 12, ES 5.1, Section G, Determine Quality of Submitted Examination, contains the criteria for determining the quality of both the written examinations and operating tests. The regional office will also consider post-examination deletions and changes when evaluating the quality of the facility licensees proposed examination for documentation in the examination report.

If 20 percent or fewer of the test items for the submitted operating test, RO written examination, and SRO-only section (assessed separately) required replacement or significant modification, the report will simply state that the facility licensees submittal for each portion of the examination was within the range of acceptability expected for a proposed examination. As applicable, an observation shall be included indicating that the examination changes agreed upon between the NRC and the facility were made according to NUREG-1021.

If more than 20-percent of the submitted test items (with the operating test, RO and SRO written examinations assessed separately) required replacement or significant modification, the report shall include the number and description of the types of test items that needed to be replaced and/or significantly modified as a result of the review, and the flaw(s) that were significant contributors to the unsatisfactory submittal determination (i.e. 33 questions had to be replaced, 25 of them due to K/A mismatch, etc.). The report shall also note that the overall submittal was outside the acceptable quality range expected by the NRC and that future examination submittals should incorporate any lessons learned from this effort.

Note: As long as changes were incorporated to result in satisfactory test items, do not count any pre-submittal sample test items reviewed by the NRC before the draft licensee examination submittal.

Note: The nominal 20-percent threshold may be raised or lowered

+/- 5% with regional approval (i.e. one common error that creates a higher number of unsatisfactory items; overall LOD less than 2 even if the number of unsatisfactory is lower than 20%; excessive predictability or failure to sufficiently modify modified questions).

However, a comment may be warranted based on the egregious nature of the deficiencies even though the 20-percent threshold

OLMC-510 July 2021 3

was not reached. Examinations that contribute to a greater number of unsatisfactory items require IOLB concurrence for adjustment of the acceptability threshold.

Negative observations regarding the adequacy of the facility licensees proposed examination (e.g., stating that the proposed examination was not adequate for administration) may be left undocumented in the exam report if:

1.

The examination was the facility licensees lead examination writers first examination submitted at the facility;

2.

The facility has not had this allowance applied towards any of the last three examinations submitted to the NRC (including retake examinations if applicable to the portion of the examination that is evaluated as unsatisfactory); and

3.

IOLB has been informed.

Any delay in administering either the written examination or operating test and the reason for the delay. No more than 30 days elapsed between operating test administration and written examination administration.

Include any extensions of the written examination time beyond the nominal time limits specified in ES 2.1 and the reason for the extension.

The results of the written examination, including any significant grading deficiencies if the facility licensee graded the examinations.

The results of the operating test that documents the number of applicants that passed and the number that failed.

Generic weaknesses identified on the examination should be documented on the examination report.

Chief examiner shall review written examination results and facility licensees performance analysis (if applicable) for indications of deficiencies in training program, and poor question construction, so that applicants are not graded unfairly. Any significant problems with either should be addressed in the examination report.

Facility comments pertaining to the operating test and written examination properly evaluated and resolved with the resolution and justification documented. Master file should be updated accordingly.

OLMC-510 July 2021 4

An overview of the examination security measures and activities evaluated while preparing and administering the examinations and any examination security issues and incidents or other matters requiring facility attention.

Note: Initial examination security issues will generally be documented in the examination report if (1) the potential or actual compromise was discovered while developing or administering the examination and resulted in replacing or modifying any proposed test item(s), (2) the potential or actual compromise was discovered after the examination was administered, but it would have resulted in replacing or modifying test items if the NRC had known about it earlier, (3) two or more lesser security issues were discovered, but they did not necessitate the replacement of test material, or (4) other security issues were discovered with extenuating circumstances (with concurrence from IOLB).

General description documenting findings and violations associated with the examination as it pertains to NRC Inspection Manual Chapter 0611, Power Reactor Inspection Reports. Detailed information on the findings will be included in the appropriate inspection report.

Documentation demonstrating approximately 10% of license applications were audited.

Any other issues or findings discussed with the licensee facility staff at the exit meeting.

b.

The report shall include (or cite the ADAMS accession number of) the following examination files. When using ADAMS packages, reference the accession number(s) for the package(s) in the report and ensure that all necessary files are included into the package(s):

A copy of the final written examination(s) and answer key(s) with all changes, made during and after the examination, incorporated.

A copy of the final operating test(s) with references and answer key(s) to include all changes, made during and after examination, incorporated.

A copy of the facility licensees (and applicants) specific comments and recommended changes regarding the operating tests and written examination that were administered.

With respect to applicant comments, redact the applicants name and docket number from the examination report. The NRC regional office shall retain a nonredacted version, indicating applicant docket numbers,

OLMC-510 July 2021 5

until any adjudicatory hearings are complete (refer to OLMC-520, Operator Licensing Examination Records Documentation).

The specific NRC explanation for accepting or rejecting each facility licensee and applicant comment, and a specific justification for every additional item deletion or change (refer to Section C of ES-4.4 for instructions on NRC review of grading and examination changes).

A simulator fidelity report (as described below, when applicable).

Generic comments submitted by the facility licensee and/or applicant(s) about the examinations or the administration process should also be included in the report, accompanied by regional office responses, as appropriate.

c.

The simulator fidelity report shall document the NRC examiners evaluation of the performance or fidelity of the simulation facility during the preparation or conduct of the operating tests.

All previously undocumented simulator deficiencies encountered while preparing or conducting the operating tests should be described in sufficient detail to allow follow up the next time the NRC staff conducts Inspection Procedure 71111.11, Licensed Operator Requalification Program and Licensed Operator Performance at the facility. The NRC examiners may include in the simulator fidelity report any concerns about physical fidelity (hardware or equipment discrepancies) or functional fidelity (performance of the simulation facility during normal, surveillance, abnormal, or emergency events). Each deficiency should include a description of the operation, event, or transient that was in progress, and how the simulation facility failed to accurately model the expected performance of the reference plant.

d.

The applicants names and specific grades (i.e., Form 5.1-3, Power Plant Examination Results Summary) shall not be published in the examination report.

e.

The NRCs regional office shall send the final examination report to the facility licensee and ensure that a copy is made available to the public. Distribute the final examination report to the facility licensee using ListServ and send a carbon copy to the Facility Training Manager through e-mail.

f.

The NRC regional office shall ensure that the examination report is updated to reflect changes that occurred following an appeal or hearing. The examination report will need to incorporate changes such as question deletions, or acceptance of multiple answer choices in the form of an errata. The errata file should be uploaded into ADAMS as a new document no later than 90 days after

OLMC-510 July 2021 6

the appeal or hearing decision and should reference the original examination file Accession Number.

OLMC-510 July 2021 7

D.

Attachments

, Sample Facility/Applicant Comments and NRC Resolutions

, Sample Simulator Fidelity Report

OLMC-510 July 2021 1

Attachment 1 ATTACHMENT 1: Sample Facility/Applicant Comments and NRC Resolutions Question #28 Provide the question, answer, and reference.

Facility Comment: The question asks for the required method of securing a diesel generator and ensuring that an automatic restart does not recur following automatic initiation on receipt of a valid loss-of-coolant accident signal with offsite power still available to its associated emergency bus. The question is recommended for deletion because the system operating procedure directs that the diesel be unloaded, verifying that the 4-kilovolt (kV) bus automatic transfer annunciator is reset and then secured by placing the handswitch in pull to lock. Therefore, the key answer (i.e., ensure that the 4KV UTO TRANSFER INOP annunciator is lit before placing the control switch in PULL TO LOCK) is incorrect.

NRC Resolution: Recommendation accepted. The question is deleted because there is no correct answer. The intended answer specified that the annunciator be confirmed as lit when it should have specified reset in accordance with System Operating Procedure No. 123, Section 5.1 (Revision 29).

Question #51 Provide the question, answer, and reference.

Applicant Comment (include applicants docket number): The question asks for a description of the operation of the residual heat removal (RHR) Loop B outboard injection valve if the level rapidly decreases to 119.5 inches with RHR Loop B operating in the shutdown cooling mode. The question should be deleted because the outboard injection valve reopens automatically when the Group 4 isolation is reset if a low-pressure coolant injection loop selection is sealed in. Therefore, the key answer (i.e., the operator must reset the shutdown cooling isolation and manually reopen the RHR Loop B outboard injection valve) is incorrect.

Facility Position on Applicant Comment: The facility licensee concurs with the applicants comment.

NRC Resolution: Recommendation not accepted. The RHR Loop B outboard injection valve will not automatically open unless the operator manually resets the shutdown cooling isolation signal. Therefore, the use of the phrase manually reopen is correct, and the key answer is correct. The facility-provided reference supports the statement that manual action is required to open the injection valve.

OLMC-510 July 2021 1 Attchment 2 ATTACHMENT 2: Sample Simulator Fidelity Report Facility Licensee:

(Facility name)

Facility Docket No.:

(number)

Operating Tests Administered on:

(date)

This form is to be used only to report observations. These observations do not constitute audit or inspection findings and, without further verification and review in accordance with Inspection Procedure 71111.11, Licensed Operator Requalification Program and Licensed Operator Performance, are not indicative of noncompliance with 10 CFR 55.46, Simulation Facilities. No licensee action is required in response to these observations.

While conducting the simulator portion of the operating tests, examiners observed the following items:

(EXAMPLES)

Item Description HPSI Header B pressure (PI-301)

The pressure instrument read midscale regardless of actual pressure.

Include deficiency number if applicable.

Head bubble During a scenario that caused a rapid depressurization during natural circulation, the vessel head level indication indicated a void (bubble).

The confirming indications (i.e., pressurizer level and pressure) failed to verify or confirm the bubble. Include deficiency number, if applicable.

Steam Generator A wide-range level The meter has been out of service for the last three operating tests (approximately 18 months). Include deficiency number, if applicable.