ML18178A559

From kanterella
Revision as of 22:07, 26 January 2019 by StriderTol (talk | contribs) (Created page by program invented by StriderTol)
Jump to navigation Jump to search
IP 71111.11 Licensed Operator Requalification Program and Licensed Operator Performance
ML18178A559
Person / Time
Issue date: 12/14/2018
From: Tindell B T
NRC/NRR/DIRS/IOLB
To:
Tindell B
Shared Package
ML18177A399, ML18348A619 List:
References
CN 18-043, DC 18-016
Download: ML18178A559 (37)
v • d • e


Text

Issue Date: 12/14/18 1 71111.11 NRC INSPECTION MANUAL IOLB INSPECTION PROCEDURE 71111 ATTACHMENT 11 LICENSED OPERATOR REQUALIFICATION PROGRAM AND LICENSED OPERATOR PERFORMANCE Effective Date: January 1, 2019 PROGRAM APPLICABILITY: IMC 2515 A CORNERSTONES: Initiating Events (10 percent) Mitigating Systems (70 percent) Barrier Integrity (10 percent) Emergency Preparedness (10 percent) INSPECTION BASES: See IMC 0308 Attachment 2 SAMPLE REQUIREMENTS: Sample Requirements Minimum Baseline Completion Sample Requirements Budgeted Range Sample Type Section Frequency Sample Size Samples Hours Licensed Operator Control Room Observations 03.01 Annual* 4 per site 4 per site 16 Licensed Operator Training/Examination Observations 03.02 Annual* 4 per site 4 per site 16 Requalification Examination Results 03.03 Annual 1 per site training program** 1 per site 2 +/- 1 Licensed Operator Requalification Program 03.04 Biennial 1 per site training program** 1 per site 96 +/- 15 *It is preferred that 4 hours4.62963e-5 days <br />0.00111 hours <br />6.613757e-6 weeks <br />1.522e-6 months <br /> of control room observations and 4 hours4.62963e-5 days <br />0.00111 hours <br />6.613757e-6 weeks <br />1.522e-6 months <br /> of training observations are performed every calendar quarter. **Some sites have more than one licensed operator requalification training program due to different reactor technologies at the same site (e.g, Arkansas Nuclear One, Millstone). For each training program at a site, the examination shall be collected annually and a requalification program inspection shall be performed biennially.

Issue Date: 12/14/18 2 71111.11 71111.11-01 INSPECTION OBJECTIVES 01.01 To evaluate licensed operator1 performance during the conduct of requalification examinations, other examinations, training exercises, and in the actual plant/main control room. 01.02 operators during the conduct of requalification examinations, other examinations, and training exercises. 01.03 annual operating tests and biennial written examinations. 01.04 To evaluate testing and maintenance of the control room simulator. 01.05 To ensure that individuals who are licensed to operate the facility satisfy the conditions of their licenses as specified in 10 CFR 55.53 and 10 CFR 55.59, and to assess the . 71111.11-02 GENERAL GUIDANCE Quarterly Control Room and Training Observations Observe and assess licensed operator performance in the actual plant/main control room during periods of heightened activity or risk. Also, Observe the facility licensee ability to administer, and licensed operator performance during, requalification training. At the discretion of Regional Management, a regional operator licensing examiner may perform this observation. For each sample, conduct a routine review of problem identification and resolution activities Although the intent is to perform samples (control room observations and requalification training observations) each schedule and inspector judgment. For example, if a plant does not perform requalification training in a given quarter or has few risk-significant control room activities in a quarter, it may be advantageous to defer the samples. Annual Requalification Examination Results In accordance with 10 CFR 55.59(a), each licensed operator must pass a comprehensive (biennial) written examination and an annual operating test2. Following the completion of each of these examinations, a Regional operator licensing examiner familiar with the facility 1 reactor operators and licensed senior reactor operators. 2 The periodicity of these examinations has been interpreted by the NRC as: annual operating tests are to be administered at least once every calendar year, and the comprehensive written examination is to be administered at least once within each 24-month requalification program preferably near the end of each program.

Issue Date: 12/14/18 3 71111.11 collects the examination results. This review is expected to be conducted in-office. Biennial Licensed Operator Requalification Program Inspection Conduct a biennial review of: 1. licensed operator performance during requalification examinations, 2. the ability of the facility licensee to properly develop and administer requalification examinations, 3. the maintenance of individual operator licenses, 4. the performance of the control room simulator, and 5. the ability of the facility licensee to identify and resolve problems related to licensed operator performance. The inspection team will consist of (1) a team leader who is a qualified operator licensing certified inspector; and (2) one or more additional examiners/inspectors.3 The biennial review is expected to require a one-week on-site visit, plus additional in-office review. Although most of these activities could be conducted by a single lead inspector administration of the requalification annual operating test, utilize a minimum of two examiners/inspectors. As an efficiency measure, it is recommended that the NRC request that the facility licensee submit specific examinations and other information prior to the on-site portion of the biennial inspection, such that any issues identified can be discussed with the facility licensee while on-site. In accordance with 10 CFR 55.59(c), facility licensees are required to make these examination records available for NRC review, and a list of the typical documents reviewed during the biennial inspection is presented in Appendix A. The biennial inspection shall be scheduled and announced to each facility licensee. Typically, facility licensees divide up each 24-month requalification program into training cycles. During each requalification training cycle, training and/or examination activities are repeated on subsequent weeks for different groups or operating crews of licensed operators. Typically, facility licensees conduct an annual operating test for licensed operators during a training cycle near the mid-point of each 24-month requalification program. Similarly, near the end of each 24-month requalification program, the facility licensees conduct both an annual operating test and a biennial written examination. With such a schedule, it is preferred that the biennial inspection be performed near the end of each 24-month cycle when the facility licensee is administering both the annual requalification operating test and the requalification written examination. For facility licensees that do not employ this typical schedule, or for facilities at which the NRC has historically completed the biennial inspection in a different way, the overall requirements are for the region to complete the biennial inspection requirements such that (1) an in-progress annual operating test is observed and reviewed, and (2) the most recently administered biennial requalification written examination is reviewed, such that at least one version of the biennial requalification written examination is reviewed successive 24-month requalification programs during each biennial inspection. 3 It is the expectation that the inspection team leader be an examiner who is a fully qualified inspector. In rare cases the inspection team leader may be an examiner qualified as a basic certified inspector, as of the team, it is preferred Issue Date: 12/14/18 4 71111.11 Generally, only the inspection requirements of this procedure will need to be conducted. However, regional managers will consider overall facility performance, allegations related to licensed operator requalification, findings from this inspection procedure (IP), and any traditional enforcement actions taken as a result of this IP in determining whether additional activities will be performed. Additional activities include observation of additional groups or crews of licensed operators during an annual requalification operating test, the performance of IP 41-conducted licensed operator requalification examination, in accordance with 10 CFR 55.59(a)(2)(iii) and NUREG-1021, -600 sections. Additional activities should be considered when any of the following conditions exist: Observe Additional Crews. At the discretion of the NRC Region, perform this activity if the inspector observes a dynamic simulator scenario crew failure during the regularly scheduled inspection, or as a possible response to allegations associated with licensed operator performance or requalification training. Perform IP 41500. This activity should be performed if the NRC is concerned with the quality of licensed operator requalification training at a facility. Initiators for this activity include: (1) significant in-plant operator performance issues that have requalification training quality as a root cause, (2) a failure rate on a requalification examination of greater than or equal to 50 percent, (3) indications of a breakdown in the systems approach to training at the facility, or (4) as a possible response to allegations associated with licensed operator performance or requalification. Prior to initiating this activity, consult with the operator licensing program office Perform an NRC-conducted licensed operator requalification examination. This activity should examinations. Initiators for this activity include: (1) white findings in both written examination quality and operating test quality; (2) in response to an actual examination compromise as ns associated with licensed operator requalification examinations. Prior to initiating this activity, consult with the operator Administrator. 71111.11-03 INSPECTION SAMPLES 03.01 Licensed Operator Performance in the Actual Plant/Main Control Room Assess licensed operator performance in the actual plant/main control room during periods of heightened activity or risk. Specific Guidance 1. Prior to observing licensed operators, review the facility expectations, and policies regarding licensed operator performance, including:

Issue Date: 12/14/18 5 71111.11 Operator compliance and use of plant procedures, including procedure entry and exit, performing procedure steps in the proper sequence, procedure place-keeping, and technical specification entry and exit. Control board/in-plant component manipulations Communications between crew members Use and interpretation of plant instruments, indications, and alarms; diagnosis of plant conditions based on instruments, indications, and alarms Use of human error prevention techniques, such as pre-job briefs and peer checking Documentation of activities, including initials and sign-offs in procedures, control room logs, technical specification entry and exit, entry into and out of service logs Management and supervision of activities, including risk management and reactivity management Pre-job briefs in various licensee to determine which procedures are applicable at a particular plant. 2. Select an activity (or activities) to observe and review the plant procedures that will be used by the operators during the observed activity. Assess licensed operator performance during periods of heightened plant activity or plant risk where the activities could impact plant safety. Select actual plant/main control room activities to observe by reviewing plant activity and/or work schedules, attending daily briefs, shift turnover briefs, refueling outage planning meetings, maintenance planning meetings, and via discussions with the facility licensee. In particular, consider observing the following activities: Plant startups, shutdowns, and mode changes Reactor power and turbine load changes, especially when licensee reactivity management policies will be in effect Infrequent plant evolutions Unplanned transients and off-normal events, including post-scram response Surveillance testing Post-maintenance testing of safety-related structures, systems, and components Pre-startup equipment line-ups, operational checks, and functional checks Changes to the line-ups or modes of operation of safety related systems, structures, and components Refueling outage preparations, such as filling the reactor cavity or entering mid-loop operations Reactor refueling activities Issue Date: 12/14/18 6 71111.11 3. Observe licensed operators conducting the selected activity, using the generic checklists contained in Appendix H as guidance. Consider modifying these checklists policies. For efficiency, perform plant status reviews (IMC 2515 Appendix D), inspection of post maintenance testing (IP 71111.19), refueling and outage activities (IP 71111.20), and surveillance testing (IP 71111.22) can be performed in parallel with this IP. The activities to be observed involve licensed operators, typically observed from the main control room. Observe any pre-job briefs held prior to the activity. During the observation period, refrain from interfering with the performance of the licensed operators being observed unless interference is warranted due to a significant safety concern. Limit questions and discussions during plant activities to prevent unnecessary distractions to the licensed operators. 03.02 Licensed Operator Requalification Training/Examinations Observe licensed operator performance during requalification training/examinations, and the ability of the facility licensee to administer requalification training/examinations. Specific Guidance 1. Review training/examination activity to be observed. Facility licensees have training procedures that cover topics such as: the administration of annual operating tests, evaluated scenarios, and how training is to be delivered. In addition, facility licensees may have training observation forms that would be useful as a guide when observing training activities. 2. Roperator performance. See Section 03.01.1. 3. Select a licensed operator requalification training/examination activity to observe. If possible, observe an annual requalification operating test required by 10 CFR 55.59 that is administered to an operating crew. Scheduling this activity may be determined from the facility licensee and/or If an annual requalification operating test required by 10 CFR 55.59 will not be observed, consider the following when selecting an activity: (a) Although not required by 10 CFR 55.59, many facility licensees also routinely evaluate their licensed operators using crew-based simulator scenarios. If so, observe evaluated simulator scenario(s) administered to an operating crew. (b) If the facility licensee does not conduct evaluated scenarios, then observe simulator training for an operating crew.

Issue Date: 12/14/18 7 71111.11 (c) Other requalification training activities may also be observed (e.g. classroom training and in-plant training.) If training is to be observed, give priority to training conducted in response to licensee-identified corrective actions, plant modifications, or upcoming plant evolutions. 4. Observe the selected training/examination activity. If a portion of an annual requalification operating test required by 10 CFR 55.59 is observed, the basic methodology for performing this inspection activity is presented in Section 03.04.c of this IP, with a focus upon completing the applicable elements of Appendix D of this IP. Based upon the limited observation time of the resident staff and their variable experience with operator licensing, it is not expected that the resident staff will conduct in-depth reviews of operating test quality (Section 03.04.b) or examination security (Section 03.04.d). During this inspection activity and upon its conclusion, any annual requalification operating test issue should be discussed with the facility licensee, to assist in confirming the issue. However, the inspector should not interfere with the facility If there are any significant concerns with the annual requalification operating test, contact regional management. In evaluating any annual requalification operating test issues for operating tests required by 10 CFR 55.59, refer to the applicable appendix of this IP. If observing licensed operator evaluations which are not a part of an annual requalification operating test required by 10 CFR 55.59, the inspector should use a similar methodology. Without process, assess: Licensed operator performance The ability of the facility licensee to administer the evaluations The quality of any post-scenario critiques Follow-up actions taken by the facility licensee for any licensed operator who failed an evaluation (e.g., removal from shift duties, remediation, re-examination) Note that 10 CFR 55.59 requires only an annual requalification operating test. Facility licensees also evaluate licensed operators in accordance with their accredited systems approach to training program. The quality of these additional evaluations and how they are administered are determined by facility licensee. Unless contained in facility licensee procedures, the details contained in Appendices B, C, D, E, and F of this IP are not applicable to additional licensed operator evaluations. If observing licensed operator requalification training, assess the quality of the training. Detailed guidance may be available via facility licensee training observation forms.

Issue Date: 12/14/18 8 71111.11 5. Observe and evaluate simulator performance if the selected training/examination activity is conducted in the control room simulator. Review simulator physical modeling and simulator performance, especially regarding recent modifications implemented in the control room. Refer to Section 03.04.g of this IP for additional information regarding this inspection activity. Based upon the limited observation time of the resident staff and their variable experience with simulators, it is not expected that the resident staff will review any facility licensee simulator corrective action or testing records. During this inspection activity and upon its conclusion, discuss any simulator performance issue with the facility licensee to assist in confirming the issue. If there are any significant concerns with simulator performance, contact regional management. In evaluating any simulator performance issues, refer to Section 03.04.g and Appendix G of this IP. 03.03 Requalification Examination Results Collect and assess requalification examination results. Specific Guidance After the facility licensee has completed and graded the exams of any training cycle which contains a licensed operator requalification annual operating test or biennial written examination required by 10 CFR 55.59, contact the facility licensee and determine the following: 1. Determine the composition of the examinations administered (e.g. written examinations, simulator scenarios, job performance measures (JPMs)). 2. Collect the examination results to complete the following table: TABLE 1: EXAMINATION RESULTS 1. Total number of licensed operators. 2. Number of licensed operators administered a requalification examination required by 10 CFR 55.59(a). 3. Number of individual licensed operators who failed any portion of a requalification examination (written, JPM, or individual simulator scenario failures). 4. Divide line 3 by line 2 to obtain the individual requalification examination failure rate. Line 3/Line 2.  % 5. Number of crews administered simulator scenarios as part of a requalification examination required by 10 CFR 55.59(a). 6. Number of crews who performed unsatisfactorily on the simulator scenarios. 7. Divide line 6 by line 5 to obtain the crew simulator scenario failure rate. Line 6/Line 5.  % 3. If any individual licensed operator failed any portion of a requalification examination (line 3) or any crew failed the simulator scenarios (line 6), determine whether the licensee remediated and the operator or crew passed a re-Issue Date: 12/14/18 9 71111.11 examination. If the inspector identifies any concerns with re-examinations which could impact whether licensed operators fulfil a condition of their license, refer to Section 03.04.f of this IP. 4. If the failure rate (individual or crew) for any requalification examination required by 10 CFR 55.59(a)(2) exceeds 20% as identified above (line 4 or line 7), this shall be considered a performance deficiency against the expected knowledge and abilities of licensed operators. This performance deficiency is typically not considered a violation of regulatory requirements. 5. Obtain the total number of licensed operators at the site Operator Licensing Assistant (RPS OL Report 9). Discuss any differences in the total number of licensed operators at the site with the total number of licensed operators who took a requalification examination with the facility licensee. Discuss any licensed operator who did not take a complete examination and any plans the licensee has for making-up any missed examinations. Determine whether each licensed operator is taking the required requalification examinations as set forth in 10 CFR 55.59(a)(2). 6. Discuss when the next requalification examinations will be administered with the facility licensee. This will allow the region to plan for future inspections. 03.04 Licensed Operator Requalification Program a. Biennial Requalification Written Examinations Review the quality of licensee-developed biennial requalification written examinations. Specific Guidance Typically, facility licensees develop multiple similar versions4 of the biennial written examination, with different versions administered on subsequent weeks for different groups or operating crews of licensed operators. To complete this inspection activity, a minimum of one complete (operator and senior operator) version of the requalification biennial written examination required by 10 CFR 55.59(a)(2) shall be reviewed using the checklists and guidance provided in Appendix B. If greater than 20 percent of the written examination questions reviewed are determined to be flawed, then a second complete version of the written examination from the same training cycle shall be reviewed. If additional written examinations are reviewed for reasons other than the percentage of flawed questions, focus on new or modified questions and review overlap between examinations. 4 Multiple versions of requalification examinations are necessary to prevent any potential examination compromise due to the different dates on which the examination is administered. See the inspection guidance contained in Section 03.04.d Issue Date: 12/14/18 10 71111.11 The biennial requalification written examinations to be reviewed should be from the training cycle that the inspectors are on-site5. However, an alternate review schedule may be used such that the biennial requalification written examinations to be reviewed would be from an examination which occurred since the last NRC biennial review, such that this IP reviews at least one version of the biennial requalification written examination from each of th-month requalification programs. Upon the completion of this review, discuss any written examination quality issue with the facility licensee to assist in confirming the issue. However, the inspector should not interfere by suggesting modifications to test items or examination schedules. If there are significant concerns with the quality of the written examinations reviewed, contact regional management. b. Annual Requalification Operating Tests Review the quality of licensee-developed annual requalification operating tests. Specific Guidance Review the quality of a minimum of 10 JPMs and 4 simulator scenarios associated with an annual requalification operating test required by 10 CFR 55.59(a)(2), using the checklists and guidance provided in Appendix C. Review the JPMs and scenarios that will be observed during the on-site portion of this inspection. If more JPMs and scenarios are required to be reviewed, select those JPMs and scenarios from other weeks within the same training cycle. In addition, check for excessive test item repetition between operating tests administered during different weeks within a training cycle (see Section 03.04.d). Upon the completion of this review, discuss operating test quality issues with the facility licensee to assist in confirming the issue. However, the inspector should not interfere modifications to test items or examination schedules. If there are significant concerns with the quality of the operating tests reviewed, contact regional management. c. Administration of an Annual Requalification Operating Test Observe the administration of simulator scenarios and JPMs during the conduct of an annual requalification operating test required by 10 CFR 55.59(a)(2). Specific Guidance This inspection activity should be conducted during a one week on-site visit. However, the region may re-visit the site and observe the annual requalification operating test for a different group or crew of licensed operators. Observe simulator scenarios as a team and separately observe different licensed operators and facility evaluators during the conduct of JPMs. 5 When reviewing an exam yet to be administered, inspectors shall be careful to protect the examination from inadvertent disclosure to unauthorized facility personnel examination security procedures and policies.

Issue Date: 12/14/18 11 71111.11 The basic methodology for performing this inspection activity is to observe licensed operator and facility licensee evaluator and administrator performance during an annual requalification operating test. Additionally, observe post-simulator scenario critiques and other facility licensee operating test grading activities. Assess the following items: 1. Licensed operator performance, including: Crew performance in terms of clarity and formality of communication Ability to take timely action in the safe direction Prioritizing, interpreting, and verifying alarms Correct use and implementation of procedures, including the alarm response procedures Timely control board operation and manipulation, including high-risk operator actions Oversight and direction provided by the shift supervisor, including ability to identify and implement appropriate technical specifications actions such as reporting and emergency plan actions and notifications Group dynamics involved in crew performance 2. test (refer to Appendix D of this IP). 3. operators (refer to Appendix D of this IP). 4. The adequacy of plant procedures. 5. The quality of the annual requalification operating test scenario guides and JPMs (see Section 03.04.b). 6. Examination security (see Section 03.04.d). 7. Simulator performance (see Section 03.04.g). During this inspection activity and upon its conclusion, discuss any operating test administration issues with the facility licensee to assist in confirming the issue. examination process. administration of the operating test, contact regional management. d. Requalification Examination Security Evaluate requalification examination security measures.

Issue Date: 12/14/18 12 71111.11 Specific Guidance Examination security encompasses all practices taken by the facility licensee to ensure compliance with 10 CFR 55.49, which states, in part: Applicants, licensees, and facility licensees shall not engage in any activity that compromises the integrity of any application, test, or examination required by this part. The integrity of a test or examination is considered compromised if any activity, regardless of intent, affected, or, but for detection, would have affected the equitable and consistent administration of the test or examination. To perform this inspection activity, complete the checklist contained in Appendix E which includes: (1) checking for excessive test item repetition, (2) observing examination security practices during the administration of an annual operating test, and (3) reviewing examination security incidents. Discuss any excessive test item repetition with the facility licensee, who should have spreadsheets and/or test outlines which show the written examination questions, JPMs, and scenarios to be used during the entire requalification examination testing cycle. Discuss any requalification examination security incidents that have occurred since the last biennial inspection with the facility licensee. e. Remedial Training and Re-Examinations Verify the facility licensee properly implements remedial training and re-examinations. Specific Guidance Since the last biennial inspection, determine from the facility licensee if any individual licensed operators or crews failed any portion (written, JPM, or simulator scenario examination) of an NRC requalification examination required by 10 CFR 55.59(a)(2), and determine what type of failure(s) occurred - individual failures or crew failures; written, JPM or simulator scenario examination failures. For each examination failure, complete the checklist contained in Appendix F. Upon completion of this review, discuss any remedial training and re-examination issues with the facility licensee to assist in confirming the issue. If there are significant concerns . f. Operator License Conditions Review a sampling of individual licensed operator records, to verify that the facility licensee has effective processes for ensuring the conformance with operator license conditions. Specific Guidance Operator license conditions are contained in 10 CFR 55.53 and 10 CFR 55.59, and include requalification training attendance, maintaining an active license, and medical fitness. This inspection activity should be performed primarily during the on-site portion of this inspection. However, prior to the on-site review: (1) obtain a list of all the Issue Date: 12/14/18 13 71111.11 licensed operators at the site (RPS OL Report 9) and a list of all licensed operator medical restrictions, if any (RPS OL Report 14); (2) preselect licensed operators for review of their medical records, (refer to paragraph 4 below); and (3) review the individual docket files (available from the Operator Licensing Assistant). Additional guidance for performing this inspection activity is presented in NUREG--605. 1. For one complete operating crew of licensed operators6, review the following: (i) Records that indicate the participation of licensed operators in the facility CFR 55.53(h), 10 CFR 55.59(a)(1), 10 CFR 55.59(c)(5)(i)). Determine if all requalification training is completed on schedule or made up in accordance with the facility's program. (ii) Records that indicate the performance of licensed operators on annual requalification operating tests and biennial requalification written examinations (10 CFR 55.59(a)(2), 10 CFR 55.59(c)(5)(i)). (iii) Records that indicate that licensed operators are properly maintaining an active license (10 CFR 55.53(e)). Include a determination of which shift crew positions the facility licensee credits towards maintaining an active license. (iv) effect at the time of the inspection and any supporting documentation that 2. Determine from the facility licensee if any licensed operator with an inactive license reactivated their license since the last NRC biennial inspection. If license reactivation did occur, select one or more licensed operators who reactivated their license(s) and review records which indicate that their license(s) was (were) properly reactivated in accordance with 10 CFR 55.53(f). 3. Determine from the facility licensee if any licensed operator did not pass a biennial requalification written examination or annual requalification operating test required by 10 CFR 55.59(a)(2) since the last NRC biennial inspection. If examination failure did occur, select one or more licensed operators who failed an examination, and review records which indicate that the operator(s): (1) received remedial training, (2) was (were) re-examined, and (3) passed their re-examination(s). 4. Review a sample (approximately 10%) of licensed operator medical records, t system. Approximately 50% of this sample, should include licensed operators whose licenses contain medical restrictions. Determine which version of - 6 For multi-unit sites, review records for all licensed operators assigned to a shift for all of the units.

Issue Date: 12/14/18 14 71111.11 Certification and Monitoring of Personnel Requiring Operator Licenses for Nuclear Power Plants In performing this review, check that: (i) The required medical examinations are conducted biennially (10 CFR 55.53(i)). (ii) The results of medical examinations agree with any license medical restrictions (10 CFR 55.23(b)). (iii) that is in effect at the time of the inspection including any supporting medical documentation, is accurate and complete. (iv) Any medical status reports, if required by NRC Form 396, are submitted to the regional office in a timely fashion. During this inspection activity and upon its conclusion, discuss any issue associated with the conformance with operator license conditions with the facility licensee to assist in confirming the issue. If it is determined that any licensed operator has not properly conformed with the conditions of their license, discuss with the facility licensee: 1. Any immediate actions to take, such as removing the affected licensed operator(s) from on-shift licensed operator duties and notifying the regional office. 2. Any plans for restoring compliance with operator license conditions. g. Control Room Simulator Evaluate the performance of the control room simulator and review records which indicate that the facility licensee is properly testing and maintaining the control room simulator. Specific Guidance This inspection activity consists of the following elements: 1. Observe simulator modeling and performance during an in-progress annual operating test and note any simulator modeling or performance deficiencies. 2. Review a listing of open simulator deficiencies as maintained by the facility licensee in their simulator corrective action program. Sample the facility 3. Review a listing of simulator deficiencies closed by the facility licensee since the last biennial inspection. Sample the corrective actions taken by the facility licensee in closing simulator deficiencies.

Issue Date: 12/14/18 15 71111.11 4. Review a sample of records associated with any simulator testing performed by the facility licensee since the last biennial inspection. 5. Review a sample of records associated with any simulator modifications made by the facility licensee since the last biennial inspection. In performing this inspection activity, determine which version of Regulatory Guide -Plant Simulators for Use in Operator Trlicensee is committed to. Determine if the facility licensee is using the simulator to meet the control manipulation eligibility requirements contained in 10 CFR 55.31(a)(5). In performing this inspection activity, answer the following three questions: 1. Is simulator modeling and performance satisfactory? 2. Does the facility licensee effectively correct identified simulator deficiencies? 3. Does the facility licensee properly perform required simulator testing? In answering this question, check for correct simulator testing periodicity, proper evaluations of the simulator against the reference plant or best-estimate data, and correct documentation and retention of simulator test records. 10 CFR 55.46 provides additional details towards answering these questions, and 10 CFR 55.46 presents a checklist in Appendix G. Regulatory Guide 1.149 and ANSI/ANS-3.5 provide additional details. During this inspection activity and upon its conclusion, discuss any simulator performance issue with the facility licensee to assist in confirming the issue. If there are significant concerns regarding simulator performance, contact regional management.. In evaluating any simulator performance issues, refer to Appendix G, 10 CFR 55.46, the applicable versions of Regulatory Guide 1.149, and ANSI/ANS-3.5. 71111.11-04 REFERENCES https://www.nrc.gov/reading-rm/doc-collections/cfr/part055/ NUREG- https://www.nrc.gov/reading-rm/doc-collections/nuregs/staff/sr1021/ https://www.nrc.gov/reading-rm/doc-collections/insp-manual/manual-chapter/index.html https://www.nrc.gov/reading-rm/doc-collections/insp-manual/inspection-procedure/index.html Medical Assessment of Licensed Operators or Applicants for Operator Licenses at Nuclear Power Plants Issue Date: 12/14/18 16 71111.11 https://www.nrc.gov/reading-rm/doc-collections/reg-guides/power-reactors/rg/division-1/division-1-121.html ANSI/ANS-3.4, el Requiring Operator Licenses http://www.internal.nrc.gov/TICS/library/standards/ihs.html Nuclear Power Plant Simulation Facilities for Use in Operator Training and License Examinations https://www.nrc.gov/reading-rm/doc-collections/reg-guides/power-reactors/rg/division-1/division-1-141.html ANSI/ANS- http://www.internal.nrc.gov/TICS/library/standards/ihs.html END List of Appendices A. Typical Documents Reviewed During Biennial Inspection B. Biennial Requalification Written Examination Quality Checklist and Worksheet C. Annual Requalification Operating Test Quality Checklist and Worksheets D. Operating Test Administration Checklist E. Requalification Examination Security Checklist F. Remedial Training and Re-Examination Checklist G. Checklist for Evaluating Plant-Referenced Simulators Operating Under 10 CFR 55.46(c) and (d) H. Generic Licensed Operator Observation Checklists Attachment 1: Revision History Issue Date: 12/14/18 A-1 71111.11 APPENDIX A TYPICAL DOCUMENTS REVIEWED DURING BIENNIAL INSPECTION 1*. Biennial requalification written examination(s) administered during the training cycle. 2*. JPMs and simulator scenarios associated with an annual requalification operating test required by 10 CFR 55.59(a)(2). The JPMs and scenarios to be reviewed should include those that will be or have been observed during the onsite portion of this inspection, with any remaining JPMs and scenarios reviewed selected from other weeks within the same training cycle. 3*. Spreadsheets and/or test outlines which show the usage of written examination questions, JPMs, and simulator scenarios. 4*. A list (or lists) of licensee-identified issues associated with licensed operator errors or other licensed operator performance problems which have occurred in the actual plant/main control room since the last biennial requalification program inspection, their associated corrective actions, and if these issues were incorporated into requalification training. 5*. Facility licensee procedures for licensed operator requalification training and examination, including examination security procedures. 6*. -site inspection week. 7**. A list of all the licensed operators at the site, a list of all licensed operator medical restrictions at the site, and selected individual licensed operator 10 CFR 55 docket files. 8**. NRC records which document licensed operator performance issues and the facility m performance since the last biennial requalification inspection. These records include: NRC biennial PI&R team inspection reports, NRC Annual Performance reports, NRC inspection findings, Plant Issues Matrix, and NRC operating experience information. 9. Licensed operator pass/fail statistics for any NRC-required requalification examination that has been completed (see Table 03.03-1, Examination Results). 10. st biennial inspection (cycle weeks, training topics, etc.) 11. A list and descriptions of any examination security problems since the last biennial inspection (likely documented in training department condition/problem identification reports).

Issue Date: 12/14/18 A-2 71111.11 TYPICAL DOCUMENTS REVIEWED DURING BIENNIAL INSPECTION (Continued) 12. Records for licensed operator requalification training attendance, licensed operator performance in requalification, licensed operator remedial training, individual licensed operator medical records, and records for maintaining an active license and license re-activation. 13. Simulator testing, maintenance, modification, and performance records. Simulator corrective action records, including lists of open/closed simulator deficiencies, and corrective actions taken. *It is recommended that the NRC request that the facility licensee submit these items prior to the on-site portion of the biennial inspection, such that portions of the biennial inspection can be reviewed and discussed with the facility licensee while on-site, and for inspection planning purposes. **These items should be reviewed prior to the on-site portion of the biennial inspection.

Issue Date: 12/14/18 B-1 71111.11 APPENDIX B BIENNIAL REQUALIFICATION WRITTEN EXAMINATION QUALITY CHECKLIST Written examinations reviewed or dates administered: ____ ____ YES NO 1. Biennial requalification written examinations are administered approximately every 2 years, such that each 24-month requalification program contains at least one biennial requalification written examination (10 CFR 55.59(a)(2)). 2. Review the biennial requalification written examination questions from at least one complete written examination using the Requalification Written Examination Questions Review Worksheet (see page B-2), and determine how many questions had flaws, how many questions had no flaws, and determine the percent of questions that had flaws. Note: if a complete examination contains fewer than 30 questions, add an additional flaw based upon the number of questions that the exam has that is less than 30 (i.e., a 28 question exam would be assigned 2 flaws just for having 2 questions less than 30). Total # of Qs without flaws: Total # of Qs with flaws:  % of all Qs reviewed with flaws: 3. From line 2, less than or equal to 20% of the total number of reviewed written examination questions contained flaws. deficiency against the expected quality standards for a licensed operator requalification written examination required by 10 CFR 55.59(a)(2), and as a possible finding.

Issue Date: 12/14/18 B-2 71111.11 Requalification Written Examination Question Review Worksheet Written examination reviewed or date administered: ________ Q# 1. LOD (1-5) 2. Question Flaws 3. Q flawed or not flawed? (F/NF) 4. Explanation LOD Direct L/U Correct Answer Cues T/F Cred. Dist. Instructions: 1. Enter the level of difficulty (LOD) of each question using a 1-5 (easy difficult) rating scale. 2. Check the appropriate block if a question flaw is identified: The question is a direct look-up. For open-reference questions, if the question stem provides the reference title and/or number and the correct answer requires no understanding, application, analysis, or problem solving, but simply requires the ability to find the information in the reference, then the question is likely a direct look-up. See NUREG-1021, ES-602 Attachment 1 for additional guidance. The question has no correct answer or more than one correct answer. Short-answer questions must contain objective scoring, with clear guidance on granting partial and full credit. In addition, check the following items for all multiple choice questions (see NUREG-1021, Appendix B): The stem or answer choices contain cues as to the correct answer (i.e., clues, specific determiners, phrasing, length, etc.) The answer choices are a collection of unrelated true/false statements such that the question can be answered correctly without reading the question stem. Two or more distractors are not credible. For open-reference questions, if the question stem provides the reference title and/or number and a distractor can be easily eliminated because it is not located anywhere in the reference, then that distractor should be considered not credible. 3. Based on review item 2 above, the question as written is either flawed (F) or not flawed (NF) . 4. Provide a brief explanation for all questions that are determined to be flawed (F).

Issue Date: 12/14/18 B-3 71111.11 Requalification Written Examination Questions Review Worksheet (Continued) Written examination reviewed or date administered: ________ Q# 1. LOD (1-5) 2. Question Flaws 3. Q flawed or not flawed? (F/NF) 4. Explanation LOD Direct L/U Correct Answer Cues T/F Cred. Dist.

Issue Date: 12/14/18 C-1 71111.11 APPENDIX C ANNUAL REQUALIFICATION OPERATING TEST QUALITY CHECKLIST Operating test # or date(s) administered: _______ YES NO 1. An annual requalification operating test is administered at least once every calendar year (10 CFR 55.59(a)(2). 2. The operating test consists of at least five JPMs for each individual. 3. Review at least ten JPMs using the Requalification JPMs Review Worksheet (see page C-3), and determine how many JPMs had flaws, how many JPMs had no flaws, and the percent of JPMs with flaws. Total # of JPMs without flaws: Total # of JPMs with flaws:  % of all JPMs reviewed with flaws: 4. From line 3, less than or equal to 20% of the reviewed JPMs contained flaws. 5. The mix of JPMs per JPM set is appropriate: two simulator or control room JPMs, two in-plant JPMs, and one JPM may be of any kind simulator, in-plant, or administrative. 6. At least 40% (e.g., two out of five) of the JPMs per JPM set are alternate path. 7. The operating test contains an adequate number of simulator scenarios, such that each licensed operator is evaluated using at least two simulator scenarios. 8.a. Review at least four scenarios using the Simulator Scenario Review Worksheet (see page C-4), and determine for each scenario the number of events that each scenario is deficient by. For example, if a scenario contains the minimum number of events (or more than the minimums per page C-5) in all categories, then this figure would be zero for that scenario. However, if a scenario contained only three malfunctions, two flaws would be assigned, since it is less than the minimum number of malfunctions by two. If this same scenario also did not exercise technical specifications, that would count as one additional flaw. After all the scenarios have been reviewed, add up the number of deficient events and enter the total number of deficient events as flaws. Total # of scenario flaws from deficient # of events:

Issue Date: 12/14/18 C-2 71111.11 ANNUAL REQUALIFICATION OPERATING TEST QUALITY CHECKLIST (continued) deficiency against the expected quality standards for a licensed operator requalification operating test required by 10 CFR 55.59(a)(2), and as a possible finding. YES NO 8.b. Review each of the scenario events from the four (or more) scenarios selected in step 8.a above using the Simulator Scenario Review Worksheet (see page C-4), and identify how many scenario events had flaws. To the number of event flaws add in line 8.a for deficient number of events (i.e., one deficient event is treated as one event flaw) and record it. Then determine and record the number of scenario events that had no flaws. From these two numbers, determine the percent of scenario events with flaws. # of scenario events without flaws: # of scenario events with flaws (add in any deficit from line 8a):  % of scenario events reviewed with flaws: 9. From line 8.b, less than or equal to 20% of the reviewed scenario events contained flaws.

Issue Date: 12/14/18 C-3 71111.11 Requalification Job Performance Measure Review Worksheet Operating test # or date(s) administered: _______ Instructions: 1. Enter the type of JPM (S)imulator, (P)lant, or (O)ther. 2. Enter (Y)es or (N)o whether the JPM is alternate path or not. 3. Enter the level of difficulty (LOD) of each JPM using a 1-5 (easy difficult) rating scale. (LOD > 1 and < 5 are acceptable). Also see below. 4. Check the appropriate block if a JPM flaw is identified: Simple one-step JPMs, or a JPM that tests solely for recall or memorization, or a JPM which requires directly looking-up a single correct answer is likely LOD = 1 and too easy. Conversely, a JPM with over 30 steps or that takes in excess of 45 minutes to complete is likely LOD = 5 and too hard. The JPM lacks adequate initial conditions or lacks an adequate initiating cue. The JPM lacks adequate evaluator cues to allow the examinee to complete the task. The JPM lacks adequate performance standards and/or contains errors in designating critical steps. The JPM lacks an appropriate validation time or lacks a time for completion standard. 5. Based on the review of item 4 above, the JPM as written is either flawed (F) or not flawed (NF). 6. Provide a brief explanation for all JPMs that are determined to be flawed (F).JPM # or title 1. Type? (S/P/O) 2. Alt. Path? (Y/N) 3. LOD (1-5) 4. JPM Flaws 5. JPM flawed or not flawed? (F/NF) 6. Explanation LOD IC Cues Perf. Stds & crit.steps Time Limit Issue Date: 12/14/18 C-4 71111.11 Simulator Scenario Review Worksheet 1. Scenario ID (e.g., # or title) Week Used: Total Malf. Malf. After EOPs ABNs MTs EOPs beyond scram CTs TS 2. Minimum number of events 5 1 2 1 1 2 1 5. Total deficit: Explanation: 3. Actual number and type of events in the scenario 4. Deficit 6. Scenario Event ID/Name: 7. Scenario event flawed (F) or not flawed (NF)? 8. Explanation Performance Standards Critical Task Instructions: 1. Enter the scenario identifier and week within the training cycle that the scenario will be (or has been) administered. 2. Minimum number of events per scenario are listed. 3. Enter the actual number and types of events in the scenario. In order for a scenario event to be counted towards the listed minimums, the scenario event must contain verifiable operator action(s) and an associated performance standard(s) to mitigate or address the event. 4. Subtract each column in line 3 from line 2. Enter zero if the minimum number of events for that event type is equaled or exceeded. 5. Add up the columns from line 4, and determine the total number of events less than the minimums that the scenario has. Any deficit will be used in computing line 8.a of the Requalification Operating Test Quality Checklist (pages C-1, C-2), when totaled with the deficits from other scenarios. 6. Enter the scenario event name and description. 7. Review the individual events contained in each scenario, and evaluate for event flaws: The scenario guide event description lacks adequate crew/operator performance standards. The scenario guide event description incorrectly designates an event as a critical task (i.e., a non-critical task labeled as critical or a critical task labeled as non-critical). The number of scenario events that are flawed and not flawed will be used in computing line 8.b of the Requalification Operating Test Quality Checklist (page C-2), when totaled with the flawed/not flawed events from other scenarios. 8. Provide a brief description for any scenario event determined to be flawed (F). Abbreviations: Malf. = Malfunction EOPs = Emergency Operating Procedures ABNs = Abnormals MTs = Major Transients CTs = Critical Tasks TS = Technical Specifications Issue Date: 12/14/18 D-1 71111.11 APPENDIX D OPERATING TEST ADMINISTRATION CHECKLIST Date(s) operating test observed: _______ deficiency against the expected standards for the administration of a licensed operator requalification operating test required by 10 CFR 55.59(a)(2), and as a possible finding. 1 examinee errors. The licensee personnel who are administering the examinations, i.e., the simulator booth operator, examinee evaluators, and other licensee personnel who are assisting in administering the examination. YES NO 1. JPMs are administered with one facility evaluator per examinee. Exception: administrative JPMs can be administered in a group format. 2. There were no uncorrected examination administration errors observed during the performance of the JPMs and all of the JPMs were conducted as planned1. Examples of examination administration errors include improper simulator set-up, improper action taken by the simulator booth operator in response to an exexaminee. 3. There were no uncorrected examination administration errors observed during the performance of the simulator scenarios and all of the scenarios were conducted as planned1. Examples of examination administration errors include improper simulator set-up, improper action taken by the simulator cuing of one or more crew members. 4. Examinee performance errors during JPMs are detected by facility evaluators, such that you agree with all of the pass/fail determinations for JPMs that you observed. 5. Examinee performance errors during simulator scenarios are detected by facility evaluators, such that you agree with all of the pass/fail determinations for scenarios that you observed. When evaluating this item, the inspector should consider whether the facility licensee utilized an adequate number of evaluators for the given crew size.

Issue Date: 12/14/18 E-1 71111.11 APPENDIX E REQUALIFICATION EXAMINATION SECURITY CHECKLIST against the expected standards for examination integrity and as a possible finding, typically associated with a regulatory violation of 10 CFR 55.49. If the equitable and consistent administration of an examination was actually affected, traditional enforcement should also be considered. YES NO 1. Biennial requalification written examinations administered to different licensed operators during different weeks within a training cycle repeat < 50% of examination questions that have previously been administered during that same training cycle. 2. Biennial requalification written examinations differ from one 24-month requalification program to the next so that 55.49 requirements and licensee program requirements for equitable and consistent examinations are met. 3. Annual requalification operating tests administered to different licensed operators during different weeks within a training cycle repeat < 50% of JPMs that have previously been administered during that same training cycle. 4. Annual requalification operating tests administered to different licensed operators during different weeks within a training cycle repeat < 50% of scenario events that have previously been administered during that same training cycle. In addition, operators tested during different weeks within a training cycle are exposed to a variety of major transients. 5. Annual requalification operating tests differ from year to year so that 55.49 requirements and licensee program requirements for equitable and consistent examinations are met. 6. During the annual requalification operating test, some examinees may have completed a test item or items prior to other examinees during the testing week. In such instances, the facility licensee implemented proper controls (e.g., sequestering, monitoring) which prevented communication of examination information between examinees who had completed a test item or items from those examinees who had yet to complete the same test item or items. 7. During the annual requalification operating test, examination materials - e.g., JPMs, scenario guides, hand-outs to examinees, procedures marked-up or used by examinees, logs kept by examinees - were properly controlled by the facility licensee, such that examinees were not exposed to any examination materials prior to exam administration. 8. During the annual requalification operating test, access to the control room simulator was properly controlled by the facility licensee (e.g., posted signs, locked doors), such that examinees were not exposed to any examination information prior to exam administration. 9. During this inspection and since the last biennial inspection, no incidents of examination compromise have occurred. As defined in 10 CFR 55.49, the integrity of a test or examination is considered compromised if any activity, regardless of intent, affected, or but for detection, would have affected the equitable and consistent administration of the test or examination. 10. If licensed individuals were used to develop or validate requalification examinations, or to administer requalification examinations to other licensed operators, then those licensed developed, validated, or administered to others.

Issue Date: 12/14/18 F-1 71111.11 APPENDIX F REMEDIAL TRAINING AND RE-EXAMINATION CHECKLIST YES NO 1. Re-examinations administered by the facility licensee are commensurate with the original failures. To be considered commensurate, determine the nature of the original failure (written, JPM, or scenario examination) and apply the criteria contained in Appendix B of this IP for written re-examinations (i.e., a minimum of 30 questions) and Appendix C of this IP for JPM and simulator scenario re-examinations (i.e., a minimum of five JPMs, at least 40% of JPMs are alternate path, each licensed operator evaluated using at least two simulator scenarios, each simulator scenario contains at least the minimum number of events). Under appropriate circumstances, an individual failure during a simulator crew examination may be retested with one or more JPMs. 2. Re-examinations do not contain any test items which exactly duplicate test items from the original examination. Similar test items may be used on re-examinations, but they must be modified from original test items as shown below: - Written examination questions used on re-examinations must include a change to at least one pertinent condition in the stem and a change to at least one answer choice when compared to questions used on the original examination. - JPMs used on re-examinations must include a substantive change to at least one condition, such that the course of action differs when compared to JPMs used on the original examination. - Simulator scenarios used on re-examinations must include a substantive change to each scenario event such that the course of action for each event differs when compared to events used from the scenarios that comprised the original examination. 3. Re-examinations repeat < 50% of test items previously administered during the training cycle. deficiency against the expected standards (either quality, integrity, or operator performance) associated with licensed operator requalification examinations required by 10 CFR 55.59(a)(2), and as a possible finding.

Issue Date: 12/14/18 G-1 71111.11 APPENDIX G CHECKLIST FOR EVALUATING PLANT-REFERENCED SIMULATORS OPERATING UNDER 10 CFR 55.46 (c) AND (d) YES NO OVERALL QUESTIONS: 1. Is simulator modeling and performance satisfactory? 2. Does the facility licensee effectively correct identified simulator deficiencies? 3. Does the facility licensee properly perform required simulator testing? At the correct simulator testing periodicity? With proper evaluations of the simulator against the reference plant or best-estimate data? With the correct documentation and retention of simulator test records? DETAILED QUESTIONS FROM 10 CFR 55.46 1. If the plant-referenced simulator is used for the administration of NRC reactor operator and senior operator operating tests, does the plant-referenced simulator demonstrate expected plant response to operator input and to normal, transient, and accident conditions to which the simulator has been designed to respond? [10 CFR 55.31(a)(5) and 55.46(c)(1)] 2. Is the plant-referenced simulator sufficient in scope and fidelity with the reference plant to allow conduct of the evolutions listed in 10 CFR 55.45(a)(1) through (13), as applicable, to the reference plant? [10 CFR 55.46(c)(1)(i)] 3. Is the plant-referenced simulator sufficient in scope and fidelity with the reference plant to allow conduct of the evolutions listed in 10 CFR 55.59(c)(3)(i)(A) through (AA), as applicable to the reference plant? [10 CFR 55.46(c)(1)(i)] 4. Is the plant-referenced simulator designed and implemented in a manner that allows for the completion of control manipulations for operator license applicants? [10 CFR 55.46(c)(1)(ii)] 5. If the plant-referenced simulator is used to meet experience requirements for applicants for operator and senior operator licenses, does the plant-referenced simulator utilize models relating to nuclear and thermal-hydraulic characteristics that replicate the most recent core load in the nuclear power reference plant for which a license is being sought? The phrase "most recent" means the current core or if the reference plant is in a refueling outage, the core just previous to the outage. (This question can be answered with a brief inquiry to the facility licensee, unless an issue is identified.) [10 CFR 55.31(a)(5), 55.46(c)(1), and 55.46(c)(2)(i)] 6. Has the plant-referenced simulator fidelity been demonstrated so that significant control manipulations are completed without procedure exceptions, simulator performance exceptions, or deviation from the approved training scenario sequence? [10 CFR 55.46(c)(2)(ii)] 7. There has been no lapse in the facility licensee conducted simulator performance testing throughout the life of the simulation facility [10 CFR 55.46(d)(1)]

Issue Date: 12/14/18 G-2 71111.11 CHECKLIST FOR EVALUATING PLANT-REFERENCED SIMULATORS OPERATING UNDER 10 CFR 55.46 (c) AND (d) (Continued) YES NO 8. Are the results of performance testing retained for four years after the completion of each performance test or until superseded by updated test results? [10 CFR 55.46(d)(1)] 9. Are modeling and hardware discrepancies and discrepancies identified from scenario validation and from performance testing being corrected? [10 CFR 55.46(d)(2)] 10. Are results of any uncorrected performance test failures that may exist at the time of the operating test or requalification program inspection available for NRC review? [10 CFR 55.46(d)(3)] 11. Has simulator fidelity been maintained such that license application, examination, and test integrity are consistent with 10 CFR 55.49 requirements? [10 CFR 55.46(d)(4)] be processed as potential findings typically with a regulatory violation against 10 CFR 55.46.

Issue Date: 12/14/18 H-1 71111.11 APPENDIX H GENERIC LICENSED OPERATOR OBSERVATION CHECKLISTS CONTROL ROOM/IN PLANT OBSERVATION CHECKLIST The following is presented as generic guidance for observing licensed operators in the main control room or in the plant. The listed items are not regulatory requirements. For additional plant- YES NO COMPLY WITH AND USE PROCEDURES INCLUDING TECHNICAL SPECIFICATIONS 1. Were the appropriate procedures used and referenced in a timely manner? 2. Were the procedures used correctly, including following procedure steps in the correct sequence, abiding by the precautions and limitations, selecting the correct procedure paths on decision blocks, and correctly transitioning between procedures? appropriately justified. 4. Was place-keeping in the procedures effective and in accordance with licensee practices for procedure place-keeping? 5. Were Technical Specifications appropriately entered, exited, and complied with? CONTROL BOARD/IN-PLANT COMPONENT MANIPULATIONS 1. Were the components/controls located efficiently and accurately by the operators? 2. Were the components/controls manipulated accurately and in a timely fashion? COMMUNICATIONS 1. Did the crew exchange complete and relevant information in a clear, easily understood, and accurate manner? 2. Did the crew appropriately keep personnel outside the control room informed of plant status, and were the required communications made outside of the control room in accordance with licensee policy? 3. Did the crew ensure the receipt of clear, easily understood communications from the crew and others? 4. Did the crew consistently use repeat backs, three-way communications, and the phonetic alphabet in accordance with licensee policy? INTERPRETATION, DIAGNOSIS, AND UNDERSTANDING 1. Were plant alarms and indications properly utilized and interpreted to correctly assess and diagnose plant conditions? 2. Were off-normal trends recognized in a timely fashion? 3. Did the crew demonstrate an understanding of the manner in which the plant, systems, and components operated and interacted, including the knowledge of setpoints, interlocks, and automatic functions? 4. Did the crew demonstrate an understanding of how their actions (or inaction) affected system and plant conditions?

Issue Date: 12/14/18 H-2 71111.11 CONTROL ROOM/IN PLANT OBSERVATION CHECKLIST (Continued) The following is presented as generic guidance for observing licensed operators in the main control room or in the plant. The listed items are not regulatory requirements. For additional plant- YES NO USE OF HUMAN ERROR PREVENTION TECHNIQUES 1. Pre-job and status briefs were well-conducted and in accordance with licensee policy (See separate Activity Briefing Checklist contained in this Appendix) 2. Were peer checks conducted in accordance with licensee policy? DOCUMENTATION OF ACTIVITIES 1. Were plant procedures, testing procedures, and other documents used during the activity properly initialed and signed? 2. Was the documentation of Technical Specification entries (and exits), and entries into equipment out of service logs/degraded equipment logs in accordance with licensee policy? 3. Were control room logs properly maintained in accordance with licensee procedures? 4. If a problem did occur during the activity, did the crew properly open and document the issue in a condition report? MANAGEMENT AND SUPERVISION OF ACTIVITIES 1. Did shift management ensure that the crew adhered to plant procedures (e.g., administrative, system operating, surveillance, and alarm response procedures; operations policies and management expectations)? 2. Did shift management properly consider plant safety, including performing a risk assessment if necessary, prior to and during the activity? 3. Did shift management demonstrate the ability to make sound decisions, applying conservative decision making where appropriate? 4. Did shift management demonstrate the ability to properly prioritize tasks and effectively use available personnel resources? in a position to allow proper crew oversight? 6. Did shift management effectively solicit crew feedback? for reactivity management?

Issue Date: 12/14/18 H-3 71111.11 ACTIVITY BRIEFING CHECKLIST The following is presented as generic guidance for observing activity briefs, and is primarily suited for assessing pre-job briefs, although aspects of this checklist may also apply to status briefs conducted during an evolution. The listed items are not regulatory requirements. For additional plant- YES NO Overall, did the brief adequately address the task? Task items to check include: Was the task adequately described? Were task and individual step completion criteria presented? Were key/critical steps identified and discussed? Were communication methods discussed? Were roles, responsibilities, and specific steps identified and assigned to specific individuals? Was the sequence of steps and events discussed? Were technical specifications, operability, and out of service log entries discussed (as applicable)? Was the use of any special equipment discussed (as applicable)? Were expected results, trends, and plant/system/component responses discussed? Were concerns and anticipated problems discussed, including contingencies and abort criteria if problems arose? Overall, did the brief adequately address human performance elements associated with the task? Human performance elements include: Were error likely situations discussed? Were irreversible actions discussed? Was the need for self-checking and peer-checking discussed? Was procedure adherence discussed? Other items Was the briefing well-led, with sufficient management involvement? Did individuals at the brief actively participate? Were safety concerns and effect on plant risk discussed? Were previous lessons learned, industry events, and operating experience associated with this activity discussed?

Issue Date: 12/14/18 H-4 71111.11 LICENSED OPERATOR PERFORMANCE: NOTES AND COMMENTS DATE(S): ______________ EVOLUTION(S) OBSERVED: ________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________ NOTES AND COMMENTS: ____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

Issue Date: 12/14/18 Att 1-1 71111.11 ATTACHMENT 1 - Revision History - IP 71111.11 Commitment Tracking Number Accession Number Issue Date Change Notice Description of Change Description of Training Required and Completion Date Comment Resolution and Closed Feedback Form Accession Number (Pre-Decisional, Non-Public Information) N/A ML012420444 08/16/2001 CN 01-015 Revised to clarify the original intent of the procedure as it relates to sample size selection. None N/A N/A ML022320730 08/20/2002 CN 02-031 Revised to reflect the amended 10 CFR Part 55, "Operators' Licenses," regarding operator license eligibility and the use of simulation facilities in operator licensing (66 FRN 52657, dated October 17, 2001). This revision provides specific guidance to inspector when assessing conformance with simulator requirements specified in 10 CFR 55.46. None N/A N/A ML040210317 12/16/2003 CN 03-041 Revised to include an additional section that inspects excessive test item repetition among comprehensive requalification exams that are taken by crews undergoing the same training program cycle. Excessive item repetition adversely affects validity of the exam. Clarify the original intent of the procedure as it relates to sample size selection. None N/A Issue Date: 12/14/18 Att 1-2 71111.11 Commitment Tracking Number Accession Number Issue Date Change Notice Description of Change Description of Training Required and Completion Date Comment Resolution and Closed Feedback Form Accession Number (Pre-Decisional, Non-Public Information) N/A ML053490168 01/05/2006 CN 06-001 Inspection resource was increased to 4 hrs/quarter (net increase of 4 hour4.62963e-5 days <br />0.00111 hours <br />6.613757e-6 weeks <br />1.522e-6 months <br />s/year) to more accurately reflect the time spent by resident inspectors during their quarterly observation of operator requalification activities. Completed historical CN search. None N/A N/A ML113270192 12/06/11 CN 11-040 Complete rewrite of document. Added 4 hrs/quarter for Resident Inspectors to observe operators in the control room. Replaced Operating History section with Problem Identification and Resolution section. Added an Examination Security section, taken from other parts of the existing IP. Clarified biennial inspection requirements and updated assessment methods to current practices. Training held by teleconference with Regional examiners on 11/30/11 ML113250476 N/A ML121560358 08/27/12 CN 12-018 Minor typographical errors corrected, clarified who can perform the biennial inspection, added reference to industry standards for requalification examinations, changed wording of appendices regarding performance deficiencies, and removed number of licensee evaluators used during scenarios as a metric from Appendix D. None Closed FF: 71111.11-1756 ML12240A228 1245-1757 ML12240A210 Issue Date: 12/14/18 Att 1-3 71111.11 Commitment Tracking Number Accession Number Issue Date Change Notice Description of Change Description of Training Required and Completion Date Comment Resolution and Closed Feedback Form Accession Number (Pre-Decisional, Non-Public Information) N/A ML14217A409 09/24/2014 CN 14-022 Flexibility added for the frequency of main control room observations, clarified individual examination failure rate, changed the methodology for assessing simulator scenario quality (including a new worksheet), and eliminated the 10% re-take exam failure rate metric. None Closed FF: 71111.11-1850, 71111.11-1920, 71111.11-1950 N/A ML18178A559 12/14/18 CN 18-043 Flexibility added for the frequency of licensed operator training observations by resident inspectors, provided guidance to review overlap between written exams. Revised review guidance for open reference questions (direct look-up and credible distractors). Removed inspection bases statements. Removed detailed PI&R section, added reference to IP 71152. Added an inspection element to check for repeating test items from a previously administered examination. Eliminated redundancy and improved for plain writing. Relocated optional requirements to the guidance section to better align with IMC 2515 Section 8.04 sample completion requirements. Streamlined IP formatting. None ML18177A416 Closed FF: 71111.11-1556 ML18178A143 71111.11-2106 ML18178A149 71111.11-2116 ML18178A194

Retrieved from "https://kanterella.com/w/index.php?title=ML18178A559&oldid=1225560"