IR 05000250/2024301
| ML24276A200 | |
| Person / Time | |
|---|---|
| Site: | Turkey Point  |
| Issue date: | 10/03/2024 |
| From: | Tom Stephen NRC/RGN-II/DRS/OLB |
| To: | Coffey B Florida Power & Light Co |
| References | |
| IR 2024301 | |
| Download: ML24276A200 (1) | |
Text
SUBJECT:
TURKEY POINT NUCLEAR PLANT - NRC OPERATOR LICENSE EXAMINATION REPORT 05000250/2024301 AND 05000251/2024301
Dear Bob Coffey:
During the period of August 12 - 20, 2024, the Nuclear Regulatory Commission (NRC)
administered operating tests to employees of your company who had applied for licenses to operate the Turkey Point Nuclear Plant. At the conclusion of the tests, the Chief Examiner discussed preliminary findings related to the examination submittal with those members of your staff identified in the enclosed report. The written examination was administered by your staff on August 23, 2024.
Five Reactor Operator (RO) and five Senior Reactor Operator (SRO) applicants passed both the operating test and written examination. One applicant passed the operating test but did not pass the written examination. Two applicants passed the written examination but did not pass the operating test. There were no post-administration comments concerning the operating test or written examination. A Simulator Fidelity Report is included as Enclosure 2.
The initial SRO written examination and the operating test walkthrough submittal failed to meet the guidelines for quality contained in NUREG-1021, Operator Licensing Examination Standards for Power Reactors, Revision 12. The initial RO written examination and operating test dynamic simulator scenario submittals were within the range of acceptability expected for a proposed examination. All examination changes agreed upon between the NRC and your staff were made according to NUREG-1021.
During the NRCs review of your facilitys proposed NRC initial license examinations, it was discovered that most Job Performance Measures (JPMs) proposed by your facility, as part of the walkthrough potion of the operating test, had been repeated from those administered three examinations earlier. The NRC also discovered that the same issue existed during your facilitys 2023 NRC initial license examination, whereby your facility repeated the majority of JPMs from the examination that had been administered at your facility three examinations earlier. Therefore, a pattern had been established where the examination content could be predicted based on the test items contained in the examinations administered three examinations prior to the current examination. The ability to predict examination content could allow an applicant to limit their scope of study, thereby negatively impacting the ability of the examination to infer mastery of knowledges and skills that were not being directly tested.
October 3, 2024 The NRC has completed its review of this examination security issue. NRC license examiners documented one finding of very low safety significance (Green) in this report. The finding involved a violation of NRC requirements. The NRC is treating this violation as a non-cited violation (NCVs) consistent with Section 2.3.2 of the Enforcement Policy. The details of the NCV are in Enclosure 1 of this report. We have determined that additional follow-up inspection for this NCV is not warranted because the cause of the NCV is clearly understood by your facility, and the issue is being addressed within your corrective action program.
If you contest the violation or the significance of the violation documented in this examination report, you should provide a response within 30 days of the date of this examination report, with the basis for your denial, to the U.S. Nuclear Regulatory Commission, ATTN: Document Control Desk, Washington, DC 20555-0001; with copies to the Regional Administrator, Region II; the Director, Office of Enforcement; and the NRC Senior Resident Inspector at the Turkey Point Nuclear Plant.
If you disagree with a cross-cutting aspect assignment in this report, you should provide a response within 30 days of the date of this inspection report, with the basis for your disagreement, to the U.S. Nuclear Regulatory Commission, ATTN: Document Control Desk, Washington, DC 20555-0001; with copies to the Regional Administrator, Region II; and the NRC resident inspector at the Turkey Point Nuclear Plant.
In accordance with 10 CFR 2.390 of the NRCs Rules of Practice, a copy of this letter and its enclosures will be available electronically for public inspection in the NRC Public Document Room or from the Publicly Available Records (PARS) component of the NRCs document system (ADAMS). ADAMS is accessible from the NRC Website at http://www.nrc.gov/reading-rm.adams.html (the Public Electronic Reading Room).
If you have any questions concerning this letter, please contact me at (404) 997-4703.
Sincerely, Thomas A. Stephen, Chief Operations Branch 1 Division of Reactor Safety Docket Nos: 50-250, 50-251 License Nos: DPR-31, DPR-41
Enclosures:
1. Simulator Fidelity Report
REGION II==
Docket No.:
50-250, 50-251 License No.:
05000250/2024301, 05000251/2024301 EPID No.:
L-2024-OLL-0040 Licensee:
Florida Power & Light Company (FP&L)
Facility:
Turkey Point Nuclear Plant, Units 3 and 4 Location:
Homestead, FL Dates:
Operating Test - August 12 - 20, 2024 Written Examination - August 23, 2024 Examiners:
M. Bates, Chief Examiner, Senior Operations Engineer N. Lacy, Operations Engineer V. Furr, Operations Engineer K. Schaaf, Operations Engineer Approved by:
Thomas A. Stephen, Chief Operations Branch 1 Division of Reactor Safety
SUMMARY ER 05000250/2024301,05000251/2024301; August 12 - 20, 2024 & August 23, 2024; Turkey Point Nuclear Plant; Operator License Examinations.
Nuclear Regulatory Commission (NRC) examiners conducted an initial examination in accordance with the guidelines in Revision 12 of NUREG-1021, "Operator Licensing Examination Standards for Power Reactors." This examination implemented the operator licensing requirements identified in 10 CFR §55.41, §55.43, and §55.45, as applicable.
Members of the Turkey Point Nuclear Plant staff developed both the operating test and the written examination. The initial Senior Reactor Operator (SRO) written examination and walkthrough portion of the operating test submittals did not meet the quality guidelines contained in NUREG-1021. The Reactor Operator (RO) written examination and the operating test dynamic simulator scenario submittals met the quality guidelines contained in NUREG-1021.
The NRC administered the operating tests during the period of August 12 - 20, 2024. Members of the Turkey Point Nuclear Plant training staff administered the written examination on August 23, 2024. Five RO and five SRO applicants passed both the operating test and written examination. One RO applicant passed the operating test but failed the written examination.
One SRO applicant and one RO applicant passed the written examination but failed the operating test. Ten applicants were issued licenses commensurate with the level of examination administered.
There were no post-examination comments.
List of Findings and Violations Cornerstone: Mitigating Systems Green: An NRC-identified Green non-cited violation (NCV) of 10 CFR 55.49, Integrity of Examinations and Tests, was identified because the licensee engaged in an activity that compromised the integrity of examinations. Specifically, the licensee utilized an examination development practice where the Job Performance Measures (JPMs) were largely repeated from the NRC initial license examination that had been administered three examinations earlier for both the 2023 and 2024 examinations.
This performance deficiency was more than minor because it was associated with the human performance attribute of the mitigating systems cornerstone, and adversely affected the cornerstone objective of ensuring the availability, reliability, and capability of systems that respond to initiating events to prevent undesirable consequences. Specifically, using predictable exam development and administration techniques could have adversely affected the integrity of the administration of the operating tests, which evaluates operator performance to ensure timely and correct mitigating actions during an event. Using the NRCs Significance Determination Process, this finding was determined to be of very low safety significance (Green)
because no known compromise of the examinations occurred. The inspectors determined the finding had a cross-cutting aspect of procedure adherence in the crosscutting area of human performance because the licensee failed to ensure that examination development practices complied with the criteria in NUREG-1021. [H.8]
The failure to comply with examination security requirements of NUREG-1021 required the licensee to replace 8 out of 9 Administrative JPMs, and 9 out of 11 Systems JPMs, with a 10th Systems JPM being significantly modified, for the 2024 examination. The replacement and modification of the JPMs resulted in the applicants receiving an examination that fully complied with the criteria of NUREG-1021. The licensee documented the issue in condition report (CR)
02495191.
REPORT DETAILS 4.
OTHER ACTIVITIES 4OA5 Operator Licensing Examinations
.1 License Applications a.
Scope The examiners reviewed all license applications submitted by the licensee to ensure the applications reflected that each applicant satisfied relevant license eligibility requirements. The applications were submitted on NRC Form 398, Personal Qualification Statement, and NRC Form 396, Certification of Medical Examination by Facility Licensee. The examiners also audited 10 percent of the license applications in detail to confirm that they accurately reflected the subject applicants qualifications. This audit focused on the applicants experience and on-the-job training, including control manipulations that provided significant reactivity changes.
b.
Findings No findings were identified.
.2 Operator Knowledge and Performance a.
Scope The NRC examiners evaluated seven Reactor Operator (RO) and six Senior Reactor Operator (SRO) applicants on both the written examination and operating test using the guidelines contained in NUREG-1021. The licensee proctored the administration of the written examinations to all applicants on August 23, 2024. The NRC examination team administered the various portions of the operating examination to all applicants from August 12 through 20, 2024.
A review of associated documentation was performed to determine if the applicants met the requirements specified in 10 CFR Part 55, Operators Licenses.
b.
Findings Five RO and five SRO applicants passed both the operating test and written examination. One RO applicant passed the operating test but failed the written examination. One SRO applicant and one RO applicant passed the written examination but failed the operating test. Ten applicants were issued licenses commensurate with the level of examination administered. The three applicants that did not pass all portions of the examination were issued preliminary denials of their license applications.
Copies of all individual examination reports were sent to the facility Training Manager for evaluation of weaknesses and determination of appropriate remedial training
.3 Initial Licensing Examination Development a.
Scope The NRC developed the written examination outline and provided it to your staff on September 20, 2023. The licensee developed the examinations in accordance with NUREG-1021, Revision 12. All licensee facility training and operations staff involved in examination preparation and validation were listed on a security agreement. The licensee submitted the operating tests and written examinations to the NRC. The Chief Examiner reviewed the outlines and draft examination materials against the requirements of NUREG-1021 and provided comments to the licensee. The NRC conducted an onsite validation of the operating examinations and provided further comments during the week of July 8, 2024. Examination changes agreed upon between the NRC and the licensee were made in accordance with NUREG-1021 and incorporated into the final version of the examination materials.
The NRC evaluated the submitted operating test by combining the scenario events and Job Performance Measures (JPMs) in order to determine the percentage of submitted test items that required replacement or significant modification. The NRC also evaluated the submitted written examination questions (RO and SRO questions considered separately) in order to determine the percentage of submitted questions that required replacement or significant modification, or that clearly did not conform with the intent of the approved knowledge and ability (K/A) statement. Any questions that were deleted during the grading process, or for which the answer key had to be changed, were also included in the count of unacceptable questions. The percentage of submitted test items that were unacceptable was compared to the acceptance criteria of NUREG-1021.
b.
Findings The NRC determined that the licensees initial reactor operator (RO) written examination and the dynamic simulator portion of the operating test submittal were within the range of acceptability expected for a proposed examination. However, the NRC determined that the licensees senior reactor operator (SRO) written examination submittal was outside the range of acceptable quality specified by NUREG-1021. The initial SRO written examination submittal was outside the range of acceptable quality because more than 20% of the SRO questions contained unacceptable flaws. Individual questions were evaluated as unsatisfactory for the following reasons:
questions failed to meet the randomly selected K/A statement
questions contained two or more implausible distractors
question did not contain one and only one correct answer
questions did not contain an acceptable level of discrimination validity
questions on the SRO examination were not written at the SRO level The Job Performance Measures (JPMs) portion of the operating test also did not meet the quality standards of NUREG-1021 due to examination development practices that allowed for the examination content to be predictable, which would allow applicants to narrow their focus of study. Because of the predictability issue, the majority of the JPMs were replaced. This issue is discussed in more detail in Section 5 of this report
The licensee submitted no post-examination comments. A copy of the final written examinations and answer keys, with all changes incorporated, may be accessed not earlier than September 16, 2026, in the ADAMS system (ADAMS Accession Numbers ML24255A007 and ML24255A008).
.4 Simulation Facility Performance a.
Scope The NRC evaluated the performance, or fidelity, of the simulation facility during the preparation and conduct of the operating tests.
b.
Findings No findings were identified. Enclosure 2 contains a Simulator Fidelity Report.
.5 Examination Security a.
Scope The NRC reviewed the licensees examination security measures while preparing and administering the examinations to ensure compliance with 10 CFR §55.49, Integrity of examinations and tests.
b.
Findings Introduction: An NRC identified Green NCV of 10 CFR 55.49, Integrity of Examinations and Tests, was identified because the licensee engaged in an activity that, if not for detection by the NRC, could have compromised the integrity of examinations. The integrity of a test or examination is considered compromised if any activity, regardless of intent, affected, or, but for detection, would have affected the equitable and consistent administration of the test or examination. Specifically, the licensee utilized an examination development practice where the Job Performance Measures (JPMs) were largely repeated from the NRC initial license examination that had been administered three examinations earlier.
Description: While reviewing the licensees examination submittal for their 2024 NRC initial license examination, it was noted that the JPMs were either identical to the JPMs used on the 2019-302 NRC initial license examination, or in many cases when designated as a modified JPM, the changes did not make the task materially different.
The tasks were repeated for 10 of the 11 Systems JPMs from the 2019-302 examination. Administrative JPM tasks were repeated for all 9 of the Administrative JPMs as originally presented on the facility licensees initial operating test outline submittal, however, the facility replaced one of those JPMs prior to submitting their full draft submittal at the 75-day point in the examination process.
The 2023 NRC initial examination was also reviewed to determine if the same or similar issue had previously existed. Again, the NRC identified that JPMs were either identical to the JPMs used on the 2019-301 NRC initial examination, or when designated as a modified JPM, the changes did not make the task materially different. The tasks on 10
of 11 Systems JPMs had been repeated from those administered on the 2019-301 examination. The tasks chosen for the Administrative JPMs were also repeated for all 9 Administrative JPMs.
Because the same practice was utilized for two consecutive examinations, a pattern existed that could be used to accurately predict the JPM operating test content on the 2024 NRC initial license examination. The failure to comply with examination security requirements of NUREG-1021 required the licensee to replace 8 out of 9 Administrative JPMs, and 9 out of 11 Systems JPMs, with a 10th Systems JPM being significantly modified, for the 2024 examination. The replacement and modification of the JPMs resulted in the applicants receiving an examination that fully complied with the criteria of NUREG-1021. The licensee has entered the issue into their corrective action program as Condition Report (CR) Number 02495191.
10 CFR 55.40, Implementation, requires that all examinations be developed, administered, and graded in accordance with the criteria in NUREG-1021, Operator Licensing Examination Standards for Power Reactors, in effect six months before the examination date, which for this examination was Revision 12. NUREG-1021, ES-1.3, Examination Security, provides a list of prohibited activities for those staff with knowledge of the examination content. In part ES-1.3, states, The license applicants should not be able to predict or narrow the possible scope or content of the licensing examination based on the facility licensees examination practices.
Analysis: The inspectors determined that developing and utilizing predictable examination material for the NRC initial operating tests was a performance deficiency.
The performance deficiency was determined to be more than minor because it was associated with the human performance attribute of the mitigating systems cornerstone, and adversely affected the cornerstone objective of ensuring the availability, reliability, and capability of systems that respond to initiating events to prevent undesirable consequences. Specifically, consistently and predictably re-using JPM tasks during both the 2023 and 2024 NRC initial license operating tests, affected, but for detection, the integrity of the administration of the operating tests, which evaluates the licensed operator applicant performance in order to ensure timely and correct mitigating actions during an event.
The significance determination was performed in accordance with Manual Chapter 0609, Significance Determination Process, Appendix A, Determining Significance of Reactor Inspection Findings for At-Power Situations. Answers to all screening questions for the Mitigating Systems cornerstone were answered, no; therefore, this finding was characterized as having very low safety significance (Green). The finding was related to the cross-cutting aspect of procedure adherence in the cross-cutting area of human performance because the licensee failed to ensure that examination development practices complied with NUREG-1021. [H.8]
Enforcement: 10 CFR 55.49, Integrity of Examinations and Tests, states that applicants, licensees, and facility licensees shall not engage in any activity that compromises the integrity of any application, test, or examination required by this part.
The integrity of a test or examination is considered compromised if any activity, regardless of intent, affected, or, but for detection, would have affected the equitable and consistent administration of the test or examination. This includes activities related to the preparation and certification of license applications and all activities related to the
preparation, administration, and grading of the tests and examinations required by this part. Activities covered by this part include examination development activities covered by 10 CFR 55.40, Implementation, which requires NRC initial license operator examinations be developed in accordance with NUREG-1021, Operator Licensing Examination Standards for Power Reactors. NUREG-1021, ES-1.3, Examination Security, states that license applicants should not be able to predict or narrow the possible scope or content of the licensing examination based on the facility licensees examination practices.
Contrary to the above, the licensee predictably re-used JPM tasks from three examinations prior to the 2023 and (as submitted) 2024 NRC initial license examinations. The re-use of the JPMs in a reliably predictable manner could have allowed applicants to limit the possible scope, or content of the licensing examination.
Because this issue is of very low safety significance and has been entered into the licensees corrective action program, Condition Report (CR) Number 02495191, the violation is being treated as a Non-Cited Violation consistent with Section 2.3.2. of the NRC Enforcement Policy. [NCV 05000250, 251/2024301-01, Failure to Comply with 10 CFR 55.49]
4OA6 Meetings, Including Exit Exit Meeting Summary On August 16, 2024, the NRC Chief Examiner discussed generic issues associated with the operating test with Mr. Michael Strope, Site Vice President, and members of the Turkey Point Nuclear Plant staff. The examiners asked the licensee if any of the examination material was proprietary. No proprietary information was identified.
On September 18, 2024, the NRC Chief Examiner discussed the examination results with Mr. Sean Wylie, Fleet Training Supervisor.
On October 2, 2024, The NRC discussed the details of the non-cited violation (NCV),
which is detailed in this report, with Mr. Michael Strope, Site Vice President.
KEY POINTS OF CONTACT Licensee personnel M. Strope, Site Vice President R. Hess, Training Manager R. Montgomery, Operations Director C. Santos, Director of Organizational Effectiveness - Training E. Juergens, Assistant Operations Manager - Training L. Pineiro, Continuing Training Supervisor C. Machado, Operations Supervisor V. Miklausich, Training Supervisor J. Hinze, Fleet Training Manager S. Wylie, Fleet Training Supervisor T. Hodge, Exam Developer E. Socias. Exam Developer
LIST OF ITEMS OPENED, CLOSED, AND DISCUSSED Opened and Closed 05000250, 251/2024301-01 NCV Failure to comply with 10 CFR 55.49 DOCUMENTS REFERENCED ADAMS ACCESSION NUMBER TITLE / DESCRIPTION ML19108A177 2019-301 NRC Initial Examination ML19275G834 2019-302 NRC Initial Examination ML23191A865 2023-301 NRC Initial Examination ML23191A868 2023-301 NRC Initial Examination ML23191A863 2023-301 NRC Initial Examination TBD 2024-301 NRC Initial Examination
Enclosure 2 SIMULATOR FIDELITY REPORT Facility Licensee: Turkey Point Nuclear Plant Units 3 and 4 Facility Docket No.: 05000250, 05000251 Operating Test Administered: August 12 - 20, 2024 This form is used only to report observations. These observations do not constitute audit or inspection findings and, without further verification and review in accordance with Inspection Procedure 71111.11 are not indicative of noncompliance with 10 CFR §55.46. No licensee action is required in response to these observations.
No simulator fidelity or configuration issues were identified.