IR 05000348/2023040
| ML24022A059 | |
| Person / Time | |
|---|---|
| Site: | Farley  |
| Issue date: | 01/24/2024 |
| From: | Alan Blamey Division Reactor Projects II |
| To: | Coleman J Southern Nuclear Operating Co |
| References | |
| EA-23-080, EPID I-2023-040-0009 IR 2023040 | |
| Download: ML24022A059 (11) | |
Text
SUBJECT:
JOSEPH M. FARLEY NUCLEAR PLANT - 95001 SUPPLEMENTAL INSPECTION REPORT 05000348/2023040 AND FOLLOW-UP ASSESSMENT LETTER
Dear Jamie Coleman:
On December 14, 2023, the U.S. Nuclear Regulatory Commission (NRC) completed a supplemental inspection using Inspection Procedure 95001, "Supplemental Inspection Response to Action Matrix Column 2 (Regulatory Response) Inputs, and discussed the results of this inspection and the implementation of your corrective actions with you and other members of your staff.
The NRC performed this inspection to review your stations actions in response to a White finding in the Mitigating Systems cornerstone which was documented and finalized in NRC Inspection Report 05000348/2023091. On November 14, 2023, you informed the NRC that your station was ready for the supplemental inspection.
The NRC determined that your staffs evaluation identified the cause of the White finding.
Specifically, your staffs evaluation identified two root causes. The first root cause was that the Unit 1 'B' emergency diesel generator (EDG) circulating lube oil pump discharge piping restraint was inadequate to prevent coupling separation. The second root cause was that troubleshooting process deficiencies and implementation weaknesses led to not adequately identifying and correcting the Unit 1 'B' EDG circulating lube oil system failure mode in November 2022.
The inspectors determined that the root cause evaluation was documented at a sufficient level of detail, included relevant operating experience, and identified the root causes, extent of conditions, and extent of causes of the performance issue. Based on the results of the inspection, the inspectors concluded the objectives of the inspection procedure were met.
The NRC determined that completed and planned corrective actions were sufficient to address the performance issue that led to the White finding. Therefore, the performance issue will be closed and no longer considered as an Action Matrix input as of the date of the exit meeting and regulatory performance meeting. Based on the results of this inspection and our Action Matrix assessment, the NRC made the determination to transition Joseph M. Farley Nuclear Plant (Farley) Unit 1 to the Licensee Response Column (Column 1) of the Action Matrix on December 14, 2023, considering the absence of additional Action Matrix inputs.
No findings or violations of more than minor significance were identified during this inspection.
January 24, 2024 This letter, its enclosure, and your response (if any) will be made available for public inspection and copying at http://www.nrc.gov/reading-rm/adams.html and at the NRC Public Document Room in accordance with Title 10 of the Code of Federal Regulations 2.390, Public Inspections, Exemptions, Requests for Withholding.
Sincerely, Alan J. Blamey, Chief Reactor Projects Branch 2 Division of Reactor Projects Docket No. 05000348 License No. NPF-2
Enclosure:
As stated
Inspection Report
Docket Number:
05000348
License Number:
Report Number:
Enterprise Identifier:
I-2023-040-0009
Licensee:
Southern Nuclear Operating Company, Inc.
Facility:
Joseph M. Farley Nuclear Plant
Location:
Columbia, AL
Inspection Dates:
December 11, 2023, to December 14, 2023
Inspectors:
P. Boguszewski, Senior Resident Inspector
M. Donithan, Senior Operations Engineer
R. Easter, Resident Inspector (Trainee)
Approved By:
Alan J. Blamey, Chief
Reactor Projects Branch 2
Division of Reactor Projects
SUMMARY
The U.S. Nuclear Regulatory Commission (NRC) continued monitoring the licensees performance by conducting a 95001 supplemental inspection at Joseph M. Farley Nuclear Plant, in accordance with the Reactor Oversight Process. The Reactor Oversight Process is the NRCs program for overseeing the safe operation of commercial nuclear power reactors. Refer to https://www.nrc.gov/reactors/operating/oversight.html for more information.
List of Findings and Violations
No findings or violations of more than minor significance were identified.
Additional Tracking Items
Type Issue Number Title Report Section Status NOV 05000348/2023002-01 Emergency Diesel Generator Lube Oil Coupling Leak EA-23-080 95001 Closed
INSPECTION SCOPES
Inspections were conducted using the appropriate portions of the inspection procedures in effect at the beginning of the inspection unless otherwise noted. Currently approved inspection procedures with their attached revision histories are located on the public website at http://www.nrc.gov/reading-rm/doc-collections/insp-manual/inspection-procedure/index.html.
Samples were declared complete when the inspection procedure requirements most appropriate to the inspection activity were met consistent with Inspection Manual Chapter (IMC)2515, Light-Water Reactor Inspection Program - Operations Phase. The inspectors reviewed selected procedures and records, observed activities, and interviewed personnel to assess licensee performance and compliance with Commission rules and regulations, license conditions, site procedures, and standards.
OTHER ACTIVITIES
- TEMPORARY INSTRUCTIONS, INFREQUENT AND ABNORMAL 95001 - Supplemental Inspection Response to Action Matrix Column 2 (Regulatory Response)
Inputs From December 11 to December 14, 2023, the inspectors conducted an inspection to verify all objectives of the inspection procedure were met. The inspectors reviewed and selectively challenged aspects of the licensees problem identification, causal analysis, and corrective actions in response to the licensee's failure to identify and correct a condition adverse to quality resulting in the inoperability of the Unit 1 B train emergency diesel generator (EDG) between December 7, 2022, and March 3, 2023, as documented in NRC Inspection Reports 05000348/2023090 (ADAMS Accession No. ML23229A118) dated August 31, 2023, and 05000348/2023091 (ADAMS Accession No. ML23263B166) dated October 19,
INSPECTION RESULTS
Assessment 95001
1. Problem Identification.
a. Identification. Southern Nuclear Operating Company, Inc. (Southern Nuclear) identified that the circulating lube oil pump discharge piping on the Unit 1 B EDG was inadequately restrained, which allowed movement of the piping and resulted in a failure of the pressure boundary when the piping separated from an in-line pipe coupling. This issue was self-revealed on February 26, 2023, during a technical specification one-hour surveillance run, when an oil leak of approximately 22 to 50 gallons per minute was identified. The Unit 1 'B' EDG was rendered inoperable based on the rate of the oil leak. A similar failure at the same location occurred previously, while Unit 1 was shutdown on November 4, 2022, following a planned replacement of the coupling. The licensee determined that troubleshooting activities conducted in response to the November 2022 oil leak failed to identify and correct the improperly restrained piping.
b. Exposure Time. The issue was assumed to have existed since the completion of the 24-hour surveillance test on November 7, 2022, following replacement of the coupling until the completion of repairs to the Unit 1 'B' EDG on March 2, 2023. This was approximately 115 days. The inspectors determined that the licensee appropriately assessed the exposure time.
c. Identification Opportunities. The licensee appropriately considered prior occurrences and identification opportunities. The licensee determined that the Unit 1 'B' EDG lube oil leak was caused by a latent design issue involving the inadequate restraints of the piping adjacent to the circulating lube oil pump discharge coupling connection. Neither the piping nor coupling had been replaced prior to November 2022, when Farley performed a first-time coupling replacement. The licensee did not identify any opportunities to identify this latent design issue prior to November 2022. However, on November 4, 2022, during testing after the first-time coupling replacement, the circulating lube oil pump discharge piping and coupling connection separated. The licensee identified that the troubleshooting efforts after the coupling separation on November 4, 2022, did represent an identification opportunity prior to the coupling failure on February 26, 2023.
d. Risk and Compliance. The licensees evaluation demonstrated an appropriate understanding of the risk and compliance aspects of the issue. Specifically, the licensee documented that from November 4, 2022, until February 26, 2023, when a near-identical event occurred, the Unit 1 'B' EDG was vulnerable to initiating events, which could adversely affect its availability, reliability, or capability. Furthermore, from December 7, 2022, to March 3, 2023, while the plant was in the modes that required the Unit 1 B EDG to be operable, the EDG was inoperable.
The NRCs risk evaluation of Farleys White performance issue was documented in NRC Inspection Report 05000348/2023090.
NRC Assessment: The inspectors determined that the licensee appropriately evaluated and documented identification of the problem, including adequate considerations of identification credit, how long the condition had existed, missed opportunities for self-identification, and risk insights.
2. Causal
Analysis.
a. Methodology. The root cause evaluation employed systematic, evidence-based methodologies to determine the root and contributing causes of the White performance issue.
The methodologies included Event and Causal Factors Analysis, Equipment Reliability Checklist, Safety Culture Attributes Assessment, Barrier Analysis, Organizational and Programmatic Assessment, and Change
Analysis.
The root cause evaluation captured two root causes and one contributing cause:
- Root Cause 1 (RC1): Piping restraint was inadequate to prevent coupling separation.
- Root Cause 2 (RC2): Troubleshooting process deficiencies and implementation weaknesses.
- Contributing Cause 1 (CC1): Less than adequate oversight by the station leaders.
b. Level of Detail. The inspectors determined the root cause evaluation was performed commensurate with the safety significance and complexity of the performance issue and was of sufficient detail to identify the root and contributing causes, extent of conditions, and extent of causes.
c. Operating Experience. The inspectors determined that the licensee appropriately considered prior occurrences and operating experience. Southern Nuclear reviewed both external and internal sources of operating experience to determine if the same or similar conditions had occurred either at the site or industry. The operating experience review did not identify cases where piping loosened or shifted out of alignment over time due to maintenance activities, nor did it identify cases where original design piping restraints were determined to be inadequate. Southern Nuclear also reviewed external operating experience for same or similar root or contributing causes related to inadequate troubleshooting and less than adequate management oversight of troubleshooting activities.
d. Extent of Condition and
Cause.
The inspectors determined that the licensee appropriately considered the extent of condition and extent of causes. Southern Nuclear used the same-same, same-similar, similar-same, and similar-similar methodologies to evaluate the extent of condition and the extent of cause. The extent of condition review considered any size Flexmaster couplings installed on EDG systems, other flexible couplings of all styles and from all vendors on EDG systems, and flexible couplings on non-EDG systems. Southern Nuclear performed corrective actions to address any items identified from the extent of condition review that were determined to be susceptible to the same failure mode as the Unit 1 'B' EDG circulating lube oil pump discharge coupling connection. The extent of cause review considered extent of cause of both root causes: inadequate piping restraint (RC1) and troubleshooting process and implementation weaknesses (RC2).
e. Safety Culture. The inspectors determined that the safety culture traits in NUREG-2165, Safety Culture Common Language, referenced in IMC 0310-06, were appropriately considered.
NRC Assessment: Based on review of the root cause evaluation, information related to the event, and interviews with Farley staff, the inspectors determined the licensee adequately ensured the White performance issue was evaluated using a systematic methodology to identify the root and contributing causes. The inspectors determined the root cause evaluation was conducted to a level of detail commensurate with the significance of the performance issue and included relevant operating experience. The licensee's evaluation considered extent of condition and extent of causes of the performance issue. Additionally, the root cause evaluation appropriately considered the safety culture traits in NUREG-2165.
3. Corrective Actions.
a. Corrective Actions
to Preclude Repetition
- (1) Completed
- Corrective Actions to Preclude Repetition (CAPRs) 1 and 2 (associated with RC1):
Design change package and implementation work order for Unit 1 'B' EDG circulating lube oil piping restraints.
- CAPRs 3 through 8 (RC1): Design change package and implementation work orders to eliminate the Unit 1 'B', Unit 1-2 'A', and Unit 2 'B' EDGs circulating lube oil pump discharge coupling.
- CAPR 9 (RC2) (CC1): Revise NMP-GM-020-001, "Issue Response Team."
- CAPR 10 (RC2) (CC1): Revise NMP-AD-002, "Conduct of Problem Solving and Troubleshooting."
- CAPR 11 (RC2): Update NMP-MA-050, Work Package Preparation.
- CAPR 12 (RC2): Update NMP-AD-002-F04, Troubleshooting Log.
- (2) Planned
- No planned CAPRs existed since all actions were completed before the end of the inspection period.
- (3) Effectiveness Review
- The effectiveness review actions established by Southern Nuclear for CAPRs were appropriate. In general, each planned corrective action to preclude repetition identified a corresponding effectiveness review, which contained quantitative and qualitative measures of effectiveness. Additionally, the inspectors concluded the actions, owners, and due dates were appropriate and commensurate with the corresponding corrective action to preclude repetition. Southern Nuclear's effectiveness reviews included the following:
Effectiveness Review 1 (CAPRs 1-8) (Completed): Verify that physical modifications to the emergency diesel lube oil system were made on time and did not introduce new failure modes.
Effectiveness Review 2 (CAPRs 9-12) (Planned): Review Issue Response Team (IRT) and troubleshooting events completed since October 6, 2023, which used focused and complex troubleshooting methods, to determine the cause and implement timely corrective actions.
b. Corrective Actions
- (1) Completed
- Corrective Action (CA) 1 (RC2): Modify the job plans or develop a procedure for EDG Flexmaster coupling replacement maintenance to include steps that document as-found and as-left torque, alignment, and insertion depth.
- CA 3 (CC1): Revise NMP-GM-006, "Work Management."
- CA 4 (CC1): Revise NMP-GM-002-001, "Corrective Action Program Instructions."
- CA 7 through CA 11 and CA 16 (RC2) (CC1): Perform teaching and learning sessions in accordance with INPO 23-001, "Guidelines for Advancing Teaching and Learning in the Nuclear Power Industry," to cover this root cause and procedure revisions for alignment and awareness of the following staff:
Maintenance managers, supervisors, and craft personnel (mechanical, electrical, instrumentation and control (I&C))
Management Review Committee (MRC) members and Corrective Action Program Coordinator personnel
Engineering personnel
Maintenance planning personnel
Farley Refueling Services managers, supervisors, and refueling technicians
Operations senior reactor operators
- CA 12 (RC2): Incorporate lessons learned from this root cause evaluation to mechanical, I&C, and electrical system master plans for initial and continuing training.
- (2) Planned
- CA 2 (RC2) (CC1): Develop and implement a case study in accordance with NMP-GM-002-GL03, "Cause Analysis and Corrective Actions Guideline," for Unit 1 'B' EDG circulating lube oil piping coupling separation that includes adequate ownership of the IRT, adherence to the fleet troubleshooting standard, effective questioning attitude, and effective barriers that would prevent gaps.
- CA 5 (CC1): MRC to review closed Priority 1 CAPR work orders and review samples of closed Priority 2 Corrective Action Work Orders to ensure closure criteria are being documented in corrective action work orders.
- CA 6 (CC1): Perform a self-assessment on closed corrective action program work orders.
- CA 13 (RC2): Perform IRT-simulated event scenarios with the following members of the Farley Duty Team: Station Duty Manager, Operations Duty Manager, Maintenance Duty Manager, Engineering Duty Manager, and Engineering Duty Supervisor. The scenarios will be used to observe, coach, and critique station leadership behaviors during a simulated plant scenario.
- CA 14 (RC2, CC1): Monthly cross-functional observations by engineering managers and supervisors of focused and simple troubleshooting activities.
- CA 15 (RC2, CC1): Monthly cross-functional observations by maintenance managers and supervisors of focused and simple troubleshooting activities.
NRC Assessment: The inspectors determined that Southern Nuclear implemented or planned appropriate and timely corrective actions to preclude repetition of a significant condition adverse to quality associated with the White performance issue. Southern Nuclear also identified appropriate effectiveness reviews for these corrective actions. The inspectors identified the following minor weakness:
Minor Weakness - Other Corrective Actions, Section 3.b: Inspection Procedure 95001, Section 02.04, "Corrective Actions," requires that the inspection team determine if appropriate measures of success have been developed for determining the effectiveness of corrective actions. The team determined that CAs 14 and 15, associated with monthly cross-functional observations of focused and simple troubleshooting activities, were planned to be stopped in March 2024 if determined successful, but did not contain well-defined success criteria. Farley revised the corrective actions to include appropriate measures of success.
4. Conclusion.
The inspectors concluded the corrective actions to preclude repetition of the root and contributing causes (causal factors) of the White performance issue were effective and adequately prioritized considering safety significance and regulatory compliance. In addition, the inspectors determined evaluations were documented at a sufficient level of detail, included relevant operating experience, and identified the root causes, extent of conditions, and extent of causes of the performance issue. Based on the results of the inspections, the inspectors concluded the objectives of the inspection procedure were met.
EXIT MEETINGS AND DEBRIEFS
The inspectors verified no proprietary information was retained or documented in this report.
- On December 14, 2023, the inspectors presented the 95001 supplemental inspection results to Edwin Dean III, Site Vice President, and other members of the licensee staff.
- At the conclusion of the exit meeting on December 14, 2023, Alan J. Blamey, Chief, Reactor Projects Branch 2, conducted a regulatory performance meeting with Edwin Dean III, Site Vice president, and other members of the licensee staff.
DOCUMENTS REVIEWED
Inspection
Procedure
Type
Designation
Description or Title
Revision or
Date
Condition Reports (CRs)
10951589, 10920885, 10557856, 10933059
Corrective Action
Reports (CARs)
275242, 275244, 275243, 406924, 275244, 275281, 481906
Corrective Action
Documents
Technical Evaluations
(TEs)
1083706, 1083711, 1093573, 1083714, 1136991, 1029102,
1031944, 1039576, 1039575, 1123337, 1136979, 1136975,
1136976, 1136977, 1105232, 1138201, 1131703
Corrective Action
Documents
Resulting from
Inspection
Condition Reports (CRs)
11031156, 11031451
F-ME-DL-501.019.02
Dynamic Learning Activity
N/A
95001 Tabletop Scenarios
Miscellaneous
U184852
Diesel Engine Generators 1-2A, 1B, and 2B Operations and
Maintenance Manual Volumes I, II and III
Ver. 64.0
NMP-AD-002
Conduct of Problem Solving and Troubleshooting
Ver. 13.7
NMP-AD-002-F04
Troubleshooting Log
NMP-AD-039
Preparation for NRC Supplemental Inspections
Ver. 3.0
NMP-GM-002
Corrective Action Program
Ver. 16.0
NMP-GM-002-001
Corrective Action Program Instructions
Ver. 47.0
NMP-GM-002-002
Effectiveness Review Instructions
Ver. 6.0
NMP-GM-002-GL03
Cause Analysis and Corrective Actions Guideline
Ver. 33.0
NMP-GM-003
Self-Assessment and Benchmark Procedure
Ver. 35.0
NMP-GM-006
Work Management
Ver. 21.1
NMP-GM-006-001
Minor Maintenance and Tool Pouch Work Procedure
Ver. 9.0
NMP-GM-020-001
Issue Response Team
Ver. 8.0
NMP-MA-050
Work Package Preparation
Ver. 13.0
NMP-MA-052
Repair/Replacement Program
Ver. 8.0
NMP-MA-055
Conduct of FIN
Ver. 2.2
Procedures
NMP-OS-003
Operational Decision Making Issue Evaluation Process
Ver. 10.2
95001
Work Orders
Work Orders (WOs)
1156372, 1447993, 1091597, 1449078, 1399361, 1480258,
1480257, 1480256, 1480255, 1482052, 1448735, 1449078,
1462848, 1462849, 1470864, 1478963