IR 05000395/2023040
ML23114A094 | |
Person / Time | |
---|---|
Site: | Summer |
Issue date: | 05/01/2023 |
From: | David Dumbacher NRC/RGN-II/DRP/RPB3 |
To: | Stoddard D Dominion Energy Co |
References | |
EA-22-039 IR 2023040 | |
Download: ML23114A094 (15) | |
Text
May 1, 2023
SUBJECT:
VIRGIL C. SUMMER NUCLEAR STATION - 95001 SUPPLEMENTAL INSPECTION SUPPLEMENTAL REPORT 05000395/2023040 AND FOLLOW-UP ASSESSMENT LETTER
Dear Mr. Daniel G. Stoddard:
On March 23, 2023, the U.S. Nuclear Regulatory Commission (NRC) completed a supplemental inspection using Inspection Procedure 95001, "Supplemental Inspection Response to Action Matrix Column 2 (Regulatory Response) Inputs, and discussed the results of this inspection with Mr. Robert Justice and other members of your staff.
The NRC performed this inspection to review your stations actions in response to a White finding in the Mitigating Systems cornerstone which was documented and finalized in NRC Inspection Report 05000395/2022091. On February 13, 2023, you informed the NRC that your station was ready for the supplemental inspection.
The NRC determined that your staffs evaluation identified the cause of the White finding.
Specifically, the root cause of the failure to identify and correct the oscillations on the B EDG was that the organization did not adequately challenge unanticipated kW oscillations observed during the 'B' EDG Surveillance Test Procedure (STP) run, instead rationalizing the occurrences to grid fluctuations attributed to winter weather and kW oscillations being previously documented in CR-22-00134, and therefore did not identify the unresolved condition adverse to quality in the Corrective Action Program (CAP).
The inspectors determined that the root cause evaluation was documented at a sufficient level of detail, included relevant operating experience, and identified the root causes, extent of conditions, and extent of causes of the performance issue. Based on the results of the inspection, the inspectors concluded the objectives of the IP were met.
The NRC determined that completed or planned corrective actions were sufficient to address the performance issue that led to the White finding. Therefore, this issue is closed on March 23, 2023, the date of the exit meeting. V.C. Summer will remain in the Regulatory Response Column of the NRCs Action Matrix due to an open White finding in the Security Cornerstone.
No findings or violations of more than minor significance were identified during this inspection. This letter, its enclosure, and your response (if any) will be made available for public inspection and copying at http://www.nrc.gov/reading-rm/adams.html and at the NRC Public Document Room in accordance with Title 10 of the Code of Federal Regulations 2.390, Public Inspections, Exemptions, Requests for Withholding.
Sincerely, David E. Dumbacher, Chief Reactor Projects Branch 3 Division of Reactor Projects Docket No. 05000395 License No. NPF-12
Enclosure:
As stated
Inspection Report
Docket Number: 05000395
License Number: NPF-12
Report Number: 05000395/2023040
Enterprise Identifier: I-2023-040-0002
Licensee: Dominion Energy
Facility: Virgil C. Summer Nuclear Station
Location: Jenkinsville, SC
Inspection Dates: March 20, 2023 to March 23, 2023
Inspectors: J. Bundy, Senior Operations Engineer M. Riley, Senior Project Engineer
Approved By: David E. Dumbacher, Chief Reactor Projects Branch 3 Division of Reactor Projects
Enclosure
SUMMARY
The U.S. Nuclear Regulatory Commission (NRC) continued monitoring the licensees performance by conducting a 95001 supplemental inspection at Virgil C. Summer Nuclear Station, in accordance with the Reactor Oversight Process. The Reactor Oversight Process is the NRCs program for overseeing the safe operation of commercial nuclear power reactors.
Refer to https://www.nrc.gov/reactors/operating/oversight.html for more information.
List of Findings and Violations
No findings or violations of more than minor significance were identified.
Additional Tracking Items
None.
INSPECTION SCOPES
Inspections were conducted using the appropriate portions of the inspection procedures (IPs) in effect at the beginning of the inspection unless otherwise noted. Currently approved IPs with their attached revision histories are located on the public website at http://www.nrc.gov/reading-rm/doc-collections/insp-manual/inspection-procedure/index.html. Samples were declared complete when the IP requirements most appropriate to the inspection activity were met consistent with Inspection Manual Chapter (IMC) 2515, Light-Water Reactor Inspection Program - Operations Phase. The inspectors reviewed selected procedures and records, observed activities, and interviewed personnel to assess licensee performance and compliance with Commission rules and regulations, license conditions, site procedures, and standards.
OTHER ACTIVITIES
- TEMPORARY INSTRUCTIONS, INFREQUENT AND ABNORMAL
95001 - Supplemental Inspection Response to Action Matrix Column 2 (Regulatory Response)
Inputs
The inspectors reviewed and selectively challenged aspects of the licensees problem identification, causal analysis, and corrective actions in response to a White finding and associated Notice of Violation (NOV) of 10 CFR Part 50, Appendix B, Criterion XVI, for the licensee's failure to identify and correct erratic governor behavior of the 'B' emergency diesel generator that caused inoperability of the diesel. Notification of the NRC's updated assessments were documented in inspection reports 05000395/2021001 and 05000395/2022091 (ADAMS Accession Nos. ML22132A192 and ML22287A184).
Supplemental Inspection Response to Action Matrix Column 2 (Regulatory Response) Inputs (1 Sample)
- (1) From March 20 - 23, 2023, the inspectors conducted an onsite review to verify all aspects of the inspection procedure were met.
INSPECTION RESULTS
Assessment 95001 1. Problem Identification
a. Identification. On January 10, 2022, the licensee removed the B emergency diesel generator (EDG) from service for maintenance. During post-maintenance testing on January 15, 2022, the diesel experienced erratic indications of frequency and kilowatt (kW) loading which were documented in CR-22-00134. Repairs were performed and a three-hour maintenance run was performed prior to the surveillance test on January 16, 2022, at approximately 0423. During the maintenance run, oscillations occurred again but were not identified because the licensee did not review the trend data. The licensee then performed the surveillance test on January 16, 2022, at approximately 2045. Indications of the surveillance test performed showed that the oscillations had not been corrected. Despite the condition, operators declared the EDG operable based on meeting the acceptance criteria of the surveillance test. The licensee performed their next B EDG surveillance test on February 9, 2022. During the surveillance test, kW oscillations were observed in the control room; however, all acceptance criteria were met, similar to the January 16th surveillance test. The diesel was declared operable and condition report (CR) 1191016 was generated to document the observed oscillations and to request an operability review from engineering. After a review of test data, engineering communicated to operations that they did not have reasonable assurance that the B EDG could perform its function due to the increasing number and magnitude of the kW oscillations. Operations then declared the B EDG inoperable on February 9, 2022, at 1518. The licensee determined that a broken pin within an Amphenol plug connector was interrupting the electronic governor output signal. After installation of a new Amphenol connector, the B EDG kW oscillations were eliminated, and the EDG successfully met the surveillance test acceptance criteria. The B EDG was declared operable on February 11, 2022, at 0120. The NRC identified a violation of 10 CFR Part 50, Appendix B, Criterion XVI, Corrective Action, for the licensees failure to identify and correct this condition adverse to quality resulting in the inoperability of the B EDG.
Exposure Time. The root cause evaluation (RCE) determined that the failure to identify and correct erratic frequency and loading oscillations on the B EDG resulted in inoperability of the diesel from January 16 to February 9, 2022. The inspectors determined that the licensee appropriately assessed the exposure time.
Identification opportunities. In general, the licensee appropriately considered prior occurrences and identification opportunities. RCE CA11316973 documented that missed opportunities existed to identify and correct EDG B oscillations prior to January 16, 2022. CR-22-00134 identified oscillations during a maintenance run on the B EDG January 15, 2022. A similar missed opportunity existed during a maintenance run on January 16, 2022, where trending data was not reviewed prior to performing the surveillance test. The licensee reviewed trending data of EDG surveillances prior to these instances in January 2022 and did not identify any instances of unanticipated oscillations on the diesels. The licensee also reviewed condition reports generated since 2020 which did not reveal any other instances of kW oscillations prior to January 2022.
Risk and Compliance. The RCE determined that the B EDG was declared inoperable and incapable of performing its intended safety function due to the failure to identify and correct the oscillations on the diesel. The RCE also documented the qualitative consequences of the issue with respect to radiological safety, industrial safety, challenging the unknown, adequate oversight, and adequate level of detail.
Based on their review, the inspectors concluded the RCE demonstrated an understanding of the significant plant consequences and compliance concerns associated with the event and the White performance issue. The NRCs risk evaluation of the White performance issue was documented in inspection report 05000395/2022091 (ML22287A184).
NRC Assessment. The inspectors determined that the licensee appropriately evaluated and documented problem identification, including adequate considerations of identification credit, how long the condition had existed, missed opportunities for self-identification, and risk insights. The inspectors had the following observation:
The RCE did not provide quantitative risk insights when assessing the safety consequences related to the loss of the B EDG.
2.Causal
Analysis.
a. Methodology. The RCE employed systematic evidence-based methodologies to determine the root cause and contributing causes of the White finding and EDG B inoperability including, Event and Causal Factors Analysis, Fishbone Chart Analysis, and Programmatic and Organizational Chart
Analysis.
Root
Cause.
The RCE determined that the root cause of the failure to identify and correct the oscillations on the B EDG was that the organization did not adequately challenge unanticipated kW oscillations observed during the 'B' EDG Surveillance Test Procedure (STP) run, instead rationalizing the occurrences to grid fluctuations attributed to winter weather and kW oscillations being previously documented in CR-22-00134, and therefore did not identify the unresolved condition adverse to quality in the corrective action program.
Contributing Causes. The RCE determined there were two contributing causes:
1. The organization failed to provide adequate oversight when emergent issues related to the B EDG arose outside of the planned maintenance work scope, which caused the maintenance window to extend beyond the normal work week (CC1).
2. The organization was not consistently providing adequate level of details for narrative log entries to ensure the organization is kept aware of events occurring, current equipment status, critical updates, and abnormal plant conditions. This would have provided an additional opportunity to challenge whether CRs are written for issues documented in narrative logs (CC2).
b. Level of Detail. The inspectors determined the RCE was performed commensurate with the safety significance and complexity of the performance issue and was of sufficient detail to identify the root and contributing causes, extent of conditions, and extent of causes. The RCE team utilized a formal cause analysis process to identify the problems and determine corrective actions.
c.
Operating Experience. The inspectors determined that the licensee appropriately considered prior occurrences and operating experience. For the RCE, the licensee considered prior occurrences and operating experience of events where failures to identify conditions adverse to quality were a result of inadequate challenges, inadequate oversight, and/or inadequate level of detail. The RCE determined that the differences between this condition and other similar events was that this event was attributed to human performance errors across multiple departments that led to the failure to identify the condition adverse to quality.
d. Extent of Condition and
Cause.
The inspectors determined that the licensee appropriately considered the extent of condition and extent of cause. The licensee used the same-same, same-similar, similar-same, and similar-similar methodologies to evaluate the extent of condition and the extent of cause. For the RCE, the licensees extent of condition evaluated whether employees and supplemental personnel failed to identify, evaluate, and correct conditions adverse to quality in the corrective action program. The licensee also reviewed a sample of CRs to verify non-conditions adverse to quality were properly categorized in the CAP. The licensees extent of cause evaluated whether operators and plant engineering inadequately challenged unanticipated kW oscillations during A and B EDG operations and emergent issues regarding other risk significant systems. There were no instances of inadequately challenging kW oscillations for the A EDG. The licensee also validated that test deficiencies were documented when unexpected conditions occur during surveillance testing.
e. Safety Culture. The inspectors reviewed the safety culture components referenced in NUREG-2165, Safety Culture Common Language, to determine if these were appropriately considered during the licensees evaluations of the root causes, extent of conditions, and extent of causes.
NRC Assessment: The inspectors review determined the licensees evaluations were documented at a sufficient level of detail, included relevant operating experience, and identified the root causes, contributing causes, extent of conditions, and extent of causes of the performance issue. Additionally, the inspectors determined the licensees RCE appropriately considered the safety culture aspects related to the failure to identify.
3. Corrective Actions.
a. Corrective Actions
to Prevent Recurrence (CAPR)
Completed
i. The licensee identified the following CAPRs for the root cause:
CAPR1: Revise EDG surveillance test procedures to include a review of kW/KVAR oscillations data and add a step to ensure unexpected oscillations are documented as a test deficiency.
CAPR2-CA1A: Communicate the requirements for documenting adverse conditions in accordance with licensee procedures through a station article.
CAPR2-CA1B: Communicate the nuclear safety culture trait of questioning attitude through a station article.
CAPR2-CA2A: Reinforce the expectations of the CAP procedure regarding the documentation of conditions adverse to quality in a condition Report and reinforce the expectations for behaviors of a questioning attitude.
2. Planned
i. To address the root cause, the following CAPRs are planned to be completed:
CAPR-CA2B: Document employee attendance at an All-Hands meeting reviewing the requirements of the CAP, specifically regarding documenting deviating/adverse conditions in a condition report, and the expectations for behaviors of questioning attitude in challenging the unknown.
CAPR2-CA3: Station leadership to perform observations over at least a 3-month period to confirm that staff are challenging the unknown, challenging assumptions, and documenting adverse conditions in a condition report.
CAPR2-CA4: Consider modifying and reimplementing the actions described in CAPR2-CA1A/B, CAPR2-CA2A/B, and CAPR2-CA3 if at least 10% of the observations are identified as needing improvement regarding challenging the unknown and documenting conditions adverse to quality in a Condition Report. Additionally, the licensee is considering sending site wide communications of examples of conditions that were not identified or documented properly.
b. Other Corrective Actions
1. Completed The licensee performed a performance gap analysis to determine why operations and engineering have not consistently provided detailed narrative logs, revised operations expectations and responsibilities, and developed communication to the organization about expectations and attributes to be utilized for at least a three-month observation period. The licensee also revised communication procedures to contact system control to validate grid conditions. Operations conducted performance gap analysis regarding operational decision making and consequence bias. Site Engineering performed performance gap analysis regarding consequence bias and technical conscience standards.
2. Planned Present RCE findings to the site through site-wide communications and discussions.
CC1: Outage & Planning (O&P) to communicate work management requirements in procedure WM-AA-3000 to operations and O&P departments.
CC1: Revise procedures SSP-007, SSP-010 and SSP-010 CWARG-01 to remove guidance that conflicts with WM-AA-3000.
CC2: Operations management to communicate conduct of operations requirements and reinforce standards for narrative logs.
CC2: Engineering Services managers to communicate the requirements of CM-AA-5001, Engineering Logs, and reinforce standards for narrative logs.
Extent of Cause (EOCa): Review narrative logs from chemistry, radiation protection, and NSS to determine if the departments are consistently providing adequate levels of detail in the logs.
ECOa: Provide expectations to maintenance for narrative log entries.
NRC Assessment: The inspectors concluded the dates for implementation and completion of the planned root and contributing cause corrective actions were reasonable, effective, and prioritized with consideration for risk significance and regulatory compliance. The inspectors also concluded the licensee established reasonable measures of success to evaluate the effectiveness of the corrective actions. When complete, the NRC plans to inspect and assess the planned corrective action to prevent recurrence identified in Section 3.a.2 above. The inspectors had the following observations:
CA11490869, Effectiveness review for CAPR-1, list several prescriptive effectiveness reviews such as: Operations personnel understand the purpose and content of the procedure revisions and Grid oscillations impacting diesel generator operations are validated by System Control. However, the criteria for evaluating the effectiveness of these goals were not well-defined.
The effectiveness measure of ensuring 95% attendance at the all-hands meeting described as acceptable in the RCE was not translated into the licensees CAP.
The revised EDG surveillance procedures include a bias toward EDG oscillations being induced by grid instability. The RCE noted that the staff rationalized the fluctuations due to grid instability, however, revisions to the surveillance procedures included the following precaution: If excessive, unexpected EDG KW/KVAR oscillations occur, a Test Deficiency should be generated and System Control should be contacted to determine if oscillations are related to grid conditions. Additionally, the inspectors noted that the term excessive was not clearly defined. Based upon discussions with the EDG system engineer, oscillations of 200kW from expected would be captured for review in the CAP.
When the CAPRs were considered and evaluated with the additional corrective actions, including corrective actions from the contributing causes, the inspectors determined that the actions were reasonable to address the root cause. The licensee placed the observations above in their corrective action program as CR 1222385.
4. Old Design Issue Evaluation. The inspectors did not evaluate the finding and associated violation for treatment as an Old Design Issue as it did not satisfy the criteria specified in IMC 0305 Section 11.05
5. Conclusion. The inspectors concluded the corrective actions to preclude repetition of the root and contributing causes (causal factors) of the White performance issue were effective and adequately prioritized considering safety significance and regulatory compliance. In addition, the inspectors determined evaluations were documented at a sufficient level of detail, included relevant operating experience, and identified the root causes, extent of conditions, and extent of causes of the performance issue. Based on the results of the inspections, the inspectors concluded the objectives of the inspection procedure were met.
EXIT MEETINGS AND DEBRIEFS
The inspectors verified no proprietary information was retained or documented in this report.
On March 23, 2023, the inspectors presented the 95001 supplemental inspection results to Mr. Robert Justice and other members of the licensee staff.
DOCUMENTS REVIEWED
Inspection Type Designation Description or Title Revision or
Procedure Date
95001 Corrective Action 11181302
Documents 1191016
200203
2106387
21620
21623
21636
21640
21661
22385
2-00134
2-00136
2-00139
2-00141
2-00151
2-00246
Corrective Action 1221967 Administrative Error on VCS TS Page 3/4 8-3 03/17/2023
Documents 1222312 CA11178458 Priority Incorrect in PAMS 03/22/2023
Resulting from 1222385 NRC Observations During 95001 Inspection 03/23/2023
Inspection
Engineering VC-22-00038 Emergency Diesel Generator Governor Amphenol Connector Rev. 1
Changes Strain Relief
Engineering EIR Number EDG Droop Setting Rev. 2
Evaluations 81431
Miscellaneous Diesel Generator Plant System Engineering Excel list of EDG System PMs 03/10/2023
System PM list
Email PSE email notes on _B_ DG loaded KW dips on 1-15-22
Communications
Email PSE Notes on _B_ DG Oscillations Investigations from 1-14-
Communications 2022 at 2345 to 1-16-22 at 0730
Email DG Engineer email reply to OPS STA on _B_ EDG
Communications performance and a few spikes, 1-17-2022
Inspection Type Designation Description or Title Revision or
Procedure Date
Level of Effort CR1191016 LEE FINAL COMPILED REV. 1.pdf 03/09/2023
Evaluation
Narrative Log OPS Narrative Logs from 1-14-2022 to 1-17-2022
Narrative Log Operations Narrative Logs from 2-8-2022 to 2-12-2022
Narrative Log Operations Narrative Logs from 3-12-2023 to 3-15-2023
Plant Computer B EDG running kW Data for 3-14-23
Data
RCE-Final Root Cause Evaluation Report: Failure to identify and 02/23/2023
CA11316973 correct a condition adverse to quality associated with B
EDG testing
VCSNS - DBD Design basis document diesel generator engine support and Rev. 15
control systems (DG)
Vendor Manual Colt Industries Operations and Maintenance Manual for VC Rev. 37
Summer Standby Diesel Generator Set
COLT-13-206152
Work Week Work Week 2202 B Diesel Generator Post Job Critique 01/20/2022
Critique
Procedures CM-AA-5001 Engineering Logs Rev. 5
MA-AA-103 Conduct of Troubleshooting Rev. 17
MMP-180.004 Emergency Diesel Generator Engine Governor and Controls Rev. 12
Maintenance
MMP-180.038 EDG A Periodic Maintenance Checks Rev. 7
MMP-180.045 EDG B Injection Pump Racks and Fuel Control Linkage Rev. 2
Inspection
MW-AA-3000 Managing Projects and Other Complex Work Rev. 1
OAP-100.4 Communications Rev. 4
OAP-100.6 Control Room Conduct and Control of Shift Activities Rev. 7
OAP-102.1 Conduct of Operations Scheduling Unit Rev. 9
OAP-106.1 Operating Rounds Rev.19
OP-AA-100 Conduct of Operations Rev. 43
OP-AA-102 Operability Determination Rev. 16
PI-AA-200 Corrective Action Program Rev. 39
PI-AA-300 Cause Evaluation Rev. 18
Inspection Type Designation Description or Title Revision or
Procedure Date
PI-AA-300-3000 Emergent Issue Response Rev. 6
PI-AA-300-3001 Root Cause Evaluation Rev. 14
SAP-0999 Corrective Action Program Rev. 19
SOP-306 Emergency Diesel Generator Rev. 20
STP-125.002B Diesel Generator B operability test Rev. 3
STP-125.004B Diesel Generator B Load Rejection Test Rev. 4
STP-125.008 Diesel Generator A 24 hour2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br /> load test Rev. 7
STP-125.009 Diesel Generator B 24 Hour Load Test Rev. 9
STP-125.010 Integrated Safeguards Test Train A Rev. 15
STP-125.013B Diesel Generator B Semiannual Operability Test Rev. 1
STP-125.017 Diesel Generator A Loss of Offsite Power Test Rev. 8
STP-125.018 Diesel Generator B Loss of Offsite Power Test Rev. 10
WM-AA-3000 Managing Projects and Other Complex Work Rev. 1
Work Orders 2011734-001
2011735-001
2011735-002
2111533-001
202845-002
202846-001
88101052415
88101052433
88101052856
88101052875
201646035
201647585
201648997
201667470
201675543
201676480
2