IR 05000416/2019040
ML19330G192 | |
Person / Time | |
---|---|
Site: | Grand Gulf ![]() |
Issue date: | 11/26/2019 |
From: | Anton Vegel NRC/RGN-IV/DRP |
To: | Emily Larson Entergy Operations |
References | |
IR 2019040 | |
Download: ML19330G192 (35) | |
Text
ber 26, 2019
SUBJECT:
GRAND GULF NUCLEAR STATION - NRC SUPPLEMENTAL INSPECTION REPORT 05000416/2019040 AND ASSESSMENT FOLLOW-UP LETTER
Dear Mr. Larson:
On October 16, 2019, the U.S. Nuclear Regulatory Commission (NRC) completed a supplemental inspection at your Grand Gulf Nuclear Station using Inspection Procedure 95001, Supplemental Inspection Response to Action Matrix Column 2 Inputs. On October 17, 2019, the NRC inspection team discussed the results of this inspection and the implementation of your corrective actions with you and other members of your staff. The results of this inspection are documented in the enclosed report.
The NRC performed this inspection to review your stations actions in response to White performance indicators (PIs) for Unplanned Scrams per 7,000 Critical Hours (Initiating Events Cornerstone) and Unplanned Power Changes per 7,000 Critical Hours (Initiating Events Cornerstone), which you reported for the third and fourth quarters of 2018.
On July 3, 2019, you informed the NRC that your station was ready for the supplemental inspection. This supplemental inspection was conducted to provide assurance that the root causes and contributing causes of the events resulting in the White performance indicators were understood, that the extent of condition and extent of cause were identified, and that the corrective actions addressed the causes and were sufficient to prevent recurrence. Additionally, the supplemental inspection was conducted to provide assurance that the collective (multiple)
White inputs were evaluated for common cause in order to consider the potential for programmatic weaknesses in your performance.
Your staffs evaluation identified two common root causes of the unplanned scrams and unplanned power changes. First, station leadership failed to ensure that equipment vulnerabilities were being adequately addressed. Second, the stations corrective action program was not effective in addressing conditions related to critical systems and components.
The stations corrective actions included additional oversight of engineering changes, observation and feedback requirements at key meetings to ensure leadership behaviors achieve desired outcomes, and actions to improve the quality of corrective action program processes.
The inspection team identified general weaknesses in cause evaluations and corrective actions, which were addressed by your staff during the inspection. As a result, these weaknesses were not considered significant. The inspection teams results and observations are included in the enclosed report.
The NRC determined that your actions to address the performance issues that led to the White performance indicators were sufficient to meet the objectives of Inspection Procedure 95001.
Additionally, the NRC determined that the Unplanned Power Changes per 7,000 Critical Hours performance indicator returned to Green in the fourth quarter of 2018, and the Unplanned Scrams per 7,000 Critical Hours performance indicator returned to Green in the third quarter of 2019. Based on the results of this inspection and our Action Matrix assessment, the NRC has determined that Grand Gulf will be transitioned to the Licensee Response Column (Column 1) of the NRCs Action Matrix as of the date of this letter. The next NRC Biennial Problem Identification and Resolution Inspection is currently scheduled for the first quarter of 2020, and it will provide the NRC the opportunity to follow up on the status of station actions to address gaps in the effectiveness of the corrective action program.
The NRC inspection team documented two findings of very low safety significance (Green) in this report. Neither of these findings involved a violation of NRC requirements.
If you disagree with a cross-cutting aspect assignment, or a finding not associated with a regulatory requirement in this report, you should provide a response within 30 days of the date of this inspection report, with the basis for your disagreement, to the U.S. Nuclear Regulatory Commission, ATTN: Document Control Desk, Washington, DC 20555-0001; with copies to the Regional Administrator, Region IV; and the NRC resident inspector at the Grand Gulf Nuclear Station.
In accordance with 10 CFR 2.390 of the NRCs Agency Rules of Practice and Procedure, a copy of this letter, its enclosure, and your response (if any) will be made available electronically for public inspection in the NRC Public Document Room and in the NRCs Agencywide Documents Access and Management System (ADAMS), accessible from the NRC Web site at http://www.nrc.gov/reading-rm/adams.html.
Please contact Ms. Patricia Vossmar at 817-200-1144 with any questions you have regarding this letter.
Sincerely,
/RA/
Anton Vegel, Director Division of Reactor Projects Docket No. 05000416 License No. NPF-29
Enclosure:
As stated
Inspection Report
Docket Number: 05000416 License Number: NPF-29 Report Number: 05000416/2019040 Enterprise Identifier: I-2019-040-0003 Licensee: Entergy Operations, Inc.
Facility: Grand Gulf Nuclear Station, Unit 1 Location: 7003 Baldhill Road Port Gibson, MS 39150 Inspection Dates: July 29 through October 16, 2019 Inspection Team: D. Bradley, Senior Resident Inspector R. Deese, Senior Reactor Analyst G. Kolcum, Senior Resident Inspector Approved By: Anton Vegel Director Division of Reactor Projects Enclosure
SUMMARY
The U.S. Nuclear Regulatory Commission (NRC) continued monitoring the licensees performance by conducting a follow-up supplemental inspection - Inspection Procedure 95001 at Grand Gulf Nuclear Station in accordance with the Reactor Oversight Process. The Reactor Oversight Process is the NRCs program for overseeing the safe operation of commercial nuclear power reactors. Refer to https://www.nrc.gov/reactors/operating/oversight.html for more information. Findings and violations being considered in the NRCs assessment are summarized in the table below.
List of Findings and Violations Failure to Identify and Correct Animal Intrusion Vulnerabilities in a Timely Manner Cornerstone Significance Cross-cutting Report Aspect Section Initiating Green [H.12] - Avoid 71153 Events FIN 05000416/2019040-01 Complacency Closed The inspection team reviewed a self-revealed, Green finding for the licensees failure to identify and correct animal intrusion into electrical component vulnerabilities in a timely manner. Specifically, the licensee experienced an unplanned downpower in December 2017 due to a loss of balance-of-plant transformer 23 from a bird shorting a phase to ground. The related cause evaluation did not adequately identify the cause per Procedure EN-LI-102,
Corrective Action Program, and corrective actions did not meet the timeliness standard of Procedure JA-PA-01, Analysis Manual. As a result, the licensee experienced a scram in May 2019 from the same direct cause when a bird caused a loss of balance-of-plant transformer 23 by shorting a phase to ground.
Failure to Identify and Correct Turbine Control System Issues in a Timely Manner Cornerstone Significance Cross-cutting Report Aspect Section Initiating Green [H.12] - Avoid 71153 Events FIN 05000414/2019040-02 Complacency Closed The inspection team reviewed a self-revealed, Green finding for the licensees failure to identify and correct turbine control system issues in a timely manner. Specifically, the licensee experienced both a scram and unplanned downpower in 2016 due to deficiencies associated with the turbine control system. The related corrective actions included plans for replacement of the turbine controls in 2020 with a new, digital system. These corrective actions, however, did not meet the timeliness standard of Procedure EN-LI-102, Corrective Action Program, and Procedure JA-PA-01, Analysis Manual, because the interim mitigating actions were insufficient. The associated corrective actions did not adequately reduce potential for recurrence or provide adequate mitigating actions to manage the severity of a subsequent event to an acceptable level. As a result, the licensee experienced a scram in January 2018 due to unexpected turbine control valve movements and a complicated scram in December 2018 from unexpected turbine bypass valve movement.
Additional Tracking Items Type Issue number Title Report Status Section LER 05000416/2018-009-01 Reactor Manual Shutdown Due 71153 - Closed to Feedwater Level Control Event Report Changes LER 05000416/2018-010-01 Reactor Manual Scram Due to 71153 - Closed Main Steam Bypass Stop and Event Report Control Valve A Drifting Open LER 05000416/2019-002-00 Manual Reactor Shutdown Due 71153 - Closed to Loss of Service Water Event Report
INSPECTION SCOPE
Inspections were conducted using the inspection procedure (IP) in effect at the beginning of the inspection unless otherwise noted. Currently approved IPs with their attached revision histories are located on the public website at http://www.nrc.gov/reading-rm/doc-collections/insp-manual/inspection-procedure/index.html. Documents reviewed by the inspection team are listed in the documents reviewed section of this report. The inspection team used the Commissions rules and regulations as the criteria for determining compliance along with established licensee standards as the criteria for assessing licensee performance.
OTHER ACTIVITIES
- TEMPORARY INSTRUCTIONS, INFREQUENT AND ABNORMAL
71153 - Follow-up of Events and Notices of Enforcement Discretion Event Report (IP Section 03.02)
The inspection team evaluated the following licensee event reports (LERs) which can be accessed at https://lersearch.inl.gov/LERSearchCriteria.aspx:
LER 05000416/2018-009-01, "Reactor Manual Shutdown due to Feedwater Level Control Changes" (ADAMS Accession ML19219B109). The circumstances surrounding this licensee event report are documented in the inspection results section of Inspection Report 05000416/2018003 (ADAMS Accession ML18317A240). This licensee event report is closed.
LER 05000416/2018-010-01, "Reactor Manual Scram due to Main Steam Bypass Stop and Control Valve A Drifting Open" (ADAMS Accession ML19192A062). The circumstances surrounding this licensee event report are documented in the inspection results section of this inspection report. This licensee event report is closed.
LER 05000416/2019-002-00, "Manual Reactor Shutdown due to Loss of Service Water" (ADAMS Accession ML19190A218). The circumstances surrounding this licensee event report are documented in the inspection results section of this inspection report. This licensee event report is closed.
95001 - Supplemental Inspection Response to Action Matrix Column 2 Inputs
.01. Inspection Scope
This inspection was conducted in accordance with NRC Inspection Procedure 95001, Supplemental Inspection Response to Action Matrix Column 2 Inputs, dated August 24, 2016, to assess the licensees evaluation of White performance indicators, which affected the Initiating Events Cornerstone in the reactor safety strategic performance area. The inspection objectives were to:
- Assure that the root causes and contributing causes of the significant performance issue are understood
- Independently assess and assure that the extent of condition and extent of cause of significant performance issues are identified
- Assure that corrective actions taken to address and preclude repetition of significant performance issues are prompt and effective
- Assure that corrective action plans direct prompt actions to effectively address and preclude repetition of significant performance issues Five scrams caused the Unplanned Scrams per 7,000 Critical Hours (Initiating Events Cornerstone) performance indicator to cross the Green-White threshold.
Seven downpowers caused the Unplanned Power Changes per 7,000 Critical Hours (Initiating Events Cornerstone) performance indicator to cross the Green-White threshold. In preparation for the inspection, the licensee performed two root cause evaluations to evaluate common-cause concerns between both unplanned downpowers and scrams. Additionally, 12 cause evaluations directly associated with the scrams and downpowers were performed by the licensee. These events and their direct causes are summarized in the list below.
1) First Scram (S1) January 2018: CR-GGN-2019-03081, Manual Shutdown Due to Unidentified Pressure Control Valve Swings, Revision 2 On January 30, 2018, the reactor operator noted unexpected turbine generator oscillations of 20 MWe and reactor power oscillations of approximately 2 percent peak-to-peak. The operators entered Off Normal Event Procedure ONEP 05-1-02-V-21, Pressure Control Malfunctions, and determined the plant parameters did not require an immediate reactor scram by procedure. The operators, however, manually scrammed the reactor after approximately 20 minutes since there was no guidance on sustained operations with low-level power oscillations. Subsequent licensee investigation determined the direct cause of the event to be an inappropriately high gain setting on a resonance compensator circuit for the turbine control valves.
This event was previously reviewed by NRC inspectors as documented in NRC Integrated Inspection Report 2018001 (ADAMS Accession ML18134A007), which included identification of a Green non-cited violation of NRC requirements. Licensee Event Report 05000416/2018-001, which is associated with this event, was previously reviewed by NRC inspectors as documented in NRC Integrated Inspection Report 2018002 (ADAMS Accession ML18215A026).
The 95001 inspection team independently reviewed the licensees actions to address this event per the inspection requirements and objectives in the 95001 inspection procedure.
2) Second Scram (S2) September 2018: CR-GGN-2018-10441, Manual [Reactor]
Scram From Heater Drain System, Revision 2 On September 14, 2018, during planned maintenance on a level transmitter for feedwater heater 6A, the licensee experienced a loss of condensate booster pump A and reactor feedwater pump A. Based on the loss of a reactor feedwater pump, the reactor recirculation pumps received an automatic runback signal and began reducing reactor power. The reactor operator then performed a manual scram of the reactor when the runback was complete since reactor water level continued to decrease. Subsequent licensee investigation determined the direct cause of the event to be that controllers for feedwater heater level control valves were not adequately (dynamically) tuned to control level during system perturbations.
Specifically, the planned maintenance on one of the level transmitters included opening equalization valves and backfilling sensing lines which led to oscillations from the other (controlling) level control circuit. These feedwater heater level oscillations increased in magnitude until the condensate booster pump tripped on low suction pressure from runout, which caused an automatic reactor feedwater pump trip and manual scram.
This event was previously reviewed by NRC inspectors as documented in NRC Integrated Inspection Report 2018003 (ADAMS Accession ML18317A240), which included identification of a Green finding.
The 95001 inspection team independently reviewed the licensees actions to address this event per the inspection requirements and objectives in the 95001 inspection procedure. Licensee Event Report 05000416/2018-009-01, which is associated with this event, is being closed with this inspection report.
3) Third Scram (S3) December 2018: CR-GGN-2018-13032, A Bypass Valve Drifting Open with Complicated Scram, Revision 2 On December 12, 2018, the reactor operator noted main turbine bypass valve A unexpectedly began opening as noted by a decrease in main generator output of 30 MWe and bypass valve position. The operators entered Off Normal Event Procedure ONEP 05-1-02-V21, Pressure Control Malfunctions, and contacted engineering and instrumentation and control staff. The turbine bypass valve began to slowly modulate between the normally shut position to less than 10 percent open over the next 90 minutes. During this time, the control room briefed reactor scram procedures, including contingencies for shutting main steam isolation valves (MSIVs)if the bypass valve were to remain open post-scram. The turbine bypass valve then opened at an increased rate to approximately 60 percent open, and operators inserted a manual reactor scram, including shutting the MSIVs. Subsequent licensee investigation determined the direct cause of the event was a failed linear variable differential transformer (LVDT) in the actuator for the train A turbine bypass control valve.
This event was previously reviewed by an NRC special inspection team due to questions about operator response to a complicated scram and issues with the reactor core isolation cooling system. The special inspection team inspection efforts, which began the week after the event occurred, were focused on the items listed in the associated charter located at the end of the NRC Special Inspection Report 05000416/2018050 (ADAMS Accession ML19088A335).
The 95001 inspection team reviewed the special inspection report and independently reviewed the licensees actions to address this event per the inspection requirements and objectives in the 95001 inspection procedure. Licensee Event Report 05000416/2018-010-01, which is associated with this event, is being closed in this report.
4) Fourth Scram (S4) February 2019: CR-GGN-2019-01504, Automatic Reactor Scram Due to Generator Lockout, Revision 2 On February 23, 2019, the main generator protective relay system actuated, causing a main generator lockout, turbine trip, and automatic reactor scram. Specifically, the main generator protective relays sensed a negative phase signal. Subsequent licensee investigation determined the direct cause of the event was an electric grid load imbalance that exceeded the setpoint for negative phase sequence.
This event was previously reviewed by NRC inspectors as documented in NRC Integrated Inspection Report 2019001 (ADAMS Accession ML19130A223).
Licensee Event Report 05000416/2019-001, which is associated with this event, was previously reviewed by NRC inspectors as documented in NRC Integrated Inspection Report 2019002 (ADAMS Accession ML19226A236), which included identification of a Green finding.
The 95001 inspection team independently reviewed the licensees actions to address this event per the inspection requirements and objectives in the 95001 inspection procedure.
5) Fifth Scram (S5) May 2019: CR-GGN-2019-03822, Loss of PSW Flow Reactor Manual Scram, Revision 0 On May 12, 2019, the balance-of-plant transformer 23 (BOP 23) lost power due to an overcurrent protection relay actuation, and four of eight radial well pumps lost power.
The control room operators entered Off Normal Event Procedure ONEP 04-1-01-P44-1, Plant Service Water/Radial Well System, and attempted to restore plant service water (PSW) flow with standby radial well pumps but were unsuccessful. Within 20 minutes, loads cooled by plant service water, such as turbine building cooling water and component cooling water, had a sustained an increasing trend in temperature, and a manual reactor scram was performed.
Subsequent licensee investigation determined the direct cause of the event was animal intrusion (bird) that grounded one phase of the overhead powerline to BOP 23.
This event was previously reviewed by NRC inspectors as documented in NRC Integrated Inspection Report 2019002 (ADAMS Accession ML19226A236).
The 95001 inspection team independently reviewed the licensees actions to address this event per the inspection requirements and objectives in the 95001 inspection procedure. Licensee Event Report 05000416/2019-002-00, which is associated with this event, is being closed with this report.
6) First Downpower (D1) November 2017: CR-GGN-2018-09889, Reactor Recirculation Pump B Mechanical Seal Failure, Revision 1 On November 15, 2017, the reactor recirculation pump B seal failed, causing an increase in unidentified drywell leakage and requiring a plant shutdown for repair.
The control room operators noted an increase in temperature and pressure for the recirculation pump B seal cavity and followed their operational decision-making issue process to shut down the plant for repairs. Subsequent licensee investigation determined the direct cause of the event was fracturing of the second stage mechanical seal from extended operation at low pressure, such as during forced outages.
7) Second Downpower (D2) December 2017: CR-GGN-2018-09890, BOP Transformer 23 Feeder Breaker Trip, Revision 1 On December 17, 2017, BOP 23 lost power due to an overcurrent protection relay actuation, and four of eight radial well pumps lost power. The control room operators entered Off Normal Event Procedure ONEP 04-1-01-P44-1, Plant Service Water/Radial Well System, and attempted to restore plant service water flow with standby radial well pumps. The operators were successful in stabilizing the plant after reducing reactor power from 68 percent to 47 percent. Subsequent licensee investigation determined the direct cause of the event was animal intrusion (bird) that grounded one phase of the overhead power line to BOP 23.
8) Third Downpower (D3) January 2018: CR-GGN-2018-00218, MSL A Vent Line Steam Leak, Revision 1 On January 8, 2018, control room operators investigated a step change in room temperature and an associated alarm for the condenser bay. Operations and radiation protection staff identified a steam leak that was unisolable from main steam line A. The licensee initiated a plant shutdown to stop the leak and complete repairs.
Subsequent licensee investigation determined the direct cause of the event was high cyclic fatigue due to mechanical resonance.
9) Fourth Downpower (D4) July 2018: CR-GGN-2018-08706, Reactor Recirculation Pumps Downshift to Slow Speed, Revision 1 On July 30, 2018, both reactor recirculation pumps unexpectedly downshifted from high speed to slow speed, and operations personnel stabilized the plant at 40 percent reactor power. Subsequent licensee investigation determined the direct cause of the event was a failure of the recirculation cavitation interlock circuitry that spuriously inserted a downpower command.
10) Fifth Downpower (D5) August 2018: CR-GGN-2018-09632, 6B High Pressure
[Feedwater] Heater [Level Control Valve] Failure, Revision 2 On August 21, 2018, the high pressure feedwater heater 6B drain valve failed open, which caused a high pressure heater isolation. Licensed operators stabilized the plant at a reduced reactor power of 71 percent. Subsequent licensee investigation determined the direct cause of the event was the failure of a positioner air relay for the level control valve actuator.
11) Sixth Downpower (D6) November 2018: CR-GGN-2018-12370, Condensate Pump A Vibration Resulting in Unplanned Downpower to Approximately 73%,
Revision 2 On November 16, 2018, indicated vibrations on condensate pump A suddenly increased to shutdown limits. Control room operators responded by lowering reactor power to 73 percent and securing the condensate pump. Subsequent licensee investigation determined the direct cause of the event was the failure of a power supply for the vibration monitoring system and not an actual high vibration condition.
12) Seventh Downpower (D7) April 2019: CR-GGN-2019-02908, Unplanned Downpower Due to Temporary Leak Repair Failure, Revision 0 On April 9, 2019, a steam leak on high pressure feedwater heater 6B was identified through an alarm for high temperatures in the room. Control room operators reduced power to 66 percent in order to secure the feedwater heater and allow for repairs.
Subsequent licensee investigation determined the direct cause of the event was a failed temporary leak repair from a previous leak in this location. The failure was due to relaxing of the sealant from cyclic pressure transients during downpowers and forced outages.
13) Common Cause Review:
(CC1) CR-GGN-2018-13042, White Performance Indicator Exceeded: Scrams per 7,000 Critical Hours, Revision 1 and (CC2) CR-GGN-2018-09645, White Performance Indicator Exceeded: Unplanned Power Changes per 7,000 Critical Hours, Revision 3 Root cause or apparent cause evaluations were performed for the five unplanned scrams and seven unplanned downpower events that input into the associated performance indicators. The licensee conducted a common cause analysis of these events to determine if commonality existed between the events, within and across performance indicators, and to identify any underlying causes of the trends of similar and repetitive events.
The licensees evaluation identified two common root causes of the White performance indicators. First, station leadership failed to align site standards to focus on resolution of equipment vulnerabilities that could result in downpowers or scrams. Specifically, leaders did not drive elimination of equipment vulnerabilities at the first available opportunity. Instead, oversight and reliability processes, such as Plant Health, tolerated single-point vulnerabilities and did not monitor changes in operating margin of systems. Second, the station did not properly classify, screen, and evaluate conditions in the corrective action program related to critical systems and components that could result in downpowers or scrams. Specifically, the station did not appropriately screen condition reports for trends, did not evaluate the conditions to an adequate depth, and did not resolve known equipment issues in a timely manner. The extent of condition was determined to be actual or potential conditions which could result in additional initiating events. This included a review of cause evaluations for the previous two years for equipment issues that could have resulted in a downpower or scram to ensure the corrective actions would correct identified deficiencies and the evaluation quality is adequate to identify underlying causes.
The stations short-term corrective actions included implementing risk classification for issues that can cause downpowers or scrams to ensure work is scheduled while providing bridging strategies, reviewing major pump control circuits for single-point vulnerabilities, reviewing operating experience (OE) and vendor communications for similar reactors for the previous 10 years, implementing a weekly site-wide communication strategy to reinforce personal roles and responsibilities, and implementing organizational behavior change plans for engineering and key committees, such as plant health.
The stations long-term corrective actions to prevent recurrence included providing additional oversight of engineering changes, including critical attribute lists and third-party reviews, observation and feedback requirements at key meetings to ensure leadership behaviors are achieving the desired outcomes, and behavioral change observations and feedback requirements at several stages of the corrective action program including screening, evaluation, and resolution. In addition, the NRC is also aware that the station has translated these long-term corrective actions and several other actions into a performance improvement action plan aimed at addressing these programmatic and common cause performance gaps across the station.
For each cause evaluation, the inspection team interviewed licensee personnel to determine whether the root and contributing causes were understood and whether corrective actions taken or planned were appropriate to address the causes and preclude repetition. The inspection team attended meetings associated with corrective actions to prevent recurrence and walked-down systems associated with the events.
Overall, the inspection team determined the licensees actions met the objectives of Inspection Procedure 95001. The following evaluation describes the inspection teams assessment of whether the licensees actions were adequate to meet the inspection requirements, including observations and weaknesses.
.02 Evaluation of the Inspection Requirements
.02.0 1 Problem Identification
a. Determine that the evaluation documented who identified the issues (i.e., licensee-identified, self-revealed, or NRC-identified) and under what conditions the issues were identified Each of the events described in Section
.01 of this report were the result of self-revealed
issues. The inspection team determined that the licensees evaluations documented who identified the issues and under what conditions the issues were identified.
b. Determine that the evaluation documented how long the issues existed and prior opportunities for identification The licensee documented when the issues originated, the circumstances in which each issue could have been previously identified, and documented the conditions, when applicable, involving similar events that had occurred at the station.
Of note, the licensee identified that animal intrusion vulnerabilities were documented in condition reports as early as 2003. Similarly, the licensee identified that the turbine control system was vulnerable to failures, including those due to obsolescence, in condition reports from 2010 and 2011. These condition reports were missed opportunities to correct material deficiencies related to the following scrams and downpowers: S1, S3, S5, and D2.
The inspection team determined that the licensees evaluations documented how long the issues existed and prior opportunities for identification.
c. Determine that the evaluation documented significant plant-specific consequences, as applicable, and compliance concerns associated with the issues The licensees evaluations included a plant-specific, risk-informed safety significance evaluation of the issues. In each evaluation, the licensee discussed the consequences of each event for nuclear safety, industrial safety, and radiological safety, including safety of the public. The inspection team concluded that the licensee appropriately documented the risk consequences and compliance concerns associated with each issue.
d. Conclusions
No weaknesses were identified by the inspection team in the area of problem identification.
.02.0 2 Root Cause, Extent of Condition, and Extent of Cause Evaluation
a. Determine that the problem was evaluated using a systematic methodology to identify the root and contributing causes The inspection team determined that the licensees root cause evaluations (RCEs)employed a combination of the following evaluation techniques: Event and Causal Factor Charting, Barrier Analysis, Common Cause Analysis, Organizational and Programmatic Analysis, Comparative Time Line, Why Staircase, and Equipment Failure Modes
Analysis.
A general weakness associated with this requirement was identified by the inspection team and is further discussed in Section 02.02.g of this report.
The inspection team determined that, in general, the licensee selected appropriate analysis methods to ensure thorough and complete evaluations.
b. Determine that the root cause evaluation was conducted to a level of detail commensurate with the significance of the problem The licensees RCEs included enough information for each event, such as timelines, descriptions, previous occurrences, missed opportunities, and analysis of consequences. Each RCE used multiple evaluation methodologies, as discussed in Section 02.02.a, to ensure the level of detail matched the significance of each event.
The inspection team determined that the RCEs were conducted to a level of detail commensurate with the significance of the problems discussed.
c. Determine that the root cause evaluation included a consideration of prior occurrences of the problem and knowledge of prior OE The licensees RCEs included a review of internal and external OE.
The licensee identified that they were not fully participating in industry working groups.
Under Corrective Action 73 in Condition Report CR-GGN-2018-09645 for common cause CC2, the licensee lists 15 working groups they intend to establish full participation in, such as the diesel owners group and the Terry turbine users group. This represents a missed opportunity to incorporate OE across scrams and downpowers identified by the licensees cause evaluation work.
The inspection team noted that common cause review CC2 under Condition Report CR-GGN-2018-09645, White Performance Indicator Exceeded: Unplanned Power Changes per 7,000 Critical Hours, discussed OE for scrams. The team questioned the lack of discussion of OE for unplanned downpowers. The licensee clarified that the unplanned downpowers OE search results were located within the individual cause evaluations for each unplanned downpower and that no additional items were identified that warranted entry in Condition Report CR-GGN-2018-09645.
The inspection team determined that root cause evaluations included a consideration of prior occurrences of the problem and knowledge of prior OE.
d. Determine that the root cause evaluation addressed the extent of condition and the extent of cause of the problem A general weakness associated with the licensees evaluation of extent of condition and extent of cause was identified by the inspection team and is further discussed in Section 02.02.g of this report.
The inspection team determined, in general, that the root cause evaluations addressed the extent of condition and extent of cause of the problem.
e. Determine that the root cause, extent of condition, and extent of cause evaluation appropriately considered the safety culture traits in NUREG-2165, Safety Culture Common Language, referenced in Inspection Manual Chapter (IMC) 0310, Aspects within Cross-Cutting Areas The licensees root cause evaluations included a review of safety culture components and how they may have contributed to the issues identified. The licensees evaluations identified weaknesses in safety culture components that were related to the identified root causes and contributing causes. The licensee established adequate corrective actions to address the safety culture weaknesses that were identified. The inspection team concluded that the licensees evaluations appropriately considered safety culture traits in the root causes, extent of condition, and extent of cause evaluations.
f. Examine the common cause analyses for potential programmatic weaknesses in performance when a licensee has a second White input in the same cornerstone The licensee conducted common cause analysis of the scrams and unplanned downpowers to determine if commonality existed between the events, within and across performance indicators, and to identify any underlying causes of trends of similar or repetitive events. The licensee identified several programmatic issues across the performance indicators in the areas of the corrective action program, use of operating experience, and managing risk. For example, bridging or mitigating strategies for known equipment deficiencies were not adequately developed, prioritization of corrective actions did not always include risk ranking, programs related to vibration monitoring were not fully developed, operating experience was not consistently used to identify potential failure modes, and known deficiencies were not prioritized for corrective action given the possibility of the condition worsening. The inspection team concluded the licensee appropriately identified and evaluated programmatic weaknesses for a second White input in the same cornerstone.
g. Conclusions
The NRC determined the station identified the two common root causes of the performance issues that led to the White performance indicators. First, station leadership failed to align site standards to focus on resolution of equipment vulnerabilities that could result in downpowers or scrams. Specifically, leaders did not drive elimination of equipment vulnerabilities at the first available opportunity. Instead, oversight and reliability processes, such as Plant Health, tolerated single-point vulnerabilities and did not monitor changes in operating margin of systems. Second, the station did not properly classify, screen, and evaluate conditions in the corrective action program related to critical systems and components that could result in downpowers or scrams. Specifically, condition reports were not appropriately screened for trends, did not evaluate the conditions to an adequate depth, and did not resolve known equipment issues in a timely manner. The extent of condition was determined to be actual or potential conditions which could result in additional initiating events. This included a review of cause evaluations for the previous two years for equipment issues that could have resulted in a downpower or scram to ensure the corrective actions would correct identified deficiencies and the evaluation quality is adequate to identify underlying causes.
Within the individual event root causes, the team identified General Weakness No. 1 associated with the root cause and corrective actions for scram S3 under Condition Report CR-GGN-2018-13032, A Bypass Valve Drifting Open with Complicated Scram.
Similarly, the team identified General Weakness No. 2 associated with the extent of condition/cause and corrective actions for downpower D3 under Condition Report CR-GGN-2018-00218, MSL A Vent Line Steam Leak. These observations are documented in the inspection results section of this report.
In addition, the NRC determined that both the licensee and the NRC had identified several programmatic issues affecting licensee performance. These programmatic issues, licensee management oversight gaps, and other deficiencies have resulted in several long-term actions to address performance at the station. The NRC determined that these programmatic and leadership gaps were properly identified in the licensees causal evaluations.
.02.0 3 Corrective Actions Taken
a. Determine that appropriate corrective actions are specified for each root and contributing cause or that the licensee has an adequate evaluation for why no corrective actions are necessary The licensees root cause evaluations (RCEs) identified corrective actions to address root and contributing causes. The inspection team reviewed the corrective actions and determined they adequately addressed the identified root and contributing causes.
b. Determine that the corrective actions have been prioritized with consideration of significance and regulatory compliance The inspection team noted that the licensee self-identified weaknesses in their corrective action program as defined in the second root cause associated with the common cause evaluations. Specifically, the station previously did not properly classify, screen, and evaluate conditions in the corrective action program related to critical systems and components that could result in downpowers or scrams.
The inspection team reviewed the prioritization of the corrective actions and concluded that, in general, the licensee adequately prioritized the corrective actions with consideration of the risk significance and regulatory compliance.
c. Determine that corrective actions taken to address and preclude repetition of significant performance issues are prompt and effective The licensees RCEs identified several corrective actions to preclude repetition (CAPRs).
A general weakness associated with corrective actions was identified by the inspection team, which is further discussed in Section 02.03.e of this report.
The inspection team determined that, in general, the CAPRs were adequate to address the adverse conditions and were considered prompt and effective.
d. Determine that each Notice of Violation related to the supplemental inspection is adequately addressed, either in corrective actions taken or planned The NRC staff did not issue a Notice of Violation related to this supplemental inspection to the licensee; therefore, this inspection item was not applicable.
e. Conclusions
As previously discussed in Section 02.02.g of this report, General Weaknesses No. 1 and No. 2 involved corrective action aspects for scram S3 and downpower D3. The team identified General Weakness No. 3 associated with corrective actions for scram S5 under Condition Report CR-GGN-2019-03822, Loss of PSW Flow Reactor Manual Scram. These observations are documented in the inspection results section of this report.
Overall, the team concluded that the licensees corrective actions were adequate to address the performance issues that led to the White performance indicators.
.02.0 4 Corrective Action Plans
a. Determine that appropriate corrective action plans are specified for each root and contributing cause or that the licensee has an adequate evaluation for why no corrective actions are necessary. Determine that the corrective action plans have been prioritized with consideration of significance and regulatory compliance.
The inspection team determined that the licensees corrective action plans adequately addressed each of the root and contributing causes in the root cause evaluations.
Further, the inspection team determined that, in general, the corrective actions plans appropriately prioritized significant issues and regulatory compliance.
b. Determine that corrective plans direct prompt actions to effectively address and preclude repetition of significant performance issues A general weakness associated with corrective action plans was identified by the inspection team, which is further discussed in Section 02.04.e of this report.
The inspection team determined that, in general, the corrective action plans directed prompt actions to effectively address and preclude repetition of significant performance issues.
c. Determine that appropriate quantitative or qualitative measures of success have been developed for determining the effectiveness of planned and completed corrective actions The inspection team determined that the licensee developed effectiveness review plans for the associated CAPRs and root cause evaluations. These plans included quantitative and qualitative measures of success to determine the effectiveness of the corrective actions to prevent recurrence. The team determined that these actions were appropriate and adequate to determine the effectiveness of planned and completed corrective actions.
d. Determine that each Notice of Violation related to the supplemental inspection is adequately addressed in corrective actions taken or planned The NRC staff did not issue a Notice of Violation related to this supplemental inspection to the licensee; therefore, this inspection item was not applicable.
e. Conclusions
For the common cause evaluations, the stations short-term corrective actions included implementing risk classification for issues that could cause downpowers or scrams to ensure work is scheduled while providing bridging strategies, reviewing major pump control circuits for single-point vulnerabilities, reviewing OE and vendor communications for similar reactors for the previous 10 years, implementing a weekly site-wide communication strategy to reinforce personal roles and responsibilities, and implementing organizational behavior change plans for engineering and key committees, such as plant health.
Similarly, the corrective actions to prevent recurrence included providing additional oversight of engineering changes including critical attribute lists and third-party reviews, observation and feedback requirements at key meetings to ensure leadership behaviors and desired outcomes, and behavioral change observations and feedback requirements at several stages of the corrective action program including screening, evaluation, and resolution.
As previously discussed in Section 02.02.g of this report, General Weaknesses No. 1 and No. 2 involved corrective action aspects for scram S3 and downpower D3.
The team identified General Weakness No. 4 associated with corrective action plans for scram S4 under Condition Report CR-GGN-2019-01504, Automatic Reactor Scram Due to Generator Lockout, common cause evaluation CC1 under Condition Report CR-GGN-2018-13042, White Performance Indicator Exceeded: Scrams per 7,000 Critical Hours, and CC2 under Condition Report CR-GGN-2018-09645, White Performance Indicator Exceeded: Unplanned Power Changes per 7,000 Critical Hours.
These observations are documented in the inspection results section of this report.
The NRC is also aware that the station has translated these long-term corrective actions and several other actions into a performance improvement action plan aimed at addressing the programmatic and common cause performance gaps across the station.
Overall, the team assessed the sustainability and planned effectiveness reviews including concerns documented in General Weaknesses #1 and #4. The inspection team determined, in general, that the CAPRs were reasonable to prevent recurrence, were sustainable, and had adequate effectiveness reviews including interim reviews.
.02.0 5 Evaluation of Inspection Manual Chapter 0305 Criteria for Treatment of Old Design
Issues The licensee did not request credit for self-identification of an old design issue; therefore, the risk-significant issues were not evaluated against Inspection Manual Chapter 0305, Operating Reactor Assessment Program, criteria for treatment of an old design issue.
INSPECTION RESULTS
Failure to Identify and Correct Animal Intrusion Vulnerabilities in a Timely Manner Cornerstone Significance Cross-cutting Report Aspect Section Initiating Green [H.12] - 71153 Events FIN 05000416/2019040-01 Avoid Open/Closed Complacency The inspection team reviewed a self-revealed, Green finding for the licensees failure to identify and correct animal intrusion into electrical component vulnerabilities in a timely manner. Specifically, the licensee experienced an unplanned downpower in December 2017 due to a loss of balance-of-plant transformer 23 from a bird shorting a phase to ground. The related cause evaluation did not adequately identify the cause per Procedure EN-LI-102, Corrective Action Program, and corrective actions did not meet the timeliness standard of Procedure JA-PA-01, Analysis Manual. As a result, the licensee experienced a scram in May 2019 from the same direct cause when a bird caused a loss of balance-of-plant transformer 23 by shorting a phase to ground.
Description:
On May 12, 2019, the licensee experienced a partial loss of plant service water due to a loss of balance-of-plant transformer 23 (BOP 23). This loss reduced the capacity of the plant service water system. Operators responded to the associated alarms by entering Off Normal Event Procedure ONEP 04-I-01-P44-1, Plant Service Water/Radial Well System. Although operators attempted to restore plant service water flow with the remaining pumps, they were unable to adequately increase flow due to the latent degraded capacity of associated radial well pumps. Within approximately 20 minutes, operators determined that the temperatures in systems cooled by plant service water, such as the turbine building cooling water and component cooling water systems, were likely to exceed their control band with a constant increasing trend. At this time, the reactor operator manually scrammed the reactor and initiated safety-related standby service water to ensure cooling to engineered safety features.
As documented in the associated LER 05000416/2019-002-00, "Manual Reactor Shutdown due to Loss of Service Water," (ADAMS Accession ML19190A218), the partial loss of plant service water resulted in a lockout of drywell chillers and the drywell temperature briefly exceeded the technical specification limit of 135°F. The licensee stabilized the plant in Mode 3 and initiated Condition Report CR-GGN-2019-03822 to document the event.
The licensee performed walkdowns and identified that that BOP 23 was shorted by a bird providing a ground path from an energized line to the power pole. Specifically, the power pole contained a metallic mesh around sections of the pole that provided the ground path when the bird came in simultaneous contact with the phase line. The licensee formed a cause evaluation team and reviewed the similarities of this event to a downpower on December 17, 2017, where a bird shorted BOP 23 in a similar manner.
Ultimately, the licensee determined that they failed to fully investigate the cause of the December 2017 unplanned downpower. As a result, the licensee experienced a repeat failure from the same direct cause leading to the May 2019 scram. The extent of condition review identified a total of eight animal intrusion issues since 1997 on BOP 23 alone.
The inspection team, in parallel and independent of the licensees efforts, performed a review of condition reports associated with animal intrusion and associated procedural requirements for corrective actions. The inspection team noted that Procedure EN-LI-102, Corrective Action Program, Revision 36, requires:
Corrective action plans should . . . address the causes/factors identified by the evaluation/analysis . . . The corrective action content should be specific, measurable, achievable, realistic, and timely (SMART). Further guidance on each of these criteria can be found in JA-PI-01.
Licensee Procedure JA-PI-01, Analysis Manual, Revision 8, further defines SMART corrective actions including timeliness:
Timely: Due date is established to account for risk and time needed to implement with quality. Actions need to have a specific time for completion that is both realistic . . . and commensurate with the risk associated with delays in implementing necessary changes. In general, actions are implemented before the next reasonable opportunity for an event to occur. If opportunities for recurrence are identified, then interim or compensatory actions should be completed to reduce the potential for recurrence and/or mitigate the severity of recurrence to an acceptable level.
The inspection team noted that identifying and correcting earlier issues associated with animal intrusion vulnerabilities in a timely manner could have provided interim actions between the December 2017 unplanned downpower and the May 2019 scram.
The inspection team concluded the licensee failed to identify and correct animal intrusion into electrical component vulnerabilities in a timely manner. Specifically, the licensee experienced an unplanned downpower in December 2017 due to a loss of BOP 23 from a bird shorting a phase to ground. The related cause evaluation did not adequately identify the cause per Procedure EN-LI-102, Corrective Action Program, and corrective actions did not meet the timeliness standard of Procedure JA-PA-01, Analysis Manual. As a result, the licensee experienced a scram in May 2019 from the same direct cause when a bird caused a loss of BOP 23 by shorting a phase to ground.
Corrective Actions: The licensee stabilized the plant in Mode 3, performed walkdowns, removed metal mesh on power poles that provided a grounding pathway, and entered issues into the corrective action program.
Corrective Action Reference: Condition Report CR-GGN-2019-03822.
Performance Assessment:
Performance Deficiency: The failure to identify and correct animal intrusion into electrical component vulnerabilities in a timely manner was a performance deficiency.
Screening: The inspection team determined the performance deficiency was more than minor, and therefore a finding, because it was associated with the Equipment Performance attribute of the Initiating Events Cornerstone and adversely affected the cornerstone objective to limit the likelihood of events that upset plant stability and challenge critical safety functions during shutdown as well as power operations. Specifically, the licensee experienced an unplanned downpower in December 2017 due to a loss of BOP 23 from a bird shorting a phase to ground. The related cause evaluation did not adequately identify the cause per Procedure EN-LI-102, Corrective Action Program, and corrective actions did not meet the timeliness standard of Procedure JA-PA-01, Analysis Manual. As a result, the licensee experienced a scram in May 2019 from the same direct cause when a bird caused a loss of BOP 23 by shorting a phase to ground.
Significance: Using Inspection Manual Chapter 0609, Attachment 4, Initial Characterization of Findings, and Appendix A, The Significance Determination Process (SDP) for Findings At-Power, Exhibit 1, Initiating Event Screening Questions, dated June 19, 2012, the inspection team determined that the finding required a detailed risk evaluation because the finding involved the partial loss of a support system, which caused an initiating event and affected mitigation equipment. Specifically, the loss of BOP 23 caused a partial loss of plant service water and affected interfacing cooling systems such as component cooling water and turbine building cooling water.
A regional senior reactor analyst performed a detailed risk evaluation and determined that the finding associated with the partial loss of plant service water event was of very low safety significance (Green). An initiating event analysis as called for in Section 8.0, Initiating Event Analyses, of Volume 1, Internal Events, of the RASP Handbook was performed by modeling this event as a general plant transient (scram) with a loss of power to half of the radial well pumps from the plant service water system along with a remaining degraded radial well pump.
The plant service water system uses eight radial well pumps to supply cooling water to various components in the plant. Four of these pumps are powered from load center 28AG and four are powered from load center 18AG. Since load center 28AG was lost in the event, the analyst added logic to better represent loss of load center power events. Under sub-fault trees for each of the individual pumps (e.g., sub-fault tree PSW-PMPS-1J, Plant Service Water MDP 1J Fails to Operate) the analyst added two events. The first event added under each sub-fault tree was either basic event ACP-BAC-LP-18AG, Loss of Power to Load Center 18AG, or ACP-BAC-LP-28AG, Loss of Power to Load Center 28AG, which modeled a loss of power to the specific pump due to loss of its specific load center. These two load center failure events were constructed using template event ZT-BAC-LP, AC Bus Fails to Operate. The second added event under every individual pumps sub-fault tree was a common cause loss of power event constructed using the events ACP-BAC-LP-18AG and ACP-BAC-LP-28AG and generic rate common cause alpha factors ZA-CCF-RATE-02A01 and ZA-CCF-RATE-02A02, Alpha Factor 1 in Group Size 2 for component CCF with Failure Mode Rate, and Alpha Factor 2 in Group Size 2 for component CCF with Failure Mode Rate. This common cause event modeled the probability of a loss of power to both pertinent load centers.
To model the event, a loss of power to half of the radial well pumps, the analyst set basic event ACP-BAC-LP-28AG to TRUE which logically made four radial well pumps non-functional and increased the common cause failure probability due to a loss of power to the other four radials well pumps to 3.06E-2 based on the proximate cause. Also, before the event radial well pump C had a degraded suction source and had been considered only partially capable. To model this component issue, the analyst set basic event PSW-MDP-TM-1C, Plant Service Water Pump 1C Is In Test or Maintenance, to TRUE.
These assumptions resulted in an increase in core damage frequency of 4.4E-7/year for the finding. The analyst ran an uncertainties analysis on the results of the SPAR model in SAPHIRE. Of the 5000 cases run in a Monte Carlo analysis, approximately 90 percent of the results were less than 1.0E-6/year, giving confidence that the finding was of very low safety significance (Green). Plant transients comprised the dominant core damage sequences. The reactor core isolation cooling and high pressure core spray systems remained available for mitigation of the dominant sequences. The analyst ran the Grand Gulf SPAR model, Revision 8.59, on SAPHIRE, Version 8.2.0, to calculate the conditional core damage probability using a cutset truncation of 1.0E-12.
The analyst assumed that external events would be an insignificant contributor to the increase in core damage frequency because the probability of any external event coinciding with the partial loss of service water event would be extremely low. As a result, only the increase in core damage frequency from the initiating event was used in the final estimate.
After reviewing Inspection Manual Chapter 0609, Appendix H, Containment Integrity Significance Determination Process, the analyst determined that the dominant sequences (high reactor coolant system pressure sequences) for a BWR Mark III containment warranted application of a large early release frequency factor of 0.2 per Table 6.1, Phase 1 Screening - Type A Findings at Full Power. Application of this factor yielded an increase in large early release frequency of 8.8E-8/year for the finding and therefore the finding was of very low safety significance (Green) for large early release frequency.
Cross-cutting Aspect: The finding had a cross-cutting aspect in the area of human performance associated with avoiding complacency because the licensee failed to recognize and plan for the possibility of mistakes, latent issues, and inherent risk, even while expecting successful outcomes including implementing appropriate error reduction tools [H.12].
Specifically, the licensee failed to recognize the risk of degraded radial well capacity combined with a temporary modification for power to the plant service water pumps and did not plan for the possibility or latent risk of additional animal intrusion events causing a plant transient.
Enforcement:
The inspection team did not identify a violation of regulatory requirements associated with this finding.
Failure to Identify and Correct Turbine Control System Issues in a Timely Manner Cornerstone Significance Cross-cutting Report Aspect Section Initiating Green [H.12] - 71153 Events FIN 05000416/2019040-02 Avoid Open/Closed Complacency The inspection team reviewed a self-revealed, Green finding for the licensees failure to identify and correct turbine control system issues in a timely manner. Specifically, the licensee experienced both a scram and unplanned downpower in 2016 due to deficiencies associated with the turbine control system. The related corrective actions included plans for replacement of the turbine controls in 2020 with a new, digital system. These corrective actions, however, did not meet the timeliness standard of Procedure EN-LI-102, Corrective Action Program, and Procedure JA-PA-01, Analysis Manual, because the interim mitigating actions were insufficient. The associated corrective actions did not adequately reduce potential for recurrence or provide adequate mitigating actions to manage the severity of a subsequent event to an acceptable level. As a result, the licensee experienced a scram in January 2018 due to unexpected turbine control valve movements and a complicated scram in December 2018 from unexpected turbine bypass valve movement.
Description:
On January 30, 2018, the licensee experienced unexpected turbine control valve movement resulting in main generator power oscillations of approximately 20 MWe. Licensed operators responded by entering the Off Normal Event Procedure ONEP 05-1-02-V-21, Pressure Control Malfunctions, and manually scrammed the reactor due to the unexpected power oscillations. This event is documented in the associated Licensee Event Report 05000416/2018-001-01, "Reactor Manual Shutdown due to Turbine Pressure Control Valve Position Changes" (ADAMS Accession ML19219B328). The licensee stabilized the plant in Mode 3 and initiated Condition Report CR-GGN-2018-00918. This condition report was later closed to Condition Report CR-GGN-2019-03081.
The licensee determined the cause of the oscillating power to be improper settings for gain associated with the turbine control valve resonance compensator circuit due to errors in the associated calibration procedure.
On December 12, 2018, the train A main turbine bypass valve unexpectedly began opening as noted by a decrease in main generator output of 30 MWe and bypass valve position indications observed by reactor operators. The operators entered Off Normal Event Procedure ONEP 05-1-02-V-21, Pressure Control Malfunctions, and contacted engineering and instrumentation and control staff. The turbine bypass valve began to slowly modulate between the normally shut position to less than 10 percent open over the next 90 minutes.
During this time, the control room operators briefed reactor scram procedures, including contingencies for shutting main steam isolation valves (MSIVs) if the bypass valve were to remain open post-scram. The turbine bypass valve then opened at an increased rate to 60 percent open, and operators inserted a manual reactor scram including shutting MSIVs.
The licensee stabilized the plant after the complicated reactor scram and initiated Condition Report CR-GGN-2018-13032.
Subsequent licensee investigation determined the direct cause of the event was a failed LVDT in the actuator for the train A turbine bypass control valve.
The licensee reviewed these scrams as part of the common cause evaluation for the 95001 inspection. The licensee identified a scram on June 25, 2016, and an unplanned downpower on July 9, 2016, that had also resulted from issues associated with the turbine control system. The related corrective actions under Condition Report CR-GGN-2016-04998 included plans for replacement of the turbine controls in 2020 with a digital system, with the goal of eliminating vulnerabilities.
Ultimately, the licensee determined that the station did not prioritize activities to eliminate critical component risk or develop bridging/mitigating strategies until the critical component risk could be eliminated, including the turbine control system.
The inspection team, in parallel and independent of the licensees efforts, performed a review of condition reports associated with the turbine control system and associated procedural requirements for corrective actions. The inspection team noted that Procedure EN-LI-102, Corrective Action Program, Revision 36, requires:
Corrective action plans should . . . address the causes/factors identified by the evaluation/analysis . . . The corrective action content should be specific, measurable, achievable, realistic, and timely (SMART). Further guidance on each of these criteria can be found in JA-PI-01.
Licensee Procedure JA-PI-01, Analysis Manual, Revision 8, further defines SMART corrective actions including timeliness:
Timely: Due date is established to account for risk and time needed to implement with quality. Actions need to have a specific time for completion that is both realistic . . . and commensurate with the risk associated with delays in implementing necessary changes. In general, actions are implemented before the next reasonable opportunity for an event to occur. If opportunities for recurrence are identified, then interim or compensatory actions should be completed to reduce the potential for recurrence and/or mitigate the severity of recurrence to an acceptable level.
The inspection team noted that identifying and correcting issues associated with the turbine control systems vulnerabilities in a timely manner could have provided mitigating actions between the 2016 turbine control events and the 2018 scrams. Specifically, the licensee could have provided additional reviews on control circuit maintenance procedures and off-normal event procedures that may have helped reduce the potential or mitigate the severity of additional turbine control system malfunctions.
The inspection team concluded the licensee failed to identify and correct turbine control system issues in a timely manner. Specifically, the licensee experienced both a scram and unplanned downpower in 2016 due to issues associated with the turbine control system. The related corrective actions included replacement of the turbine controls in 2020 to a new, digital system. These corrective actions, however, did not meet the timeliness standard of Procedure EN-LI-102, Corrective Action Program, and Procedure JA-PA-01, Analysis Manual, because the interim mitigating actions were insufficient. The associated corrective actions did not adequately reduce potential for recurrence or provide adequate mitigating actions to manage the severity of a subsequent event to an acceptable level. As a result, the licensee experienced a scram in January 2018 due to unexpected turbine control valve movements and a complicated scram in December 2018 from unexpected turbine bypass valve movement.
Corrective Actions: The licensee stabilized the plant and entered issues into the corrective action program.
Corrective Action References: Condition Reports CR-GGN-2018-00918, CR-GGN-2018-13032, and CR-GGN-2019-03801.
Performance Assessment:
Performance Deficiency: The failure to identify and correct issues associated with the turbine control system in a timely manner was a performance deficiency.
Screening: The inspection team determined the performance deficiency was more than minor, and therefore a finding, because it was associated with the Equipment Performance attribute of the Initiating Events Cornerstone objective and adversely affected the cornerstone objective to limit the likelihood of events that upset plant stability and challenge critical safety functions during shutdown as well as power operations. Specifically, the licensee experienced both a scram and unplanned downpower in 2016 due to issues associated with the turbine control system. The related corrective actions did not meet the timeliness standard of Procedure EN-LI-102, Corrective Action Program, and Procedure JA-PA-01, Analysis Manual, because the interim mitigating actions were insufficient. The associated corrective actions did not adequately reduce the potential for recurrence or provide adequate mitigating actions to manage the severity of a subsequent event to an acceptable level. As a result, the licensee experienced a scram in January 2018 due to unexpected turbine control valve movements and a complicated scram in December 2018 from unexpected turbine bypass valve movement.
Significance: Using Inspection Manual Chapter 0609, Attachment 4, Initial Characterization of Findings, and Appendix A, The Significance Determination Process (SDP) for Findings At-Power, Exhibit 1, Initiating Event Screening Questions, dated June 19, 2012, the inspection team determined that the finding required a detailed risk evaluation because the finding involved a reactor trip and the loss of mitigation equipment relied upon to transition the plant from the onset of the trip to a stable shutdown condition (e.g., loss of condenser).
Specifically, the complicated scram on December 12, 2018, involved a loss of the main condenser since operators were required to shut the MSIVs to regain reactor pressure control. This action removed the main condenser from service as an available heat sink.
A regional senior reactor analyst performed a detailed risk evaluation and determined that the finding associated with the turbine bypass valve failing open event was of very low safety significance (Green). The analyst performed an initiating event analysis as called for in Section 8.0, Initiating Event Analyses, of Volume 1, Internal Events, of the RASP Handbook. A loss of condenser heat sink event was modeled since the MSIVs were closed and the main feedwater pumps and ability to dump steam to the condenser had been lost due to the event. During the actual event, some mitigating equipment was degraded, but that degradation was not assumed to be caused by the performance deficiency associated with this finding. As a result, only the mitigating equipment rendered nonfunctional by the main steam valve closure was assumed to have been degraded by the performance deficiency.
This modeling resulted in an increase in core damage frequency of 4.3E-7/year for the finding. The analyst ran an uncertainties analysis on the results of the SPAR model in SAPHIRE. Of the 5,000 cases run in a Monte Carlo analysis, approximately 90 percent of the results were less than 1.0E-6/year giving confidence that the finding was of very low safety significance (Green). Loss of condenser heat sink events comprised the dominant core damage sequences. The ability to depressurize and the high pressure core spray system remained available for mitigation of the dominant sequences. The analyst ran the Grand Gulf SPAR model, Revision 8.59, on SAPHIRE, Version 8.2.0, to calculate the conditional core damage probability using a cutset truncation of 1.0E-12.
The analyst assumed that external events would be an insignificant contributor to the increase in core damage frequency because the probability of any external event coinciding with the loss of condenser heat sink event would be extremely low. As a result, only the increase in core damage frequency from the initiating event was used in the final estimate.
After reviewing Manual Chapter 0609, Appendix H, Containment Integrity Significance Determination Process, the analyst determined that the dominant sequences (high reactor coolant system pressure sequences) for a BWR Mark III containment warranted application of a large early release frequency factor of 0.2 per Table 6.1, Phase 1 Screening - Type A Findings at Full Power. Application of this factor yielded an increase in large early release frequency of 8.6E-8/year for the finding and therefore the finding was of very low safety significance (Green) for large early release frequency.
Cross-cutting Aspect: The finding had a cross-cutting aspect in the area of human performance associated with avoiding complacency because the licensee failed to recognize and plan for the possibility of mistakes, latent issues, and inherent risk, even while expecting successful outcomes including implementing appropriate error reduction tools [H.12].
Specifically, the licensee failed to adequately plan for the latent risk of additional turbine control system issues prior to system replacement including reviewing and improving related procedures.
Enforcement:
The inspection team did not identify a violation of regulatory requirements associated with this finding.
Observation 95001 General Weakness No. 1: Root Cause, Extent of Condition, and Extent of Cause Evaluation; and Corrective Actions Taken Background: The team reviewed scram S3 under Condition Report CR-GGN-2018-13032, A Bypass Valve Drifting Open with Complicated Scram, and identified a general weakness associated with the root cause and corrective actions. Specifically, the direct cause was identified as a failed LVDT in the actuator for the train A bypass control valve that resulted in the inability of the valves primary controller to properly position the valve. The root cause was identified as procedural guidance for turbine controls omitted available functions to utilize the installed, available auxiliary controller to drive the valve to its proper closed position. The CAPR was revising the off-normal event procedure to allow manual swapping to the auxiliary controller.
The inspection team noted that the licensee had not determined the cause of the LVDT failure. The licensee, instead, had removed the failed LVDT from service electrically and connected an installed spare LVDT. This was done for several reasons. First, the effort involved to remove the failed LVDT was significant as it required disassembly of the turbine bypass valve. Second, the licensee had a planned CAPR from a 2016 scram, not within the scope of this 95001 inspection, to replace the turbine control system with an updated, digital design in the next refueling outage. This planned major modification would replace the LVDTs in question within 18 months of scram S3. Third, the licensee had not identified any similar failures of this model of LVDT in both internal and external (industry) OE.
The inspection team also noted that the turbine bypass system was inoperable for a separate reason associated with issues in the test circuit. For this reason, the licensee was no longer testing the turbine bypass valves, although they were available to automatically respond to plant events.
Weakness: The inspection team concluded that the licensees root cause determination and the associated corrective actions may not preclude repetition of a turbine bypass valve failure.
Since the cause of the LVDT failure is not known, there is a possibility of a similar latent fault existing in the turbine bypass valves. The CAPR, which involves actions to be taken in response to a turbine bypass valve failure, relies on the capability of the inoperable, untested turbine bypass valves to be operated using an alternate controller. This set of information called into question the efficacy of the action to prevent recurrence since the ability to shut turbine bypass valves is not tested. Stated differently, the CAPR assumes the turbine bypass valves will successfully change state on demand, but the licensee is not testing this capability because they were considered inoperable.
Further, the inspection team questioned the sustainability of the CAPR since the planned major modification to digital turbine controls would include replacement of both the bypass valve controls and the associated off-normal event procedures in the near future. In short, the inspection team questioned if the licensee had created a CAPR that would be physically eliminated within the next year.
At the time these questions were raised, the inspection team considered this issue to be a significant weakness, defined per the 95001 procedure as a substantial inadequacy in the
- (a) evaluation of the root causes,
- (b) determination of the extent of the performance issues, or
- (c) actions taken or planned to correct the issue.
Corrective Actions: In response to the inspection teams questions, the licensee added a corrective action to re-implement periodic testing of the turbine bypass valves. The licensee has an assigned action to perform inspection of the failed LVDT when it is removed in the next planned refueling outage and use the resulting information to inform any response for other LVDTs located in the plant.
The licensee added a CAPR to ensure the upcoming digital turbine control modification provides controls to manually operate each of the turbine bypass valves in the event of a failure within the primary control circuit. Further, the reactor pressure controls malfunction ONEP will be updated with instructions to use available features of the new system to manually close an individual turbine bypass valve in the event of a failure in the primary control circuit. These actions were taken during the period of inspection including changes to CAPRs that involved approval by the stations Performance Improvement Review Group.
Conclusion: Since the licensee was able to take adequate actions during the period of inspection to ensure appropriate CAPR actions were in place, this issue was recharacterized from a significant weakness to a general weakness. Specifically, the licensee identified adequate measures to ensure the turbine bypass valves are capable of being operated, consistent with the assumption of the procedure CAPR, and will ensure the intent of the CAPR will be carried forward into the new digital turbine control system design.
Observation 95001 General Weakness No. 2: Root Cause, Extent of Condition, and Extent of Cause Evaluation; and Corrective Actions Taken Background: The team reviewed downpower D3 under Condition Report CR-GGN-2018 0218, MSL A Vent Line Steam Leak, and identified a general weakness associated with the apparent cause and corrective actions. Specifically, the direct cause was identified as a cantilevered line break resulting in an unplanned down-power caused by high cyclic fatigue due to mechanical resonance. The possible causes were:
(PC-1) The operating conditions were a possible direct cause to the failure of the cantilevered small-bore piping. GGNS has experienced multiple forced outages for Cycle 21 - this means that the station experienced multiple startups at low power, high vibration. Additionally, during Cycle 21, the station experienced 11 scrams.
(PC-2) The other possible direct cause was postulated to be some other pre-existing defect leading to failure of the piping.
Causal Factor 1 (CF1) was identified as a less than adequate implementation of the corrective action program by the station, in that conditions were not identified in some cases and others not appropriately screened, evaluated, and conditions resolved in a timely manner to address piping vibration. Causal Factor 2 (CF2) identified that responsible individuals did not perform roles and responsibilities to ensure identification, evaluation, and elimination of station susceptibility to vibrational fatigue failures in cantilevered piping. Causal Factor 3 (CF3)determined that engineering management had a lack of commitment to identify and resolve the risk of flow induced vibration on cantilevered small-bore piping. Causal Factor 4 (CF4)identified that engineering did not utilize industry OE to ensure the vulnerabilities in cantilevered small-bore piping were addressed.
Weakness: The inspection team noted that the licensee had not adopted fleet guidance, and had not resolved the weaknesses in their small-bore piping program. The licensee had multiple actions from previous corrective actions that did not address implementation of the program and known vibration issues.
The following condition reports revealed repeated failures of the main steam line 1st stage pressure sensing lines. Multiple apparent cause evaluations and RCEs performed pre- and post-extended power uprate did not evaluate vibration conditions in the main steam system:
CR-GGN-1999-01961/01964/01963/01684; CR-GGN-2000-00135; CR-GGN-2005-04552; CR-GGN-2007-01249; CR-GGN-2010-01854; CR-GGN-2012-08314; CR-GGN-2014-02824.
In Condition Report CR-GGN-2005-04932, mechanical standard GGNS-MS-58, Cantilevered Vent and Drain Piping Vibration Monitoring Program, was developed and issued on January 30, 2003, in accordance with Condition Report CR-GGN-2001-00194, but the program has not been fully implemented.
In Condition Report CR-GGN-2017-09586, during a review of the Grand Gulf Nuclear Station response to IER 14-30, Analysis of Vibration-Induced Piping and Tubing Leaks, it was discovered that MS-58 Grand Gulf Nuclear Station Design Engineering Mechanical Standard for Cantilevered Vent and Drain Piping Vibration Monitoring Program, has not been fully implemented.
Further, the inspection team noted in Condition Report CR-GGN2018-09645, corrective action CA-73 did not contain any relevant users groups for small bore cantilevered piping vibration.
In short, the licensee had not implemented effective corrective actions from their history of small-bore piping failures and had not identified the gap in use of industry OE.
At the time these observations were raised, the inspection team considered them to be a general weakness per the 95001 procedure.
Corrective Actions: In response to the inspection teams questions, the licensee determined that the applicable users groups would be EPRI Pressurized Water Reactor Materials Reliability Program and Welding and Repair Technology Center. These users groups cover topics such as flow-induced vibration and fatigue failures. The licensee also initiated Condition Report CR-GGN-2019-07048 for design engineering to contact available subject matter experts for support in resolution of Grand Gulf Nuclear Station small-bore piping flow-induced vibration issues. The inspection team also noted that Condition Report CR-GGN-2018-00218, Corrective Action CA-25, was initiated to perform benchmarking at Arkansas Nuclear One to review implementation of the Arkansas Nuclear One vibration monitoring program and incorporate learning into the Mechanical Standard GGNS-MS-58, Cantilevered Vent and Drain Piping Vibration Monitoring Program, and SEP-VIB-GGN-001, Grand Gulf Vibration Monitoring Program.
Conclusion: The licensee took actions to address this general weakness during the period of inspection. Specifically, the licensee identified actions to effectively utilize OE and resolve gaps in their small-bore piping program.
Observation 95001 General Weakness No. 3: Corrective Actions Taken Background: The team reviewed scram S5 under Condition Report CR-GGN-2019-03822, Loss of PSW [plant service water] Flow Reactor Manual Scram, and identified a general weakness associated with the corrective actions taken. Specifically, the team performed a walkdown of balance-of-plant transformer 23 (BOP 23) and other areas of the plant for material condition and equipment used to prevent animal intrusion into electrical components.
The inspection team identified a number of concerns including inconsistent use of animal barriers, animal intrusion barriers out of position or fallen onto the ground, inadequate preventative maintenance tasks to inspect the equipment, the lack of a checklist of the equipment for use during licensee maintenance activities or walkdowns, and a lack of a technical basis for the use of particular animal intrusion devices including placement locations.
Weakness: The inspection team concluded that the licensees corrective actions including mitigating actions may not preclude repetition of animal intrusion into BOP 23 and a subsequent plant transient. The material condition of barriers was poor, the system used to validate their correct positioning was lacking, and the technical basis for the placement of devices was not clear. This set of information called into question how the licensee would prevent recurrence.
At the time these questions were raised, the inspection team considered this issue to be a significant weakness per the 95001 procedure.
Corrective Actions: In response to the inspection teams questions, the licensee generated 15 condition reports and 16 corrective actions, including walkdowns, fixing material deficiencies, establishing a checklist of what is expected on walkdowns, and incorporating existing Entergy transmission guidance on preventing animal intrusion, which was not previously known to the nuclear station. These actions were taken during the period of inspection, including changes to CAPRs that involved approval by the stations Performance Improvement Review Group.
Conclusion: Since the licensee was able to take adequate actions during the period of inspection to ensure that appropriate corrective actions were being implemented, this issue was recharacterized from a significant weakness to a general weakness. Specifically, the licensee identified actions to holistically review their animal intrusion plan and correct deficiencies from a technical basis for BOP 23 and other electrical components vulnerable to intrusion.
Observation 95001 General Weakness No. 4: Corrective Action Plans Background: The team reviewed scram S4 under Condition Report CR-GGN-2019-01504, Automatic Reactor Scram due to Generator Lockout, Common Cause Evaluation CC1 under Condition Report CR-GGN-2018-13042, White Performance Indicator Exceeded: Scrams per 7,000 Critical Hours, and Common Cause Evaluation CC2 under Condition Report CR-GGN-2018-09645, White Performance Indicator Exceeded: Unplanned Power Changes per 7,000 Critical Hours. The team identified a general weakness associated with corrective action plans for these evaluations.
For the generator lockout, the direct cause was identified as a grid load imbalance which exceeded the negative phase sequence protective setpoint. The root cause was identified as engineering leadership not ensuring standards including a systematic configuration change process that considered operating margins. The CAPRs included implementing a standard for performing design reviews for engineering changes and critical attributes for third party review for engineering changes.
For the common cause evaluations, Root Cause No. 1 was identified as the stations leadership team failed to align site standards to focus on resolution of equipment vulnerabilities including quality of engineering changes with adequate performance testing and monitoring to detect changes in operating margin. The CAPRs included crediting corrective actions for the generator lockout.
When reviewing the corrective action plans, the team noted the licensee had located some of the CAPRs in desktop guides, such as the Design Review Board Guideline. These informal documents were located on internal webpages within the licensees network and not in the controlled document library.
Weakness: The inspection team concluded that the corrective plans may not preclude repetition of unplanned downpowers or scrams. Specifically, the inspection team challenged the sustainability of the CAPRs located in desktop guides. The team questioned the vulnerability of these files to be inadvertently moved, edited, or deleted through uncontrolled processes. Further, the team questioned the ability for new licensee staff to locate and implement the CAPRs embedded in the desktop guides if they were not tied to a formally established site-wide procedure or process.
At the time these questions were raised, the inspection team considered this issue to be a significant weakness per the 95001 procedure.
Corrective Actions: In response to the inspection teams questions, the licensee moved the associated CAPRs from desktop guides to management standards. Similar to a site-wide standing order, management standards are contained in the controlled document library and have strict controls to move, modify, or delete content. Specifically, Management Standard 50, GGN Design Review Board Guideline, and Management Standard 51, GGN Technical Product Quality Standard, were created to ensure CAPRs are carried forward sustainably.
These actions were taken during the period of inspection, including changes to CAPRs that involved approval by the stations Performance Improvement Review Group.
Conclusion: Since the licensee took adequate actions during the period of inspection to ensure that CAPR actions would be appropriately sustainable, this issue was recharacterized from a significant weakness to a general weakness. Specifically, the licensee ensured CAPRs were contained in controlled documents that were visible to all site staff.
EXIT MEETINGS AND DEBRIEFS
On October 17, 2019, the inspection team and Mark Shaffer, the Region IV Deputy Regional Administrator presented the supplemental inspection results to Mr. E. Larson, Site Vice President, and other members of the licensee staff. The inspection team verified no proprietary information was retained or documented in this report.
DOCUMENTS REVIEWED
Inspection Type Designation Description or Title Revision or
Procedure Date
95001 Corrective Condition Reports CR-GGN-2005-02446, CR-GGN-2007-0773,
Action CR-GGN-2008-01476, CR-GGN-2010-05889,
Documents CR-GGN-2011-01565, CR-GGN-2011-09029,
CR-GGN-2014-5906, CR-GGN-2014-6277,
CR-GGN-2014-7429, CR-GGN-2015-0416,
CR-GGN-2015-0462, CR-GGN-2015-2495,
CR-GGN-2015-2846, CR-GGN-2016-4834,
CR-GGN-2016-4850, CR-GGN-2016-1186,
CR-GGN-2016-6067, CR-HQN-2017-01541,
CR-GGN-2017-1299, CR-GGN-2017-9910,
CR-GGN-2018-9889, CR-GGN-2018-0218,
CR-GGN-2018-00239, CR-GGN-2018-08706,
CR-GGN-2018-9281, CR-GGN-2018-09632,
CR-GGN-2018-09645, CR-GGN-2018-9679,
CR-GGN-2018-9725, CR-GGN-2018-09890,
CR-GGN-2018-10117, CR-GGN-2018-10600,
CR-GGN-2018-11055, CR-GGN-2018-12370,
CR-GGN-2018-13032, CR-GGN-2019-02908,
CR-GGN-2019-03822, CR-GGN-2019-03962,
CR-GGN-2019-06779, CR-GGN-2019-07048,
CR-GGN-2019-06783, CR-GGN-2019-06789,
CR-GGN-2019-07027, CR-GGN-2019-07030,
CR-GGN-2019-07039, CR-GGN-2019-07042,
CR-GGN-2019-07045, CR-GGN-2019-07046,
CR-GGN-2019-07047, CR-GGN-2019-07050,
CR-GGN-2019-07051, CR-GGN-2019-07107,
CR-GGN-2019-07109, CR-GGN-2019-07111,
CR-GGN-2019-07134, CR-HQN-2019-01815,
Calculations EC-N1N41-17001 Generator Protection 0
PR0204 Generator Negative Sequence Protection 0, 1, and 2
Work Orders WO486066 1N23N059A/1N23N061A Backfill Level Transmitter 10/23/2018
WO509173 Perform STI to Tune 6a Feed Water Heater 4/10/2019
Controllers
WO519151 1N41L701 - Install Temporary Modification EC81801 3/12/2019
Miscellaneous Post-trip analysis Various
ACMP from POD report 8/27/2019
Animal Intrusion Plan 2019
CAP Backlog Report Various
19-0003 Standing Order: Radial Well Switchgear Lineup 0
EC7471 Need Electrical Design Group Evaluation of General 0
Electric Relay Model 12SGC21B1A to Replace
2SGC12B1a
EC23022 Grand Gulf Nuclear Station Extended Power Uprate 3/10/2011
Feed Water Heater Drain System Modification
EC29327 Modification to Feed Water Heater 2 dump valves 6/01/2011
N1N23F508A, B, and C
EC67304 Main Generator Protective Relay Replacement 0
EC78855 RECIRCULATION PUMP CAVITATION 7/31/2018
INTERLOCK IN BYPASS
EC81801 Temporary Modification EC for Negative Sequence 0
Relay Setpoint
FSAR Section 1.2.2.6.1 and 9.2.10.2 on radial well pumps
EC65556 Temporary Modification for BOP23 Overhead Line 2016
EC81511 Install New Overhead Line From the Switchyard to 2019
BOP23
GFIG-OPS-N2335 Training Material: Heater and Moisture Separator 8/26/2016
Reheater Vents and Drains System - N23/N25
Figures
GFIF-OPS-N3202 Training Material: Main Turbine EHC Control 6/22/2106
System
GLP-OPS-N2325 Training Material: Heater and Moisture Separator 12/10/2018
Reheater Vents and Drains - N23/N25
GLP-OPS-N3202 Training Material: Main Turbine EHC Control 7/05/2016
System - N32-2
N23-002-1N23FX382 Tagout Clearance for Feedwater Heater 6A Level 8/6/2019
Instrument Isolation
STI-1803 Feedwater Heater Drain System Tuning 4/23/2018
Maintenance Rule Functional Failure Evaluations Various
Operator Logs 9/12/2018 -
9/14/2018
Operator Aggregate Index Various
Plant Data System screenshot for steam dome 12/12/2018
pressure, APRM level, Bypass control valve A
position, Bypass control valve A demand signal
Licensed Operator Continuing Training Schedule Various
Post-trip Reports Various
Scram Storyboards and Roadmaps 0
Site Staffing Levels Various
System Health Reports Various
95001 Self Assessments Various
Temporary Modifications List Various
Procedures EN-HU-104 Technical Risk and Rigor 9
EN-HU-106 Procedure and Work Instruction Use and Adherence 7
EN-MA-100 Maintenance Fundamentals 5
EN-LI-102 Corrective Action Program 36
EN-LI-118 Cause Evaluation Process 29
EN-LI-121 Trending and Performance Review Process 25
04-1-02-1H13-P680- Alarm response Instruction for CNDS PMP A VIBR 240
1A-E1 HI
05-1-02-V-21 Reactor Pressure Control Malfunctions 0, 3, and 5
Drawings M-1055B Heater Vents and Drains System Piping and 14
Instrumentation Diagram
M-1055C Heater Vents and Drains System Piping and 32
Instrumentation Diagram
2