IR 05000311/2017011
| ML17297A437 | |
| Person / Time | |
|---|---|
| Site: | Salem  |
| Issue date: | 10/24/2017 |
| From: | Bower F L Reactor Projects Branch 3 |
| To: | Sena P P Public Service Enterprise Group |
| References | |
| IR 2017011 | |
| Download: ML17297A437 (21) | |
Text
UNITED STATES NUCLEAR REGULATORY COMMISSION REGION I 2100 RENAISSANCE BLVD., Suite 100 KING OF PRUSSIA, PA 19406
-2713 October 24, 2017 Mr. Peter P. Sena, III President and Chief Nuclear Officer PSEG Nuclear LLC - N09 P.O. Box 236 Hancocks Bridge, NJ 08038
SUBJECT: SALEM GENERATING STATION UNIT 2 - SUPPLEMENTAL INSPECTION REPORT 05000311/2017011 AND ASSESSMENT FOLLOW-UP LETTER
Dear Mr. Sena:
On September 15 , 2017, the U.S. Nuclear Regulatory Commission (NRC) completed a supplemental inspection pursuant to Inspection Procedure 95001, "Supplemental Inspection Response to Action Matrix Column 2," at your Salem Generating Station Unit 2. On September 15 , 2017, the NRC inspection team and an NRC Region I Branch Chief discussed the inspection results and the implementation of your corrective actions with Mr. C. McFeaters and other members of your staff during an inspection exit meeting and subsequent Regulatory Performance Meeting. The results of the inspection are documented in the enclosed report.
In accordance with the NRC Reactor Oversight Process (ROP) Action Matrix, this supplemental inspection was conducted within the Regulatory Response Column of the NRC's ROP Action Matrix because of one White performance Indicator (PI) for Unplanned Reactor Scrams per 7,000 hours0 days <br />0 hours <br />0 weeks <br />0 months <br /> Critical. Specifically, Salem Unit 2 crossed the threshold from Green to White in the second quarter of 2016 following one reactor trip in August 2015, two reactor trips in February 2016, and a fourth reactor t rip in June 2016. The PI remained White when a fifth reactor trip occurred in August 2016. The details behind each reactor trip are discussed in this inspection report. In response to this Action Matrix input, the NRC informed you in the 2016 mid-cycle letter dated August 31, 2016 (Reports 05000272/2016005 and 05000311/2016005; ML16242A0991), that a supplemental inspection using Inspection Procedure 95001, "Supplemental Inspection for One or Two White Inputs in a Strategic Performance Area," would be required.
The NRC staff was informed on July 27 , 2017, of your staff's readiness for this supplemental inspection.
The objectiv es of this supplemental inspection were to provide assurance that: (1) the root causes and the contributing causes of risk-significant performance issues were understood; (2) the extent of condition and extent of cause of risk-significant performance issues were identified; and (3) corrective actions, taken or planned, for risk-significant performance issues are sufficient to address the root and contributing causes and prevent recurrence.
The inspection consisted of examination of activities conducted under your license as they related to safety, compliance with the Commission's rules and regulations, and the conditions of your operating license. Based on the results of this inspection, the NRC concluded that PSEG's evaluation of the root and contributing causes associated with the five unplanned reactor scrams at Salem Unit 2, as well as the overall adverse scram trend, i s appropriate.
PSEG appropriately identified the root cause s to include inadequate management focus on processes and programs to ensure equipment reliability (ER), such as the mitigation or elimination of single point vulnerabilities that led to several of these reactor scrams, as well as weaknesses in the implementation of the corrective action program. Overall, PSEG satisfactorily identified the individual and collective performance issues associated with th is White PI and has appropriate corrective actions either implemented or planned to address the issues that led to this White PI. Key corrective actions included: 1) replacing certain high consequence protective relays and/or converting them to digital; 2)
implementing causal evaluation training for the engineering staff; 3) installing a modification to prevent future water hammers when returning containment fan cooler units to service; 4) mitigating or eliminating a large number of single point vulnerabilities; 5) implementing management tools to focus on monitoring and improving ER
- and, 6) increasing corrective action closure oversight
. Based on the results of this inspection, the NRC concluded that PSEG performed a satisfactory evaluation of the White PI, the supplemental inspection objectives were met, and no significant weaknesses or findings of significance were identified. Based on the guidance in Inspection Manual Chapter (IMC) 0305, "Operating Reactor Assessment Program," and the results of this inspection, the White PI will be closed. Additionally, since Salem Unit 2 has successfully completed this inspection, a Regulatory Performance Meeting has been held, and the PI returned to Green during the first quarter of 2017, Unit 2 will transition from the Regulatory Response Column of the NRC's ROP Action Matrix to the Licensee Response Column effective the date of this inspection report. In accordance with Title 10 of the Code of Federal Regulations (10 CFR) 2.390 of the NRC's "Rules of Practice,"
a copy of this letter, its enclosure, and your response (if any), will be available electronically for public inspection in the NRC Public Document Room or from the Publicly Available Records System component of the NRC's Agencywide Documents Access and Management System (ADAMS), accessible from the NRC Web site at http://www.nrc.gov/reading
-rm/adams.html (the Public Electronic Reading Room).
Sincerely,/RA/ Fred L. Bower, Chief Reactor Projects Branch 3 Division of Reactor Projects Docket No. 50-3 11 License No. DPR-75
Enclosure:
Inspection Report 050003 11/2017011
w/Attachment:
Supplementary Information cc w/encl: Distribution via ListServ SUBJECT: SALEM GENERATING STATION UNIT 2 - SUPPLEMENTAL INSPECTION REPORT 050003 11/2017011 AND ASSESSMENT FOLLOW-UP LETTER DATED OCTOBER 24, 2017 DISTRIBUTION w/encl: DDorman, RA (R1ORAMail Resource) DLew, DRA (R1ORAMail Resource) RLorson, DRP (R1DRPMail Resource) DPelton, DRP (R1DRPMail Resource) JYerokun, DRS (R1DRSMail Resource) BWelling, DRS (R1DRSMail Resource) FBower, DRP RBarkley, DRP MDraxton, DRP PFinney , DRP, SRI AZiedonis , DRP, R I COtt, DRP, AA JBowen, RI, OEDO RidsNrrPMSalem Resource RidsNrrDorlLpl1 Resource ROPassessment Resource ROPReports DOCUMENT NAME:
G:\DRP\BRANCH3\Inspection Reports
\Salem\2017\95001\Salem IR2017011
-IP95001_FINAL.docx ADAMS ACCESSION NUMBER:
ML17297A437 SUNSI Review Non-Sensitive Sensitive Publicly Available Non-Publicly Available OFFICE RI/DRP RI/DRP NAME RBarkley FBower DATE 10/23/17 10/24/17 OFFICIAL RECORD CO PY 1 Enclosure U.S. NUCLEAR REGULATORY COMMISSION REGION I Docket No. 50-3 11 License No. DPR-75 Report No. 050003 11/2017011 Licensee: PSEG Nuclear LLC (PSEG) Facility: Salem Generating Station (Salem) Unit 2 Location: Hancock's Bridge, New Jersey Dates: September 11 - 15 , 2017 Inspectors:
R. Barkley , PE, Senior Project Engineer, Team Leader N. Warnek, Senior Allegation Coordinator M. Hardgrove , Project Engineer (Observer)
Approved by: Fred L. Bower, Chief Reactor Projects Branch 3 Division of Reactor Projects 2
SUMMARY
Inspection
Report 050003 11/2017011; 0 9/11/2017 - 09/1 5/2017; Salem Generating Station (Salem) Unit 2; Supplemental Inspection- Inspection Procedure (IP) 95001 A Senior Project Engineer from the Division of Reactor Projects, U.S. Nuclear Regulatory Commission (NRC) Region I and the Senior Allegation Coordinator (inspector qualified) from the Office of the Regional Administrator
, NRC Region I, performed this inspection.
The NRC's program for overseeing the safe operation of commercial nuclear power reactors is described in NUREG-1649, "Reactor Oversight Process," Revision 6.
Cornerstone: Initiating Events
The NRC staff performed this supplemental inspection pursuant to IP 95001, "Supplemental Inspection Response to Action Matrix Column 2 Inputs," because of a White performance indicator (PI)associated with the Initiating Events Cornerstone. The PI involved Unplanned Reactor Scrams per 7,000 hours0 days <br />0 hours <br />0 weeks <br />0 months <br /> Critical. The PI turned White following one reactor trip in August 2015, two reactor trips in February 2016, and a fourth reactor trip in June 2016. The PI remained White when a fifth reactor trip was incurred in August 2016. The PI returned to Green as of the first quarter of 2017. The details behind each reactor trip is discussed within this inspection report. The NRC staff was informed on July 27 , 2017, of your staff's readiness for this supplemental inspection.
Based on the results of the inspection, no significant weaknesses or findings were identified.
The inspectors concluded that PSEG had adequately performed a total of six (6)root cause analys e s for the five event s as well as the overall adverse scram trend. PSEG appropriately identified the root cause s to include inadequate management focus on processes and programs to ensure equipment reliability (ER), such as the mitigation or elimination of single point vulnerabilities (SPVs) that led several of these reactor scrams, as well as weaknesses in the implementation of the corrective action program (CAP). With regard to the safety culture of the organization, the most significant issues were noted in the attributes of problem identification , decision-making , leadership safety values and actions, and questioning attitude. Key corrective actions to address these root and contributing causes included
- 1) placing the heater drain pumps (HDPs) in the Long Term Asset Management (LTAM) program; 2) replacing certain high consequence protective relays and/or converting them to digital (42 in total); 3) implementing plans for the replacement of multiple 25 kV monitoring system components that are obsolete or out of service; 4) implementing Causal Evaluation training for the Engineering staff; 5) installing a 1
-inch bypass line on the service water (SW) supply line to the containment fan cooler units (CFCUs) to prevent future water hammers when returning the system to service; 6) installing modifications to eliminate a number of SPVs (61 in total);
7) implementing management tools to focus on monitoring and improving ER, including an ER Heat Map; 8) providing Line-Of-Sight review for CAP evaluations; and, 9) performing Corrective Action Closure Boards on a monthly basis.
3 The team also reviewed the corrective actions that addressed the White PI for Unplanned Scrams per 7000 Hours Critical at Salem Unit 1 since the conclusion of its 95001 inspection in August 2015.
The station identified that the corrective actions implemented to address that White PI had not effectively corrected the significant conditions adverse to quality because the station received a repeat White PI for Unplanned Scrams on Unit one year later.
PSEG's review noted that the four reactor trips at Unit in 2014-2015 were for different reasons, and the focus of the prior efforts was on the recovery of margin and how to prevent crossing the threshold for this White PI, versus improving ER to prevent future scrams/trips. PSEG found that some of the corrective actions implemented were either too broad or were not effective in the long-term. Moreover, some corrective actions were changed over time such that they did not meet the intent of the action originally planned. These changes were made without adequate review and approval by the PSEG senior management team.
Th e root cause evaluation s (RCEs) for the Unit 2 White PI did touch on the issues noted at Unit in 2 014-2015, but focused predominately on enhancing the corrective action process and improving ER in order to prevent reactor scrams/trips. The RCEs performed in support of this Unit 2 supplemental inspection also looked back to 2010 to ensure that the causes identified were aligned with the reactor scram/trip issues noted at both Salem units since then. The numerous corrective actions, both completed and planned, for this Unit 2 White PI were reasonable to address the related issues. Based on the guidance in I MC 0305, "Operating Reactor Assessment Program," dated November 17, 2016, and the results of this inspection, the White PI will be closed. Additionally, a regulatory performance meeting was held immediately following the inspection exit meeting on September 15
, 2017. Since Salem has successfully completed this inspection, a Regulatory Performance Meeting has been held, and the PI returned to Green effective the first quarter of 2017
, Salem Unit 2 will transition to the Licensee Response Column of the reactor oversight process (ROP) Action Matrix effective the date of this inspection report. (Section 4OA4)4
REPORT DETAILS
OTHER ACTIVITIES
4OA3 Follow-up of Events and Notices of Enforcement
Discretion (Closed) Licensee Event Report (LER) 05000311/2015 01: Reactor Trip Due to Loss of 4kV Non
-Vital Group Bus
a. Inspection Scope
This LER (05000311/2015 00) was originally submitted by PSEG in October 2015, following an August 5, 2015, Unit 2 reactor trip. The LER presented the results of PSEG's RCE for this event, which concluded that there was no definitive cause. PSEG had determined that the most probable cause was a ground fault on the 21 HDP that was not isolated by its associated neutral overcurrent relay, resulting in the loss of the 2H bus, including the 21 reactor coolant pump (RCP), and leading to the subsequent reactor trip. The LER was reviewed and closed by the NRC under inspection report 05000311/2016001 (ML16133A087).
In preparation for the 95001 inspection, PSEG revised the RCE for this event and definitively concluded that the cause of the trip was that the 21 HDP motor neutral overcurrent relay did not actuate in response to a ground fault on the 21 HDP motor. The relay failed to actuate due to aging or foreign material. PSEG submitted a revised LER (05000311/2015 01) to ensure alignment with th is root cause. The inspectors reviewed the revised LER, the revised RCE and corrective actions, and interviewed PSEG staff. No findings or violations of NRC requirements were identified. This LER is closed.
b. Findings
No findings were identified (Closed) Licensee Event Report 05000311/2016 01: Automatic Reactor Trip due to Main Turbine Trip
a. Inspection Scope
This LER (05000311/2016 00) was originally submitted by PSEG in April 2016, following a February 4, 2016, Unit 2 reactor trip. The LER presented the results of PSEG's original RCE for this event, which concluded that the reactor trip was caused by a main generator protective relay (model STV1) that actuated unexpectedly due to setpoint drift. The LER was reviewed and closed by the NRC under inspection report 05000311/2016002 (ML17101A407), which documented a Green, self
-revealing FIN for PSEG work orders not specifying the appropriate modification testing for the protective relay.
In preparation for the 95001 inspection, PSEG revised the RCE for this event. The revised RCE considered the inadequate work order to be a contributing cause, but pointed to a new root cause:
that PSEG did not develop and implement adequate corrective actions after Unit 1 experienced a similar trip in December 2012 due to STV1 relay setpoint drift. The inspectors reviewed the revised LER (05000311/2016 01), the revised RCE and corrective actions, and interviewed PSEG staff. No additional findings or violations of NRC requirements were identified. This LER is closed.
b. Findings
No findings were identified (Closed) Licensee Event Report 05000311/201 6-00 5-0 0: Automatic Reactor Trip Due to Main Generator Trip
a. Inspection Scope
This LER presented the results of PSEG's RCE for this event, which found that this June 2016 generator trip occurred due to a fault internal to the recently replaced
'A' main transformer. The new transformer had wiring running from a current transformer to a connection point.
The wiring was wrapped with an adhesive cotton tape. However, over time, the ground wire in this bundle worked loose due to an inadequate termination on one end (which was not visible at fabrication) as well as eventual loosening of the cotton tape that held the wiring bundle together. The wire was suspended in the transformer's oil and moved in response the oil flow in this area. The wire eventually caused a n intermittent ground fault when it touched the low voltage connections inside the transformer bushing compartment that ultimately result ed in a main generator trip.
PSEG performed extensive monitoring and troubleshooting as well as equipment cleaning (i.e., carbon dioxide pressure washing of the 25 kV bus bar support dielectric coils) and replacement of select component s to identify the source of this generator/reactor trip. As the ground fault was intermittent, it was difficult to locate. Electrical experts from outside PSEG were brought in to assist with the troubleshooting effort. The source of the ground fault was readily identifiable once the decision was made to drain and enter the
'A' main transformer following the elimination of numerous other possible causes over a multi
-week troubleshooting effort.
The inspectors reviewed the LER, the final RCE on this scram and pictures of the wiring bundle: 1) at the end of fabrication, 2) following opening of the faulted transformer, and 3) following repairs to the wiring. He also spoke with the PSEG staff member involved in the oversight of the main transformer procurement. No findings or violations of NRC requirements were identified. This LER is closed.
b. Findings
No findings were identified
4OA4 Supplemental
Inspection (IP 95001)
.1 Inspection Scope
The NRC staff performed this supplemental inspection in accordance with IP 95001, "Supplemental Inspection Response to Action Matrix Column 2 Inputs,"
to assess PSEG's evaluation of a White PI, which affected the Initiating Events cornerstone in the Reactor Safety strategic performance area.
The inspection objectives were:
To assure that the root causes and contributing causes of individual and collective significant performance issues are understood.
To independently assess and assure that the extent of condition and extent of cause of significant individual and collective performance issues are identified.
To assure that corrective actions taken to address and preclude repetition of significant performance issues are prompt and effective.
To assure that corrective plans direct prompt actions to effectively address and preclude repetition of significant performance issues.
In accordance with the NRC ROP, t he inspectors performed a supplemental inspection since Salem Unit 2 was in the Regulatory Response Column of the Action Matrix because of one White PI for Unplanned Reactor Scrams per 7,000 Critical hours. Specifically, Salem Unit 2 crossed the Green
-White threshold in the second quarter of 2016 following one reactor trip in August 2015, two reactor trips in February 2016, and a fourth reactor trip in June 2016. The PI remained White when a fifth reactor trip was incurred in August 2016. The five
Rx Trip #1:
On August 5, 2015, Unit 2 experienced an automatic reactor trip from full power due to a loss of the 21 RCP. The 21 RCP
, which is powered from a breaker on the 2H bus, tripped when power to the 2H 4 kV non
-vital bus was lost following opening of the infeed breaker from the 2B auxiliary power transformer due to a fault on 21 HDP motor. Rx Trip #2:
On February 4, 2016, Unit 2 automatically tripped from 75 percent power on a generator trip initiated by a main generator automatic voltage regulator (AVR) volts/hertz over
-excitation protection (STV1) relay trip. Power had been reduced to 75 percent to support removal of a 500 kV line offsite. The STV1 relay was found to have a manufacturing defect which caused the relay setpoint to drift low and actuate when the volts/hertz ratio increased during the power reduction.
Rx Trip #3:
On February 14, 2016, Unit 2 experienced a main generator protection trip signal which led to a reactor trip from 100 percent power. The trip was later found to be caused by water dripping into the stator water cooling trip Agastat relay (62
-C1), causing an electrical short.
Rx Trip #4:
On June 28, 2016, Unit 2 experienced a main generator and subsequent reactor trip.
The cause was ultimately traced to a loose current transformer wire on the 'A' main power transformer, which caused an intermittent ground.
Rx Trip #5:
On August 31, 2016, Unit 2 automatically tripped due to the loss of the 21 RCP. The breaker for the 21 RCP opened due to a fault within the inner enclosure assembly of its electrical penetration. The fault was caused by brackish service water falling on the enclosure, which occurred after a water hammer on the 22 CFCU blew out a section of rubber gasket on its motor cooler. The water hammer occurred a s flow was being reestablished to the cooler following SW accumulator discharge valve testing.
In each of the reactor scrams/trips, safety systems performed as expected and the operators responded to the events properly. Thus none of the five events above involved operator performance issues.
In response to each of these five reactor scrams/trips, PSEG created five RCEs to evaluate the root and contributing causes for these events as well as proposed corrective actions. PSEG also performed an RCE on the overall adverse trend in reactor scrams/trips between August 2015 and August 2016
. The inspectors reviewed all six of PSEG's RCEs, reviewed applicable CAP documents, and interviewed licensing, engineering, maintenance, and operations management personnel to ensure that the root and contributing causes were understood and corrective actions taken or in progress were appropriate.
The inspectors also conducted an in-plant walkdown of the affected components in the Unit 2 turbine building, noting modifications that had been completed since these reactor trips occurred.
.2 Evaluation of the
95001 Supplemental Inspection Requirements
.2.1 Problem Identification
a. IP 95001 requires that the inspection staff determine that PSEG's evaluation of the issue documents who identified the issue (i.e., licensee
-identified, self
-revealing, or NRC identified) and the conditions under which the issue was identified.
In each of the fiv e reactor trips that caused the PI to turn White, PSEG documented who identified the cause of the trip and the conditions under which the issue was identified. In each case, the cause of the trip was either self
-revealing or identified by PSEG after troubleshooting. The exception was the first reactor trip, which involved an electrical failure of the 21 HDP and subsequent loss of the 2H 4kV bus. The reason why the protective relay (which should have isolated the 21 HDP by opening its breaker before the 2H bus was lost) did not operate properly could not be definitely identified, although the potential causes were narrowed to aging (the relay was original equipment) or foreign material on a key internal component. As a precaution, the relay was replaced.
The inspectors determined this inspection objective was met.
b. IP 95001 requires that the inspection staff determine that PSEG's evaluation of each issue documents how long the issue existed and prior opportunities for identification.
In each of the five reactor trips, PSEG documented how long the issue which caused the reactor trip existed and prior opportunities for identification. PSEG ultimately determined the following:
In the first reactor trip, the testing data over several years indicated that the 21 HDP motor was acceptable, but showed a clear trend that the motor insulation was degrading; In the second reactor trip, the generator STV 1 relay had experienced performance problems since 2005, with multiple replacements over the years. This same relay had caused a reactor trip on Unit 1 in 2012. During the most recent Unit 2 STV 1 relay replacement, a few months before the 2016 trip, problems were experienced during acceptance testing that could have led to the identification of the degraded relay; In the third reactor trip, no definitive time was found for when the water leak on a relay developed, although the design of the panel that housed the co
-located stator water line and the relay dated to the replacement of the Unit 2 main generator in 1992; In the fourth reactor trip, the current transformer wire came loose at some point following its entry into service in November 2015 and the trip in June 2016, without prior, definitive warning; and, The fifth reactor trip was caused by a CFCU motor cooler gasket leak that led to the wetting of the RCP penetration immediately prior to the trip. However, the system was susceptible to service water (SW)-induced water hammers since a 1998 modification
, and apparent cause evaluations (ACEs) had been performed on four separate occasions for significant leaks since that time.
The inspectors determined this inspection objective was met.
c. IP 95001 requires that the inspection staff determine that PSEG's evaluation documents the plant specific risk consequences, as applicable, and compliance concerns associated with the issue(s).
In each of the five trips , the operators responded as expected and the trips were uncomplicated. The trips all involved equipment that was not safety
-related, with the exception of the safety
-related SW leak from the CFCU which sprayed down on a non
-safety related reactor coolant pump electrical penetration and caused a short to ground. Specific regulatory compliance concerns, if any, were addressed by the resident inspectors in their quarterly inspection reports.
The increased frequency of reactor trips during this 12 month period (August 2015
- August 2016) does impact the Probabilistic Risk Assessment (PRA) model for Salem , but not to a significant degree as uncomplicated trips are a small percentage (~10 percent) of the PRA initiating events that result in increasing the core damage frequency.
The inspectors determined this inspection objective was met.
d. Findings
No findings were identified.
.2.2 Root Cause, Extent of Condition, and Extent of Cause Evaluation
a. IP 95001 requires that the inspection staff determine that PSEG evaluated the issue using a systematic methodology to identify the root and contributing causes PSEG used a systematic methodology to review each event, identifying the root and contributing causes of each of the five reactor trips as well as the overall adverse scram trend. The following root cause evaluation (RC E) methodologies were utilized for the PSEG evaluation:
The WHY Staircase Barrier Analysis Failure Modes and Effects Analysis Event and Casual Factor (E&CF) Chart Safety Culture Analysis Organizational and Programmatic (O&P) Review Extent of Condition (EOCo) and Extent of Cause (EOCa) Review:
o Same - Same o Same - Similar o Similar - Same o Similar - Similar Operating Experience (OE) Assessment Review of Previous Internal Occurrences The inspectors determined this inspection objective was met.
b. IP 95001 requires that the inspection staff determine that PSEG's RCE s were conducted to a level of detail commensurate with the significance of the issue.
The six RCEs performed by PSEG were conducted to a level of detail commensurate with the significance of the issue. Each RCE used a variety of techniques to identify the root and contributing causes, and had documentation as necessary to support the conclusions drawn. The amount of detail varied by the complexity of the issue, with the RCEs on the CFCU leak that ultimately resulted in Rx trip #5 and t he adverse scram trend RCE s being the most lengthy and involved.
The inspectors determined this inspection objective was met.
c. IP 95001 requires that the inspection staff determine that PSEG's RCE included a consideration of prior occurrences of the issue and knowledge of Operating Experience.
The RCEs on each of the five reactor trips included a review of industry operating experience and prior occurrences of the issue. Prior occurrences varied by event, with the CFCU leak that ultimately resulted in a reactor trip (Rx trip #5) having the most, followed by the inadvertent actuation of the STV 1 relay (Rx trip #2) being the second. However some events had little or no precedent, such as the stator cooling water dripping on a relay that caused Rx trip #3. Industry experience was also considered in each case, but provided limited information due to either the uniqueness of the event and/or the uniqueness of the system design (e.g., Salem SW system, which is the only plant in the country equipped with accumulators
). The inspectors determined this inspection objective was met.
d. IP 95001 requires that the inspection staff determine that PSEG's RCE addresses the extent of condition and extent of cause of the issue.
PSEG's six RCEs documented considerable work to address the extent of condition and the extent of cause. For example, inspections were conducted on a range of similar equipment in the plant, such as the remaining two main transformers and a spare at Unit 2, the six HDPs at both units (including a spare in storage), and multiple electrical penetrations that were wetted by SW recently during past events, causing corrosion). In the case of the electrical penetrations, cleaning and repairs were performed on those penetrations
' boxes found to have significant corrosion. While additional inspections are planned (i.e., internal inspection of the Unit 1 main power transformers (MPTs) in the fall 2017 outage
), most were completed at the time of this inspection and discrepancies were documented and addressed. For the adverse scram trend, the extent of cause was focused on reviewing an expanded sample of RCEs and ACEs since 2013 to ensure the evaluations and resulting actions met procedural requirements. For instance, a large number of conditions requiring corrective action (CRCAs) were reviewed for proper closure and implementation. In addition, corrective actions to preclude recurrence (CAPRs) which had a change of intent since their creation received additional management scrutiny.
The inspectors determined this inspection objective was met.
e. IP 95001 requires that the inspection staff determine that PSEG's root cause, extent of condition, and extent of cause evaluations appropriately considered the safety culture components as described in IMC 0305.
An analysis was performed which looked at each of the 10 safety culture traits described in IMC 0305. While behaviors were found to have been met in 3 of the 10 traits , weaknesses were identified in the remaining seven traits, with the most significant weaknesses found to be in the areas of: 1) Problem Identification; 2) Decision
-Making; 3) Leadership Safety Values and Actions
- and, 4) Questioning Attitude.
PSEG concluded that the various corrective actions proposed for each of the five individual reactor trip RCEs will address most of the safety culture attributes, although an additional performance analysis was performed and corrective actions will be taken t o address issues with leadership decision-making.
The inspectors determined this inspection objective was met.
f. Findings
No findings were identified.
.3 Corrective Actions
a. IP 95001 requires that the inspection staff determine that
- (1) PSEG specified appropriate corrective actions for each root and/or contributing cause, or
- (2) an evaluation that states no actions are necessary is adequate.
PSEG specified appropriate corrective actions for each root and/or contributing cause
- none of the evaluations stated that no action was necessary. Key corrective actions included
Placing the HDPs in the LTAM program Replacing certain high consequence protective feeder relays and/or converting them to digital (42 in total)
Implementing plans for the replacement of multiple 25 kV monitoring system components that are obsolete or out of service Implementing causal evaluation training for the Engineering staff Placing a 90
-day limit for Long Term Asset (LTA)manager items to be in draft status Installing a 1
-inch bypass line on the SW supply line to the CFCUs to prevent future water hammers when returning the system to service Installing modifications to eliminate a number of SPV s (61 SPVs in total) Implementing management tools to focus on monitoring and improving ER , including an ER Heat Map Providing Line-Of-Sight review for CAP evaluations Performing Corrective Action Closure Boards on a monthly basis.
The inspectors determined this inspection objective was met. a. IP 95001 requires that the inspection staff determine that PSEG prioritized corrective actions with consideration of risk significance and regulatory compliance.
Each of these reactor trips involved balance of plant or non-safety related equipment (the exception being the CFCU SW leak), so the risk significance was low and regulatory compliance was not an issue. However, PSEG completed (or will soon complete) a number of the corrective actions involving equipment changes. In the case of the CFCU SW leak, which was the only one involving safety
-related equipment that is inside containment (which limits both access and available outage work windows), interim corrective actions have been implemented to prevent another water hammer until a small bypass line is installed to allow slow refilling of the CFCUs.
The risk significance and regulatory compliance issues for the overall root cause s were addressed in the five individual root causes. An overall risk significance for the multiple reactor trips was provided in the sixth adverse trend RCE. The prioritization of corrective actions was generally subjective, with the exception of actions to address SPVs. The SPVs are individual components which, if they fail or actuate, lack redundancy (e.g., 2 out of 3 in a logic circuit) and thus trip a component that will subsequently lead to a reactor trip.
PSEG procedure ER
-AA-2004, System Vulnerability Review Process Description, Attachment 1 contain ed a SPV Prioritization Matrix for risk ranking vulnerabilities.
PSEG used this structured prioritization matrix to rank SPVs and help determine which ones should be eliminated (versus mitigated), as well as which SPVs should be addressed first.
The inspectors determined this inspection objective was met.
b. IP 95001 requires that the inspection staff determine that PSEG established a schedule for implementing and completing the corrective actions.
PSEG established a comprehensive schedule for implementing and completing the roughly 150 corrective actions (CAPRs, CRCAs)and action items (ACITs) laid out between the six RCEs. Almost all of the CAPRs, CRCAs and ACITs involving non
-equipment issues have been completed or are scheduled for completion before the end of 2017.
The nature of the equipment modifications planned in response to these events (or other related plant or industry initiatives) are, by their nature, long
-term since they can only be performed during outages and on only one train at a time. For example, PSEG plans to perform action s to mitigate the trip risk posed by the 700+ SPVs identified by their reviews, a time consuming, long
-term action. The following other equipment modifications are planned f or the following number of plant cycles (on both units)
- SW to CFCU 1 inch fill bypass lines
- 2 cycles CFCU gasket design changes
- 5 cycles Relay conversions to digital 3 cycles Eliminating 61 SPV s - 2-3 cycles 25 kV monitoring equipment replacements
- 2 cycles Although, t he progress of the implementation of the above modifications, as well as the completion of key procedural and process CAPRs and CRCAs and their subsequent effectiveness reviews are expected to be inspected during the conduct of future inspections , the inspectors determined this inspection objective was met.
c. IP 95001 requires that the inspection staff determine that PSEG developed quantitative and/or qualitative measures of success for determining the effectiveness of the CAPRs.
Multiple effectiveness reviews (EFFRs) were planned to confirm the successful implementation of the corrective actions to address each RCE. The EFFRs used a balance of qualitative versus quantitative measures based on the nature of the corrective action (e.g., a definitive hardware change versus training or a programmatic or process change). Although many of the EFFRs cannot be completed until the completion of planned plant modifications , interim EFFRs were done where possible
. The EFFRs that were not completed at the time of this inspection are expected to be inspected during the conduct of future inspections; therefore, the inspectors determined this inspection objective was met.
d. IP 95001 requires that the inspection staff determine that PSEG's planned or taken corrective actions adequately address a Notice of Violation (NOV) that was the basis for the supplemental inspection, if applicable.
This supplemental inspection was based on a White PI versus an inspection finding. However, three Green NCVs were issued by the resident inspectors in 2016 for performance issues related to the cause s of three of the five reactor trips.
e. Findings
No findings were identified.
.4 Evaluation
of IMC 0305 Criteria for Treatment of Old Design Issues PSEG did not request credit for self
-identification of an old design issue involving any of these five reactor trips.
.5 Review of PSEG Corrective Actions taken in Response to the 95001 Inspection
Conducted at Salem Unit 1 in August 2015 The team reviewed the corrective actions that addressed the White PI for Unplanned Scrams per 7000 Hours Critical at Salem Unit 1 since the conclusion of the 95001 inspection (05000272/2015
-009; ML15258A467)in August 2015.
The station identified that the corrective actions implemented to address that White PI had not effectively corrected the significant conditions adverse to quality because the station received a repeat White PI for Unplanned Scrams on Uni t 2 one year later.
PSEG's review noted that the four reactor trips at Unit 1 in 2014-2015 were for different reasons, and the focus of the prior efforts was on the recovery of margin and how to prevent crossing the threshold for this White PI, versus improving ER to prevent future scrams/trips.
The involvement of PSEG staff in the prior RCE efforts were also heavily weighted with Regulatory Affairs staff, versus a representative cross
-section of the station Operations, Maintenance
, and Engineering staff who a re focused on equipment issues.
In the 2015 RCEs for the Unit 1 scrams/trips, PSEG identified two root causes: 1) "behaviors of site leadership did not effectively enforce existing processes to ensure specific and adequate actions were developed, challenged, and implemented to prevent operational challenges leading to reactor scrams
- " and , 2) "ineffective knowledge and use of existing processes to develop plans and actions that could have mitigated the risk of the reduced scram margin."
PSEG determined that "governance and oversight was ineffective in identifying weaknesses in the application of fundamentals to prevent reactor scrams" was also a contributing cause. Thus the corrective actions for the previous White PI were focused on enhancing procedural compliance, especially for new employees, as well as enhancing leadership and accountability. However, PSEG found that some of the corrective actions implemented were either too broad or were not effective in the long
-term. Moreover, some corrective actions were changed over time such that they did not meet the intent of the action originally planned. These changes were made without adequate review and approval by the PSEG senior management team.
By contrast, the RCEs prepared for the Unit 2 White PI involved a cross
-section of station staff to ensure the inclusion of viewpoints and buy
-in of the involved departments. The RCEs did touch on the issues noted at Unit 1 in 2014-2015, but focused predominately on enhancing the corrective action process and improving ER in order to prevent reactor scrams/trips.
The RCEs performed in support of this supplemental inspection also looked back to ensure that the causes noted were aligned with the reactor scram/trip issues noted at both Salem units since 2010.
In response to the observation that some corrective actions for the prior White finding experienced significant changes of intent over time, PSEG elected to impose management controls on changes of intent to ensure that such changes receive the proper level of management review and scrutiny. In addition, in contrast to the corrective actions taken for the prior White PI, PSEG is making efforts to improve the reliability of the equipment involved in each of the five reactor trips that contributed to this PI. Thus the team had sufficient basis to conclude that the corrective actions taken in response to this White PI will be more effective than the scram prevention efforts previously undertaken by PSEG. However, since the effectiveness reviews for many of these corrective actions will not be completed for some time, the full effectiveness of these measures are expected to be inspected during the conduct of future inspections.
4OA6 Exit Meetings
On September 15, the inspectors presented the inspection results to Mr. C. McFeaters , Salem Site Vice President, and other members of the PSEG staff. The inspectors verified that no proprietary information was retained by the inspectors or documented in this report.
PSEG management acknowledged and did not dispute the findings.
The results of the inspection were visually summarized using slides that can be found in ADAMS (ML17291B082
).
SUPPLEMENTARY INFORMATION
KEY POINTS OF CONTACT
Licensee Personnel
- P. Sena, Chief Nuclear Officer
C. McFeater
s, Site Vice President
- P. Martino, Plant Manager
- P. Davison, Engineering Vice President
- R. DeNight, Engineer Manager
- K. Anderson, Salem Operations, RCE team leader
- J. Wearne, Regulatory Assurance
Manager
- S. Nevelos, Manager
- Salem Organizational Effectiveness
- M. Brummitt, I & C Supervisor
- F. Possessky, Salem Performance Improvement
- S. Markos, Design Engineering Manager
- K. Hernandez, Salem Engine ering
- J. Fleming, Regulatory Assurance
Director
- R. Flessner, Contractor, Landis Consulting
- K. Landis, Contractor, Landis Consulting
- P. Essner, Engineer
- J. Stead, Engineer
- A. Zheng, Engineer
Other
- E. Rosenfeld, State of New Jersey Bureau of Nuclear Engin eering
LIST OF ITEMS
OPENED, CLOSED AND DISCUSSED
Closed
-002-01 LER Reactor Trip Due to Loss of 4kV Non
-Vital Group Bus (Section 4OA3)
-002-01 LER Automatic Reactor Trip due to Main Turbine Trip
(Section 4OA3)
-005-00 LER Automatic Reactor Trip Due to Main Generator Trip
(Section 4OA3)
LIST OF DOCUMENTS REVIEWED
Procedures
- CC-AA-112, Temporary Configuration Changes, Revision 14
- ER-AA-10, Equipment Reliability Process Description, Revision 3
- ER-AA-2001, Plant Health Committee, Revision 17
- ER-AA-2001-1001, Evaluation of Equipment Reliability Strategies, Revision 1
- ER-AA-2004, System Vulnerability Review Process, Revision 8
- ER-AA-2030, Conduct of Plant Engineering Manual, Revision 14
- ER-AA-1001, Component Classification, Revision 4
- LS-AA-125, Corrective Action Program, Revision 5
- OP-SA-102-1001, Operations Review of Design Changes, Revision 0
- PIA-005, Apparent Cause Evaluation Template, Revision 3
- PIA-020, Corrective Action Closure Board, Revision 1
- PI-AA-120, Issue Identification and Screening Process, Revision 0 S2.OP-AR.GEN-0003(Z), Voltage Regulator Panel Local Alarm Response, Revision 1
- S2.OP-ST.SW-0016 (Q), Inservice Testing of Service Water Accumulator Di scharge Valves, Revision 7
- Orders
- 60132600, Salem Unit 2 Electrical Penetration Minor Repairs
- 70171797, Effectiveness Review for the 2015 White PI at Salem Unit 1
- 70179457, RCE - Salem Unit 2 Reactor and Turbine Trip due to a Loss of the '2H' 4kV
- Non-Vital Bus, Rev. 1
- 70183932, RCE - Unit 2 Reactor Trip on Generator Protection STV1 Relay Actuation, Rev. 3
- 70184453, RCE
- Unit 2 GW relay (62
-C1) trip, Revision 1
- 70187998, RCE
- Main Power Transformer CT wire trip, Revision 2
- 70195447, SPV vulnerability reviews (deep dives)
Notifications
(* written as a result of NRC inspection
)
- 20759233
- 20760362
- 20760363
- 20760364
- 20760366
- 20760367
- 20739025
- 20739026
- 20739027
- 20774108*
- 20774788*
- 20774858* LERs Licensee Event Report 2015
-002-01, Reactor Trip Due to Loss of 4kV Non
-Vital Group Bus Licensee Event Report 2016
-002-01, Automatic Reactor Trip due to Main Turbine Trip Licensee Event Report 2016
-003-00, Automatic Reactor Trip due to
a Loose CT wire in a Main Power Transformer
- Other Change No.
- 8018892, Salem Unit 1 & 2 Turbine Building Roof Drain Bypass Modification Change No.
- 80119078, 62
-C1/C3, 63-C2A, 63-X13 Relay Relocation, Revision 0
- Equipment Reliability Indicator (ERI) trend since 2014
- INPO Event Report 11
-2, 2009 - 2010 SCRAM Analysis
- NSRB Meeting Minutes (1/2015, 5/2015, 9/2015 & 10/2016)
- Nuclear Parts Quality Initiative Test Results
- 2017 YTD
- PIA-023, Performance Analysis
- Gaps in Leadership Decision Making and Questioning Attitude, Revision 1
- Salem 95001 Briefing Book Summer 2017
- Salem Plant Health Committee Meeting Agenda, 9/11/17
- Salem
- ER-AA-2001 Score Card, current as of 9/11/17
- Salem Station Heat Map, current as of 9/11/17
- System Vulnerability Report, Salem Unit 2 Main Generator, April 2016
- List of Acronyms Used
- Apparent Cause Evaluation
Action Item
- Agencywide Documents Access and Management System AVR
- Automatic Voltage Regulator
- C AP
- Corrective Action Program
- Corrective Actions to Preclude Repetition CFCU
- Containment Fan Cooler Unit CFR
- Code of Federal Regulations
- Component Maintenance Optimization
- CRCA
- Conditions Requiring Corrective Action E&CF
- Event and Casual Factor
- EFFR
- Effectiveness Reviews EPRI
- Electric Power Research Institute
- ER
- Equipment Reliability
- HDP
- Heater Drain Pumps
- IMC
- Inspection Manual Chapter
- IP
- Inspection Procedure
- IR
- Inspection Report LER
- Licensee Event Report
- Long Term Asset LTAM
- Long Term Asset M
anag ement MPT
- Main Power Transformer
- Management Review Committee
- NOTF
- Notification
- Non-Cited Violation
- Notice of Violation NRC
- U.S. Nuclear Regulatory Commission
Operating Experience
- Performance Indicator
- Problem Identification and Resolution PRA
- RC
- Root Cause
- Root Cause Evaluation
- Reactor Coolant Pump ROP
- Reactor Oversight Process
- SP V
- Single Point Vulnerability
- Work Order