IR 05000293/2017009

Problem Identification and Resolution Inspection Report 05000293/2017009

ML17355A479
Person / Time
Site:	Pilgrim
Issue date:	12/21/2017
From:	Arthur Burritt NRC/RGN-I/DRP/PB5
To:	Brian Sullivan Entergy Nuclear Operations
Burritt A
References
IR 2017009
Download: ML17355A479 (35)

Text

December 21, 2017

SUBJECT:

PILGRIM NUCLEAR POWER STATION - PROBLEM IDENTIFICATION AND RESOLUTION INSPECTION REPORT 05000293/2017009

Dear Mr. Sullivan:

On September 29, 2017, the U.S. Nuclear Regulatory Commission (NRC) completed onsite activities associated with a problem identification and resolution inspection at your Pilgrim Nuclear Power Station (Pilgrim) and the results of this inspection were discussed with you and other members of your staff. During this discussion, your staff provided additional information for our consideration. The NRC staff performed an in-office review of the information provided; and upon completion of our review a telephonic exit meeting was conducted on November 13, 2017, with Mr. Dave Noyes, Recovery Manager, and other members of your staff. The results of this inspection are documented in the enclosed report.

The NRC inspection team reviewed the stations corrective action program to evaluate its effectiveness in identifying, prioritizing, evaluating, and correcting problems, and to confirm that the station was complying with NRC regulations and licensee standards for corrective action programs. However, the team noted in several cases that the level of detail associated with the documentation of issues being entered into the corrective action program was less than adequate and as a result challenged Pilgrims ability to effectively classify and evaluate issues.

The team also noted that Pilgrim has received a large number of non-cited violations (NCVs)

since the last problem identification and resolution inspection. This volume has created a challenge at times for Entergy staff to ensure issues are fully evaluated and appropriate corrective actions developed. In addition, the team noted the longstanding issues associated with operability determinations and operator fundamentals continue to challenge performance at the site.

The team also evaluated the stations processes for use of industry and NRC operating experience information and the effectiveness of the stations audits and self-assessments.

Based on the samples reviewed, the team determined that your staffs performance in each of these areas adequately supported nuclear safety.

Finally, the team reviewed the stations programs to establish and maintain a safety conscious work environment, and interviewed station personnel to evaluate the effectiveness of these programs. Based on the teams observations and the results of these interviews, the team found no evidence of challenges to your organizations safety conscious work environment. Your employees appeared willing to raise nuclear safety concerns through at least one of the several means available.

The NRC inspection team documented four findings of very low safety significance (Green) in this report. Three of these findings involved violations of NRC requirements. The NRC is treating these violations as NCVs consistent with Section 2.3.2.a of the NRC Enforcement Policy.

If you contest the violations or significance of these NCVs, you should provide a response within 30 days of the date of this inspection report, with the basis for your denial, to the U.S. Nuclear Regulatory Commission, ATTN: Document Control Desk, Washington, DC 20555-0001; with copies to the Regional Administrator, Region I; the Director, Office of Enforcement; and the NRC resident inspector at Pilgrim. If you disagree with a cross-cutting aspect assignment or a finding not associated with a regulatory requirement in this report, you should provide a response within 30 days of the date of this inspection report, with the basis for your disagreement, to the U.S. Nuclear Regulatory Commission, ATTN: Document Control Desk, Washington, DC 20555-0001; with copies to the Regional Administrator, Region I; and the NRC resident inspector at Pilgrim.

This letter, its enclosure, and your response (if any) will be made available for public inspection and copying at http://www.nrc.gov/reading-rm/adams.html and at the NRC Public Document Room in accordance with Title 10 of the Code of Federal Regulations (10 CFR) 2.390, Public Inspections, Exemptions, Requests for Withholding.

Sincerely,

/RA/

Arthur L. Burritt, Chief Reactor Projects Branch 5 Division of Reactor Projects

Docket No. 50-293 License No. DPR-35

Enclosure:

Inspection Report 05000293/2017009 w/Attachment: Supplementary Information

cc w/encl: Distribution via ListServ

SUMMARY

Inspection Report 05000293/2017009; Pilgrim Nuclear Power Station (Pilgrim); Biennial

Baseline Inspection of Problem Identification and Resolution. The inspectors documented four findings in the area of corrective action program (CAP) - two in problem evaluation and two in timely and effective corrective actions.

This U.S. Nuclear Regulatory Commission (NRC) team inspection was performed by two regional inspectors and three resident inspectors. The inspectors identified three non-cited violations (NCVs) and one finding (FIN), all of which were of very low safety significance (Green). The significance of most findings is indicated by their color (i.e., greater than Green, or Green, White, Yellow, Red) and determined using Inspection Manual Chapter (IMC) 0609,

Significance Determination Process, dated October 28, 2016. Cross-cutting aspects are determined using IMC 0310, Aspects Within the Cross-Cutting Areas, dated December 4, 2014. All violations of NRC requirements are dispositioned in accordance with the NRCs Enforcement Policy, dated November 1, 2016. The NRCs program for overseeing the safe operation of commercial nuclear power reactors is described in NUREG-1649, Reactor Oversight Process, Revision 6.

Problem Identification and Resolution

Based on the samples selected for review, the inspection team concluded that Entergy was generally effective in identifying, evaluating, and resolving problems. Entergy personnel were typically effective at identifying problems and entering them into the CAP at a low threshold. In general, Entergy prioritized and evaluated issues commensurate with the safety significance of the problems. Corrective actions were generally implemented in a timely manner. However, the team noted in several cases that the level of detail associated with the documentation of issues being entered into the CAP was less than adequate and as a result challenged Pilgrims ability to effectively classify and evaluate issues. The team also noted that Pilgrim has received a large number of NCVs since the last problem identification and resolution inspection. This volume has created a challenge for Entergy at times for staff to ensure issues are fully evaluated and appropriate corrective actions developed. In addition, the team noted the longstanding issues associated with operability determinations and operator fundamentals continue to challenge performance at the site.

In addition to implementation of the CAP, the inspectors also reviewed Entergys use of operating experience, conduct of self-assessments, and safety conscious work environment at the station. Based on the samples selected for review, the inspectors did not identify any issues with Pilgrims use of industry operating experience. The inspectors concluded that the self-assessments reviewed were generally effective in identifying issues and improvement opportunities. Finally, the inspectors found no evidence of significant challenges to Entergys safety conscious work environment at Pilgrim. Based on the inspectors observations, Entergy staff are willing to raise nuclear safety concerns through at least one of the several means available.

Cornerstone: Mitigating Systems

Green.

An NRC-identified Green NCV of Title 10 of the Code of Federal Regulations (10 CFR) Part 50, Appendix B, Criterion XVI, Corrective Action, was identified because Entergy staff did not promptly identify and correct a condition adverse to quality associated with an issue previously documented in NRC NCV 05000293/2017002-03, Inaccurate Suppression Pool Water Level Instrument Not Identified during Post-Event Prompt Investigation. Specifically, the condition report (CR) for this NCV was closed to the root cause evaluation of the torus level overfill event. However, the performance deficiency was that the post transient review did not identify a problem with one channel of the torus level instrumentation. During the onsite portion of this inspection, no corrective actions were identified which addressed the performance deficiency identified in the NCV; therefore, the condition adverse to quality was not corrected and compliance was not restored. Following identification, Entergys corrective actions included writing CR-2017-8115 and revising station procedure 1.3.34, Operations Administrative Policies and Processes, to ensure that following events in which technical specification (TS) related structures, systems, and components (SSCs) are made inoperable and a prompt investigation is required, all necessary data-gathering to ensure appropriate equipment response is performed within 24 hours2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br />. Additionally, Entergy conducted training to ensure these standards are understood.

These corrective actions restored compliance with the original violation and addressed the identified performance deficiency.

This performance deficiency was more than minor because if left uncorrected, the performance deficiency had the potential to lead to a more significant safety concern.

Specifically, since a post transient review is necessary to identify if operators and equipment responded properly to an event, if the review is not completed in a timely manner, inoperable safety-related equipment may not be identified within TS-allowed outage times.

In accordance with IMC 0609.04 Initial Characterization of Findings, issued October 7, 2016, and IMC 0609, Appendix A, The Significance Determination Process for Findings At-Power, issued June 19, 2012, the inspectors determined that the finding was of very low safety significance (Green), because the finding did not represent a loss of system and/or function, did not represent an actual loss of function of a single train for greater than its TS-allowed outage time, or represent an actual loss of function of one non-TS train designated as high safety significance. The finding had a cross-cutting aspect in the area of Problem Identification and Resolution, Evaluation, because Entergy did not thoroughly evaluate issues to ensure that resolutions address causes and extent of conditions commensurate with their safety significance. Specifically, Entergy did not properly evaluate CR-2017-8115 to ensure that corrective actions addressed the performance deficiency in NCV 05000293/2017002-03. [P.2] (Section 4OA2.1.c.(1))

Green.

An NRC-identified Green finding without an associated regulatory violation (FIN)against Entergy procedure EN-LI-102, Corrective Action Program, was identified because Entergy staff did not ensure that corrective actions were developed to adequately address identified performance issues. Specifically, from January 12, 2017, through the present,

Entergy has not established corrective actions for several performance issues associated with NCV 05000293/2016011-04, Programmatic Issues with Implementation of the Operability Determination Process, and CR-2017-0626. Upon identification, Entergy entered this issue into their CAP as CR-2017-9672 and CR-2017-9673 and conducted an evaluation.

The inspectors determined that this performance deficiency was more than minor because if left uncorrected, it could lead to a more significant safety issue. Specifically, the failure to enter and document a basis for operability could lead to not recognizing inoperable safety-related equipment, and place the reactor at a higher risk of core damage in a design basis accident. In accordance with IMC 0609.04 Initial Characterization of Findings, issued October 7, 2016, and IMC 0609, Appendix A, The Significance Determination Process for Findings At-Power, issued June 19, 2012, the inspectors determined that the finding was of very low safety significance (Green), because it did not affect the design or qualification of a mitigating SSC, represent a loss of system and/or function, involve an actual loss of function of at least a single train or two separate safety systems for greater than its TS-allowed outage time, or represent an actual loss of function of one or more non-TS trains of equipment designated as high safety significant. The inspectors determined that the finding had a cross-cutting aspect in the area of Human Performance, Consistent Process, because Entergy staff did not use a consistent, systematic approach to make decisions, incorporating risk insights as appropriate. Specifically, Entergy staffs overreliance on a complicated series of administrative closures for CRs and corrective actions being closed to other corrective actions hindered their ability to identify that performance issues were not adequately addressed. [H.13]. (Section 4OA2.1.c.(2))

Green.

A self-revealing Green NCV of 10 CFR Part 50, Appendix B, Criterion XVI with two examples was identified because Entergy did not identify and correct a condition adverse to quality associated with the salt service water (SSW) pumps. Specifically, corrective actions developed for previous SSW pump failures did not adequately correct known conditions adverse to quality, resulting in the failure of the B SSW due to shaft spider bearing failures and E SSW due to pump baseplate degradation. Both failures occurred in 2017 and led to unplanned inoperability of these safety-related pumps. Entergy entered both failures into their CAP, conducted evaluations, and repaired the affected pumps, returning them to an operable status.

The inspectors determined that this performance deficiency was more than minor because it adversely affected the Equipment Performance attribute of the Mitigating Systems cornerstone and impacted the cornerstone objective of ensuring the availability, reliability, and capability of systems that respond to initiating events to prevent undesirable consequences (i.e. core damage). Specifically, both the B and E SSW pumps were determined to be inoperable due to vibrations and may not have been able to perform their safety function for the pumps mission times on various occasions. The inspectors evaluated this finding using IMC 0609.04, Initial Characterization of Findings, and IMC 0609, Appendix A, Exhibit 2, Mitigating Systems Screening Questions. The inspectors determined that this finding was of very low safety significance (Green) because the performance deficiency was not a design or qualification deficiency, did not involve an actual loss of safety function, did not represent actual loss of a safety function of a single train for greater than its TS-allowed outage time, and did not screen as potentially risk-significant due to a seismic, flooding, or severe weather initiating event. The finding had a cross-cutting aspect in the area of Problem Identification and Resolution, Resolution, because Entergy did not take effective corrective actions to address issues in a timely manner commensurate with their safety significance. Specifically, timely and effective corrective actions were not developed or implemented for the other SSW pumps as an extent of condition following the identification of the baseplate design being a contributing cause for B SSW failures in 2012, 2014, 2015, and 2017. [P3] (Section 4OA2.1.c.(3))

Green.

An NRC-identified Green NCV of 10 CFR Part 50, Appendix B, Criterion XVI was identified because Entergy did not identify and correct a condition adverse to quality associated with a previous failure of the 125 volt direct current (VDC) control power to the B emergency diesel generator (EDG) which resulted in a similar failure on April 29, 2017.

Specifically, the B EDG experienced a loss of control power when a fuse opened due to degradation over time due to an elevated current surge condition during EDG starts. This same condition occurred on November 6, 2012, however, Entergy did not effectively identify and correct the degraded condition. Specifically, Entergy did not complete corrective actions to perform additional monitoring and troubleshooting during the next refueling outage or maintenance outage window following the November 2012 failure because the work order developed for this task was removed from the outage plan and cancelled.

Entergy entered this issue into their CAP as CR-2017-4563 and developed corrective actions which include monitoring the control power circuit with a recorder to assess for variations in circuit voltage and amperage. Corrective actions also include quarterly replacement of the fuses in the circuit to assess the degradation rate of the fuses, and to increase the replacement rate if any degradation is found. Actions also include weekly analysis of data captured from the recorder. The inspectors also noted that operators received just-in-time training on the manual actions that could be taken to start the EDG using procedure 2.4.16, Distribution Alignment Electrical System Malfunctions.

The inspectors determined that this performance deficiency was more than minor because it adversely affected the objective of the Equipment Performance attribute of the Mitigating Systems cornerstone to ensure the reliability of systems that respond to initiating events to prevent undesirable consequences (i.e. core damage). Specifically, the loss of control power to the B EDG in 2012 was not adequately addressed and a similar issue occurred again in May 2017 which prevented the ability of the EDG to automatically start on a loss of offsite power. The inspectors evaluated this finding using IMC 0609.04, Initial Characterization of Findings, and IMC 0609, Appendix A, Exhibit 2, Mitigating Systems Screening Questions. The inspectors determined that this finding is of very low safety significance (Green) because the performance deficiency was not a design or qualification deficiency, did not involve an actual loss of safety function, did not represent actual loss of a safety function of a single train for greater than its TS-allowed outage time, and did not screen as potentially risk-significant due to a seismic, flooding, or severe weather initiating event. The finding does not have a cross-cutting aspect because the performance deficiency occurred in 2012 and is not indicative of current performance. (Section 4OA2.1.c.(4))

REPORT DETAILS

4OA2 Problem Identification and Resolution

This inspection constitutes one biennial sample of problem identification and resolution as defined by Inspection Procedure 71152, Problem Identification and Resolution, dated February 26, 2015. All documents reviewed during this inspection are listed in the to this report.

.1 Assessment of Corrective Action Program Effectiveness

a. Inspection Scope

The inspectors reviewed the procedures that described the CAP at Pilgrim and evaluated CAP performance since the last NRC biennial problem identification and resolution inspection completed in August 2015. To assess the effectiveness of the CAP, the inspectors reviewed performance in three primary areas: problem identification, prioritization and evaluation of issues, and corrective action implementation. The inspectors compared performance in these areas to the requirements and standards contained in 10 CFR Part 50, Appendix B, Criterion XVI, Corrective Action, and Entergy procedure EN-LI-102, Corrective Action Program (CAP), Revisions 25 through 30.

For each of these areas, the inspectors considered risk insights from the stations risk analysis and reviewed CRs and work tasks selected across the seven cornerstones of safety in the NRCs Reactor Oversight Process. Additionally, the inspectors attended multiple CAP screening committee (Department Performance Improvement Coordinator (DPIC)) meetings, Performance Review Group (PRG) meetings, departmental screening meetings, and operability determination challenge boards. The inspectors selected items from the following functional areas for review: engineering, operations, maintenance, emergency preparedness, radiation protection, chemistry, physical security, nuclear oversight, and the CAP.

(1) Effectiveness of Problem Identification

In addition to the items described above, the inspectors reviewed system health reports, a sample of completed corrective and preventative maintenance work orders, completed surveillance test procedures, operator logs, and periodic trend reports. The inspectors also completed field walkdowns of various systems on site, such as the SSW system, the 4160 and 480 volts alternating current distribution systems, the high pressure coolant injection (HPCI) system, the reactor core isolation cooling (RCIC) system, plant physical security systems, the EDGs, and the diesel fuel oil transfer systems.

Additionally, the inspectors reviewed a sample of CRs written to document issues identified through internal self-assessments, audits, emergency preparedness drills, and the operating experience program. The inspectors completed this review to verify that Entergy staff entered conditions adverse to quality into their CAP as appropriate.

(2) Effectiveness of Prioritization and Evaluation of Issues

The inspectors reviewed the evaluation and prioritization of a sample of CRs issued since the last NRC biennial problem identification and resolution inspection completed in August 2015. The inspectors also reviewed CRs that were assigned lower levels of significance that did not include formal cause evaluations to ensure that they were properly classified. The inspectors review included the appropriateness of the assigned significance, the scope and depth of the causal analysis, and the timeliness of resolution. The inspectors assessed whether the evaluations identified likely causes for the issues and developed appropriate corrective actions to address the identified causes. Further, the inspectors reviewed equipment operability determinations, reportability assessments, and extent-of-condition reviews for selected problems to verify these processes adequately addressed equipment operability, reporting of issues to the NRC, and the extent of the issues.

(3) Effectiveness of Corrective Actions

The inspectors reviewed Entergys completed corrective actions through documentation review and, in some cases, field walkdowns to determine whether the actions addressed the identified causes of the problems. The inspectors also reviewed CRs for adverse trends and repetitive problems to determine whether corrective actions were effective in addressing the broader issues. The inspectors reviewed Entergys timeliness in implementing corrective actions and effectiveness in precluding recurrence for significant conditions adverse to quality. The inspectors also reviewed a sample of CRs associated with selected NCVs and findings previously identified by the NRC to verify that Entergy personnel properly evaluated and resolved these issues. The inspectors expanded the corrective action review to five years to evaluate Entergys actions related to deficiencies associated with the SSW and EDG systems. In accordance with Inspection Procedure 71152, the inspectors also reviewed the status of corrective actions related to greater-than-Green findings that were not completed by the end of the associated supplemental inspection, and were not otherwise reviewed.

(4) Trending

The inspectors reviewed Entergys processes for identifying and addressing emergent and existing adverse trends in equipment and human performance. The inspectors conducted interviews with plant staff who conducted the department trend reviews, reviewed department trend reports, site quarterly trend reports, maintenance rule performance monitoring reports, and a(1) action plans and evaluations as required by 10 CFR 50.65, Requirements for monitoring the effectiveness of maintenance at nuclear power plants. The inspectors also reviewed the minutes from System Health Committee meetings.

b. Assessment

(1) Effectiveness of Problem Identification

Based on the selected samples, plant walkdowns, and interviews of site personnel in multiple functional areas, the inspectors determined that, in general, Entergy identified problems and entered them into the CAP at a low threshold. Entergy staff initiated approximately 28,000 CRs between August 2015 and August 2017. The inspectors observed staff and supervisors at the Plan-of-the-Day meetings, departmental CAP screening meetings, operability determination challenge boards, and DPIC and PRG meetings appropriately questioning and challenging CRs to ensure clarification of the issues.

Based on the samples reviewed, the inspectors determined that, in general, Entergy identified equipment and programmatic issues appropriately and entered these issues into the CAP as CRs. The inspectors verified that conditions adverse to quality and significant conditions adverse to quality identified through this review generally were entered into the CAP as appropriate. In response to several questions and minor equipment observations identified by the inspectors during plant walkdowns, Entergy personnel promptly initiated CRs and/or took immediate action to address the issues.

The inspectors also observed that the DPIC screening committee members went back to the originators of several CRs in order to obtain additional details so the issue was clearly documented in the CAP and could be appropriately evaluated.

However, the inspectors noted in several cases that the level of detail associated with the documentation of issues being entered into the CAP was less than adequate and as a result challenged Pilgrims ability to effectively classify and evaluate issues at times.

Entergy had not identified this potential trend or taken actions to evaluate and correct this weakness. Examples supporting this observation include the following:

CR-2017-9152: A surveillance test on the B EDG could not be completed due to a damper alarm. The CR did not include the fact that there were approximately 45 minutes from the receipt of the alarm until the EDG was shut down as required by the Alarm Response Procedure. This was due to equipment needed to carry out the first step of the Alarm Response Procedure not being available.

CR-2017-09686: Site personnel documented an electrical fire from a lighting fixture near the independent spent fuel storage installation pad. Relevant details such as how long the fire was considered to be burning, that it took over 40 minutes to locate the breaker and de-energize the fixture, and that the fire was not considered out until this occurred as documented in the operator logs were not included in the CR. This information was needed for determining operability, evaluating emergency action level entry criteria, and determining reportability.

CR-2017-9609: A contaminated spill from the RCIC system was not flagged for control room review even through it was a level 3 tag-out error and crew clock reset item. Only the discovery of contamination in a clean area was documented.

CR-2017-9684: Written to document the error from CR 2017-9609 (see above), but identified it as a procedure error which prevented the post-maintenance test from being performed. The fact that the error resulted in an abnormal system configuration, a spill of torus water into the RCIC room, and created radiological and potential operability concerns which needed to be evaluated was not clearly documented. As a result, the operability of containment was not evaluated until the NRC inspection team brought this concern to the stations attention.

In each of these examples, relevant details were not provided to allow Entergy staff to properly screen and/or classify issues. Pilgrims Nuclear Oversight organization identified similar concerns as documented in CR-2017-9183, which independently confirmed the teams observation. Each example was evaluated using IMC 0612, Appendix B, Issue Screening, and IMC 0612, Appendix E, Examples of Minor Issues.

Each CAP violation above was determined to be of minor significance. The teams observations were entered into Entergys CAP.

(2) Effectiveness of Prioritization and Evaluation of Issues

The inspectors determined that, in general, Entergy appropriately prioritized and evaluated issues commensurate with the safety significance of the identified problem.

Entergy staff screened CRs for operability and reportability, categorized the CRs by significance, and assigned actions to the appropriate department for evaluation and resolution. The CR screening process considered human performance issues, radiological safety concerns, repetitiveness, adverse trends, and potential impact on the safety conscious work environment.

Based on the sample of CRs reviewed, the inspectors noted that the guidance provided by Entergy CAP implementing procedures was sufficient to ensure consistency in categorization of issues. Operability and reportability determinations were generally performed when conditions warranted, and, in most cases, the evaluations supported the conclusion. Causal analyses appropriately considered the extent of condition or problem, generic issues, and previous occurrences of the issue. Root cause evaluations and apparent cause evaluations reviewed were completed when required and received management review prior to approval.

The team recognized that Pilgrim has received a large number of NCVs since the last problem identification and resolution inspection (68 NCVs). This volume has created a challenge for Entergy staff to ensure issues are fully evaluated and appropriate corrective actions developed. In many cases, multiple NCVs were closed to the same master CR. In some cases, the inspectors found that this practice contributed to development of corrective actions which did not fully address the underlying issue or did not adequately address the performance deficiency and violation. Examples include the following:

NCV 05000293/2017-002-03: The CR for this NCV was closed to the root cause evaluation of the torus level overfill event. However, the performance deficiency was that the post transient review did not identify a problem with one channel of the torus level instrumentation. During the onsite portion of this inspection, no corrective actions were identified which addressed this performance deficiency; therefore, the condition adverse to quality was not corrected and compliance was not restored.

After the onsite portion of this inspection, corrective actions were developed and implemented restoring compliance. See Section 4OA2.1.c.(1) for additional information.

NCV 05000293/2017-002-04: The CR for this NCV was closed to the root cause evaluation done for the torus water level overfill event. The root cause report identified a significant condition adverse to quality related to operations leadership not reinforcing consistent application of operator fundamentals throughout the operations department. Corrective actions were developed which included fleet level training, but did not address weaknesses with operator fundamentals and the use of human performance error prevention techniques which were the direct cause of the event. Given that there have been other notable operator fundamental errors and failures to apply human performance error prevention techniques recently, the underlying weakness still needs to be addressed.

NCV 05000293/2016-011-04: This NCV identified a weakness with the operability determination process. This NCV was closed to a master CR which numerous issues were closed to. The NCV identified four examples of inadequate operability determinations; however, only one of the four was addressed in the corrective actions. Further review identified that many corrective actions were closed to multiple products in a circular fashion and no actual actions were taken. Given that there were several additional examples of continuing weaknesses in the operability determination process, including an unexplained oil loss from the HPCI pump, EDG jacket water leaks, and moving fuel with inoperable source range monitors, it appears these corrective actions have not been fully effective. See 4OA2.1.c.(2) for additional details.

The team identified that the apparent cause evaluation for the failure of the E SSW pump was inadequate. CR-2017-7096 failed to identify all of the contributing causes to the failure of the E SSW pump and therefore did not develop corrective actions as required. Specifically, the installation of the original style motor which had a different vibration and damping signature than the newer model motor, was identified by two engineers as one of the reasons the baseplate failed so soon after repairs. However, this was not listed as a contributing cause and the old style motor remained in the warehouse and was available for use with no restrictions. CR-2017-9648 was written to capture the teams observation, confirmed the motor had not been used since it was removed from the E SSW pump, and placed administrative controls restricting the future use of the pump. There were no adverse consequences from this error. The issue was evaluated using IMC 0612, Appendix B, Issue Screening, and IMC 0612, Appendix E, Examples of Minor Issues, and was determined to be of minor significance.

The inspectors identified two findings associated with Entergys prioritization and evaluation of issues. In addition, the team noted the longstanding issues associated with operability determinations and operator fundamentals continue to challenge performance at the site.

Continued Weaknesses Associated With Operator Fundamentals

In Inspection Report 05000293/2017002, the NRC documented a self-revealing Green NCV of TS 5.4.1.a, Procedures, associated with improper system restoration, which resulted in the torus being declared inoperable. Entergy documented this finding in their CAP under CR-2017-8117, which was closed to evaluations and actions taken within CR-2017-2785, which contained the root cause analysis for the event.

Entergy determined that the root cause for the torus water level overfill event was the failure of operations leadership to reinforce consistent application of operator fundamentals throughout the operations department. Contributing causes included how Entergy monitored and assessed crew and individual performance in the area of operator fundamentals and gaps in previous evaluations of human performance related events. Operator fundamentals include: a) closely monitoring plant conditions; b) precisely controlling the plant evolutions; c) a bias toward conservatism; d) working effectively as a team; and e) having a solid understanding of plant design and theory.

The use of human performance error prevention techniques such as peer and self-checking and STAR (stop, think, act, review) are used to ensure precise control of plant events and avoid configuration control errors.

Although one-time training was performed as a corrective action to address weaknesses in operator fundamentals, the inspection team noted that a number of recent events illustrate that actions taken to date may not have been effective at addressing the issue.

The inspection team recognized that, in this case, Entergy had not yet evaluated the effectiveness of their corrective actions per their CAP guidance, and that several programmatic corrective actions related to monitoring and assessing crew performance were still being implemented and may need some time to show results. The team determined this did not constitute a violation of 10 CFR Part 50, Appendix B, Criterion XVI at this time. Recent examples include:

On August 16, 2017, while operators were isolating instrument air dryer X-105 for a tagout, they partially closed the instrument air filters combined outlet valve instead of the individual instrument air filter outlet block valve as called for on the tagging sheet.

This resulted in instrument air header pressure lowering, the receipt of the associated annunciators in the main control room, and the diesel air compressor to start. This is an example of failure to use human performance error prevention techniques of peer and self-checking to ensure the proper equipment is operated.

On September 27, 2017, during a RCIC maintenance window, operators attempted to align the RCIC torus suction valve for power supply breaker post-maintenance testing, under a temporary tagout lift. The operators did not recognize the system as not in its normal configuration and that the post-maintenance testing evolution could not be performed in that line up. As a result, when operators opened the torus suction valve, they observed water spilling out of a danger tagged open and uncapped drain valve onto the reactor building floor. This is an example of failure to use human performance error prevention techniques of STAR to ensure expected system response is understood prior to operating primary containment isolation valves in an off-normal system configuration.

On October 14, 2017, control room operators received multiple alarms including Main Stack Rad Hi, Hi-Hi, and the B 24 VDC Undervoltage alarms in the main control room. Operators identified degraded voltage on the 24 VDC Bus 'B' and that the bus was aligned to the temporary battery charger which had been deenergized.

The cause of the loss of 24 VDC battery charger 'B' was when a tagout was cleared on breaker B2232B (temporary power for the battery charger), the breaker was opened inappropriately. This event was still under review by the resident staff. This is an example of failure to conduct peer and self-checking and STAR to ensure the plant equipment was configured correctly with equipment in an off normal line up.

The NRC will review the licensees progress in addressing the weakness in operator fundamentals and evaluating the effectiveness of the corrective actions and corrective actions to preclude repetition for this significant condition adverse to quality as part of the confirmatory action letter close-out inspections.

Continued Weakness Associated with the Operability Determination Process

The inspection team identified that the weaknesses in the Operability Determination Process continue based on several recent examples. See Section 4OA2.1.c.(2) for additional details.

(3) Effectiveness of Corrective Actions

The inspectors concluded that corrective actions for identified deficiencies were generally timely and adequately implemented. For significant conditions adverse to quality, Entergy identified actions to prevent recurrence. The inspectors concluded that, in most cases, corrective actions to address the sample of NCVs and findings since the last problem identification and resolution inspection were timely and effective.

Nonetheless, the inspectors did identify two findings related to inadequate or incomplete corrective actions which did not correct longstanding conditions adverse to quality. See the five year review section below for details

Five Year Reviews

Inspectors conducted five year reviews for the EDG and SSW systems. This review identified a number of long-term conditions with both systems. The five year review for the EDGs identified one finding related to failure to correct a condition adverse to quality related to the B control power circuit resulting in repetitive failures for the EDG to start on demand. See Section 4OA2.1.c.(4) for additional details.

Additionally, the team observed that Entergys apparent cause evaluation for the May 2017 B EDG failure had established an action to perform an extent of condition review for the May 2017 failure. During that review, it was identified that the station did not adequately address frequently occurring ground alarms. One example of this included a January 2017 CR, CR-2017-0042, which identified a ground alarm on the 125 VDC battery system. The CR was closed by Entergy personnel without considering the impact on supported systems or performing additional investigation to determine the cause. The causal evaluation identified that the CR was closed to a work order, and the ground alarm continued to occur randomly over 1000 times during the following three month period. The cause of the grounds was never determined but ground alarms did precede the control power fuses blowing for the B EDG in both 2012 and 2017, so Entergy missed an opportunity to troubleshoot further.

The five year review of the SSW system identified concerns with a number of long term corrective actions related to repetitive problems with SSW due to vibrations.

Vulnerabilities associated with SSW pump vibrations have been long-term design weaknesses for this application, such as the fact the pump is operated at a frequency with low margin to a natural resonance frequency and the pump baseplate design not being robust; however, compensatory measures and recent corrective actions have not been effective. See Section 4OA2.1.c.(3) for additional details.

Review of Corrective Actions related to greater-than-Green Findings that were not Completed by the End of the Associated Supplemental Inspection

Long term corrective actions related to a White findings and performance indicators, such as effectiveness reviews, which were not completed before the conclusion of the associated supplemental inspection were reviewed by the problem identification and resolution team. No issues were identified.

(4) Trending

The inspectors reviewed Entergys processes for identifying and addressing emergent and existing adverse trends in equipment and human performance. In general, Entergy was able to identify trends at a low level using their department trending process. These trends were rolled up to station level on a quarterly basis, and action and monitoring plans were developed as appropriate. The CAP screening team and system engineers also identified potential trends during their screening meeting and elevated the significance of low-level issues based on the identification of potential trends.

The inspectors noted that Entergy has made significant improvements in its maintenance rule program since the last problem identification and resolution team inspection and the Inspection Procedure 95003 Phase C team inspection. These improvements include revising and updating the maintenance rule basis documents and performance criteria, and revising the process for which maintenance rule determinations are tracked in the CAP. However, some process changes were not well communicated to plant staff and gave the impression that CAP items could be closed with open corrective actions related to maintenance rule functional failure determinations. Additionally, the inspectors identified some instances of systems having a maintenance rule a(1) action plan where the corrective actions were delayed by up to 12 months, despite having the equipment necessary to make associated repairs on hand. 10 CFR 50.65 does not have any requirements related to timeliness or interface with the CAP, so neither observation would be considered a violation of NRC requirements. Therefore, as of the time of this inspection, the stations maintenance rule performance monitoring program was generally effective in evaluating system performance and identifying trends.

c. Findings

(1) Untimely Corrective Actions to Address Previously Identified NCV

Introduction:

An NRC-identified Green NCV of 10 CFR Part 50, Appendix B, Criterion XVI, Corrective Action, was identified because Entergy staff did not promptly identify and correct a condition adverse to quality associated with an issue previously documented in NCV 05000293/2017002-03, Inaccurate Suppression Pool Water Level Instrument Not Identified during Post-Event Prompt Investigation. Specifically, the CR for this NCV was closed to the root cause evaluation of the torus level overfill event.

However, the performance deficiency was that the post transient review did not identify a problem with one channel of the torus level instrumentation. During the onsite portion of this inspection, no corrective actions were identified which addressed this performance deficiency; therefore, the condition adverse to quality was not corrected and compliance was not restored. After the onsite portion of this inspection, corrective actions were developed and implemented restoring compliance.

Description:

On March 31, 2017, following a pre-refueling outage flush of the A core spray system piping, operators positioned a valve (MO-1400-3A) out of sequence while aligning the system for standby operation. The mispositioned valve allowed approximately 55,000 gallons of water to gravity-drain due to the height difference between the condensate storage tank and the torus. This resulted in the torus water level rising by approximately 10 inches. The increase in torus water level exceeded the torus water level limits, and operators entered TS 3.7.A.1.b for being above the maximum torus water volume of 94,000 cubic feet. Operators also entered TS 3.7.A.8.a, Drywell and Suppression Chamber Differential Pressure, for the differential pressure being below 1.17 pounds per square inch between the drywell and suppression chamber. During the event, the A channel of torus water level deviated from the B channel as much as 14.75 inches. After the flow from the condensate storage tank was secured, the indicated levels converged.

As part of the response to this event, Entergy staff performed a prompt Investigation using the guidance in Procedure EN-FAP-OM-012, Prompt Investigations and Notifications. As part of the NRCs follow-up to this event, inspectors determined that Entergy staff did not identify, during their post transient review, that the A torus water level wide range instrument had provided inaccurate level indications during the event.

When informed of this, operators declared the A torus water level wide range instrument inoperable on April 6, 2017, and performed corrective maintenance to restore the instrument to operability. The NRC issued NCV 05000293/2017002-03 for failure to identify and correct a condition adverse to quality. The performance deficiency documented was that Entergy failed to identify the condition. This was due to the fact the post transient review was initiated but never completed. Entergy staff did not complete this review based on a misunderstanding that this review was not required, given that a reactor transient had occurred, this transient had not resulted in a reactor trip. The purpose of the post transient review is to gather all relevant equipment and human performance data to ensure immediate performance issues are identified and corrective and/or compensatory actions can be developed. Since an apparent cause evaluation can take up to 30 days to be completed, the post transient review is necessary to identify whether operators and equipment responded as expected and take timely actions.

Entergy staff entered the torus overfill event into the CAP as CR 2017-02965, conducted a root cause analysis, and corrected the torus level indication problem on April 10, 2017.

Following the issuance of NCV 05000293/2017002-03, Entergy entered the issue into the CAP as CR-2017-8115. This CR was closed to the root cause analysis performed for the event. While this is allowed, corrective actions must be added to the CR it is being closed to in order to ensure the issue is fully resolved. In this case, the corrective actions associated with the closure of CR-2017-8115 did not address the specific performance deficiency identified in NCV 05000293/2017002-03, nor were there any planned additional actions to fully address the conditions described. CR closure reviews did not identify this. Thus, no corrective actions were developed to correct the cause of the performance deficiency and compliance had not been restored as required.

Following the on-site portion of this inspection, Entergy re-opened and revised CR-2017-8115 and created additional corrective actions to revise station procedure 1.3.34, Operations Administrative Policies and Processes, to require that, following events in which TS related SSCs are made inoperable and a prompt investigation is required, all necessary data-gathering to ensure appropriate equipment response is performed within 24 hours2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br />, in accordance with EN-FAP-OM-012. Additionally, Entergy conducted training on the expectations to document any equipment issues observed during a post transient review. These actions restored compliance for NCV 05000293/2017002-03.

Analysis:

The inspectors determined that Entergy staff did not correct the performance deficiency associated with previously issued NCV 05000293/2017002-03, in accordance with procedure EN-LI-102, Corrective Action Program. Specifically, CR-2017-8115 documented NCV 05000293/2017002-03, but the associated corrective actions did not ensure that compliance was restored. The inspectors determined that this was a performance deficiency reasonably within Entergys ability to foresee and correct, and should have been prevented. This performance deficiency was more than minor because if left uncorrected, the performance deficiency had the potential to lead to a more significant safety concern. Specifically, failure to address deficiencies associated with the implementation of guidance contained in procedure EN-FAP-OM-012, Prompt Investigations and Notifications, could result in equipment malfunctions not being identified following plant events, and subsequently not being corrected to ensure availability and reliability, as was the case during the event on March 31, 2017. In accordance with IMC 0609.04 Initial Characterization of Findings, issued October 7, 2016, and IMC 0609, Appendix A, The Significance Determination Process for Findings At-Power, issued June 19, 2012, the inspectors determined that the finding was of very low safety significance (Green), because the finding did not represent a loss of system and/or function, an actual loss of function of a single train for greater than its TS-allowed outage time, or an actual loss of function of one non-TS train designated as high safety significance. Inspectors determined that the finding had a cross-cutting aspect in the area of Problem Identification and Resolution, Evaluation, because Entergy did not thoroughly evaluate issues to ensure that resolutions addressed causes and extent of conditions commensurate with their safety significance. Specifically, Entergy did not properly evaluate CR-2017-8115 to ensure that corrective actions addressed the performance deficiency in NCV 05000293/2017002-03. This performance deficiency involved Entergy not identifying that one of the torus level instruments was inoperable during the post event review. [P.2]

Enforcement:

10 CFR Part 50, Appendix B, Criterion XVI requires, in part, that measures shall be established to assure that conditions adverse to quality, such as failures, malfunctions, deficiencies, deviations, defective material and equipment, and non-conformances are promptly identified and corrected. Contrary to the above, from August 14, 2017, to October 17, 2017, Entergy did not promptly identify and correct the performance deficiency identified in NCV 05000293/2017002-03. Specifically, Entergy did not develop corrective actions to correct a previously documented vulnerability associated with the implementation of guidance in procedure EN-FAP-OM-012, Prompt Investigations and Notifications, when performing station procedure 1.3.34, Operations Administrative Policies and Processes. Entergys corrective actions included revising procedure 1.3.34 to ensure that all necessary data gathering associated with prompt investigations was performed within 24 hours2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br />, and in accordance with procedure EN-FAP-OM-012. Because this violation was of very low safety significance (Green) and was entered into Entergys CAP as CR-2017-8115, the NRC is treating this as an NCV, consistent with Section 2.3.2.a of the NRC Enforcement Policy. (NCV 05000293/2017009-01, Untimely Corrective Actions to Address Previously Identified Non-Cited Violation)

(2) Failure to Correct Programmatic Issues with Implementation of the Operability Determination Process

Introduction:

An NRC-identified Green finding without an associated regulatory violation (FIN) against Entergy procedure EN-LI-102, Corrective Action Program, was identified because Entergy staff did not ensure that corrective actions were developed to adequately address identified performance issues. Specifically, from January 12, 2017, through the present, Entergy has not established corrective actions for several performance issues associated with NCV 05000293/2016011-04, Programmatic Issues with Implementation of the Operability Determination Process, and CR-2017-0626.

Description:

In Inspection Report 05000293/2016011 (ML17129A217), the inspectors identified a Green NCV of 10 CFR Part 50, Appendix B, Criterion V, Instructions, Procedures, and Drawings, (NCV 05000293/2016011-04, Programmatic Issues with Implementation of the Operability Determination Process). To support this NCV, the inspectors documented four examples where Entergy did not enter the operability determination process and/or did not document operability determinations with sufficient detail to reach an operability conclusion. The underlying issues involved the lack of technical rigor, inadequate common cause evaluations, over reliance on engineering judgement rather than applying the design bases criteria, and inadequate documentation. Entergy captured this NCV within their CAP as CR-2017-0626.

CR-2016-1340 was a master CR which contained the apparent cause evaluation completed for the programmatic weaknesses in the operability determination process identified during pre-inspection self-assessments for the NRC Inspection Procedure 95003 Phase C team inspection. It was a common practice for CRs documenting operability determination process issues to be closed to this master CR. At the time of this problem identification and resolution inspection, CR-2016-1340 was in excess of 7000 pages and contained over 120 corrective actions.

The NRC problem identification and resolution inspection team observed that CR-2017-0626 was closed to CR-2016-1340, CA-104. The inspectors observed that while all four NRC examples were captured within a corrective action (CA-104), CA-104 was administratively closed to other corrective actions.

Entergy wrote CR-2017-0902, which self-identified that examples 1 and 2 had not been addressed, however examples 3 and 4 had also not been addressed. Entergy identified that some corrective actions written to address the NCV examples were closed to each other without any actual actions taken. Subsequently, CR-2016-1340, CA-110, established and implemented a corrective action plan to brief qualified causal evaluators on the lesson learned that if new information is revealed concerning a failure mechanism that has not been previously considered in the operability determination, a CR must be generated. This corrective action only addresses the performance issues documented in example 2. Many of the corrective actions associated with the closure of NCV 05000293/2016011-04 examples were closed by April 5, 2017.

The inspectors identified that CR-2016-1340, CA-115 and CA-120 were actions that are also associated with NCV 05000293/2016011-04. CA-115 was a detailed comparative analyses incorporated into licensed operator continuing training which discusses the finding, the associated examples, as well as the internal and regulatory expectations.

CA-120 directs the performance of lessons learned briefings to the engineering staff, supervisors, and management. However, these corrective actions were not intended to address the specific performance issues and were not linked to the corrective actions associated with addressing them. CA-115 and CA-120 were both closed in August 2017. Thus, with the closure of corrective actions in April and August 2017, the performance issues documented in examples 1, 3, and 4 were never corrected. This was not identified during closure reviews as required by Entergy procedure EN-LI-102, Corrective Action Program (CAP).

During the course of this inspection, the inspectors discovered multiple additional examples demonstrating that the performance issue identified in the NCV have not been corrected. The team based its conclusion on the following recent issues:

CR-2017-3541: During fuel movements conducted on April 14 and 15, 2017, in reactor quadrant B, source range monitor channel B was inoperable due to not meeting the signal to noise ratio of no less than 2 to 1, as required by Final Safety Analysis Report, Section 7.5.4.1. On April 28, Entergy acknowledged that, on April 14, the channel had been inoperable and fuel movements should not have been permitted. This issue was documented in Inspection Report 05000293/2017002, Section 4OA7 as a licensee-identified NCV. This is similar to examples 1 and 4 from NCV 05000293/2016011-03.

CR-2017-05256: On May 5, 2017, Entergy identified that water was overflowing from a catch containment that had been installed to capture water leaking through the spent fuel pool concrete structural wall as observed by Entergy during the spring 2017 refueling outage. The associated operability determination evaluated the effects of the overflowing water on impacted cable trays; however, it did not address the effects of the water passing through the structural concrete or the impact of the leakage upon the spent fuel pool and cooling system. Entergy wrote CR-2017-9637 as a result of the inspectors questions on September 28, 2017, and performed an adequate operability assessment which supported current operability. This is similar to example 3 of NCV 05000293/2016011-03.

CR-2017-9122: On September 11, 2017, the inspectors observed a meeting of the Operational Safety Review Committee (OSRC) which reviewed a procedure change to 8.M.2-2.10, Diesel Generator B Initiation by Loss of Offsite Power Logic - Critical Maintenance, scheduled for execution that week. The A train equivalent test procedure had been revised and performed the previous week. The change to the B train initiation by loss of offsite power logic test was primarily a copy of the A train change. OSRC rejected this B train product on September 11, 2017. These concerns called into question the common cause impact of the test the previous week on the A train. CR-2017-9122 was generated for the rejection of the test procedure but did not evaluate the potential impact of operability on the A train.

CR-2017-9185 was written to address this operability concern; however, this CR did not provide adequate information to determine operability. CR-2017-9631 was written to capture the teams observation. This is similar to example 2 of NCV 05000293/2016011-03.

CR-2017-9683: During performance of Operations weekly lubrication procedure, it was discovered that the HPCI turbine lube oil reservoir level was below specification and several inches below the level observed the previous week. Operators added 26 gallons of oil to the reservoir to restore level. The operability determination did address the minimum oil level to support operability. However, it did not evaluate the impact of the leakage rate on the capability of the HPCI pump turbine to support mission time, or assess the impact of the lost oil inventory internally to the system as no external leakage was identified. This is similar to example 1 of NCV 05000293/2016011-03.

Given that the corrective actions associated with CR-2017-0626 were improperly closed without verifying they were adequate to address the identified performance issues, and based upon recent evidence that the performance issues addressed by these corrective actions have not been corrected, Entergy has not met the standards of EN-LI-102. This constitutes a performance deficiency. Entergy has documented the teams observations within their CAP as CR-2017-9672 and CR-2017-9673.

Analysis:

Inspectors determined that Entergys failure to correct performance issues associated with programmatic issues with implementation of the operability determination process was a performance deficiency reasonably within licensees ability to foresee and prevent. The performance deficiency was more than minor because, if left uncorrected, it could lead to a more significant safety issue. Specifically, the failure to enter and document a basis for operability could lead to not recognizing inoperable safety-related equipment, and place the reactor at a higher risk of core damage in a design basis accident. Additionally, IMC 0612, Appendix E, Examples of Minor Issues, dated August 11, 2009, example 4a, associated with the routine failure to perform engineering evaluations, informed more than minor screening for this issue of concern.

In accordance with IMC 0609.04 Initial Characterization of Findings, issued October 7, 2016, and IMC 0609, Appendix A, The Significance Determination Process for Findings At-Power, issued June 19, 2012, the inspectors determined that the finding was of very low safety significance (Green), because it did not affect the design or qualification of a mitigating SSC, represent a loss of system and/or function, involve an actual loss of function of at least a single train or two separate safety systems for greater than its TS-allowed outage time, or represent an actual loss of function of one or more non-TS trains of equipment designated as high safety significant.

The inspectors determined that the finding had a cross-cutting aspect in the area of Human Performance, Consistent Process, because Entergy staff did not use a consistent, systematic approach to make decisions, incorporating risk insights as appropriate. Specifically, Entergy staffs overreliance on a complicated series of administrative closures for CRs and corrective actions being closed to other corrective actions hindered their ability to identify that performance issues were not adequately addressed. [H.13].

Enforcement:

Entergy did not ensure corrective actions were developed which adequately addressed identified performance issues in accordance with EN-LI-102, Corrective Action Process, Revisions 28 and 29. The NRC team did not identify a violation of regulatory requirements associated with this finding since EN-LI-102 is not a procedure specifically required by TSs or NRC Regulatory Guide 1.33, Revision 2.

Because this finding does not involve a violation and is of very low safety or security significance (Green) it is identified as a finding (FIN). (FIN 05000293/2017009-02, Failure to Correct Programmatic Issues with Implementation of the Operability Determination Process)

(3) Inadequate Corrective Actions for SSW Pump Issues Result in Multiple Pump Failures

Introduction:

A self-revealing Green NCV of 10 CFR Part 50, Appendix B, Criterion XVI with two examples was identified because Entergy did not identify and correct a condition adverse to quality associated with the SSW pumps. Specifically, corrective actions developed for previous SSW pump failures did not adequately correct known conditions adverse to quality, resulting in the failure of the B SSW due to shaft spider bearing failures and E SSW due to pump baseplate degradation. Both failures occurred in 2017 and led to unplanned inoperability of these safety-related pumps.

Description:

The safety-related SSW system at Pilgrim consists of two loops, each containing two SSW pumps, along with a fifth pump that can be connected to either loop.

The safety function of the SSW system is to provide cooling to the reactor building closed cooling water system, which cools emergency and safety-related systems, during all modes of operation. The pumps are a vertical deep well style pump design with a 42 foot long vertical shaft running from the service water bays to the pump motors inside the service water building. The shaft is supported by a single casting with an inner hub with five spokes and an outer ring. A bearing is fitted inside the inner ring and the outer ring is pinned at the column flange to prevent rotation of the bearing assembly. The bearing retainers are designed to allow a limited amount of float. This assembly is referred to as a spider bearing.

Since the 1980s, the Pilgrim station has identified concerns that this pump design was a poor choice for the SSW system application. The natural resonance frequencies for the pump shaft assemblies are very close to the running frequency of the pumps, with the B and E SSW pumps having the least margin. This was confirmed in 2015 following a study performed by a vendor. Operation at or near a pumps natural frequency can result in amplified vibrations and subject the pump shaft components to high levels of vibration related stresses, accelerated degradation, and premature failures.

One of the means to ensure these stresses are minimized is to ensure the pump is securely fastened to a sturdy baseplate and keep the pumps balanced. However, in the mid 1990s, it was identified that the baseplate design was inadequate in that it was not sized large enough to ensure it would not degrade due to the vibrations undergone during normal operations. The baseplate has an aluminum-bronze 3/4 thick frame and is filled with concrete. Over time, the concrete is pulverized and voids form in the baseplate which creates a soft foot condition. This makes the pump very difficult to balance and keep balanced. Instead of addressing the underlying design issues, Pilgrim developed a preventative maintenance task to re-grout the baseplates during the pumps 5 year overhauls, where grout was injected into the baseplate to fill any voids which had formed during the operating cycle. While this compensatory measure was effective for a number of years, there was evidence that this measure was becoming less and less effective. Evaluations following SSW pump failures in 2010, 2012, 2014, and 2015 all identified the baseplate design as a contributing factor to those failures.

On November 7, 2015, the A SSW pump was declared inoperable after divers discovered debris at the bottom of the SSW bay, identified to be pieces of the pumps spider bearings. Entergy completed an equipment apparent cause evaluation (CR-2015-09189) and developed a corrective action to establish as a requirement the recommendation of 3.M.4-14.2, Salt Service Water Pumps: Routine Maintenance, 6, which recommended the installation of up to three additional anti-rotation pins per vendor drawings. This modification was completed on the A, B, and E SSW pumps during their 5 year overhauls in 2016.

However, when the additional anti-rotation pins were installed, the bearing housing retainer was now firmly fixed in place and not allow to float. This stiffened and subjected the bearing retainer to the shaft vibrations, resulting in wear and fatigue. Given the known susceptibility to vibrations due to the natural frequency and known baseplate degradation issues, it was reasonable that Entergy staff should have identified and evaluated this change for new potential failure mechanisms as part of the engineering change package done for this modification.

For the B SSW pump, the modification was installed in February 2016. The pump was declared inoperable on May 6, 2017, due to vibration levels exceeding the action-required level of the pumps in-service testing (IST) program. On May 9, 2017, divers discovered several pieces of the spider bearings on the service water bay seabed. It was determined that the corrective actions to address extent of condition from the spider bearing failure on the A SSW pump (after 6 years in service) actually made the condition worse and resulted in premature failure of the B SSW pump spider bearing after just over a year in service. Additionally, voids were identified in the baseplate for the B SSW in 2014 and 2015 and following the 2017 spider bearing failure. Again, these voids were considered a contributing cause to the failure. However, following this failure, the baseplate for the B SSW pump was replaced with a more robust design in May 2017. No extent-of-condition review was done for this issue, and no actions were developed or scheduled to be performed for the other four SSW pump baseplates.

The E SSW pump was overhauled in November 2016. The anti-rotation pin modification was installed and the baseplate was re-grouted in accordance with 3.M.4.4-14.2. Following this maintenance, Entergy personnel had issues balancing the pump.

On five occasions, between February and July 2017, the E SSW pump vibrations were measured to be out of tolerance, twice exceeding the IST program action-required levels, resulting in the pump being declared inoperable. The pump had to be rebalanced multiple times until voids of up to 25 percent were discovered in the E SSW baseplate in July 2017. Entergy concluded that the re-grouting evolution had been performed properly and there were no performance issues associated with the maintenance. It appears that the baseplate began significantly degrading within three months of the re-grouting process.

The similarity in margin to natural resonance for the B and E SSW pumps documented in the 2015 vendor study and the repetitive identification of premature void formation in the B SSW pump baseplate in 2014, 2015, and 2017 indicated that the compensatory measure was no longer effective. Entergy had the opportunity to address the baseplate issue for the both the B and E pumps during their 5 year overhauls in 2016.

Additionally, corrective actions implemented for the B SSW pump in 2017 for the same condition were neither implemented nor scheduled for the E SSW pump. The opportunity was available during a re-performance of the overhaul in August 2017, during which Entergy replaced the pump motor and re-grouted the baseplate, but did not replace/upgrade the baseplate, as had been done for the B pump. Entergy initiated an apparent cause evaluation and scheduled the baseplate to be replaced with the more suitable design in early 2018.

Corrective actions were also developed to plan replacement of the baseplates for the remaining SSW pumps on an as-needed basis using IST program monitoring data. The inspectors reviewed these actions and concurred that the current IST program monitoring is adequate to ensure the pumps remain operable until the baseplates are replaced due to the fact that the remaining pumps have additional margin and are being monitored at an increased periodicity such that baseplate degradation can be identified prior to operability being challenged.

Analysis:

The inspectors determined that Entergys failure to correct the condition adverse to quality, in accordance with EN-LI-102, associated with the B and E SSW pumps was a performance deficiency within Entergys ability to foresee and correct and should have been prevented. The inspectors determined that this performance deficiency was more than minor because it adversely affected the Equipment Performance attribute of the Mitigating Systems cornerstone and impacted the cornerstone objective of ensuring the availability, reliability, and capability of systems that respond to initiating events to prevent undesirable consequences (i.e. core damage). Specifically, both the B and E SSW pumps were determined to be inoperable due to vibrations and may not have been able to perform their safety function for the pumps mission times on various occasions.

The inspectors evaluated this finding using IMC 0609.04, Initial Characterization of Findings, and IMC 0609, Appendix A, Exhibit 2, Mitigating Systems Screening Questions. The inspectors determined that this finding is of very low safety significance (Green) because the performance deficiency was not a design or qualification deficiency, did not involve an actual loss of safety function, did not represent actual loss of a safety function of a single train for greater than its TS-allowed outage time, and did not screen as potentially risk-significant due to a seismic, flooding, or severe weather initiating event. Specifically, the SSW systems safety function was not lost, due to the fact that the required number of other pumps in the system remained operable.

The finding had a cross-cutting aspect in the area of Problem Identification and Resolution, Resolution, because Entergy did not take effective corrective actions to address issues in a timely manner commensurate with their safety significance.

Specifically, timely and effective corrective actions were not developed or implemented for the other SSW pumps as an extent of condition following the identification of the baseplate design being a contributing cause for B SSW failures in 2012, 2014, 2015, and 2017. [P3]

Enforcement:

10 CFR Part 50, Appendix B, Criterion XVI requires, in part, that measures shall be established to assure that conditions adverse to quality are promptly identified and corrected. Contrary to the above, two examples were identified were corrective action did not correct the known condition adverse to quality.

(1) From February 2016 until May 2017, corrective actions implemented to address an identified spider bearing issue for the SSW pumps were never properly evaluated, and these actions created a new failure mechanism, and therefore did not correct the issue.

This resulted in the premature failure of the B SSW pump spider bearings.

(2) From November 2016 to August 1, 2017, corrective actions were not developed or implemented to address a known issue with the compensatory measures to address baseplate degradation, which resulted in the E SSW being declared inoperable several times due to high vibrations in excess of IST program action levels.

In both cases, Entergys corrective actions did not ensure that an identified condition adverse to quality associated with the SSW pumps was corrected. In accordance with NRC Enforcement Manual, Section 1.3.4, Documenting Multiple Examples of a Violation, multiple examples of a single violation are allowed to be documented as a single violation bounded by the characterization of the most significant example. In this case, each example screens to very low safety significance (Green). Entergy entered these issues into their CAP as CR-2017-05046, CR-2017-07096, and CR-2017-09325, repairs and modifications have been scheduled or implemented for the affected pumps, and compliance has been restored. Anti-rotation pins for the B and E SSW pumps were removed, the baseplate for the B pump was replaced, and replacement is scheduled for the E pump. The remaining pumps continue to be monitored by the IST program until modifications can be performed during their next scheduled overhauls.

This violation is being treated as an NCV, consistent with Section 2.3.2.a of the NRC Enforcement Policy. (NCV 05000293/2017009-03, Inadequate Corrective Actions for SSW Pump Issues Result in Multiple Pump Failures)

(4) Loss of Control Power to EDG not Corrected in accordance with CAP

Introduction:

An NRC-identified Green NCV of 10 CFR Part 50, Appendix B, Criterion XVI was identified because Entergy did not identify and correct a condition adverse to quality associated with the failure of the 125 VDC control power to the B EDG.

Specifically, on April 29, 2017, the B EDG experienced a loss of control power when a fuse opened due to degradation over time due to an elevated current surge condition during EDG starts. This same condition resulted in the B EDG failing to start on November 6, 2012, however, Entergy did not effectively identify and correct the degraded condition.

Description:

The emergency power distribution system design for Pilgrim contains two EDGs, A and B, which serve as standby AC power sources in the event of a loss of normal offsite power for the two safety-related 4160 volt vital buses. Each generator is designed to start within 10 seconds to supply a reliable source of power and to ensure the design basis of maintaining the reactor core covered with water. Each of the EDGs control systems is powered by its respective division of 125 VDC control power. The 125 VDC battery system ensures that a reliable source of power is provided to support the starting circuit and controls of the EDG system and other systems important to reactor safety.

On April 29, 2017, the B EDG experienced a loss of control power when a fuse in the starting circuit opened. Operators in the main control room also received an alarm for 125 VDC ground. Entergy entered this issue into the CAP is CR-2017-4563, began troubleshooting, and successfully replaced the fuse. Troubleshooting, however, was not successful at determining the cause of the ground and opening of the fuse. However, it was identified that the starting current surge during a B EDG start was intermittently higher than expected but below the short term current rating for the fuse. The cause of this condition is not known at this time and requires additional troubleshooting when plant conditions would support testing such as a bus outage window.

The inspectors review of the cause evaluation performed by Entergy revealed that a loss of B EDG control power also occurred on November 6, 2012. This issue was documented as CR-2012-5034; however, a cause for the loss of control power was not identified. At the time, the same fuse failed, and the same 125 VDC ground alarms were received. The inspectors review determined that Entergy had developed a troubleshooting plan to be performed per Work Order 00332585. This plan provided instructions directing additional troubleshooting tasks to be completed during the next available opportunity (i.e. the next EDG maintenance outage). Between 2012 and 2017, there were numerous opportunities during system and bus maintenance periods where the additional troubleshooting tasks could have been performed per Work Order 00332585, but were not performed since the work order was removed from the next bus outage and cancelled. Since the cause evaluation in 2012 did not identify a cause, compensatory measures such as increased monitoring, more frequent fuse replacement, and standing orders also should have been developed but were not.

Entergy procedure EN-LI-102, Corrective Action Program, Section 5.5[3] provides direction for addressing Category B - Adverse Conditions, which includes formulation of a corrective action plan to correct the adverse condition and to address other factors that are identified and warrant additional action. Contrary to this, Entergy did not adequately correct the adverse condition associated with the loss of control power to the B EDG in 2012. Although Entergy had developed troubleshooting which could have conducted more intrusive testing, it was not performed.

Following the April 2017 loss of control power, Entergy developed corrective actions which include monitoring the control power circuit with a recorder to assess for variations in circuit voltage and amperage. Corrective actions also include quarterly replacement of the fuses in the circuit to assess the fuse degradation rate, and to increase the replacement rate if any degradation is found. Additional actions include weekly analysis of data captured from the recorder and compensatory actions to ensure that operators are briefed to take manual actions to start the EDG using procedure 2.4.16, Distribution Alignment Electrical System Malfunctions if the automatic start fails. This would allow for recovery of the EDG. The inspectors reviewed these corrective and compensatory actions and determined that they adequately address the fuse degradation concern until the cause of the excessive starting surge on the B EDG can be identified and corrected.

Analysis:

The inspectors determined that Entergys failure to correct the condition adverse to quality, in accordance with EN-LI-102, associated with the loss of control power for the B EDG between November 6, 2012, and April 29, 2017, was a performance deficiency within Entergys ability to foresee and correct and should have been prevented. Specifically, the cause was never determined after the 2012 failure, corrective actions to conduct more detailed troubleshooting per Work Order 00332585 when plant conditions allowed were never scheduled or completed, and no compensatory measures were developed to mitigate the impact or likelihood of future failures due to the same cause. The inspectors determined that this performance deficiency was more than minor because it adversely affected the Equipment Performance attribute of the Mitigating Systems cornerstone and its associated cornerstone objective to ensure the availability, reliability, and capability of systems that respond to initiating events to prevent undesirable consequences (i.e. core damage).

Specifically, the loss of control power to the B EDG in 2012 was not adequately addressed and corrected to prevent future failures. This issue occurred again in May 2017 and prevented the ability to automatically start the EDG on a loss of offsite power.

The inspectors evaluated this finding using IMC 0609.04, Initial Characterization of Findings, and IMC 0609, Appendix A, Exhibit 2, Mitigating Systems Screening Questions. The inspectors determined that this finding was of very low safety significance (Green) because the performance deficiency was not a design or qualification deficiency, did not involve an actual loss of safety function, did not represent actual loss of a safety function of a single train for greater than its TS-allowed outage time, and did not screen as potentially risk-significant due to a seismic, flooding, or severe weather initiating event. Furthermore, since the component which failed was electronic (a fuse), the magnitude of the starting current surges was variable, and the fuse only saw elevated currents for a short period during an EDG start, fault exposure time began at the time of discovery (i.e. when the fuse blew). Operability was only impacted while the fuse was blown.

This finding did not have a cross-cutting aspect because the performance deficiency was a historical issue with the actions taken in 2012 and is not indicative of current licensee performance.

Enforcement:

10 CFR Part 50, Appendix B, Criterion XVI, Corrective Action, requires in part that measures shall be established to assure that conditions adverse to quality, such as failures, malfunctions, deficiencies, deviations, defective material and equipment, and non-conformances are promptly identified and corrected. Contrary to the above, from November 6, 2012, to April 29, 2017, Entergy did not adequately identify and correct the cause for a loss of control power to the B EDG. Specifically, Entergy failed to identify and correct non-conforming conditions which resulted in the loss of control power to the B EDG on November 6, 2012, which occurred again on April 29, 2017, and resulted in the inoperability of the B EDG. Entergy entered the issue into their CAP and conducted an evaluation. Compensatory measures have been established to install additional monitoring equipment and quarterly fuse replacement until the cause can be determined through more detailed troubleshooting. Because this issue is of very low safety significance (Green) and Entergy entered this issue into their CAP as CR-2017-4563, this finding is being treated as an NCV consistent with Section 2.3.2.a of the Enforcement Policy (NCV 05000293/2017009-04, Loss of Control Power to EDG not Corrected in accordance with CAP)

.2 Assessment of the Use of Operating Experience

a. Inspection Scope

The inspectors reviewed a sample of CRs associated with review of industry operating experience to determine whether Entergy staff appropriately evaluated the operating experience information for applicability to Pilgrim and had taken appropriate actions, when warranted. The inspectors also reviewed evaluations of operating experience documents associated with a sample of NRC generic communications to ensure that Entergy staff adequately considered the underlying problems associated with the issues for resolution via their CAP. In addition, the inspectors observed various plant activities to determine if the station considered industry operating experience during the performance of routine and infrequently performed activities.

b. Assessment

The inspectors determined that Entergy staff appropriately considered industry operating experience information for applicability, and used the information for corrective and preventive actions to identify and prevent similar issues when appropriate. The inspectors determined that operating experience was appropriately applied and lessons learned were communicated and incorporated into plant operations and procedures when applicable. Moreover, the inspectors found that Entergy utilized benchmarking of industry peers as a tool for both the gathering of operating experience and self-assessment. The inspectors determined that Entergy effectively implemented this technique as a means of continuous improvement. The inspectors also observed that industry operating experience was routinely discussed and considered during the conduct of PRG and Plan-of-the-Day meetings, during pre-job briefs, and included in work packages.

c. Findings

No findings were identified.

.3 Assessment of Self-Assessments and Audits

a. Inspection Scope

The inspectors reviewed a sample of audits, including the most recent audit of the CAP, departmental self-assessments, and assessments performed by independent organizations. The inspectors performed these reviews to determine if Entergy entered problems identified through these assessments into the CAP, when appropriate, and whether Entergy staff initiated corrective actions to address identified deficiencies. The inspectors evaluated the effectiveness of the audits and assessments by comparing audit and assessment results against self-revealing and NRC-identified observations made during the inspection.

b. Assessment

Based on the inspected sample, the inspectors concluded that self-assessments, audits, and other internal Entergy assessments were critical, thorough, and effective in identifying issues. The inspectors observed that Pilgrim personnel knowledgeable in the subject completed these audits and self-assessments in a methodical manner. The inspectors observed that Nuclear Oversight was critical and identified weaknesses and areas requiring improvement. When progress in improving performance was not being accomplished in a timely manner, Nuclear Oversight escalated the issues. Entergy completed these audits and self-assessments to a sufficient depth to identify issues which were then entered into the CAP for evaluation. In general, the station implemented corrective actions associated with the identified issues commensurate with their safety significance.

c. Findings

No findings were identified.

.4 Assessment of Safety Conscious Work Environment

a. Inspection Scope

During interviews with station personnel, the inspectors assessed the safety conscious work environment at Pilgrim. Specifically, the inspectors interviewed personnel to determine whether they were hesitant to raise safety concerns to their management and/or the NRC. The inspectors conducted small group interviews with rank and file employees from the system engineering, instrumentation and controls maintenance, the Fix-it-Now team, chemistry, radiation protection, and security groups. The inspectors also interviewed the station Employee Concerns Program coordinators to determine what actions are implemented to ensure employees were aware of the program and its availability with regards to raising safety concerns. The inspectors reviewed the Employee Concerns Program files to ensure that the Entergy staff entered issues into the CAP when appropriate.

b. Assessment

During interviews, Entergy staff expressed a willingness to use the CAP to identify plant issues and deficiencies and stated that they were willing to raise safety issues. The inspectors noted that no one interviewed stated that they personally experienced or were aware of a situation in which an individual had been retaliated against for raising a safety issue. All persons interviewed demonstrated an adequate knowledge of the CAP and the Employee Concerns Program. Based on these limited interviews, the inspectors concluded that there was no evidence of an unacceptable safety conscious work environment and no significant challenges to the free flow of information.

c. Findings

No findings were identified.

4OA6 Meetings, Including Exit

On September 29, 2017, the inspection results were discussed with Mr. Brian Sullivan, Site Vice President, and other members of Entergy staff. During this discussion, Entergy staff requested to provide additional information for consideration. In-office review of the additional information continued by the NRC after the conclusion of the onsite inspection, and a telephonic exit meeting was conducted on November 13, 2017, with Mr. Dave Noyes, Recovery Manager, and other members of Entergy staff. The inspectors verified that no proprietary information was retained by the inspectors or documented in this report.

ATTACHMENT:

SUPPLEMENTARY INFORMATION

KEY POINTS OF CONTACT

Licensee Personnel

B. Sullivan, Site Vice President

D. Pitts, General Manager Plant Operations

A. Lombardo, Radiation Protection

A. Madeiras, Supervisor, Engineering

A. Niederberger, Supervisor, Engineering

A. Zelie, Radiation Protection Manager

B. Barrus, Senior Lead Engineer

B. Collins, Maintenance

B. Giorgio, Supervisor FIN

B. Naeck, Engineering

D. Eldredge, Radiation protection

D. Noyes, Director Engineering

D. Perry, Engineering

D. Swenszkowski, Supervisor Radiation

Protection

E. Perkins, Regulatory Assurance Manager

G. Blankerbille, Chemistry Manager

G. Flynn, Operations Manager

G. Pitts, NIOS

H. Corderio, Chemistry

J. Falconieri, Senior Lead Engineer

J. Freeman, FIN

J. Gerety, Recovery

J. ODonnell, Senior Engineer

J. West, Radiation Protection

K. Drown Performance Improvement

K. Woods, Supervisor, Engineering

M. Landry, Senior Engineer

M. Laundry, Engineering

M. LeFrancois, Engineering

M. Lemieux, FIN

M. Santos, Chemistry

M. Williams, Regulatory Assurance

N. Eisenmann, Supervisor, I&C Design

Engineering

N. Reece, Senior Engineer

N. Reesce, Engineering

P. Beabout, Security

P. Chase, Recovery

P. Miner, Regulatory Assurance

R. Abbott, FIN

R. Bowen, Chemistry

R. Felumb, Fleet CAP/OE

R. Probasco Supervisor FIN

S. Asplin, Engineering

S. Brewer, Supervisor Radiation Protection

T. Wheble, Maintenance

T. White, Design Engineering

W. Sharkey, Employee Concerns

NRC Personnel

E. Carfang, Pilgrim Senior Resident Inspector

P. Boguszewski, Pilgrim Resident Inspector

C. Bickett, Division of Reactor Safety, Region I

LIST OF ITEMS OPENED, CLOSED, DISCUSSED, AND UPDATED

Opened and Closed

05000293/2017009-01 NCV Untimely Corrective Actions to Address Previously Identified Non-Cited Violation (4OA2.1.c(1))

05000293/2017009-02 FIN Failure to Correct Programmatic Issues with Implementation of the Operability Determination Process (4OA2.1.c(3))

05000293/2017009-03 NCV Inadequate Corrective Actions for SSW Pump Issues Result in Multiple Pump Failures (4OA2.1.c(4))

05000293/2017009-04 NCV Loss of Control Power to EDG not Corrected in Accordance with CAP (4OA2.1.c(5))