IR 05000261/2011010

From kanterella
Jump to navigation Jump to search
IR 05000261-11-010, on 05/31/2011 - 06/10/2011, H. B. Robinson Steam Electric Plant, Unit 2; Supplemental Inspection - Inspection Procedure (IP) 95002
ML111870510
Person / Time
Site: Robinson Duke Energy icon.png
Issue date: 07/06/2011
From: Randy Musser
NRC/RGN-II/DRP/RPB4
To: Duncan R
Carolina Power & Light Co
References
IR-11-010
Download: ML111870510 (47)


Text

UNITED STATES uly 6, 2011

SUBJECT:

H. B. ROBINSON STEAM ELECTRIC PLANT - NRC INSPECTION PROCEDURE 95002 SUPPLEMENTAL INSPECTION REPORT 05000261/2011010 AND ASSESSMENT FOLLOW-UP LETTER

Dear Mr. Duncan:

On June 9, 2011, the U.S. Nuclear Regulatory Commission (NRC) completed a supplemental inspection pursuant to Inspection Procedure (IP) 95002, Inspection for One Degraded Cornerstone or Any Three White Inputs in a Strategic Performance Area, at your H.B.

Robinson Steam Electric Plant Unit 2. The enclosed inspection report documents the inspection results, which were discussed at the exit meeting on June 9, 2011, with you and other members of your staff.

As required by the NRC Reactor Oversight Process Action Matrix, this supplemental inspection was performed because three findings, all of White safety significance, were identified which placed H.B. Robinson Unit 2 in the Degraded Cornerstone Column in the third quarter of 2010.

The following issues degraded the Mitigating Systems Cornerstone: the failure to adequately implement requirements of multiple procedures required by Technical Specification 5.4.1 during a cooldown of the Reactor Coolant System and subsequent safety injection after a reactor trip on March 28, 2010; the failure to adequately implement operator training based on learning objectives and evaluate mastery of learning objectives during training as required by 10 CFR 55.59(c); and the failure to promptly correct a condition adverse to quality involving the failure of B Emergency Diesel Generator (EDG) output breaker 52/27B to close in October, 2008 as required by 10 CFR Part 50, Appendix B, Criterion XVI, Corrective Action.

These violations were previously documented in NRC Inspection Reports 05000261/2011008 dated January 31, 2011 and 05000261/2010014 dated December 7, 2010. The NRC staff was informed on February 1, 2011, of your readiness, as of May 9, 2011, for us to conduct this supplemental inspection.

The objectives of this supplemental inspection were to provide assurance that: the root causes and the contributing causes for the risk significant issues were understood; the extent of condition and extent of cause of the issues were identified; and corrective actions were or will be sufficient to address and preclude repetition of the root and contributing causes.

CP&L 2 The inspection also included an independent NRC review of the extent of condition and extent of cause for these issues and an assessment of whether any safety culture component caused or significantly contributed to the issues. The inspection consisted of examination of activities conducted under your license as they related to safety, compliance with the Commissions rules and regulations, and the conditions of your operating license.

The inspection team determined that your corrective actions, as itemized in the root cause evaluations, are appropriate to resolve the deficiencies related to the Degraded Mitigating Systems Cornerstone. The inspection team also concluded that your root cause, extent of condition, and extent of cause evaluations appropriately considered the safety culture components as described in Inspection Manual Chapter 0305, Operating Reactor Assessment Program.

Based on the results of this inspection, the three White findings are closed. However, one of the findings, specifically 05000261/2010013-01, Failure to Comply with Conduct of Operations Procedure, can still be considered for agency actions in accordance with the Action Matrix until September 30, 2011. As a result, the NRC determined the performance at Robinson Unit 2 to be in the Regulatory Response Column of the Reactor Oversight Process Action Matrix as of July 1, 2011. In addition, based on the results of this inspection, no findings were identified.

In accordance with 10 CFR 2.390 of the NRCs Rules of Practice, a copy of this letter, its enclosure, and your response (if any) will be available electronically for public inspection in the NRC Public Document Room or from the Publicly Available Records (PARS) component of the NRCs document system, Agency wide Documents Access and Management (ADAMS).

ADAMS is accessible from the NRC Web site at http://www.nrc.gov/reading-rm/adams.html (the Public Electronic Reading Room).

Sincerely,

/RA/

Randall A. Musser, Chief Reactor Projects Branch 4 Division of Reactor Projects Docket No.: 50-261 License No.: DPR-23

Enclosure:

Inspection Report 05000261/2011010 w/Attachment: Supplemental Information

REGION II==

Docket No.: 50-261 License No.: DPR-23 Report No.: 005000261/2011010 Facility: H. B. Robinson Steam Electric Plant, Unit 2 Location: 3581 West Entrance Road Hartsville, SC 29550 May 31 - June 3, 2011 (Week 1)

Dates:

June 6 - 9, 2011 (Week 2)

F. Ehrhardt, Sr. Reactor Inspector, RII, Team Leader Inspectors:

A. Allen, Special Assistant, RII (1st week)

M. Bates, Sr. Operations Engineer, RII (2nd week)

J. Brady, Sr. Resident Inspector, RII B. Caballero, Sr. Operations Engineer, RII J. Kellum, Sr. Reactor Operations Engineer, NRO (2nd week)

M. King, Sr. Project Engineer, RII R. Pelton, Training and Assessment Specialist, NRO (1st week)

J. Rivera-Ortiz, Sr. Reactor Inspector, RII Approved by: R. Musser, Chief Reactor Projects Branch 4 Division of Reactor Projects Enclosure

SUMMARY OF FINDINGS

IR 05000261/2011010, 05/31/2011 - 06/10/2011: H. B. Robinson Steam Electric Plant, Unit 2;

Supplemental Inspection - Inspection Procedure (IP) 95002.

This supplemental inspection was conducted by two senior reactor inspectors; one senior resident inspector; two senior operations engineers; one senior project engineer; one training and assessment specialist; one reactor operations specialist; and one special assistant. No findings were identified. The NRC's program for overseeing the safe operation of commercial nuclear power reactors is described in NUREG-1649, Reactor Oversight Process.

Cornerstone: Mitigating Systems

The inspection team performed this supplemental inspection in accordance with Inspection Procedure (IP) 95002, Inspection for One Degraded Cornerstone or Any Three White Inputs in a Strategic Performance Area, to assess the licensees evaluations associated with (1) failure to adequately implement requirements of multiple procedures required by Technical Specification 5.4.1, (2) failure to adequately design and implement operator training based on learning objectives and evaluate mastery of learning objectives during training as defined in 10 CFR 55.4 and as required by 10 CFR 55.59(c), and (3) failure to promptly correct a condition adverse to quality involving the failure of the B Emergency Diesel Generator (EDG) output breaker 52/27B to close in October, 2008 as required by 10 CFR Part 50, Appendix B, Criterion XVI, Corrective Action. The findings associated with failure to adequately implement requirements of multiple procedures and failure to adequately implement operator training and evaluate mastery of learning objectives during training were identified by the NRC during follow-up of Unresolved Items (URIs) arising from an Augmented Inspection. The results of this Augmented Inspection were documented in IR 05000261/2010009 (ML101830101). The finding associated with failure to promptly correct a condition adverse to quality was identified by the NRC during a Problem Identification and Resolution (PI&R) inspection. The NRC previously characterized each of these three findings as having low to moderate safety significance (White) in NRC Inspection Report (IR) numbers 05000261/2010014 dated December 7, 2010 (ML103410289) and 05000261/2011008 dated January 31, 2011 (ML110310469). Detailed descriptions of these findings were previously documented in NRC IRs 05000261/2010004 and 05000261/2010501 dated November 12, 2010 (ML103160382), 05000261/2010006 dated October 8, 2010 (ML102810633), and 05000261/2010013 dated December 27, 2010 (ML103620095). Each of these three findings was associated with the Mitigating Systems Cornerstone.

The inspection team determined that the licensee performed an adequate evaluation of the issues. The inspection team also determined that the root cause evaluations for these issues appropriately evaluated the root and contributing causes, adequately addressed the extent of condition and cause, assessed safety culture, and established corrective actions for the risk significant performance issues. In addition to assessing the licensees evaluations, the inspection team independently performed an extent of condition and extent of cause review of the three findings and a focused inspection of the site safety culture as it related to the root cause evaluations. The team concluded that the licensees root cause evaluations and corrective actions, both completed and planned, were sufficient to address the causes and prevent recurrence. The team also concluded that the licensees assessment of H.B. Robinson safety culture accurately reflected the conditions at the site. As a result the team concluded that the licensee appropriately addressed the three White findings, and in accordance with the guidance in IMC 0305, Operating Reactor Assessment Program, each of the three White findings will be considered in assessing plant performance for a total of four quarters. The licensees implementation of corrective actions will be reviewed during future inspections.

NRC-Identified and Self-Revealing Findings

None

Licensee-Identified Violations

None

REPORT DETAILS

OTHER ACTIVITIES

Cornerstones: Initiating Events, Mitigating Systems, Barrier Integrity

4OA4 Supplemental Inspection

.01 Inspection Scope

The inspection team performed this inspection in accordance with IP 95002, Inspection for One Degraded Cornerstone or Any Three White Inputs in a Strategic Performance Area, because the licensee entered the Degraded Cornerstone column of the NRC Action Matrix in the third quarter of 2010 as a result of three NRC-white inspection findings in the Mitigating Systems Cornerstone. These findings are summarized below:

  • Failure to adequately design and implement operator training based on learning objectives (Systems Approach to Training (SAT) Element 3) and failure to evaluate mastery of objectives during training (SAT Element 4), as defined in 10 CFR 55.4 and required by 10 CFR 55.59(c), in that training lesson material failed to identify the basis of a procedural action involving reactor coolant pump seal cooling in licensee procedure PATH-1 and evaluators did not identify, document, and evaluate operator weaknesses during evaluated scenarios, respectively. IR 05000261/2010004 and 05000261/2010501 contains additional details concerning this issue. The NRC characterized this finding as White based on a Phase 3 risk analysis as discussed in IR 05000261/2011008.

The objectives of this supplemental inspection included the following:

  • provide assurance that the root and contributing causes of risk-significant issues were understood
  • provide assurance that the extent of condition and extent of cause of risk significant issues were identified and to independently assess the extent of condition of risk significant issues, both individually and collectively
  • independently determine if safety culture components caused or significantly contributed to the risk significant issues
  • provide assurance that the licensees corrective actions for risk significant issues were, or will be, sufficient to address the root and contributing causes as well as preclude recurrence The licensee staff informed the NRC staff on February 1, 2011, of their readiness, as of May 9, 2011, for this supplemental inspection. In preparation for the inspection, the licensee performed root cause evaluations (RCEs) to identify weaknesses that existed in various processes and organizations that resulted in the White findings that led to a degraded Reactor Oversight Process cornerstone. As part of the RCEs the licensee also completed a safety culture self assessment report. The licensee provided the NRC inspection team with copies of their RCEs (Nuclear Condition Reports (NCRs) 419190, 422989, 438394, 438396, and 452367) and Nuclear Safety Culture Self Assessment (Quick Hit Self Assessment 461432) on May 13, 2011. The licensee provided the NRC inspection team with a copy of their common cause evaluation (NCR 454853) on May 23, 2011.

The inspection team reviewed the licensees RCEs and other evaluations the licensee conducted in support of, or as a result of, the RCEs. The inspection team reviewed corrective actions that the licensee had taken, or planned to take, to address the identified causes. The inspection team also held discussions and conducted interviews with licensee personnel to determine if the root and contributing causes, and the contribution of safety culture components, was understood as well as whether completed or planned corrective actions were adequate to address the causes and preclude recurrence. The inspection team independently assessed the extent of condition and extent of cause of the findings. In addition, the inspection team assessed whether any safety culture components caused or significantly contributed to the findings.

.02 Evaluation of the Inspection Requirements

02.01 Problem Identification a. Determine that the licensees evaluation of the issue documented who identified the issue (i.e. licensee-identified, self-revealing, or NRC-identified) and under what conditions the issue was identified.

.1 Failure to adequately implement requirements of multiple procedures

The inspectors determined that this issue had aspects that were both self-revealing and NRC-identified. In addition, the licensees RCE (NCR 438394) documented a number of other aspects to this finding through their extent of cause and extent of condition reviews.

The full scope of the identification for this issue was a collection of all of these sources.

The inspectors determined that the licensees RCE (NCR 438394) documented the issue as NRC-identified. The issue was initially documented as two URIs in NRC Augmented Inspection Team Report 05000261/2010009 and was later identified as a White finding in NRC IR 05000261/2011008.

.2 Failure to properly implement Elements 3 & 4 of the SAT

The inspectors determined that the licensees RCE (NCR 438396) documented the issue as NRC-identified. The issue was initially an unresolved item (URI) in NRC Augmented Inspection Team Report 05000261/2010009 (URI 05000261/2010009-05), and was later identified as a White finding in NRC IR 05000261/2011008.

.3 Failure to correct a condition adverse to quality involving the B EDG output breaker

The inspectors determined that the licensees RCE (NCR 419190) documented the issue as NRC-identified. The issue was first identified in NRC Inspection Report 05000261/2010006, identified as a White finding in NRC IR 05000261/2010014.

b. Determine that the evaluation documented how long the issue existed and prior opportunities for identification.

.1 Failure to adequately implement requirements of multiple procedures

The licensees RCEs identified that operations performance deficiencies existed prior to the March 28, 2010, event and were identified by the licensees Nuclear Oversight (NOS)organization as well as by an audit performed by an outside organization. Aspects of these deficiencies were identified as early as 2006. The RCE performed by the licensee following the March 28, 2010, reactor trip and safety injection (NCR 390095) documented reviews of seven prior audits concerning operations performance and concluded that operational performance gaps were not fully understood and therefore not addressed by station leadership.

The inspectors concluded that the licensees RCE accurately documented how long the issue existed and prior opportunities for identification.

.2 Failure to properly implement Elements 3 & 4 of the SAT

The licensees RCE included an event chronology/timeline summary of the issue beginning in February 2007. The licensees RCE also documented opportunities that the licensee had to identify the issue, including the following:

  • In May 2008 the National Academy for Nuclear Training (NANT) placed RNP operator training programs on probation for 180 days because of inadequate management of training, which resulted in weaknesses in the implementation of the SAT processes.
  • In August 2008 the licensees NOS organization identified that deficiencies in analysis, design, and development activities had resulted in training materials, including examination and training documentation, which did not support effective or repeatable training.

The inspectors concluded that the licensees RCE accurately documented how long the issue existed and prior opportunities for identification.

.3 Failure to correct a condition adverse to quality involving the B EDG output breaker

The licensees RCE documented a chronology of the event which began in June 2008 when B EDG output breaker 52/27B was installed and ended on April 27, 2009, when the licensee received a letter from Westinghouse confirming the breaker failure mechanism. The refurbished Westinghouse DB-100 breaker that was installed in June 2008 for the B EDG output breaker contained a modification to the part that was used to retain the control relay mechanical lift linkage during the assembly process. Specifically, the cotter pin used to retain the lift linkage in the refurbished breaker was longer than other cotter pins used in previous breaker refurbishments.

On October 15, 2008, the breaker failed to close twice during a post-modification test conducted as part of a fire protection related upgrade. Based on the observed symptoms, the licensee concluded that the breaker control relay was attempting to operate. To determine the cause of the failure to close, licensee staff checked the control relay for binding/obstruction by manually manipulating the moving-core-contact operating arm connector, which interacts with the lift linkage. Manipulation of the control relay had the potential to reposition the cotter pin from its faulted state, preventing an opportunity to identify the problem. After manipulating the control relay and observing no abnormal conditions, the licensee successfully closed the breaker on the third attempt. The licensee initiated a work request and a work order (WO) to investigate the potential control relay malfunction, but did not enter the issue into the Corrective Action Program (CAP). The licensees troubleshooting efforts were limited to cycling the breaker several times in an attempt to repeat the failure. The RCE documented that, despite opportunities by multiple work groups to enter this condition into the CAP in October 2008 an NCR was not generated to address the recognized breaker deficiency.

The licensees RCE also documented that a similar breaker failure occurred in April 2009.

The licensee entered the second failure into the CAP as an NCR and, through systematic troubleshooting and by involving the vendor, successfully identified the cause of the failure. As documented in the RCE, the licensee determined that the failure mechanism arising from the modified cotter pin was random in nature and was introduced when the breaker was closed. Since the breaker is opened through a separate mechanism from that used to close the breaker, the inability of the breaker to close would not be identified until the next attempt to close the breaker. Because the breaker successfully passed all the required surveillance tests from October 2008 to April 2009 the licensee concluded that the breaker was inoperable since the last successful breaker surveillance prior to April 2009. The RCE documented that the EDG B output breaker was inoperable from March 28 to April 23, 2009.

The inspectors concluded that the licensees RCE accurately documented how long the issue existed and prior opportunities for identification.

c. Determine that the evaluation documented the plant-specific risk consequences, as applicable, and compliance concerns associated with the issue(s) both individually and collectively.

.1 Failure to adequately implement requirements of multiple procedures

26 of the RCE documented the licensees quantitative evaluation of the risk consequences associated with the compliance issues for this White finding. Because of the broad scope of command and control deficiencies, as well as multiple examples of failure to follow multiple procedures, the potential consequence was an increased likelihood of operators failing to respond properly to postulated accidents.

The inspectors concluded that the licensees RCE appropriately characterized and documented the risk consequences and compliance concerns associated with the issue.

.2 Failure to properly implement Elements 3 & 4 of the SAT

The licensees RCE stated that the consequence of failing to properly implement SAT Elements 3 & 4 in the license operator continuing training program was that cooling for the Reactor Coolant Pump (RCP) seals was unnecessarily challenged during a crews response to an actual plant event involving a fault on 4kV Bus 5 on March 28, 2010.

The licensees RCE described the compliance concerns with 10CFR55.4 and 10CFR55.59 (c)

(4) as
(1) some aspects of learning objective PATH-1-005 (Explain the bases of steps, notes, and cautions) were not completely covered in licensed operator continuing training and
(2) a crew that had failed to identify a complete loss of RCP seal cooling during a 2007 simulator exam scenario was not adequately retrained.

Compliance with 10CFR55.4 and 10CFR55.59 was restored in October 2010 when (1)the licensee completed retraining for all licensed operators in accordance with a revised PATH-1 lesson plan for learning objective PATH-1-005 and

(2) the licensee reviewed previous examination materials and records for licensed operators with previously noted weaknesses, including details on underlying causes, to ensure that adequate retraining had been completed for these operators.

The inspectors concluded that the licensees RCE appropriately documented the risk consequences and compliance concerns associated with the issue.

.3 Failure to correct a condition adverse to quality involving the B EDG output breaker

The licensees RCE documented that the failure to initiate an NCR and resolve the output breaker issue resulted in subsequent unplanned unavailability of the EDG B. The RCE documented 44 hours5.092593e-4 days <br />0.0122 hours <br />7.275132e-5 weeks <br />1.6742e-5 months <br /> of unplanned unavailability to repair the breaker when it failed in April 2009 and 554 unplanned unavailability hours due to fault exposure hours. The licensee determined that the raw core damage probability of this event (not crediting recovery actions or other possible mitigation) would be approximately 2E-6. The licensee restored compliance of the inoperable EDG in April 2009 and removed the failure mechanism from all affected breakers on site.

The inspectors concluded that the licensees RCE appropriately documented the risk consequences and compliance concerns associated with the issue.

d. Findings and Observations

As described above, the licensees RCEs documented plant-specific risk consequences for the three individual White findings. In addition, the licensee performed three other pertinent RCEs associated with the March 28, 2010 event, one for human performance (NCR 452367), one for CAP effectiveness (NCR 422989), and one for a common cause analysis (NCR 454853). The licensees common cause evaluation documented, qualitatively, the collective risk associated with the three White findings as well as the human performance and CAP effectiveness RCEs.

A regional Senior Reactor Analyst evaluated the risk, both individually and collectively, of the original three performance deficiencies that resulted in this inspection and considered any additional findings from this inspection. The exposure period was one year with all three of the white findings overlapping for one month and the findings associated with operator training and main control room command and control overlapping for the other eleven months. There were no additional findings from the inspection that resulted in an expansion of the extent of condition/cause from the original risk evaluations.

The inspectors, in conjunction with the Senior Reactor Analyst, determined that the licensee adequately documented the collective risk associated with the three White findings. No findings were identified.

02.02 Root Cause, Extent of Condition, and Extent of Cause Evaluation a. Determine that the problem was evaluated using a systematic methodology to identify the root and contributing causes.

The licensee used a variety of methods to determine the root and contributing causes in accordance with procedure CAP-NGGC-0205, Condition Evaluation and Corrective Action Process.

.1 Failure to adequately implement requirements of multiple procedures

The licensee used the following systematic methods to complete their RCE:

  • Event and causal factor charting
  • Cause and effect analysis
  • Barrier analysis
  • Human performance evaluation
  • Organizational and programmatic evaluation The RCE also included a safety culture analysis to identify contributing safety culture components as described in NRC IMC 0310, Components with Cross-Cutting Areas.

The licensee identified the following root causes for this issue:

1. Senior management failed to implement a formal program or process to continually monitor, evaluate, and improve Operations crew performance.

2. Training failed to identify and remediate operator performance deficiencies due to programmatic weaknesses in the Operations Simulator Training Program.

3. Operations and Training failed to effectively implement the corrective action program because the threshold for NCR initiation is too high.

4. Procedure GP-004, Post Trip Stabilization, failed to provide the control room operators with the appropriate level of detail to effectively respond to plant conditions.

The licensee also identified the following contributing causes for this issue:

1. Operations and Training procedures did not contain requirements for the tracking and resolving crew and individual performance deficiencies.

2. Operations self-assessments were not sufficiently self critical.

3. Operations did not use benchmarking effectively to identify gaps to industry excellence and improve performance.

The inspectors did not identify any concerns with the systematic methods used by the licensee to identify root causes and contributing causes.

.2 Failure to properly implement Elements 3 & 4 of the SAT

The licensee used the following systematic methods to complete their RCE:

  • Event chronology
  • Cause and effect analysis
  • Barrier analysis
  • Human performance evaluation
  • Organizational and programmatic evaluation
  • Event and causal factor charting Additionally, the evaluation included a safety culture analysis to identify contributing safety culture components as described in NRC IMC 0310, Components with Cross-Cutting Areas.

The licensee identified the root cause for this issue to be that Operations, Training, Senior Site Managers, and the Training Advisory Board (TAB) did not provide the leadership necessary to ensure the integrity of the operations training infrastructure was maintained and monitored. The licensee also identified the following contributing causes for this issue:

1. Training management did not provide the oversight required to ensure corrective

action reviews were being conducted with the necessary rigor to ensure quality investigations and sustainable corrective actions were completed concerning processes pertinent to the application of a systematic approach to training.

2. The Training Supervisors reluctance to document training material weaknesses in health reports, TAB meetings, Training Performance Indicators, and NCRs fostered an environment where continuous performance improvement was not embraced.

3. Training procedures were inadequate to provide the required clarity to prevent misinterpretations and subsequent material deficiencies in lessons.

The inspectors did not identify any concerns with the systematic methods used by the licensee to identify root causes and contributing causes.

.3 Failure to correct a condition adverse to quality involving the B EDG output breaker

The licensee used the following systematic methods to complete their RCE:

  • Event chronology
  • Cause and effect analysis
  • Change analysis
  • Human performance evaluation
  • Event and causal factor charting Additionally, the evaluation included a safety culture analysis to identify contributing safety culture components as described in NRC IMC 0310, Components with Cross-Cutting Areas.

The licensee identified following root causes for this issue:

1. Employees did not understand the threshold for when issues are required to be tracked in both the work management process and the CAP process and, as a result, failed to use a systematic method for problem identification and resolution for a safety significant component failure.

2. Personnel responded differently to an EDG breaker failure during an outage as compared to when on line. A significant critical component functional failure was not recognized during refueling outage 25 (RO-25) because the equipment was under clearance and not required to be operable per Technical Specifications.

3. Latent organizational weaknesses exist in the interface between the work management processes and skill of the craft work practices for safety related structures, systems, and components.

The licensee also identified following contributing causes for this issue:

1. Inadequate standards were established and enforced for work execution and

documentation including work package quality and level of detail included in completion comments.

2. Robinson Nuclear Plant has fundamental human performance issues impacting plant performance, specifically:

a. Decision making did not demonstrate that safety is the overriding priority because assumptions are not verified and the threshold for using a structured, systematic process for resolving problems is too high.

b. Work practices did not reflect acceptable ownership of problems, questioning attitude when faced with unanticipated conditions, tolerance for poor work instruction and procedures and use of human performance tools to detect and prevent human performance errors.

The inspectors did not identify any concerns with the systematic methods used by the licensee to identify root causes and contributing causes.

b. Determine that the root cause evaluation was conducted to a level of detail commensurate with the significance of the problem.

.1 Failure to adequately implement requirements of multiple procedures

The RCE was performed by a multidisciplinary team which included RNP staff, employees from two other nuclear plants operated by the licensee and one outside consultant. The RCE contained a comprehensive description of facts associated with the issue. As described in the Section 02.02.a.1 above, the licensees evaluation team employed a variety of systematic analytical tools to identify the root and contributing causes. The licensees RCE was broad in scope and documented each technique in sufficient detail to facilitate an understanding of the rationale behind the conclusions. It included information from prior RCEs and prior corrective action documents associated with this issue. The RCE also addressed aspects of human performance, training, safety culture, CAP effectiveness, and organizational effectiveness. It considered not only the inappropriate acts from the March 28, 2010, event, but inappropriate organizational acts prior to the event that contributed to the environment that caused the conduct of operations errors.

Based on the breadth and depth of the licensees evaluation, and a review of the licensees assertions, the inspectors concluded that the licensees RCE was thorough, self-critical, and conducted to a level of detail commensurate with the significance of the issue.

.2 Failure to properly implement Elements 3 & 4 of the SAT

The RCE was performed by a multidisciplinary team which included RNP staff, employees from two other nuclear plants operated by the licensee and from the licensees corporate office, and one outside consultant. The RCE contained a comprehensive description of facts associated with the issue. As described in the Section 02.02.a.2 above, the licensees evaluation team employed a variety of systematic analytical tools to identify the root and contributing causes. The licensees RCE documented each technique in sufficient detail to facilitate an understanding of the rationale behind the conclusions.

The inspectors reviewed the basis for the root and contributing causes identified by the licensee. Based on the breadth and depth of the licensees evaluation, and a review of the licensees assertions, the inspectors concluded that the licensees RCE was thorough, self-critical, and conducted to a level of detail commensurate with the significance of the issue.

.3 Failure to correct a condition adverse to quality involving the B EDG output breaker

The RCE was performed by a multidisciplinary team from Maintenance, Operations, and Support Services. The team also included members from outside of the RNP organization. The RCE contained a comprehensive description of facts associated with the issue. As described in the Section 02.02.a.3 above, the licensees evaluation team employed multiple systematic analytical tools to identify the root and contributing causes.

The licensees RCE documented each technique in sufficient detail to facilitate an understanding of the rationale behind the conclusions.

The inspectors reviewed the basis for the root and contributing causes identified by the licensee. Based on the breadth and depth of the licensees evaluation, and a review of the licensees assertions, the inspectors concluded that the licensees RCE was thorough, self-critical, and conducted to a level of detail commensurate with the significance of the issue.

c. Determine that the root cause evaluation included a consideration of prior occurrences of the problem and knowledge of prior operating experience.

.1 Failure to adequately implement requirements of multiple procedures

The licensees RCE for this issue, as well as the related RCEs described in Section 02.02.a.1, considered prior and subsequent occurrences associated with conduct of operations problems from a very broad perspective. The licensee identified performance deficiencies associated with the threshold for use of the CAP by operations as well as inadequate use of CAP before the event by other organizations (Training and NOS) that evaluate operations performance. In addition, reviews of events from other sites were included in the RCEs from an operating experience (OE) perspective.

The inspectors determined that the licensees RCE included an appropriate consideration of prior occurrences of the problem and knowledge of prior internal and external OE.

.2 Failure to properly implement Elements 3 & 4 of the SAT

The licensees RCE included a review of the following internal and external industry OE:

  • May 2008 NANT probation of Robinson Operator Training Programs
  • Robinson NCRs 280561 (Analysis, Design, And Development Deficiencies),361039 (Training Material Deficiencies), and 418789 (Training Management Not Holding Personnel Accountable)
  • Harris NCR 324338 (Objectives not Designed to Support Lesson Content) and Crystal River NCR 338340 (Training Material Learning Objective Deficiencies)
  • Indian Point Unit 3, (Task-to-Training Matrix Deficiencies)

The licensees RCE stated that RNP corrective actions associated with the internal OE were not sustainable and allowed workarounds in training material development processes. As a result of their OE review, the licensee identified that a primary contributing cause for the failure to properly implement Elements 3 & 4 of the SAT was that previous corrective actions reviews associated with SAT processes had not been performed with the necessary rigor to ensure quality investigations and sustainable corrective actions.

The inspectors did not identify any concerns with the licensees consideration of prior occurrences of the problem and knowledge of prior internal and external OE.

.3 Failure to correct a condition adverse to quality involving the B EDG output breaker

The RCE included a review of internal and external OE to determine if the issue could have been prevented through use this information. The licensee conducted a search of internal NCRs and external OE items and determined that OE was not a significant contributor to the failure to identify and correct the first breaker failure. RNP was the first facility with Westinghouse DB-100 breakers to experience a breaker failure of this nature and the licensees review of OE did not identify any examples of similar breaker failures prior to October 2008 that could have alerted them to the failure mechanism introduced by the cotter pin.

However, the RCE identified multiple examples of internal and external OE associated with the failure to enter conditions adverse to quality into the CAP as NCRs. The licensee recognized, as documented in the RCE, that there were weaknesses in their effective implementation of the CAP. The licensee addressed this issue in a separate RCE (NCR 422989). In this separate RCE, the licensee identified several examples of internal and external OE that were applicable to the CAP performance issues. The RCE concluded that the CAP deficiencies may have been prevented, in part, if the applicable OE had been appropriately disseminated and a comparison made with other industry CAPs.

The inspectors determined that the licensees RCE included an appropriate consideration of prior occurrences of the problem, and knowledge of prior internal and external OE.

d. Determine that the root cause evaluation addresses the extent of condition and the extent of cause of the problem.

.1 Failure to adequately implement requirements of multiple procedures

The licensees RCE included an evaluation of the extent of condition, which consisted of determining whether the condition could exist in other plant equipment, processes or human performance. The licensees RCE also included an evaluation of the extent-of-cause, for each root and contributing cause, to determine the degree to which the causes had resulted in additional problems. The extent of cause evaluation examined each of the four root causes identified and determined whether each was isolated to the operating organization or more wide-spread. In their evaluation, the licensee implicated multiple levels of management across multiple organizations (Operations/Training/NOS), including fleet level support organizations and extending up to senior management. In addition, the licensees common cause assessment (NCR 454853) was critical of certain aspects of fleet management of resources.

Based on a review of the RCEs and discussions with licensee management and staff personnel the inspectors concluded that the licensees RCEs thoroughly addressed the extent of condition and the extent of cause of the problem through a disciplined process.

.2 Failure to properly implement Elements 3 & 4 of the SAT

The licensees RCE included an evaluation of the extent of condition, which consisted of determining if the issue had broader implications and the extent to which similar failures had occurred. The licensee evaluated the applicability of the failure to implement Elements 3 and 4 of the SAT to other accredited training programs. The licensees review encompassed a period of four years (January 2007 to March 2011) and included a review of 47 significant (Significance Level 1) NCRs.

The licensees RCE identified that the improper application of SAT Element 3, design and implementation of training based on learning objectives, existed across all accredited training programs at the site but was most prevalent within the operator training programs. The licensees RCE also documented that the improper application of SAT Element 4, specifically the failure to document underlying knowledge and skill gaps, existed across all training programs at H.B. Robinson.

The inspectors observed that the licensees RCE report explicitly stated that extent of condition reviews associated with SAT Element 1 (systematic analysis of the jobs to be performed), SAT Element 2 (learning objectives derived from the analysis which describe the desired performance after training), and SAT Element 5 (evaluation and revision of the training based on the performance of trained personnel in the job setting) were excluded from the RCE. The inspectors questioned licensee management regarding why these other SAT elements were not considered for extent of condition, with regard to operator training programs, in their RCE. The licensee stated that their extent of condition review considered all five SAT Elements and that the RCE had been worded improperly. The licensee subsequently generated NCR 00469883 to correct the wording in the RCE associated with the scope of the extent of condition review. The licensee also provided the inspectors with documents and records, referenced in the RCE, related to extent of condition regarding SAT Elements 1, 2, and 5. The inspectors reviewed these documents, as well as additional documents requested by the inspectors, and determined that the scope of the licensees RCE regarding extent of condition adequately included SAT elements 1, 2, and 5 within the operator training programs.

The inspectors noted that the licensee, in performing their extent of condition review for SAT Elements 1 and 2, had identified discrepancies in some operator training program task lists. The inspectors questioned the licensee about these discrepancies and the licensee stated that they were re-validating operator training program task lists as a result of the discrepancies. The licensee also stated that effort to re-validate the task lists was part of the corrective action for upgrading training materials, which included a comparison of the operator task lists to RNP plant procedures as well as to task lists from other nuclear plants (Braidwood, Byron, North Anna, and VC Summer).

The licensees RCE also included an evaluation of the extent of cause, for each root and contributing cause, to determine the degree to which the causes had resulted in additional problems. The licensees RCE documented the following with respect to extent of cause:

  • Shortcomings in management oversight affected other programs and processes at H.B. Robinson.
  • Issues with the quality of training procedures would be resolved as part of a broader corrective action associated with procedure quality across the site (NCR 452367).

Based on a review of the RCE and discussions with licensee management and staff personnel the inspectors determined that the licensees RCE adequately addressed the extent of condition and the extent of cause of the problem.

.3 Failure to correct a condition adverse to quality involving the B EDG output breaker

The licensees RCE included an evaluation of the extent of condition, which consisted of determining if the issue had broader implications and the extent to which similar failures had occurred. The licensee evaluated the extent of three conditions associated with this White finding:

1. applicability of the cotter pin failure mechanism to other breakers 2. work orders on critical components that were not entered into the CAP as NCRs 3. historical issues with alignment of the secondary contacts on breakers The licensees extent of condition evaluation identified similar breakers that were susceptible to the cotter pin issue, numerous WOs for corrective maintenance on critical components that were not entered into the CAP as NCRs, and several examples of problems with breaker secondary contacts.

The RCE also included an evaluation of the extent of cause, for each root and contributing cause, to determine the degree to which the causes had resulted in additional problems. Based on the common nature of the root and contributing causes, the licensee grouped certain causal factors together when evaluating the extent of cause.

In evaluating these groupings, the licensee identified the following with respect to extent of cause:

  • examples of lack of management ownership and oversight that had affected the effectiveness of other programs in addition to the CAP
  • differences in the way the site addressed emergent issues, within a number of processes, when the plant is online versus in an outage
  • human performance practices at the plant were weak and required development of an improvement plan to change the behavior of plant personnel The inspectors determined that the licensees RCE adequately addressed the extent of condition and the extent of cause of the problem.

e. Findings

No findings were identified.

02.03 Corrective Actions a. Determine that appropriate corrective actions are specified for each root and contributing cause or that the licensee has an adequate evaluation for why no corrective actions are necessary.

Each of the licensees RCEs included a corrective action plan in which each causal factor had corrective actions assigned to address the cause. All actions to address the root and contributing causes fell into one of the following two categories:

  • Corrective Action to Prevent Recurrence (CAPR) - A sustainable action to prevent recurrence of a Significant Condition Adverse to Quality (SCAQ) by addressing the root cause. This type of corrective action had an initial due date of 120 days from the NCR initiation date. Extension of a CAPR assignment due date must be approved by the Management Review Committee for NCRs of Significance Level
  • Corrective Action (CORR) - An action to correct an undesired condition or the identified cause of the condition. This type of corrective action had an initial due date of 150 days from the NCR initiation date. Extension of a CORR assignment due date must be approved by the Management Review Committee for NCRs of Significance Level 1.

In addition, the licensees extent of condition and extent of cause evaluations within the RCEs, as well as the common cause evaluation encompassing all RCEs, resulted in development of additional corrective actions and/or additional analysis to determine if appropriate corrective actions were in place.

.1 Failure to adequately implement requirements of multiple procedures

The inspectors determined that the 12 broad-based CAPRs documented in the licensees RCE and summarized below, were appropriate to address the root and primary contributing causes.

  • Issue and implement a procedure for monitoring and improving the performance of operations crews.
  • Revise and implement a procedure for performance planning and monitoring to require organizational effectiveness reviews.
  • A suite of related corrective actions that, in general, are designed to monitor and improve leadership skills and accountability among managers and supervisors.
  • Revise a procedure to ensure that crew and individual training performance evaluations are reviewed during shift management review meetings.
  • Revise procedures for simulator and remedial training to require complete evaluation and documentation of underlying performance gaps.
  • Establish and/or revise processes to ensure effective use of the CAP.

The licensees corrective action plan also included nine CORRs to address the primary contributing causes. In general, these corrective actions included activities designed to improve line and senior operations management oversight of operator performance, ensure balanced crew composition, and improve the quality of training and evaluation of licensed operators.

Additionally, the human performance, CAP, and common cause RCEs specified additional corrective actions that were relevant to this issue. One of actions was a broad-based project to upgrade site procedures that was a CORR for a primary contributing cause in the human performance RCE. The procedure upgrade program will establish the appropriate resources for upgrading procedures, including operating procedures, to industry standards for content and format.

The inspectors concluded that the licensee identified appropriate corrective actions for each root and contributing cause identified in the RCE.

.2 Failure to properly implement Elements 3 & 4 of the SAT

The inspectors determined that the CAPRs documented in the licensees RCE and summarized below, were appropriate to address the root cause identified.

  • Revise and implement a procedure for performance planning and monitoring to require organizational effectiveness reviews.
  • A suite of related corrective actions that, in general, are designed to monitor and improve leadership skills and accountability among managers and supervisors.
  • A suite of related corrective actions that, in general, are designed to align training resources with operating needs, encourage uncensored reporting of weaknesses, and improve accountability for training material quality.
  • Revise a procedure to ensure that crew and individual training performance evaluations are reviewed during shift management review meetings.
  • Revise procedures for simulator and remedial training to require complete evaluation and documentation of underlying performance gaps.
  • Revise a procedure to require semiannual effectiveness reviews of all operations training programs by the TAB.

The licensees corrective action plan also included four CORRs to address the primary contributing causes. In general, these corrective actions included self assessments and training for various groups.

The inspectors concluded that the licensee identified appropriate corrective actions for each root and contributing cause identified in the RCE.

.3 Failure to correct a condition adverse to quality involving the B EDG output breaker

The inspectors determined that the CAPRs documented in the licensees RCE and summarized below, were appropriate to address the root cause identified.

  • Establish and implement methods for site leadership engagement to initiate a culture shift in order to change behavior to embrace the CAP
  • Revise procedures to better align with industry best standards and to support implementation of a single point of entry process to identify adverse conditions in the CAP
  • Develop and implement procedures to establish the requirements for the Outage Command Center for resolving unplanned emergent equipment issues and managing discovery items through the use of contingency plans or by coordination of recovery actions
  • Develop and implement procedures to clearly communicate what is required to work under the skill of the craft criteria for safety related equipment
  • Revise procedures to clearly state WO instruction use requirements for safety related equipment
  • Provide training to maintenance personnel on work packages for safety related structures, systems and components and differences between skill of the craft and minor maintenance activities The corrective action plan also included 17 CORRs to address the primary contributing causes. In general, these corrective actions included procedure revisions, self assessments, training for various organizations, and development of a Human Performance Improvement Plan.

The inspectors concluded that the licensee identified appropriate corrective actions for each root and contributing cause identified in the RCE.

b. Determine that the corrective actions have been prioritized with consideration of risk significance and regulatory compliance.

The licensees corrective action plan assignments were categorized in accordance with CAP-NGGC-0205. In general, the corrective actions were not prioritized based on risk significance, but on the type of corrective action involved (CAPR or CORR).

.1 Failure to adequately implement requirements of multiple procedures

The licensee, as described in the RCE (NCR 438394) completed both immediate and interim corrective actions to address deficiencies identified as a result of the March 28, 2010 reactor trip and safety injection.

The inspectors noted that a substantial portion of the corrective actions associated with this issue were not completed at the time of the inspection. Specifically, the licensee, as part of a larger procedure upgrade project, is in the process of converting Emergency Operating Procedure (EOP) PATH-1 and PATH-2 flowcharts to the Westinghouse standard two column format. The licensee plans to implement the two column EOPs in the control room in December 2011 and will begin training on use of the two column EOPs in September 2011. Inspectors observed that the licensees priorities for upgrading other operating procedures (e.g. Abnormal Operating Procedures (AOPs),

Annunciator Panel Procedures (APPs)), as well as procedures used by other departments, were not as clearly defined. Inspectors determined that the licensee had identified those AOPs directed to be implemented by the two column EOPs with the intention of upgrading these AOPs ahead of other AOPs. The licensee indicated that some APPs will also be upgraded during this time frame based on their relationship to the two column EOPs. The scheduled completion dates for the remaining portions of the procedure upgrade project ranged from two to five years.

The inspectors concluded that, although the licensee did not prioritize all corrective actions related to this issue with respect to risk significance, the prioritization methodology was, in part, based on consequences with respect to accident mitigation and adequate to assure timely completion of the actions. The inspectors concluded that the licensee appropriately prioritized corrective actions, considering the impact of the actions on correcting and preventing recurrence of the problem as well as regulatory compliance.

.2 Failure to properly implement Elements 3 & 4 of the SAT

As described in Section 02.01.c.2, the licensee took immediate corrective action in October 2010 to restore compliance with 10 CFR 55.59.

As described in Section 02.02.d.2, the licensee identified, as part of their extent-of-condition review that the majority of training material quality problems existed in the accredited operations training programs. The licensees corrective action to upgrade initial and continuing training materials established a priority based on consequences with respect to accident mitigation. The licensees priority for upgrading training material, as described in their RCE, was as follows:

  • First, EOP material for operations training
  • Second, AOP and plant systems (with high risk importance) material for operations training
  • Third, maintenance training program material The inspectors determined that the corrective actions for this issue were appropriately prioritized based on risk significance and regulatory compliance.

.3 Failure to correct a condition adverse to quality involving the B EDG output breaker

The licensee restored compliance with the plant Technical Specifications in April 2009 when the EDG B was satisfactorily returned to service. Additionally, the licensee completed the following WOs to remove the control relay failure mechanism by replacing the cotter pins for spiral rings in all breakers identified in the extent-of-condition as recommended by the breaker vendor.

  • 1540986 - EDG "B" to 480V Bus E2 (52/27B)
  • 1540983 - EDG "A" to 480V Bus E1 (52/17B)
  • 1540987 - 480V Bus E2 to Safety Injection Pump B (52/29B)
  • 1540985 - 480V Bus E1 Supply to Safety Injection Pump "B" (52/22B)
  • 1541991 - Station Service Transformer 2F to 480V Bus E1 (52/18B)
  • 1541992 - Station Service Transformer 2G to 480V Bus E2 (52/28B)

The inspectors concluded that, although the licensee did not prioritize corrective actions specifically with respect to risk significance, the prioritization methodology was adequate to assure timely completion of the actions. The inspectors concluded that licensee appropriately prioritized corrective actions, considering the impact of the actions on correcting and preventing recurrence of the problem as well as regulatory compliance.

c. Determine that a schedule has been established for implementing and completing the corrective actions.

The inspectors observed that the licensee had entered all corrective actions identified in their RCEs into their action tracking program. The inspectors determined that each corrective action was associated with an action request (AR) number and had an associated assignment number, corrective action type, corrective action description, assignment owner, due date, and completion status.

The inspectors observed that the majority of corrective actions planned by the licensee had not been completed by the end of the onsite inspection and may warrant additional inspection in order to ensure that the completed actions adequately address the root causes of the issues discussed in this report. The inspectors noted that a number of the corrective actions in progress or planned by the licensee are designed to change long term organizational behavior.

.1 Failure to adequately implement requirements of multiple procedures

The inspectors determined that the licensee had established schedules for completing corrective actions related to procedures and hardware associated with this issue. The inspectors observed that although the completion dates for those CAPRs intended to change organizational behavior are tied to implementation of the related procedures, the effectiveness reviews of these CAPRs are linked to the intended organizational changes.

The inspectors concluded that that the licensee had established a schedule for implementing and completing the corrective actions.

.2 Failure to properly implement Elements 3 & 4 of the SAT

The inspectors determined that the licensee had established schedules for completing corrective actions related to training material associated with this issue. The inspectors reviewed a sample of corrective actions which were completed at the time of this inspection to verify that they were implemented as intended in the RCE.

The inspectors noted that, although the licensee had developed a scenario to evaluate the ability of simulator evaluators to consistently identify underlying operator knowledge and skill deficiencies, the licensee had not yet conducted the scenario. The inspectors questioned the licensee regarding the intent and schedule of this activity. The licensee stated that the intent of the corrective action was to evaluate all simulator evaluators using this activity even though the wording of the corrective action did not state this intention. The licensee subsequently initiated NCR # 00470049 to track completion of this activity for the simulator evaluators.

The inspectors concluded that that the licensee had established a schedule for implementing and completing the corrective actions.

.3 Failure to correct a condition adverse to quality involving the B EDG output breaker

The inspectors reviewed the sample of corrective actions listed below, which were completed at the time of this inspection, to verify that they were implemented as intended in the RCE.

  • The inspectors reviewed procedure CAP-NGGC-0200, Corrective Action Program, which was revised to better align with industry best practices and to support implementation of a single point of entry process to identify adverse conditions in the CAP. The inspectors also attended a screening committee meeting to verify that conditions adverse to quality were being identified and entered into the CAP as NCRs.
  • The inspectors reviewed procedure OMA-NGGC-0206, Outage Command Center Structure, Staffing, and Expectations, which was developed to establish the requirements for resolving unplanned emergent equipment issues and managing discovery items through the use of contingency plans or by coordination of recovery actions.
  • The inspectors reviewed procedure MNT-NGGC-1000, Conduct of Maintenance, which was developed to clearly communicate the criteria for applying skill of the craft work practices to safety related equipment.
  • The inspectors reviewed procedure ADM-NGGC-0104, Work Implementation and Completion, which was revised to clearly state the requirements for use of WO instructions when working on safety related equipment.
  • The inspectors reviewed training material provided to maintenance personnel for work packages on safety related structures, systems and components and differences between skill of the craft work practices and minor maintenance.

The inspectors did not identify any inconsistencies between the completed corrective actions and their description in the RCEs corrective action plan. The inspectors concluded that that the licensee had established a schedule for implementing and completing the corrective actions.

d. Determine that quantitative or qualitative measures of success have been developed for determining the effectiveness of the corrective actions to prevent recurrence.

The licensees RCEs included effectiveness reviews for all CAPRs. Each effectiveness review was entered in the licensees action tracking process with an associated AR number and had a designated assignment number, action type (i.e. EREV), description of the method to be used for performing the effectiveness review, attributes and success criteria to consider, effectiveness review owner, due date, and completion status.

.1 Failure to adequately implement requirements of multiple procedures

The inspectors reviewed the ARs associated with the effectiveness reviews for CAPRs associated with this issue. The inspectors did not identify any inconsistencies between the CAPRs, the associated effectiveness reviews as described in the licensees RCE, and the ARs associated with the effectiveness reviews.

The inspectors observed that the licensees effectiveness reviews for changes to tangible objects, such as changes to equipment and procedures, were written to ensure verification that the changes were physically completed. The licensees effectiveness reviews for more subjective issues associated with organizational change included assessing the overall goal related to the changes.

The inspectors concluded that the licensee had developed adequate quantitative and/or qualitative measures of success for determining the effectiveness of the CAPRs.

.2 Failure to properly implement Elements 3 & 4 of the SAT

The inspectors reviewed the ARs associated with the effectiveness reviews for CAPRs associated with this issue. The inspectors did not identify any inconsistencies between the CAPRs, the associated effectiveness reviews as described in the licensees RCE, and the ARs associated with the effectiveness reviews.

The inspectors concluded that the licensee had developed adequate quantitative and/or qualitative measures of success for determining the effectiveness of the corrective actions to prevent recurrence.

.3 Failure to correct a condition adverse to quality involving the B EDG output breaker

The inspectors reviewed the ARs associated with the effectiveness reviews for CAPRs associated with this issue. The inspectors did not identify any inconsistencies between the CAPRs, the associated effectiveness reviews as described in the licensees RCE, and the ARs associated with the effectiveness reviews.

The inspectors concluded that the licensee had developed adequate quantitative and/or qualitative measures of success for determining the effectiveness of the corrective actions to prevent recurrence.

e. Determine that the corrective actions planned or taken adequately address a Notice of Violation (NOV) that was the basis for the supplemental inspection, if applicable.

.1 Failure to adequately implement requirements of multiple procedures

The NRC issued an NOV (EA 2010-257) to the licensee on January 31, 2011, in IR 2011-008 for failure to implement requirements of multiple procedures.

Based on the review of the RCE, the inspectors determined that the licensee identified all the deficiencies leading to this issue and developed a corrective action plan to address the root and contributing causes to the event. These actions included changes to administrative, training, and operating procedures, changes to equipment, and additional or enhanced training for operators. The inspectors noted that some of the procedure changes are intended to eliminate weaknesses in the organizational culture that contributed to the violations.

The inspectors determined the corrective actions that the licensee has completed, and plans to complete, adequately address the NOV that was the basis for this supplemental inspection.

.2 Failure to properly implement Elements 3 & 4 of the SAT

The NRC issued an NOV (EA 2010-257) to the licensee on January 31, 2011, in IR 2011-008 for failure to adequately implement Element 3 (design and implement training based on learning objectives) of the SAT.

Based on a review of the RCE, the inspectors determined that the licensee identified all the deficiencies leading to the failure to implement Element 3 of the SAT and developed a corrective action plan to address the root and contributing causes of the deficiency.

Additionally, as described in Section 02.01.c.2, the licensee took prompt corrective actions to address the training deficiencies resulting from failure to implement Element 3 of the SAT.

The inspectors determined the corrective actions that the licensee has completed, and plans to complete, adequately address the NOV that was the basis for this supplemental inspection.

.3 Failure to correct a condition adverse to quality involving the B EDG output breaker

The NRC issued an NOV (EA 2010-205) to the licensee on December 7, 2010, in IR 2010-014 for failure to assure that a condition adverse to quality, involving an EDG output breaker 52/27B failure-to-close malfunction, was promptly corrected. Based on the review of the RCE, the inspectors determined that the licensee identified all the deficiencies leading to the failure to promptly correct this condition adverse to quality and developed a corrective action plan to address the root and contributing causes to the event. Additionally, the licensee took prompt corrective actions to remove the failure mechanism from all affected breakers on site.

The inspectors determined the corrective actions that the licensee has completed, and plans to complete, adequately address the NOV that was the basis for this supplemental inspection.

f. Findings

.

No findings were identified.

02.04 Independent Assessment of Extent of Condition and Extent of

Cause.

a. Inspection Scope

.

The inspectors independently assessed the validity of the licensees conclusions regarding the extent of condition and extent of cause of the findings. The objective of this requirement was to independently sample performance, as necessary, within the key attributes of the cornerstone that were related to the findings to ensure that the licensees evaluation regarding the extent of condition and extent of cause was sufficiently comprehensive.

The inspectors conducted independent extent of condition and extent of cause reviews of the issues associated with the three White findings. The reviews focused on the primary root causes associated with the findings as well as contributing causes related to more specific aspects of the root causes. In conducting this review, the inspectors interviewed station management and personnel, reviewed program and process documentation, reviewed station program monitoring and improvement efforts, and reviewed corrective action documents.

.1 Failure to adequately implement requirements of multiple procedures

In assessing the extent of condition and extent of cause of the failure to adequately implement requirements of multiple procedures, the inspectors reviewed the operational errors made by the control room staff on March 28, 2010, which was the basis for the finding. The inspectors evaluated whether the root and primary contributing causes identified by the licensee extended into other functions, processes, and organizations.

This verification included review of the following records:

  • administrative procedures
  • training procedures
  • operations procedures
  • training lesson plans
  • simulator evaluations
  • evaluations performed by nuclear oversight and plant management, and
  • evaluations of operating crew performance The inspectors independently evaluated operator performance during licensee evaluated dynamic simulator scenarios to assess the degree to which weaknesses in individual and operating crew performance extended beyond the individuals that were on shift during the March 28, 2010, reactor trip and safety injection. The inspectors also assessed the ability of the evaluators to accurately document any discovered weaknesses so that those weaknesses could be adequately corrected and trended. Additionally, the inspectors evaluated the performance of operators in the control room in order to independently assess SRO command and control and oversight of planned evolutions.

.2 Failure to properly implement Elements 3 & 4 of the SAT

The inspectors used applicable portions of IP 71111.11, IP 41500, and NUREG 1220 to independently assess the extent-of-condition and extent-of-cause of the failure to properly implement Elements 3 & 4 of the SAT. The inspectors evaluated whether the root and primary contributing causes identified by the licensee extended into other functions, processes and elements of the licensees programs and especially within operations training programs. To perform this evaluation, inspectors completed the following activities:

  • reviewed the licensees procedures for analyzing tasks, identifying skills/knowledge, developing learning objectives, and for conducting evaluation and remediation (SAT Element 1 & 2)
  • reviewed the licensees Reactor Operator and Senior Reactor Operator task lists and task-to-training matrix to identify potential discrepancies in training and qualification methods and to identify training materials for further evaluation (SAT Element 1 & 3)
  • evaluated a sample of 25 job tasks using NUREG 1220 (SAT Element 1)
  • evaluated the licensees training analysis of two plant modifications (SAT Element 1, 2 & 3)
  • evaluated five Shift Manager (SM) qualification card training modules (SAT Element 1, 2, & 3)
  • assessed remedial training for crews that failed their annual operating examination (SAT Element 4)
  • assessed the licensees administration and documentation of simulator examinations for licensed operators (SAT Element 4)
  • compared licensee procedures for training Shift Technical Advisors (STAs) and SMs to industry guidelines in order to identify potential discrepancies in the licensees training programs (SAT Element 5)
  • verified that all SMs had completed the licensees SM Training Program requirements
  • reviewed Training Program Committee minutes and assessed the training conducted as a result of the March 28, 2010 event (SAT Element 5)
  • interviewed operations training instructors (SAT Elements 1, 2, 3, 4, & 5)

.3 Failure to correct a condition adverse to quality involving the B EDG output breaker

The inspectors conducted an independent assessment of the extent of condition and extent of cause of the EDG B output breaker White Finding. The extent of condition focused on the identification of similar breakers that could be susceptible to the same failure mechanism introduced by the cotter pin. The inspectors reviewed plant drawings of the 480V distribution system to identify all the potential applications that could use the same type of breaker affected by the cotter pin issue (i.e. Westinghouse DB-75 and DB-100). The inspectors confirmed the information contained in the plant drawings by conducting a walk down of a sample of safety related and non-safety 480V breakers to physically confirm the breaker model. For those 480V breakers that were not DB-75 or DB-100, the inspectors conducted an inspection of the control relay operation on similar breakers that were out of service to verify that the cotter pin failure mechanism did not apply to those breaker models. The inspectors reviewed equipment data bases, WOs, and refurbishment records to verify that the cotter pin failure mechanism had been removed from all affected breakers. The inspectors also conducted an inspection of spare DB-75 and DB-100 breakers to verify that the cotter pin failure mechanism had been corrected.

The independent extent of cause focused on the review of equipment issues and work performed during the last two refueling outages to determine the degree to which the root and contributing causes could have resulted in additional problems. The inspectors reviewed a sample of corrective maintenance WOs, post-maintenance testing WOs, post-modification WOs, and Maintenance Rule Evaluations for risk significant systems.

b. Assessment.

.1 Failure to adequately implement requirements of multiple procedures

The inspectors did not identify any significant issues in the independent assessment of the extent of condition and extent of cause that would have impacted impact the licensees RCE conclusions and corrective action plan.

In assessing the contributing causes of the operating errors made on March 28, 2010, the inspectors determined that several of the errors made were errors of omission on the part of the control board operators, which directly challenged the Control Room Supervisor (CRS), SM, and STA on their ability to effectively monitor the plant and direct appropriate mitigating actions. Examples included:

  • Operators failed to diagnose that the reactor coolant temperature had decreased by over 100 oF and therefore did not take action to stop the cooldown.
  • Operators failed to diagnose key parameters and critical alarms associated with RCP seal cooling (RCP thermal barrier heat exchanger cooling and low volume control tank level). The failure to identify and respond to these alarms before the loss of adequate RCP seal cooling significantly increased the likelihood of a loss of coolant accident.

The inspectors observed that the licensee, in their RCE, identified that operations and training procedures did not contain adequate requirements for tracking and resolving both crew and individual performance deficiencies. However, the licensee did not identify that part of the contributing cause for this condition was that their methods for assessing operator performance did not individually evaluate all operators on their ability to effectively monitor the plant, maintain oversight, and direct appropriate mitigating actions.

Additionally, the inspectors observed that, per OPS-NGGC-1000, Fleet Conduct of Operations, plant management was required to review and adjust on-shift crew composition based, in part, on strengths and weaknesses of crew members. Inspectors determined it would be necessary for plant management to know, based on objective evidence, the strengths and weaknesses of individual operators in order to effectively adjust on-shift crew composition. The inspectors concluded that individually evaluating operators on fundamental competencies, such as their ability to monitor the plant, provide oversight, and provide appropriate direction to the crew, was a prerequisite to being able to effectively adjust shift composition based on individual operator strengths and weaknesses.

The inspectors concluded that the licensees assessment methods were not structured to thoroughly evaluate operators responsible for plant oversight and command and control, on an individual basis, when control board operators fail to identify and diagnose changing plant conditions. In particular, the licensees evaluation methods did not ensure that the CRS, SM, and STA were challenged to the same degree when faced with a complex event similar to the March 28, 2010 event.

The inspectors discussed their observation with licensee operations and training management and the licensee entered the observation into their CAP as AR 00469150 and 00469663. The licensees proposed corrective actions were designed to more precisely address individual evaluation of all operators, including the CRS, SM and STA, on their ability to adequately monitor the plant and provide effective direction for addressing changing plant conditions.

The inspectors concluded that the licensee conducted an adequate RCE with respect to extent of condition and extent of cause.

.2 Failure to properly implement Elements 3 & 4 of the SAT

The inspectors did not identify any significant issues in the independent assessment of the extent of condition and extent of cause that the licensee had not already identified and addressed with planned or completed corrective actions. The inspectors concluded that the licensee conducted an adequate RCE with respect to extent of condition and extent of cause.

.3 Failure to correct a condition adverse to quality involving the B EDG output breaker

The inspectors did not identify any significant issues in the independent assessment of the extent of condition and extent of cause that would have impacted impact the licensees RCE conclusions and corrective action plan. The inspectors concluded that the licensee conducted an adequate RCE with respect to extent of condition and extent of cause.

c. Findings

.

No findings were identified.

02.05 Safety Culture Consideration.

a. Inspection Scope

.

The inspection team conducted a focused inspection to independently determine that the licensees RCE appropriately considered whether any safety culture component caused or significantly contributed to any risk significant performance issue. The inspectors reviewed NCRs associated with the licensees RCEs, licensees employee concerns program, and conducted group and individual interviews with 57 licensee staff to determine if the licensee properly considered whether any safety culture component caused or contributed to the findings and to assess the validity of the safety culture components identified in the licensees RCE.

b.

Assessment.

As part of their RCEs, the licensee reviewed the identified root causes, contributing causes, and inappropriate acts against the safety culture components that could have contributed to the issues. The licensees RCEs included discussions of the 13 safety culture components described in Regulatory Issue Summary 2006-013, Information on the Changes Made to the Reactor Oversight Process to More Fully Address Safety Culture, (ADAMS Accession No. ML061880341) as they applied to the performance issues.

The inspection team independently confirmed the licensees conclusion that improving behaviors related to safety culture should be a high priority for the recovery effort. The inspection team concluded that every safety culture component was a contributor to the performance issues and agreed with the licensees assessment that weaknesses in decision making, resources, work practices, the CAP, and self and independent assessments were the most prevalent contributors. For all safety culture components identified by the licensee as contributing to the performance issues, the inspection team confirmed that the licensee had established appropriate corrective actions.

The results of the interviews conducted by the inspectors supported the conclusions documented in the licensees common cause evaluation, which stated:

Limited resources perplexed by changing management team, changing plant and fleet priorities, resulted in eroded standards. Additionally, impacts were felt with higher backlogs, poor procedures, flawed training implementation and evaluation processes, and a high threshold for writing NCRs or using a systematic problem solving method through the corrective action process.

Senior Management did not ensure that risk informed decisions were effectively factored into business planning and succession planning using metrics that provided a complete and accurate indication of declining performance. Consequently, organizational capacity did not match the long term risk-informed needs of RNP resulting in continued declining performance.

While interviewing licensee personnel, the inspectors asked questions related to a Safety Conscious Work Environment to determine if the licensees staff were reluctant to raise safety concerns or if retaliation resulted from raising safety concerns. The inspectors determined that the staff felt free to raise nuclear safety concerns without fear and were aware of multiple avenues available to report issues. The inspectors concluded that the staff was very comfortable reporting issues to first line supervisors.

The inspectors concluded that licensees RCEs, including the extent of condition and extent of cause evaluations, appropriately considered the safety culture components as described in IMC 0305, Operating Reactor Assessment Program. The inspection team also concluded that the licensee had established appropriate corrective actions to address the identified safety culture issues.

c. Findings

.

Introduction:

The team identified an unresolved item (URI) concerning the closure of a large number of engineering change requests (ECRs) as identified in NCR 417814.

Description:

The inspectors noted that NCR 417814 was written to address a condition adverse to quality associated with the cancellation of approximately 375 ECRs which were cancelled without technical justification. The NCR stated that the individual ECRs were cancelled as part of a corrective action associated with an earlier NCR (382451)which identified the ECRs as not meeting an administrative requirement to have a management sponsor. Inspectors noted that the corrective actions taken by the licensee did not ensure that each cancelled ECR was reviewed to ensure the existence of an adequate technical basis for cancellation. Inspectors also noted that the evaluation did not consider that some of the ECRs were intended to correct previously identified conditions adverse to quality as documented in NCRs. The inspectors identified that at least one open NCR corrective action was inappropriately closed due to the cancellation of the ECRs. Inspectors concluded that further review of information related to the closure of the ECRs and any related NCRs is necessary to determine if the issue is more than minor. The licensee entered this issue into their corrective action program as NCR 417814. This issue is identified as URI 0500261/2011010-01, "Simultaneous Closure of Several Engineering Change Requests Not Meeting Administrative Requirements" 02.06 Evaluation of IMC 0305 Criteria for Treatment of Old Design Issues The licensee did not request credit for self identification of an old design issue. Therefore, the subject risk significant issues were not evaluated against the IMC 0305 criteria for treatment of an old design issue.

4OA6 Meetings, Including Exit

On June 9, 2011, the lead inspector presented the inspection results to Mr. R. Duncan and members of the licensees staff. The licensee acknowledged the findings and confirmed that the inspectors did not review any proprietary information.

ATTACHMENT:

SUPPLEMENTAL INFORMATION

KEY POINTS OF CONTACT

Licensee personnel

S. Blaker, Control Room Supervisor - Operations
S. Brown, Superintendent - Nuclear Operations Performance
J. Edwards, Superintendent - Operator Training
D. Foster, Superintendent - Work Control Center - Operations
R. Hill, Supervisor - Performance Improvement
C. Kamilaris, Manager - Support Services
G. Kilpatrick, Manager - Operations
A. Pope, BNP Supervisor - Licensing/Regulatory Programs
S. Price, Employee Concerns
C. Sandifer, Lead Engineer - System Engineering
T. Simonson, Superintendent - Electrical/I&C
A. Zimmerman, NGG Lead Engineer - Nuclear Regulatory Affairs

LIST OF ITEMS OPENED, CLOSED, AND DISCUSSED

Opened

05000261/2011010-01 URI Simultaneous Closure of Several Engineering Change Requests Not Meeting Administrative Requirements (Section 4OA4.02).

Opened and Closed

None

Closed

05000261/2010013-01 VIO Failure to Comply with Conduct of Operations Procedure
05000261/2010004-05 VIO Failure to Correctly Implement a Systems Approach to Training of the Licensed Operator Requalification Program
05000261/2010006-02 VIO Failure to Correct a Condition Adverse to Quality in B Emergency Diesel Generator

LIST OF DOCUMENTS REVIEWED