IR 05000456/2002010
| ML030020255 | |
| Person / Time | |
|---|---|
| Site: | Braidwood  |
| Issue date: | 12/31/2002 |
| From: | Grant G Division Reactor Projects III |
| To: | Skolds J Exelon Generation Co, Exelon Nuclear |
| References | |
| -RFPFR IR-02-010 | |
| Download: ML030020255 (29) | |
Text
December 31, 2002
SUBJECT:
BRAIDWOOD STATION, UNIT 1 NRC SUPPLEMENTAL INSPECTION REPORT 50-456/02-10(DRP)
Dear Mr. Skolds:
On December 4, 2002, the U.S. Nuclear Regulatory Commission (NRC) completed Supplemental Inspection Procedure 95002 Inspection For One Degraded Cornerstone or Any Three White Inputs In A Strategic Performance Area at your Braidwood Station, Unit 1. The results of this inspection were discussed on December 4, 2002, with Mr. von Suskil and other members of your staff. The enclosed report presents the results of this inspection.
The NRC conducted this supplemental inspection as required by the NRC Action Matrix based on our assessment of plant performance. As stated in our August 22, 2002 Mid-Cycle Performance Review letter, plant performance at Braidwood Station Unit 1 was within the Degraded Cornerstone Column of the NRC Action Matrix based on two White issues in the Mitigation Systems Cornerstone.
The first issue was identified in the fourth quarter of 2001 when performance of the Unit 1 auxiliary feedwater system declined resulting in a White performance indicator (Safety System Unavailability, Heat Removal System, Auxiliary Feedwater System) in the Mitigation Systems Cornerstone. Two events in 2001 resulted in increased fault exposure hours for the 1B Auxiliary Feedwater pump: (1) foreign material in the control air solenoid valve for the room cooler service water discharge isolation valve and (2) fuel shutoff solenoid valve failure.
Supplemental Inspection Procedure 95001 Inspection For One or Two White Inputs In a Strategic Performance Area was conducted in February 2002 to better understand the declining performance. The inspection results were documented in NRC Inspection Report 50-456/02-04(DRP).
The second issue pertains to your staffs failure to take prompt corrective actions to prevent recurring Unit 1 pressurizer power operated relief valve (PORV) air accumulator check valves leak-through, as evidenced by repeated failures to meet testing acceptance criteria between 1991 and 2001. This resulted in several extended periods where the unit was operated in a condition where the pressurizer PORVs may not have been able to perform their intended safety function of opening following events which resulted in isolation of instrument air to the containment or loss of the service air compressors. This issue was characterized as White (low to moderate risk significance) and affected the Mitigation Systems Cornerstone. The supplemental inspection was an examination of activities conducted under your license as they relate to safety and to compliance with the Commissions rules and regulations and with the conditions of your license. Within these areas, the inspection consisted of a selective review of procedures and representative records and interviews with personnel. The purpose of this inspection was to (1) provide assurance that the root and contributing causes for the White performance indicator for the auxiliary feedwater system failures, the White inspection finding concerning inadequate corrective actions for the Unit 1 pressurizer power operated relief valve air accumulator check valves, and the overall performance issues which resulted in the Degraded Cornerstone are understood; (2) independently assess the extent of condition and generic implications; and (3) provide assurance that the corrective actions are sufficient to prevent recurrence.
Based upon the results of this inspection, the team determined that your root cause evaluation for the White performance indicator and the White inspection finding identified the primary and contributory causes for the issues. Your corrective actions which included replacing the fuel solenoid shutoff valve and revising maintenance procedures associated with the air accumulator check valve have been completed. Therefore, the White finding associated with the PORV air accumulator check valves will only be considered in assessing plant performance for a total of four quarters in accordance with the guidance in IMC 0305, Operating Reactor Assessment Program.
With respect to the Degraded Mitigation Systems Cornerstone, you attributed the primary root cause to be the inability of station personnel to identify and correct long term equipment problems and an overall tolerance for longstanding degraded material conditions. The inspection team did not identify significant weaknesses in your evaluation. The team noted that your proposed corrective actions and evaluation activities associated with the degraded cornerstone were in a developmental and investigatory phase. While the team found your approach for completing these activities to be sound, the team was not able to assess the effectiveness or completeness of these proposed actions because these actions were incomplete. The team also noted that the second corrective action, specifically, the performance of aggregate system reviews, was not yet endorsed by corporate management.
Because our assessment of your corrective actions was based on your preliminary plans, we will review and, if necessary, re-assess the effectiveness of your corrective actions during an additional Problem Identification and Resolution inspection which will be performed in accordance with Inspection Procedure 71152. The specific dates for this inspection will be communicated in the end-of-cycle assessment letter. In accordance with 10 CFR 2.790 of the NRCs "Rules of Practice," a copy of this letter and its enclosure will be available electronically for public inspection in the NRC Public Document Room or from the Publicly Available Records System (PARS) component of NRCs document system (ADAMS). ADAMS is accessible from the NRC Web site at http://www.nrc.gov/reading-rm/adams.html the Public Electronic Reading Room).
Sincerely,
/RA/
Geoffrey E. Grant, Director Division of Reactor Projects Docket No. 50-456 License No. NPF-72
Enclosure:
Inspection Report 50-456/02-10
REGION III==
Docket No:
50-456 License No:
NPF-72 Report No:
50-456/02-10(DRP)
Licensee:
Exelon Generating Company, LLC Facility:
Braidwood Station, Unit 1 Location:
35100 S. Route 53 Suite 84 Braceville, IL 60407-9617 Dates:
November 4 through December 4, 2002 Inspectors:
S. Burton, Senior Resident Inspector R. Lerch, Project Engineer N. Shah, Resident Inspector Approved by:
Ann Marie Stone, Chief Branch 3 Division of Reactor Projects
SUMMARY OF FINDINGS
IR 05000456-02-10; Exelon Generating Company, LLC; on 11/4-12/4/2002, Braidwood Station;
Unit 1. Supplemental Inspection - Mitigation Systems Cornerstone.
Cornerstone: Mitigation Systems
The U.S. Nuclear Regulatory Commission (NRC) performed this supplemental inspection to assess, both individually and collectively, the licensees root cause evaluations and corrective actions associated with a degraded Mitigation Systems Cornerstone which resulted from a White performance indicator for the auxiliary feedwater system (AFW) safety system unavailability and a White finding pertaining to inadequate corrective actions for pressurizer power operated relief valves (PORV) air accumulator check valves. Supplemental Inspection Report 50-456/02-04 documented the details and the initial review of the White AFW safety system unavailability performance indicator. The inadequate corrective actions associated with the PORV air accumulator check valves was previously characterized as White in the NRCs final significance determination letter dated July 12, 2002, and was defined as having low to moderate risk significance. Inspection Report 50-456/02-03; 50-457/02-03 documented the details and the initial review of that finding.
During this supplemental inspection, performed in accordance with Inspection Procedure 95002, the team evaluated the issues both individually and collectively. The team determined that the licensee performed a comprehensive evaluation of the issues individually and collectively. The licensee attributed the primary root cause for the degraded cornerstone to be the inability of station personnel to identify and correct long term equipment problems and an overall tolerance for longstanding degraded material conditions. The licensees planned corrective actions included a periodic review of mitigation system performance and a human performance improvement program.
The team did not identify any significant findings during their review of the licensees evaluation.
The team concluded that the primary root cause, the inability of station personnel to identify and correct long term equipment problems and an overall tolerance for longstanding degraded material conditions, represented a human performance cross cutting issue. The team identified that the proposed corrective actions for the contributing causes were in a developmental and investigatory phase. The team found the approach for completing these activities to be sound, but were unable to assess the effectiveness or completeness of these proposed actions.
Because the team could not confirm that the proposed actions will be initiated, the licensees corrective actions associated with the degraded cornerstone will be re-assessed during a subsequent Problem Identification and Resolution inspection.
Given the licensees acceptable performance in addressing the PORV air accumulator check valves, the White finding associated with this issue will only be considered in assessing plant performance for a total of four quarters in accordance with the guidance in IMC 0305,
Operating Reactor Assessment Program.
SUMMARY OF FINDINGS (contd)
A.
Inspector-Identified Findings No findings of significance were identified.
B.
Licensee-Identified Findings No findings of significance were identified.
REPORT DETAILS
INSPECTION SCOPE
The U.S. Nuclear Regulatory Commission (NRC) performed this supplemental inspection to assess, both individually and collectively, the licensees root cause evaluations and corrective actions associated with a degraded Mitigation Systems Cornerstone which resulted from a White performance indicator for auxiliary feedwater system (AFW) unavailability and a White finding pertaining to inadequate corrective actions for power operated relief valves (PORV) air accumulator check valves. The White AFW safety system unavailability performance indicator was documented in Supplemental Inspection Report 50-456/02-04. The inadequate corrective actions associated with the PORV air accumulator check valves was previously characterized as White in the NRCs final significance determination letter dated July 12, 2002.
EVALUATION OF INSPECTION REQUIREMENTS 02.01 Problem Identification
.1 Concerns with the Unit 1B Diesel Driven Auxiliary Feedwater Pump
a.
Determination of who (i.e., licensee, self revealing, or NRC) identified the issue and under what conditions In Supplemental Inspection Report 50-456/02-04, the NRC documented that the conditions leading to the White performance indicator for the AFW system were self-revealing. In the fourth quarter of 2001, the accumulated unavailability/fault exposure time for the 1B AFW pump exceeded the NRC performance indicator White threshold. This resulted from the pumps failing to start during surveillance testing between September 1999 and November 2001, and from the failure of the pump cooling water outlet valve to open in April 2001.
The primary contributor to these events was the failure of the 1B AFW pump fuel shutoff solenoid valve. Following the pumps failure to start in November 2001, the licensee identified that this valve was inappropriate for the pumps fuel control system. The valve was designed for use in a hydraulic oil versus lubricating oil applications. Therefore, the internal clearances of the valve were smaller and were not designed for the higher particle counts in the pump lubricating oil. The licensee also identified that the 1B AFW pumps particle count was within limits, but was significantly higher than the 2B AFW pump. This accounted for the increased number of failures on the 1B AFW pump.
The team identified no additional issues during their review.
b.
Determine of how long the issue existed, and prior opportunities for identification In Supplemental Inspection Report 50-456/02-04, the NRC identified that the licensee had taken inadequate action to identify and correct the problem with the fuel shutoff solenoid valve.
The team identified no additional issues during their review.
Between 1992 and 1999, the 1B AFW pump was slow to start on five occasions. The licensee failed to initiate actions to identify the cause for each event. The licensee removed individual pump components and analyzed them for failure after a slow start in September 1999. However, the licensee took no further actions to determine the root cause when problems with the individual components were not identified. After the November 2001 failure resulted in the performance indicator exceeding the White threshold, the licensee determined that the installed fuel shutoff solenoid valve was inappropriate for the pump fuel control system.
c.
Determination of the plant specific risk consequences (as applicable) and compliance concerns associated with the issue The licensees risk evaluation was reviewed and documented in Supplemental Inspection Report 50-456/02-04 and no problems were identified. A Non-Cited Violation and Green finding for the licensee failure to identify the cause and prevent recurrence of the AFW pump problems was documented in that report.
The team identified no additional issues during their review.
.2 Unit 1 Pressurizer Power Operated Relief Valve Air Accumulator Check Valve Failures
a.
Determination of who (i.e., licensee, self revealing, or NRC) identified the issue and under what conditions The failure of the pressurizer PORV air accumulator check valves was NRC-identified and documented in Inspection Reports 50-456/95010; 50-457/95010, 50-456/02-03; 50-457/02-03, and 50-456/02-07; 50-457/02-07. The PORV air accumulator check valves were designed to maintain an accumulator pressurized with operating air in the event that the normal supply of air was lost. Between 1991 and 2001, multiple failures had occurred which resulted in extended periods with the PORV air accumulator check valves potentially unable to perform their intended safety function. In many cases, past failures were not documented in a condition report (CR) or were documented without identifying the cause of the failures. Licensee Event Reports (LERs)50-456/2002-002-00 and -01 discussed this issue. The licensee initiated CR 95245 to determine the root cause of the failures.
b.
Determination of how long the issue existed, and prior opportunities for identification In 2002, the NRC identified that since 1991, the licensee had multiple PORV check valve failures, but failed to take the appropriate actions. Examples included:
- In 1992, all four check valves failed during surveillance testing. Although the licensee performed an evaluation, the evaluation was closed without identifying the cause of the failures.
- In 1994, two of the four check valves failed during surveillance testing. The licensee did not initiate a condition report for the failures and did not identify the cause for the failures.
- In January 1995, the A train check valves failed their surveillance tests and a condition report was initiated; however, the condition report was closed with no action taken and again, the licensee did not identify the cause for the failures.
- As documented in Inspection Reports 50-456/95010; 50-457/95010, the NRC noted that the licensee had not taken appropriate action to address the recurrent valve failures.
- In the fall of 1995, all four check valves failed during surveillance testing, but again, the licensee did not initiate a condition report to document the failures and again, did not identify the cause for the failures.
- In 1997, all four check valves initially failed during surveillance testing and an action item was initiated to identify the cause of the failures. However, this item was closed in 1998 without being completed.
- In 2001, all four check valves failed during surveillance testing and a condition report was initiated. This report concluded that the failures were maintenance preventable functional failures requiring an apparent cause evaluation (ACE).
However, this evaluation did not identify the root cause for the failures (incorrect disc to valve seat clearance) and concluded that the valves were operable.
c.
Determination of the plant specific risk consequences (as applicable) and compliance concerns associated with the issue The NRCs risk assessment concluded that the PORV air accumulator check valve repeated failures was of low to moderate safety significance, a White finding. The licensees risk assessment agreed with the NRCs assessment. The NRC concluded that the failures were a violation of 10 CFR Part 50, Appendix B, Criterion XVI, Corrective Action, for the failure to properly identify the reason for the failures and to take appropriate corrective action. In letters dated June 25 and July 23, 2002, the NRC summarized the results of the risk evaluation and transmitted the Notice of Violation.
On August 22, 2002, the licensee transmitted the response to the Notice of Violation, including the identified root cause and the associated corrective actions.
.3 Inability to Identify and Correct Longstanding Equipment Problems Leading to Degraded
Cornerstone a.
Determination of who (i.e., licensee, self revealing, or NRC) identified the issue and under what conditions In a letter dated July 23, 2002, the NRC stated that the combination of the White performance indicator for the 1B AFW pump and the White finding for the PORV air accumulator check valves resulted in a degraded Mitigation Systems Cornerstone under the Revised Reactor Oversight Process. This conclusion was restated by the NRC in an August 22, 2002 letter summarizing Braidwoods Mid-Cycle Performance Review. In response, the licensee initiated CR 113947 to perform a root cause evaluation to identify the issues leading to the degraded cornerstone.
The licensee concluded that the degraded cornerstone was caused by an inability to identify and correct longstanding equipment problems and an overall tolerance for longstanding degraded material conditions. This was exemplified by the multiple missed opportunities to identify and correct the problems associated with the 1B AFW pump (Section 02.01.1.a) and the PORV air accumulator check valves (Section 02.01.2.a).
The team noted that this conclusion was also consistent with licensee self-assessments of equipment reliability monitoring conducted in August and September 2002.
b.
Determination of how long the issue existed, and prior opportunities for identification The licensees root cause evaluation documented the dates of the issues and the missed opportunities for identification. The degraded Mitigation Systems Cornerstone resulted from the failure to identify and correct recurring problems with the 1B AFW pump and PORV air accumulator check valves (Sections 02.01.1.b and 02.01.2.b). The licensee concluded that the issues would have been resolved through the stations existing equipment reliability programs had opportunities for identification been recognized and appropriate actions been taken.
The team independently reviewed the stations existing equipment reliability programs, specifically, the licensees system health indicator program (SHIP) and rework monitoring programs, to assess their effectiveness. The SHIP was chosen because it provided an overall indicator of system health by integrating the results of other equipment monitoring processes, such as the maintenance rule and condition reporting programs. The rework program was chosen because of the recurring failure to correctly perform maintenance on both the 1B AFW pump fuel solenoid valve and the PORV air accumulator check valves.
The team concluded that the SHIP adequately monitored equipment performance and integrated the results from other equipment monitoring programs. However, the team noted that these programs relied heavily on the corrective action program for capturing equipment performance issues. The failure to write or fully evaluate condition reports for the AFW pump and the PORV air accumulator check valve problems contributed to the lack of assessment in the SHIP and maintenance rule programs (Sections 02.01.1.b and 02.01.2.b).
The team concluded that the rework program did not adequately monitor rework issues because the design of the program allowed some equipment performance trends to be unidentified. For example, the licensee identified that the rework identification process was limited to a one year scope. The PORV air accumulator check valve failures occurred during an 18-month routine surveillance; therefore, would not be classified as rework. The licensee planned to re-assess the definition of rework. Additional examples of programmatic design impediments identified by the team are discussed in Section 02.04.
c.
Determination of the plant specific risk consequences (as applicable) and compliance concerns associated with the issue The team performed a qualitative risk assessment of the licensees overall findings and identified no significant issues. The licensees risk evaluation for the 1B AFW pump and PORV check valve failures are discussed in Supplemental Inspection Report 50-456/02-04 and Section 02.01.2.c of this report, respectively. The licensee also identified several longstanding issues with other mitigating systems during the evaluation of the degraded cornerstone. These issues were entered into their corrective action program for resolution.
The team performed an independent review of the Unit 1 and Unit 2 125v DC battery systems to evaluate the accuracy of the licensees conclusions for the degraded cornerstone root cause. The team reviewed selected condition reports, work packages, modifications, and other relevant documentation generated since 1999 and determined that equipment issues were being appropriately handled. No new issues were identified.
02.02 Root Cause and Extent of Condition Evaluation
.1 Programmatic Concerns with the Unit 1B Diesel Driven Auxiliary Feedwater Pump
a.
Evaluation of methods used to identify root causes and contributing causes The methods used to identify the root and contributing causes were discussed in Supplemental Inspection Report 50-456/02-04. The licensee used several different analysis techniques including Event and Causal Factor charting, Barrier Analysis, Failure Modes and Effects Analysis, Change Analysis, and Tap Root methodology. The team concluded that the licensee used a formal, structured approach to perform the common cause analysis to identify root causes and contributing factors. No new issues were identified.
b.
Level of detail of the root cause evaluation Supplemental Inspection Report 50-456/02-04 documented that the level of detail of the root cause evaluation for exceeding the performance indicator threshold was adequate.
However, the level of detail of previous ACE determinations was poor. The lack of ACE quality involving the PORVs was later identified by the licensee as a causal factor for the degraded cornerstone.
c.
Consideration of prior occurrences of the problem and knowledge of prior operating experience Supplemental Inspection Report 50-456/02-04 documented that the licensees review of previous operating history of both Braidwood and Byron operating units was adequate.
The team did not identify any new issues during their review.
d.
Consideration of potential common cause(s) and extent of condition of the problem Supplemental Inspection Report 50-456/02-04 documented that the licensee appropriately identified the potential for a common cause failure mode based on the inappropriate application of the diesel fuel shutoff solenoid valve. The team did not identify any new issues during their review.
.2 Unit 1 Pressurizer Power Operated Relief Valve Air Accumulator Check Valve Failures
a.
Evaluation of methods used to identify root causes and contributing causes The team reviewed root cause evaluation CR 95245 for the PORV air accumulator check valve failures and concluded that the Failure Modes and Effects Analysis technique was appropriately used by the licensee and adequately identified the root causes. The licensee attributed the check valve failures to improper valve assembly following planned maintenance. Specifically, the valve was reassembled with an incorrect disc to valve seat clearance. This prevented the valve disc from fully engaging with the seat and caused the valve o-ring to become dislodged following post-maintenance testing. The improper maintenance was caused by not using existing vendor guidance for performing work on these check valves. The licensee could not identify why the incorrect guidance was used for the Unit 1 valves. The licensee performed a review of past work packages for Unit 2 and confirmed that the tolerances had been checked, which indicated the condition did not exist on Unit 2.
The team noted that the licensee originally intended to investigate the equipment problems and possible programmatic and institutional issues. However, during their evaluation, the licensee determined that the programmatic review was not conducted.
As stated above, the licensee concluded that this lack of quality in the ACE was a causal factor for the degraded cornerstone.
b.
Level of detail of the root cause evaluation The level of detail was adequate for the technical issues addressed and appeared to resolve the functional failure of the check valves.
c.
Consideration of prior occurrences of the problem and knowledge of prior operating experience The licensees review of prior occurrences and operating experience was adequate. All identified issues and failures of the PORV air accumulator check valves from 1992 to date were reviewed by the licensee. A review of industry experiences was also included.
d.
Consideration of potential common cause(s) and extent of condition of the problem The root cause assessment of common causes and extent of condition was adequate.
The licensees extent of condition review considered all Anderson Greenwood valves and concluded that the maintenance procedures for these valves had been previously updated. The improper maintenance issues only applied to the Unit 1 PORV air accumulator check valves. The licensee also contacted station personnel at the Byron Station, Units 1 and 2 to verify accuracy of their maintenance procedures. No concerns were identified.
.3 Inability to Identify and Correct Longstanding Equipment Problems Leading to Degraded
Cornerstone a.
Evaluation of methods used to identify root causes and contributing causes Overall, the licensee used appropriate methods to identify the root causes and causal factors. The team reviewed root cause report CR 113947 and interviewed members of the root cause analysis team. The licensee started the root cause analysis with the Event and Causal Factor charting method to describe the time lines for the two issues involved: the 1B AFW failed starts and the PORV air accumulator check valve failures.
Seventeen causal factors were identified. A barrier analysis was then performed to identify equipment or program failures common to the two issues. The licensee also used the Tap Root methodology to further identify causes due to human error, programmatic, or organizational failure modes.
The licensee concluded that the inability of the station personnel to identify and correct long term equipment problems was the root cause for the degraded cornerstone. The licensee identified numerous causal factors related to poor implementation of the corrective action program and other programs by various organizations. Additionally, the licensee concluded that prior correction of these causal factors could have prevented or significantly mitigated the degraded cornerstone. For example:
- A 1995 issue on PORV check valve failures was closed in 1998 without adequate review for additional corrective actions (Causal Factor 10).
- An engineering request to replace the check valves was inappropriately canceled in 1997 without resolving the issue (Causal Factor 6).
- The nuclear tracking system records indicate that the PORV check valves may have failed a test in the spring of 1997, but documentation was lacking. No condition report was written (Causal Factor 9).
- The maintenance rule reviews did not include work history as specified in the reliability criteria and failed to identify the functional failure of the PORV check valves (Causal Factor 7).
- Two PORV check valves failed in 2000; however, station personnel did not write condition reports (Causal Factor 9).
- A condition report written for the PORV check valve failures in the fall of 2001 was not adequately evaluated. The apparent cause evaluation failed to identify the implications of the failures. (Causal Factors 11, 12, 15 and 16).
In their evaluation, the licensee classified these issues under Human Performance Difficulty and concluded that these represented widespread human performance deficiencies which resulted in the inability of the station personnel to correct and identify long term equipment problems. The team concluded that these issues represented a human performance cross cutting issue (Section 4OA4).
b.
Level of detail of the root cause evaluation The level of detail in the root cause report, CR 113947, provided sufficient information to support the conclusions reached. Included in the report was a discussion of the licensees methodology and scope, a time line and description of events, an extent of condition assessment, the safety significance evaluation, and data and analysis for internal and external operating experiences. Additional detail was documented in the results of the review efforts conducted by the five teams established for the root cause evaluation.
c.
Consideration of prior occurrences of the problem and knowledge of prior operating experience The root cause evaluation, in building on the contributing issue root causes, adequately captured the equipment issues, prior occurrences, and operating experiences. The issues with the auxiliary feed pump and the PORVs had been determined to be long term and repetitive. All of the occurrences and prior experiences were therefore factored into this root cause analysis through the event and causal factor charting.
d.
Consideration of potential common cause(s) and extent of condition of the problem The licensee performed extensive reviews of all mitigating systems to identify outstanding or existing component problems. This included system walkdowns and an aggregate review of condition reports, work requests, and other management inputs to system status, such as maintenance rule, system health and component health reviews.
The licensee did not identify any operability issues; however, some potential equipment concerns were identified. No immediate actions were required. The team did not identify deficiencies with the licensees evaluation or any additional equipment concerns.
The team noted that although human performance was identified as a root cause, the licensee did not specifically conduct an extent of condition review on human performance. This is further discussed in Section 02.04.
02.03 Corrective Actions
.1 Programmatic Concerns with the Unit 1B Diesel Driven Auxiliary Feedwater Pump
a.
Appropriateness of corrective action(s)
Corrective actions to prevent recurrence generated as a result of the licensee root cause evaluation were reviewed and documented in Supplemental Inspection Report 50-456/02-04. The team assessed the appropriateness of several corrective actions for contributing causes which were completed subsequent to the inspections performed for Inspection Report 50-456/02-04. Corrective actions reviewed included action tracking item (ATI) 84527-22, which reviewed the parts evaluation methodology; ATI 84527-25, which provided training on the event; and ATI 84527-27, which reviewed the procurement process that allowed the purchase of a valve designed for hydraulic fluid versus oil applications. The corrective actions reviewed appeared appropriately closed and adequate to prevent recurrence.
b.
Prioritization of corrective actions The prioritization of the corrective actions for the root cause evaluation was evaluated and found acceptable as documented in Supplemental Inspection Report 50-456/02-04.
The team did not identify any new issues during their review of related action tracking items.
c.
Establishment of schedule for implementing and completing the corrective actions The licensees schedule for implementing and completing the corrective actions was determined to be acceptable as documented in Supplemental Inspection Report 50-456/02-04. The team did not identify any new issues during their review of the related action tracking items.
d.
Establishment of quantitative or qualitative measures of success for determining the effectiveness of the corrective actions to prevent recurrence The measures of success for determining the effectiveness of the corrective actions generated as a result of the licensee root cause evaluation were reviewed and documented in Supplemental Inspection Report 50-456/02-04. The team did not identify any new issues during their review of related action tracking items.
.2 Unit 1 Pressurizer Power Operated Relief Valve Air Accumulator Check Valve Failures
a.
Appropriateness of corrective action(s)
The licensees review for corrective actions and the extent of conditions for the PORV air accumulator check valve failures appeared to be adequate. The team reviewed the corrective actions to prevent recurrence and extent of condition analysis for LER 50-456/02-02-00, Failure of Pressurizer PORV Instrument Air Accumulator Isolation Check Valves Caused by Improper Maintenance Activities, and the associated White finding.
The licensee identified three corrective actions to prevent recurrence (CAPR) and multiple actions to address contributing causes associated with this issue. Corrective actions included reviewing and revising the applicable maintenance and surveillance procedures. The licensee also planned to replace the check valves during the next refueling outage. The team verified that corrective actions and extent of condition review were entered and tracked in the licensees corrective program.
b.
Prioritization of corrective actions Prioritization of the corrective actions generated from the root cause evaluation appeared to be adequate and commensurate with their regulatory and safety significance. The prioritization of corrective actions was completed as required by Procedure LS-AA-125, Corrective Action Program (CAP) Procedure, and Procedure LS-AA-125-1006, CAP Process Expectations Manual.
The team determined that the licensees process did not include probabilistic risk assessments as a quantitative method to assist with prioritization of corrective actions.
Procedures LS-AA-125 and LS-AA-125-1006 utilized a qualitative evaluation of risk and uncertainty. Risk was defined as an assessment of consequences, as identified in a list of examples for determining significance level, and the probability of recurrence if left uncorrected. Uncertainty was defined as an assessment of the lack of understanding of the issue, combined with an assessment of the potential lack of effectiveness considering the proposed corrective actions. A matrix comparing the risk and uncertainty determinations was then used to determine the final guidance for which type of evaluation to perform. The procedure then recommended a completion time for each type of causal evaluation. Actions or work assignments resulting from the causal evaluation were assigned due dates, which were mutually agreed upon between process management and the assignee, with the primary focus being on managements perception of importance and workload. No procedural guidance existed for establishing or prioritizing action due dates. Although no inappropriately prioritized actions were identified, the team concluded that the lack of a probabilistic tool as a prioritization aid could allow some subtle higher risk activities to be prioritized incorrectly.
c.
Establishment of schedule for implementing and completing the corrective actions The due dates established for implementing the corrective actions appeared sufficient to prevent recurrence of a similar event. The schedule for implementing corrective actions existed as assigned due dates within the corrective action program tracking system.
The selection of due dates was made using the process described in Section 02.03.2.b.
Corrective actions specific to the facility appeared reasonable and were scheduled for completion prior to the end of the next refueling outage.
d.
Establishment of quantitative or qualitative measures of success for determining the effectiveness of the corrective actions to prevent recurrence The licensee planned to conduct an effectiveness review of the corrective actions in December 2003. No other formal measures of success had been established. An extended discussion of this observation is included in Section 02.03.3.d.
.3 Inability to Identify and Correct Longstanding Equipment Problems Leading to Degraded
Cornerstone a.
Appropriateness of corrective action(s)
The proposed corrective actions and the interim measures for the degraded Mitigation Systems Cornerstone appeared to be adequate and were appropriate for the root cause identified. Corrective actions were established as a product of the root cause methodology utilized by the licensee and scheduled as described in Section 02.03.2.b.
The team verified that the proposed corrective actions addressed the root causes and each of the causal factors. Corrective actions were entered into the licensees corrective action program in accordance with Procedure LS-AA-125 as CR 00113947.
The licensee identified two corrective actions to prevent recurrence for the degraded cornerstone root cause assessment. The first CAPR (CAPR1) required development of an awareness improvement plan to modify behavior relative to human performance issues identified during the root cause assessment. The second CAPR (CAPR2) was the development of a process to perform a recurring review of safety significant systems that would assess the effectiveness of interfacing processes such as maintenance rule, SHIP, and corrective actions. Both CAPRs were in the developmental phase with a December 31, 2002, due date for release of the approved formal plan/program.
For each CAPR, the team reviewed the licensees proposed formal plans and methodologies and determined that the process appeared adequate for addressing the identified root causes if implemented as proposed at the time of the inspection.
For CAPR1, several interim actions had been accomplished or were in progress at the time of the inspection. These included general awareness meetings with senior station management, information distributed in site communications documents, first line supervisory meetings, recovery bulletins, and team reviews of the issues. Topics contained within these measures included discussions on causal factors, management expectations, and status of station progress in addressing the causal factors. The licensee planned additional actions such as future team reviews, meetings, and measures for success.
For CAPR2, the licensee planned to perform a vertical review of mitigating systems with respect to interfacing programs, such as maintenance rule or SHIP. This methodology which was used during the root cause evaluation appeared adequate as a format for the corrective action. The team noted that the licensee indicated that corporate management had yet to fully endorse CAPR2 and planned a trial period prior to site or fleet implementation of the process.
Because the team could not confirm that the proposed actions associated with the CAPRs will be initiated, the licensees corrective actions associated with the degraded cornerstone will be re-assessed during a subsequent Problem Identification and Resolution inspection.
b.
Prioritization of corrective actions Prioritization of the corrective actions from the root cause evaluation appeared to be adequate and commensurate with their regulatory and safety significance. Interim measures established for CAPR1 were commensurate with the safety significance of the identified root cause and appeared to be adequate until a formal plan for improving human performance could be established. Prioritization of corrective actions was completed using the process described in Section 02.03.2.b.
c.
Establishment of schedule for implementing and completing the corrective actions The licensees established due dates for implementing corrective actions appeared sufficient to prevent recurrence of a similar event. The schedule for implementing corrective actions existed as assigned due dates within the corrective action program tracking system. The selection of due dates was made using the process described in Section 02.03.2.b.
The team noted that many of the corrective actions were investigative in nature, or covered potentially generic issues. For example, ATI 113947-16 required an extent of condition review of engineering requests that may have been inappropriately canceled, ATI 113947-22 required a focused area self-assessment of the check valve program to evaluate the current condition and compliance with corporate procedures, and ATI 113947-40 required an evaluation of engineering program trending/concerns to ensure compliance with procedures. Additional actions may result if discrepancies are identified during these reviews.
The team reviewed the procedures for establishing, tracking, and closing action items.
The procedures appeared adequate; however, the team noted that the procedure did not require action items which were closed to another action item to be cross-referenced which could result in inappropriate closure of items. For example, the licensee initiated engineering request 9601111 to replace PORV air accumulator check valve seats with more appropriately designed seats. This request was placed on hold pending the outcome of corrective action E20-1/2-96-225 which was later canceled. No further action was performed. The licensee concluded that ER 9601111 was inappropriately closed. The licensee identified several additional examples including:
- A condition report initiated on May 15, 2001, documenting the auxiliary feedwater unavailability was closed to an apparent cause investigation which was in progress.
- Condition Report 456-201-95-0180 documenting a problem with maintaining PORV air system pressure was closed to a work order with no action taken.
- A 1992 condition report, CDE 20-1-92-227, to investigate PORV air accumulator check valve failures was closed to a work request.
Without cross-referencing, it was not clear to the team how inappropriate closure of items would be prevented. The team noted that at the end of the inspection, the licensee initiated actions to evaluate this cross-referencing concern.
d.
Establishment of quantitative or qualitative measures of success for determining the effectiveness of the corrective actions to prevent recurrence The team did not assess the licensees measures of success because the CAPRs remained in a developmental phase. The team noted that the licensee planned to perform an effectiveness review in December 2003. Procedure LS-AA-125 stated that effectiveness reviews should normally be performed after implementation of the final CAPR and sufficient time has elapsed to challenge the CAPR(s). The team observed that all effectiveness reviews were scheduled well after CAPRs were scheduled for completion and that no in-progress effectiveness reviews were planned.
The team observed that the licensees use of effectiveness reviews was a lagging indicator of success, only occurring after the potential for failure had passed. Through interviews, the team determined that the licensee had not considered any in-progress or leading indicators of effectiveness. However, several informal tools for measuring improvements in human performance such as SHIP, corrective and elective maintenance work order back log, configuration control event monitoring, productivity assessments, and degraded cornerstone action item completion status were utilized.
Although the licensees corrective actions to prevent recurrence were not formalized at the time of this inspection, the team found the licensees approach for completing these activities to be sound. The team was unable to assess the effectiveness or completeness of these proposed actions; however, noted that the licensee had informal tools for monitoring in-progress performance. The effectiveness of the corrective actions will be evaluated during a subsequent Problem Identification and Resolution Inspection in accordance with the Reactor Oversight Program.
02.04 Independent Assessment of Extent of Condition and Generic Implications The team performed an independent extent of condition assessment and did not identify any significant issues. The team concluded that correction of the root cause, the inability of the station to identify and correct long standing equipment problems and an overall tolerance for longstanding degraded material conditions, should address any generic implications associated with implementation of the various equipment monitoring programs.
The team noted that although human performance was identified as a root cause, the licensee did not specifically conduct an extent of condition review on human performance. The team identified that additional human performance concerns in the area of configuration control and barrier impairments had been recently identified by the licensee but were not incorporated into the licensees review for the degraded cornerstone. Further review into other area such as emergency preparedness, fire protection and inservice testing and inspection may be warranted to ensure appropriate scope to the human performance issue.
The team also identified that the design of several related programs may provide the potential for issues to go unrecognized, become delayed, or closed without the generation of a condition report as identified below:
- The team observed that a condition report was not initiated to document a black, gummy residue found inside the 1B AFW pump lube oil cooler service water inlet valve. Troubleshooting was performed under Work Request 99201097-01, dated September 2001. The licensee indicated that a condition report was not written because the residue affected the valves ability to close but did not affect the valves safety-related function to open. However, the team observed that by not writing a condition report the licensee could not evaluate some fundamental questions such as: Where did the residue originate, was the residue present in other service water systems, did the valve failure to close indicate a need to increase the frequency to clean and inspect the valve, and was the amount of the residue changing with time or season? The licensee entered this observation into their corrective action program as CR 133091.
- Step C14, of station procedure BwHS TRM 3.8.c, 125 Volt ESF [Engineered Safety Feature] Battery Bank and Rack Surveillance, allowed observations of lead-sulfate crystals identified during DC battery inspections to be closed to an action request instead of a condition report. Because operability evaluations are part of the condition reporting process, lead-sulfate crystals may not be properly evaluated by closure to an action request. The licensee initiated CR 133412 to address the DC battery procedural concern.
- Subjectivity in the rework process may not provide for proper evaluation of trends or extent of condition. Maintenance that did not involve disassembly of the affected component was not considered rework by the mechanical maintenance department, whereas the definition was different for the instrument maintenance department. For example, when comparing the equivalent task of sending a technician to re-torque a nut or reset a potentiometer, re-torqueing of a nut was not rework, but resetting a potentiometer was rework. The team concluded that this practice may provide the potential for trends such as inadequate training or procedures to go unidentified. The licensee initiated CR 131318 to evaluate the rework process.
4OA4 Cross-Cutting Findings
The team observed that the licensee attributed the cause for the degraded cornerstone to human performance deficiencies, resulting in the inability of the station to identify and correct long term equipment problems. The team concluded that this represented a human performance cross-cutting issue. This issue resulted in a degraded cornerstone for mitigation systems (Section 02.02.3.a).
MANAGEMENT MEETINGS
Exit Meeting Summary
The team presented the inspection results to Mr. von Suskil and other members of licensee management at the conclusion of the inspection on December 4, 2002. The licensee acknowledged the findings presented. No proprietary information was identified.
KEY POINTS OF CONTACT Licensee J. von Suskil, Site Vice President T. Joyce, Plant Manager K. Ainger, Licensing Manager J. Bailey, Regulatory Assurance - USNRC Coordinator R. Blaine, Radiation Protection Manager S. Butler, Regulatory Assurance Corrective Action Program Administrator G. Dudek, Operations Manager C. Dunn, Site Engineering Director A. Ferko, Regulatory Assurance Manager G. Heisterman, Maintenance Manager R. Himes, Program Engineer Manager K. Jury, Licensing Director, Exelon F. Lentine, Design Engineer Manager D. Meyers, Training Director D. Riedinger, Electrical Design Engineering Support L. Rhoden, On-Line Work Control Manager M. Smith, Plant Engineering Manager U. S. Nuclear Regulatory Commission M. Chawla, Project Manager, Office of Nuclear Reactor Regulation G. Grant, Director, Division of Reactor Safety A. Stone, Chief, Reactor Projects Branch 3 LIST OF ITEMS OPENED, CLOSED, AND DISCUSSED Opened None Closed None Discussed None LIST OF
DOCUMENTS REVIEWED
BwAR 1-21-D6; 125V DC Bus 111 Ground; Revision 7
BwAR 1-22-D6; 125V DC Bus 112 Ground; Revision 7
BwAR 2-21-D6; 125V DC Bus 211 Ground; Revision 7
BwAR 2-22-D6; 125V DC Bus 212 Ground; Revision 7
BwHP 4006-059; DC Bus Ground Location Using the Groundbuster Ground Locating
Equipment; Revision 1
BwHS 4002-136; Battery Impedance Test for Lead Acid Batteries in Stationary
Applications; Revision 2
BwHS TRM 3.8.c.4; 125 Volt ESF Battery Bank and Rack Surveillance; Revision 0
BwOP DC-15; DC Ground Isolation; Revision 1E1
1BwOS DC-1a; AAR *125V DC ESF Bus Ground; Revision 4
2BwOS DC-1a; AAR *124V DC ESF Bus Ground; Revision 4
2BwOS DC-1a; AAR *Action Chart 125V DC ESF Bus Ground; Revision 4
1BwOSR 3.8.6.1-2; Unit One 125V DC ESF Battery Bank and Charger 112 Operability
Weekly Surveillance; Revision 1
1BwOSR 3.8.4.5-2; 112 to 212 125V DC Crosstie; September 24, 2001
1BwVSR 3.8.6.6-111; Unit One 125 Volt ESF Battery Bank 111 Modified Performance
Test; Revision 0
CR A2000-00669; Battery Charger 112 Trouble Annunciator; February 12, 2000
CR A2000-00680; Problems During Troubleshooting of 125V DC Battery Charger
(Bus 112); February 12, 2000
CR A2000-00709; Potential Overtightening of Fasteners During Reterm of 1DC04E;
February 15, 2000
CR A2000-01912; 112 Battery 100V DC Negative Ground During Rain; April 16, 2000
CR A2000-02738; DC Battery 111Max Corrected Specific Gravity Deviation >
Administrative Limit; June 30, 2000
CR A2000-03445; Wrong Ground Detectors Installed; August 29, 2000
CR A2000-04650; Incorrect Assumption Used in 125V DC Battery Sizing Calculation;
December 19, 2000
CR A2001-00551; Battery Charger 223 AC Input Breaker Tripped During Battery
Recharge
CR A2001-01444; AFW System Exceeded One-Half of the NEI/NRC Green Band Goal;
April 7, 2001
CR A2001-01897; Inadvertent Opening of AC Power Breaker CB-1 on Battery
Charger 211 - Unplanned LCOAR Entry; January 25, 2001
CR Pre-Screening 131318; Enhancement CR for Rework Reduction MA-AA-716-003;
November 12, 2002 (NRC Identified)
CR 00072371; DC Bus 212 Fluctuating Ground As High As 125V Pos; August 15, 2001
CR 00072507; DC Bus 112 Reoccurring Ground, Weather Related; August 18, 2001
CR 00075743; Fuses in 250V DC System Have Inadequate DC Voltage Rating;
September 17, 2001
CR 00089360; DC Ground on Bus 212 Requiring 2BWOs DC-1a Entry; January 7, 2002
CR 00091072; Part Concern on Transformer for 0DC08J; January 16, 2002
CR 00091825; DC Bus 212 Has a +62V DC Ground; January 23, 2002
CR 00092044; +70V DC Ground on DC Bus 212; January 24, 2002
CR 00104163; Prior Inoperability of Unit 1 and Unit 2 Pressurizer PORVs; April 16, 2002
CR 00105220; Discrepancy With Fuse Obtained From Stores; April 24, 2002
CR 00113947; Braidwood Station Degraded Mitigating Systems Cornerstone Root
Cause Report; July 26, 2002
CR 00112122; Positive 130 Volt Ground Indicated on DC Bus 111/113; June 14, 2002
CR 00115279; Equipment Reliability FASA Identifies Several Deficiencies; July 11, 2002
CR 00116575; DC Battery 223 Failed Acceptance Criteria 2BwOS DC-Q3; July 20, 2002
CR 00118891; Battery 211 Terminal Corrosion; August 10, 2002
CR 00119097; Loss of FME Integrity for 111, 112, 211, and 212 EST Batteries;
August 13, 2002
CR 00120793; DC Bus 212 Ground (+125V DC) Tied to 2B DG; August 27, 2002
CR 00121683; Trend Code B4: 2ER-DC09E Indication and Alarm Found OOT;
July 10, 2002
CR 00133091; Potential Vulnerability-CRs During Work Package Closeout;
November 25, 2002 (NRC Identified)
CR 00133146; Enhancement-Due Dates and Nuclear Safety Impact Review;
November 25, 2002 (NRC Identified)
CR 133412; Potential Vulnerability in BwHS TRM 3.8.4.c Battery Surveillance;
November 22, 2002 (NRC Identified)
CR 134235; Potential vulnerabilities--Degraded Cornerstone Inspection;
December 4, 2002
Engineering Change Request (ECR) 0000042966; 125V ESF Battery Reduced Cell
Capacity Determination
ECR 0000081103; 125 V ESF Battery Reduced Cell Capacity Determination; Actioned
June 29, 2001
ECR 0000081112; 125V DC Crosstie Cables - Add Another Set of 350MCM Cables;
Canceled December 6, 2000
ECR 0000084162; Replace Ground Detector, Esterline Angus Model A601C Obsolete;
Canceled April 18, 2001
ECR 0000331612; 125V ESF Battery Reduced Cell Capacity Determination; Closed
December 4, 2001
ECR 0000337599; Temporarily Defeat the 125V DC Bus 111 Ground Alarm in the Main
Control Room; July 2, 2002
ER 00-001, PIF A2000-00669; Supporting Operability Documentation 1B 125V DC
Battery Charger 112; February 14, 2000
ER-AA-10; Equipment Reliability Process Description; Revision 1
ER-AA-2001; Material Condition Improvement Process; Revision 2
ER-AA-2002; System Health Indicator Program; Revision 1
ER-AA-2002; System Performance Monitoring and Analysis; Revision 2
IP 95002; Inspection for One Degraded Cornerstone or Any Three White Inputs in a
Strategic Area
IP 71111.12; Maintenance Effectiveness
IP 71111.15; Operability Evaluations
IP 71111.16; Operator Workarounds
IP 71111.17; Permanent Plant Modifications
IP 71111.19; Post Maintenance Testing
IP 71111.21; Safety System Design and Performance Capability
IP 71111.22; Surveillance Testing
IP 71111.23; Temporary Plant Modifications
IP 71151; Performance Indicator Verification
IP 71152; Identification and Resolution of Problems
LS-AA-125; Corrective Action Program (CAP) Procedure; Revision 4
LS-AA-125-1005; Coding and Trending Manual; Revision 3
LS-AA-125-1006; CAP Process Expectations Manual; Revision 2
MA-AA-716-011; Work Execution & Close Out; Revision 0
MA-AA-716-013; Rework Reduction; Revision 0
MA-AA-716-232; Proactive Maintenance; Revision 2
NOA-BW-01-3Q; Nuclear Oversight Continuous Assessment Report Braidwood
Generating Station; July - September 2002
WC-AA-101-1001; Work Screening and Processing; Revision 1
Work Order (WO) 99238186 01; 1dc06e Isolate Ground Bus 112; November 18, 2002
WO 99269023 01; Ground Detector Indicator Always Shows 5 Volts; June 27, 2001
WO 00370371 01; Detector Locked Up At +125V Ground Detector, DC Bus 111;
July 11, 2002
WO 00430370 01; 125V DC Bus 111 Ground; April 12, 2002
WO 00447528 01; Rescale Ground Detector Setpoint; June 21, 2002
WO 00464003 01; Isolate/Repair Ground on 1MS101A; July 17, 2002
WO 99201097; Valve Will Not Close; June 25, 2001
WO 00508150 01; Contingency W/O For Ground Busting 125V DC Bus 212;
November 19, 2002
WR 980008919 01; 125V ESF Distribution Panel Bus 112 Assembly 1DC06E Bus 112
Troubleshoot Grounds; February 13, 2002
WR 980070649 01; 125V DC ESF Distribution Center Bus Assembly 2DC06E Bus 212
Troubleshoot Grounds; November 12, 2002
WR 990068268 01; Contingency Package for Repairs at Battery Charger 112;
February 13, 2000
WR 990252122 01; 1B Auxiliary Feedwater Pump Battery Cell #18 Cap Keeps Opening;
June 14, 2001
WR 00031205; 212 Battery Ground Detector Detecting a Small Ground; January 8,
2002 (Cancel)
WR 00033335; Investigate +62V DC Ground on DC Bus 212; January 23, 2002
WR 00033539; Investigate +70V DC Ground on Bus 212; January 24, 2002
WR 00051253; Cell #21 Right-Most Positive Terminal Has Crystals; May 25, 2002
WR 00053308; Isolate Ground on DC Bus 111; June 13, 2002
Maintenance Rule Expert Panel Scoping Determination; DC System; as of
November 22, 2002
Maintenance Rule Data Request; DC Power Storage and Distribution System -
Unavailability + Reliability Graphs; November 1, 2002
Maintenance Rule - Performance Criteria; DC System; as of November 22, 2002
Maintenance Rule - Evaluation History; MR System DC; November 11, 2000 to
November 18, 2002
Braidwood Nuclear Power Plant System Monitoring Plan; Battery and DC Distribution
(DC); November 1, 2002
Braidwood Operations Narrative Logs; Unit 1; November 1, 1999 through
November 19, 2002
Braidwood Operations Narrative Logs; Unit 2; November 1, 1999 through
November 19, 2002
Braidwood Station Equipment Reliability Focus Area Self Assessment; May 29 through
July 8, 2002
SHIP Semi-Annual Report; September 2002
LIST OF ABBREVIATIONS USED
Apparent Cause Evaluation
Auxiliary Feedwater System
Action Tracking Item
Corrective Actions Program
Corrective Actions to Prevent Recurrence
CR
Condition Report
Engineering Change Request
Engineered Safety Feature
IP
Inspection Procedure
LER
Licensee Event Report
NEI
Nuclear Energy Institute
NRC
U.S. Nuclear Regulatory Commission
Problem Identification & Resolution
Power Operated Relief Valves
SHIP
System Health Indicator Program
Work Order
Work Request