IR 05000339/2007009
| ML072410359 | |
| Person / Time | |
|---|---|
| Site: | North Anna  |
| Issue date: | 08/27/2007 |
| From: | Casto C Division Reactor Projects II |
| To: | Christian D Virginia Electric & Power Co (VEPCO) |
| References | |
| IR-07-009 | |
| Download: ML072410359 (26) | |
Text
ust 27, 2007
SUBJECT:
NORTH ANNA POWER STATION - NRC SPECIAL INSPECTION REPORT 05000339/2007009
Dear Mr. Christian:
On July 20, 2007, the United States Nuclear Regulatory Commission (NRC) completed a special inspection at your North Anna Power Station, Unit 2. The inspection reviewed the facts surrounding the Unit 2 spurious actuation of the 'B' train of safety injection (SI) and reactor trip on June 29, 2007. In accordance with Management Directive 8.3, NRC Incident Investigation Program, a special inspection was warranted because the event involved significant unexpected system interactions and the estimated conditional core damage probability exceeded 1E-6. The enclosed special inspection report documents the inspection findings which were discussed on July 26, 2007, with Mr. Dan Stoddard and other members of your staff.
The inspection was performed in accordance with Inspection Procedure 93812, Special Inspection, and focused on the nine areas discussed in the enclosed charter. The inspection examined activities conducted under your licenses as they relate to safety and compliance with the Commissions rules and regulations and with the conditions of your licenses. The team reviewed selected procedures and records, conducted field walkdowns, observed activities, and interviewed personnel.
On the basis of the information reviewed during the special inspection, the team determined that, in general, the licensee responded appropriately to the event on June 29, 2007. The team determined that equipment anomalies were adequately identified and evaluated, and effective immediate corrective actions were implemented. The team noted that the operating crews response was good overall and they were able to effectively stabilize the plant and bring it to a hot shutdown condition. However, several of the actions taken by operators to place the unit in a shutdown condition were knowledge-based and skill-of-the-craft rather than procedurally driven. Also, because the licensee had chosen to run the reactor protection system logic cards to failure, they had not established a preventative maintenance program that would identify degradations of these cards. The licensee relied upon surveillance testing to identify and correct failures and reactor protection system redundancy to ensure availability and reliability of the reactor protection system. No findings of significance were identified.
VEPCO 2 In accordance with 10 CFR 2.390 of the NRC's "Rules of Practice," a copy of this letter, its enclosure, and your response, if any, will be available electronically for public inspection in the NRC Public Document Room or from the Publically Available Records (PARS) component of NRCs document system (ADAMS). ADAMS is accessible from the NRC Web-site at http://www.nrc.gov/reading-rm/adams.html (the Public Electronic Reading Room).
Sincerely,
/RA/
Charles A. Casto, Director Division of Reactor Projects Docket No.: 50-339 License No.: NPF-7
Enclosure:
NRC Special Inspection Report 05000339/2007009 w/Attachments: 1. Supplemental Information 2. Special Inspection Team Charter
REGION II==
Docket No.: 50-339 License No.: NPF-7 Report No.: 05000339/2007009 Licensee: Virginia Electric and Power Company (VEPCO)
Facility: North Anna Power Station, Unit 2 Location: P.O. Box 490 Mineral, VA 23117 Dates: July 3, 2007 through July 20, 2007 Inspectors: S. Walker, Senior Reactor Inspector J. Reece, Senior Resident Inspector F. Ehrhardt, Senior Operations Engineer B. McKay, Reactor Inspector (Training)
Approved by: Eugene Guthrie, Branch Chief Reactor Projects Branch 5 Division of Reactor Projects Enclosure
SUMMARY OF FINDINGS IR 05000339/2007-009; 07/03/2007 - 07/20/207; North Anna Power Station, Unit 2; Special Inspection Report.
The report documents special inspection activities conducted by a senior reactor inspector, senior resident inspector, and a senior operations engineer to review a spurious B train safety injection and subsequent reactor trip on Unit 2 that occurred June 29, 2007.
Special Inspection The team determined that, in general, the licensee responded appropriately to the event on June 29. The team determined that equipment anomalies were adequately identified and evaluated, and effective immediate corrective actions were implemented. The team determined the licencees preliminary cause evaluation, extent of condition review, and immediate corrective action implementation were overall adequate and timely. The team noted that the operating crews response was good overall and they were able to effectively stabilize the plant and bring it to a hot shutdown condition. However, several examples were identified where actions taken by the operators to stabilize the unit were knowledge-based and skill-of-the-craft rather than procedurally driven. The operators utilized procedural steps before transition was directed or where the entry conditions were not met. The team also noted that the licensee did not have a surveillance or maintenance program that was able to identify degradations of reactor protection system logic cards. The licensee did not have a method of trending and documenting component performance and thus identifying degraded components prior to failure. The licensee relied upon surveillance testing to identify and correct failures and reactor protection system redundancy to ensure availability and reliability of the reactor protection system.
A. NRC-Identified and Self-Revealing Findings No findings of significance were identified.
B. Licensee-Identified Violations None.
Report Details Summary of Plant Event On June 29, 2007, at approximately 17:52, North Anna Unit 2 received a spurious safety injection (SI) signal with only the 'B' train partially actuating. This spurious 'B' train SI tripped the main feedwater pumps, initiated a Containment Phase A isolation, started the 2J emergency diesel generator (EDG) and B auxiliary feedwater pump, and realigned the B operating charging pump (also the high head safety injection pump) to discharge directly into a reactor coolant system cold leg. Subsequent plant response included a main turbine generator trip and a reactor trip. The A train of SI was manually initiated per station emergency operating procedure (EOP) 2-E-0, Reactor Trip or Safety Injection. Subsequently, in accordance with procedures, the A train SI was reset and the A SI flow into the reactor coolant system (RCS) was secured. However, the 'B' train of SI could not be reset, which resulted in overfilling the pressurizer and multiple actuations of a pressurizer power operated relief valve (PORV) to limit RCS system pressure. RCS inventory from the PORV discharged to the pressurizer relief tank, which then overfilled and caused the rupture disc to rupture. The discharged water flowed into the containment sump. By closing and de-energizing a valve between the B high head safety injection pump and the RCS cold leg, injection from the B train of SI into the RCS was secured at 18:25. The normal charging and letdown system was placed into service for RCS pressure and level control. The malfunctioning B SI logic was reset/de-energized by placing the solid state protection system (SSPS) into TEST, locally de-latching some relays, and de-energizing the circuit by pulling fuses. The unit was subsequently placed into Mode 5, Cold Shutdown, using the normal operating procedures on June 30, at approximately 08:39.
NRC Response Upon notification of the event, the resident inspectors reported to the site and Region II partially staffed the Region II incident response center to monitor the event. The NRC reviewed the safety significance and risk of the event and concluded that a special inspection was warranted in accordance with Management Directive 8.3, NRC Incident Investigation Program, due to significant unexpected system interactions and the estimated conditional core damage probability exceeding 1E-6. The special inspection team conducted inspection activities in accordance with the attached Special Inspection Team Charter and Inspection Procedure 93812, Special Inspection. The objectives of the inspection were to: (1) review the facts surrounding the Unit 2 spurious actuation of the 'B' train of SI and reactor trip on June 29, 2007; (2) assess the licensees response and investigation of the event; (3) identify any generic issues associated with the event; and (4) conduct an independent extent of condition review.
The following nine charter items were inspected to meet these objectives.
(a) Develop a complete sequence of events related to the event.
(b) Verify and assess the correct operation of equipment that actuated during the event.
(c) Verify and assess that the Licensees post trip review and investigations were adequate.
(d) Review proper operation and design basis function of the power operated relief valve which cycled during the event on June 29, 2007. Verify and assess the licensees inspection of the material condition of the power operated relief valves.
(e) Identify and evaluate the effectiveness of the immediate actions taken by the licensee in response to this event including use of the EOPs related to starting the A train safety injection and the inability to reset B train safety injection.
(f) Evaluate the adequacy of the procedures used by the operators to respond to the event.
(g) Determine the cause, common cause potential, extent of condition, and corrective actions associated with the Unit 2, SSPS card failure.
(i) Review and assess the licensees testing and evaluation of the maintenance and card replacement activities in the SSPS.
(j) Determine and assess the licensees implementation of the emergency action level recommendations for appropriate emergency classification on Unit 2 during the event on June 29, 2007.
4. OTHER ACTIVITIES 4OA3 Event Followup - Special Inspection (93812)
.1 Sequence of Events (Charter Item a.)
The team developed the following sequence of events for the inadvertent SI event using control room logs and trace recorder printouts, computer event recorder data, engineering logs, and licensee event notification reports. All times are eastern daylight time.
Date /
Approximate Time Condition/Event June 29, 2007 1752 The Unit 2 main feedwater pumps tripped due to a spurious B train SI signal. The main turbine tripped and the reactor tripped from 100 percent rated thermal power. The licensee determined that the spurious SI signal was caused by a failure of Zener diode CR 34 on SSPS Universal Card A-313.
1753 The operating crew observed B train SI equipment running, but did not observe a SI First Out annunciator. The crew entered EOP 2-E-0, Reactor Trip or Safety Injection, determined that SI had actuated, and manually actuated both trains of SI. Following manual actuation of SI, both trains of Engineered Safeguards Features equipment actuated as expected.
1800 The crew attempted to reset the SI signal from the main control room (MCR) and determined that B train of SI would not reset. A train of SI was successfully reset.
1808 Pressurizer level exceeded 93 percent.
1810 Pressurizer power operated relief valve 2-RC-PCV-2455C (PORV 2455C) began to lift in response to increasing RCS pressure caused by B train high-head SI Pump flow.
1815 The rupture disk on the Pressurizer Relief Tank ruptured, resulting in approximately 2800 gallons of reactor coolant into the containment sump.
1823 - 1825 Boron Injection Tank Inlet Isolation Valve 2-SI-MOV-2867B was deenergized and manually closed, isolating B high-head SI pump flow and allowing PORV 2455C to close and remain shut. Over the course of the event, PORV 2455C cycled approximately 51 times, passing both steam and water.
1827 2-CH-MOV-2289B, Normal Charging Line Isolation Valve, was deenergized and manually opened in order to restore normal charging flow.
1828 The crew transitioned from EOP 2-E-0 to EOP 2-ES-1.1, SI Termination.
1840 The crew restored letdown flow using guidance contained in Abnormal Procedure (AP), 2-AP-3, Loss of Vital Instrumentation, Attachment 2, Letdown Restoration.
1851 The crew directed Instrumentation and Control (I&C) technicians to place the SSPS Mode Selector switch in TEST in an attempt to reset the B train SI signal. Placing the Mode Selector switch in TEST replaced the normally present 120 VAC latching voltage with a 15 VDC test voltage.
1854 The crew attempted to reset the B train SI signal from the MCR and personnel present at the Train B Safeguards cabinets heard a number of relays change position.
1854-2058 The crew realigned equipment from the MCR as directed by EOP 2-ES-1.1, including components that had been actuated by the B train SI signal.
1947 The crew was unable to secure the 2J EDG using the Normal Stop button, as directed by EOP 2-ES-1.1, Attachment 1, Stopping Emergency Diesel Generators. The 2J EDG could not be secured due to continuing to receive an emergency start signal from the spurious B SI signal. The crew secured the 2J EDG using the Emergency Stop button.
1947-2124 Operations and I&C technicians determined that relays K608, K610, K611 (SI slave relays); K6, K622 (spare relays); K636; and K647 were latched. Using skill of the craft, the I&C technicians manually unlatched relays K6, K636, and K647.
2045 The facility performed the 4 and 8 hour9.259259e-5 days <br />0.00222 hours <br />1.322751e-5 weeks <br />3.044e-6 months <br /> notifications to the NRC as required by 10 CFR Part 72.
2100 The crew transitioned from EOP 2-ES-1.1 to Normal Operating Procedure, 2-OP-3.2, Unit Shutdown From Mode 3 to Mode 4.
2124 I&C technicians manually unlatched the K608 master SI slave relay which allowed the B train SI actuation logic to be reset. However, some additional slave relays did not reset.
2130 The operating crew began to reduce pressurizer level. The crew utilized 2-AP-3, Loss of Vital Instrumentation, as guidance to re-establish letdown, however they did not meet the entry requirements.
2359 The crew commenced an RCS cooldown per Operating Procedure 2-OP-3.2.
June 30, 2007 0216 The crew closed the SI Accumulator Discharge MOVs (2-SI-MOV-2865A/B/C), per Operating Procedure 2-OP-3.2, from the MCR.
0255 The crew received annunciator 2J-D2, Accum 2A-2B-2C Hi-Lo Level, and discovered that 2-SI-MOV-2865C was open. 2-SI-MOV-2865C reopened, when the control switch was released from the close position at 02:16, because relay K628 unlatched when the Mode Selector Switch was placed in TEST at 18:51 on June 29, 2007. This unlatched relay sent an open signal to the valve. The crew closed 2-SI-MOV-2865C by holding the MCR switch in the close position and deenergizing the valve in the close position.
0418 The crew secured B Motor Driven Auxiliary Feedwater Pump, 2-FW-P-3B, by manually opening breaker 25J3. The crew was unable to stop 2-FW-P-3B from the MCR because relay K611 remained latched.
0630 The crew placed Residual Heat Removal System in service.
0839 Unit 2 entered Mode 5, Cold Shutdown.
1100 Operations personnel removed the fuses from both trains of SSPS per Operating Procedure 2-OP-3.4, Unit Shutdown from Cold Shutdown (Mode 5) at 200EF or Less to Cold Shutdown (Mode 5) at 140EF or Less With Keff of 0.95 or Less, deenergizing all SSPS master and slave relays, including those remaining that required unlatching to return equipment to a normal configuration (K610, K611).
.2 Correct Operation of Equipment (Charter Item b.)
a. Inspection Scope The team conducted an independent review of the event to determine if plant systems and components responded appropriately during the event. The team reviewed applicable plant design basis documents, procedures, narrative logs, and event recorder data. The team also conducted interviews with relevant operations and maintenance personnel.
b. Findings and Observations No findings of significance were identified.
Based on the review, the team independently verified that all equipment responded as expected considering the A-313 universal card failure mechanism and its position in the B SI train logic. The failure mechanism consisted of a shorted diode which caused down stream relays, e.g. the B SI train logic master relay, to experience a voltage sufficient to actuate the relays. All equipment affected by these relay actuations functioned as designed. Portions of the B SI train logic upstream or in parallel to this failed circuit component were unaffected. In addition, the inability to reset the partial B SI resulted from this voltage holding the relays in the actuated position when the MCR reset button was pushed and released. After the master SI relay was manually de-latched and the SSPS Mode Selector Switch was in TEST, other relays did not reset due to the 15VDC introduced by the SSPS TEST circuit.
During attempts to reset the B train SI signal per troubleshooting guidance from the Westinghouse SSPS technical manual and the licensees surveillance procedure, multiple relays did not reset as expected given the failure mechanism. The following safety-related equipment was affected by the relays failure to reset: 2J EDG; control room bottled air; B Auxiliary Feedwater Pump, 2-FW-P-3B; and the Steam Supply Valve for the Turbine Driven Auxiliary Feedwater Pump, 2-MS-TV-211B. This condition forced operators to take manual actions to control some equipment, and these actions were conducted utilizing various procedures that contained documented guidance.
.3 Licensee Post-Trip Review and Investigation of the Event (Charter Item c.)
a. Inspection Scope The team reviewed the licensees interim Post Trip Review Report, Event Report, and interim Root Cause Report (1) to independently assess the licensees investigation of the event; (2) to verify the licensee had appropriately reviewed the plant response to the trip and; (3) to identify any problems or issues with respective corrective actions. The team reviewed personnel interview statements, narrative logs, as well as conducted independent interviews of Operations, Engineering, and Maintenance personnel to verify the licensees response to the event was appropriate. The licensee completed the final post trip report and root cause report and provided them for review.
b. Findings and Observations No findings of significance were identified.
The team concluded that the licensees interim post trip investigation was adequate.
Through review of electrical schematics, event recorder data, and licensee personnel interviews, the team verified the licensee identified and adequately explained any equipment discrepancies and/or equipment anomalies that occurred during the event.
Based on review of the interim reports, the team determined the licensee adequately identified and documented causes specific to the event and immediate corrective actions for identified discrepancies. The licensees investigation determined the cause
of the event was due to the train B A-313 universal card found with a failed diode during the initial troubleshooting. During the initial troubleshooting, the licensee used a KIMKA logic card tester to test the SSPS cards and identified the failed Zener diode in the SI logic circuit. The licensee completed the root cause report and provided it to the NRC.
The team agreed with the licensees determination that the Operations staff responded correctly to the event, with priority taken to secure the SI flow once the SI was diagnosed as invalid.
.4 Condition of Cycled Power Operated Relief Valve (Charter Item d.)
a. Inspection Scope The team reviewed the licensees engineering evaluation and maintenance work orders for the PORV. The team also reviewed pictures of PORV 2455C, the relief valve that cycled, and the specific maintenance work orders to replace the degraded seal ring on this valve.
b. Findings and Observations No findings of significance were identified.
The licensees evaluation determined pressurizer PORV 2455C cycled approximately 51 times during the event in a water solid condition. The licensee subsequently disassembled PORV 2455C and discovered a degraded valve bonnet to body seal ring.
The licensee determined that the seal ring degradation of PORV 2455C was not due to the event and had not impacted the ability of PORV 2455C to perform its function, as evidenced by this event. The team determined the licensee adequately assessed the applicability to the other PORV. The team determined through review of diagnostic data and event recorder data, that the other PORV did not lift during the event because reactor pressure did not exceed its pressure setting.
During the event, the Pressurizer Relief Tank ruptured following the repeated cycling of PORV 2455C in a water solid condition. The licensee calculated that approximately 2800 gallons of coolant entered the containment sump. The licensee performed two calculations: one from the amount of water from the SI system and changes in the pressurizer relief tank and refueling water storage tank levels; the second based on containment sump pump operation. Both calculations resulted in a total of approximately 2800 gallons. The water that flowed to the containment sump was processed as designed. The team also verified no unmonitored radioactive releases occurred as a result of the event.
.5 Licensee Immediate Response to the Event (Charter Item e.)
.5.1 Effectiveness of Immediate Operator Actions (Charter Item e.1.)
a. Inspection Scope The special inspection team was asked to review the effectiveness of the licensees immediate operator actions regarding the spurious SI and the subsequent reactor trip.
The team reviewed the operator logs, event recorder data, and statements collected as a part of the apparent and root cause evaluations. The team also conducted interviews with the operators involved with the event.
b. Findings and Observations No findings of significance were identified.
The team determined that, in general, the licensee responded appropriately to the event on June 29. The team determined that equipment anomalies were adequately identified and evaluated, and effective immediate corrective actions were implemented. The team noted that the operating crews response was good overall and was effective in stabilizing the plant and bringing it to a hot shutdown condition. Several examples were identified where actions taken by the operators during stabilization of the unit were knowledge-based and skill-of-the-craft rather than procedurally driven.
.5.2 Actuation of the A train Safety Injection (Charter Item e.2.)
a. Inspection Scope The team was tasked to review the licensees actuation of the A train SI in response to this event. The team reviewed the operator logs and statements collected as a part of the apparent and root cause evaluations. The team also conducted interviews with the operators involved with the event. The team reviewed the licensees immediate response to the event relative to implementation of Technical Specifications, emergency and abnormal procedures, and timely incorporation of this event into the overall plant risk analysis used in maintenance rule risk evaluations. The team also reviewed the licensees interim reactor trip report and interim root cause evaluation and interviewed plant personnel to verify the licensee had appropriately reviewed the plants response to the trip and identified any problems or issues with respective corrective actions.
b. Findings and Observations No findings of significance were identified.
Through the review of documents and interview, the team determined that the operators immediately recognized that a spurious SI had occurred while performing the SI verification step in the EOP. During the event, Operations personnel, based on their training and understanding of a step in EOP 2-E-0 manually initiated a full SI (including A SI). The team confirmed that this action was in accordance with the licensees procedures and training. Consequently, the operators actions compounded the
problem of excess RCS inventory by injecting additional SI flow from the A train. This was an expected consequence due to the construct and usage of the EOPs.
.5.3 Inability to Reset B train Safety Injection (Charter Item e.3.)
a. Inspection Scope The special inspection team was tasked to review the licensees inability to reset the B train SI signal. The team reviewed the operator logs, event recorder data, and statements collected as a part of the apparent and root cause evaluations. The team also conducted interviews with the operators involved with the event.
b. Findings and Observations No findings of significance were identified.
The team determined that the procedures (i.e., EOPs, APs) used for this event lacked guidance to reset SI locally should normal methods (i.e., pushbutton in the MCR) not work. The actions taken by operators and I&C technicians to manually de-energize a valve to stop SI flow into the RCS, locally placing SSPS into the TEST mode, and manually de-latching relays were mainly effective in mitigating the consequences of the spurious SI. As an immediate corrective action, the licensee established a procedure 1/2-AP-0, Resetting SI Locally, prior to Unit 2, startup to mitigate a similar event.
.6 Adequacy of Procedures (Charter Item f.)
a. Inspection Scope The special inspection team was tasked to review the adequacy of the procedures used by the operators to respond to the event. The team reviewed licensee emergency and abnormal operating procedures, and operating logs from the event. The team also conducted interviews with the operators involved with the event, as well as, licensee managers and technicians.
b. Findings and Observations No findings of significance were identified.
The team determined that the procedures (i.e., EOPs, APs) used for this event lacked guidance to reset SI locally should normal methods (i.e., pushbutton in the MCR) not work. In several instances, the EOPs did not contain sufficient information in the Response Not Obtained column to respond to non-responsive equipment. For example, EOP steps that required manipulation of individual components did not provide guidance for local manipulation in the event these components could not be operated from the MCR. Thus, the operators had to rely on various procedures that contained applicable guidance, as well as, operator knowledge and support from engineering and I&C technicians for necessary actions.
Lack of formal procedural guidance also resulted in the 2-SI-MOV-2865C, C Cold leg Accumulator valve, re-opening when operators placed the SSPS B train in TEST. The Westinghouse SSPS technical manual stated that operators must hold the valve closed from the MCR when the SI train is in TEST to prevent the accumulator isolation valve from opening. The open accumulator valve resulted in injecting additional inventory into the RCS during cooldown.
The inspection team made an additional observation with regard to the licensees actions and implementation of procedures. The team noted the licensees administrative procedure on procedure usage may allow too much flexibility in moving between steps in EOPs and use of other procedures before transition is directed or if the entry conditions are not met, without having previously identified the potential consequences, such as resetting SI before directed; and use of 2-AP-3 to re-establish letdown. This could allow the operator to arrive in a configuration in which system or component interactions are unknown. Typically these interactions are evaluated and administratively controlled per procedure. The team did not find any evidence this was being done. However, this flexibility did not cause any adverse impacts during this event. Based on the teams observation, the licensee entered the feedback into their corrective action program and plans to conduct an independent review and assessment of their process.
The team identified an unresolved item (URI) to determine if the procedural guidance and procedure usage flexibility utilized by the licensee during this event was adequate.
This item will remain unresolved pending NRC review of these items and inspection of the licensees final root cause evaluation and investigation of the event. This URI is identified as 05000339/2007009-01: Lack of Procedural Guidance During Spurious Safety Injection and Procedure Usage Flexibility Acceptability.
.7 Event Cause, Common Cause Potential, Extent of Condition, and Corrective Actions (Charter Item g.)
a. Inspection Scope The inspection team was tasked to determine the cause(s), any common cause potential, extent of condition, and corrective actions associated with the failure of the Unit 2 SSPS Universal Card. The team reviewed drawings, logs, and conducted interviews with I&C technicians, program managers, and operations personnel. The team reviewed past corrective actions for logic cards (including, but not limited to SSPS)
and the corrective action plans generated as a result of this event. The team also examined the failed universal card, the SSPS cabinets, and the SI system master and slave relays for any evidence of other possible failure causes.
b. Findings and Observations No findings of significance were identified.
Event Cause Through troubleshooting activities, the licensee determined that the event was caused by a failed Zener diode on the A-313 Universal card. The Zener diode was discovered
to have shorted, inducing sufficient voltage in the SI logic which sent an invalid SI signal to the main feedwater pumps to trip, causing the main turbine to trip, and, in turn, causing the reactor to trip. The failed diode also initiated a Containment Phase A isolation signal. The shorted diode also prevented the B train from being reset from the MCR, due to the induced voltage which essentially sealed in the manual SI signal for the B train. The team reviewed electrical schematics to verify the safety systems actuated accordingly based on this failure. The team also verified the SSPS TEST Mode applied 15VDC that caused some relays to remain latched was accurate. The licensee completed the final root cause after the on-site inspection was completed and provided this information for review and evaluation.
Common Cause & Extent of Condition Based on the teams review, and supported by the licensees interim root cause report, the cause of the failure was most likely due to either age-related degradation (the card had been an original installation) or a random failure. The licensee completed the final root cause after the on-site inspection was completed and provided this information for review and evaluation. While on-site, the inspection team reviewed the conditions of the failure to determine if tin whiskers were involved. Tin whiskers are an industry phenomenon known to have adversely affected safety system logic circuits. The licensee also investigated if tin whiskers could have contributed to the failure. However, previous inspections of the card found no evidence of tin whiskers, and there was no evidence of charred whiskers on the card after the event. In addition, the licensee estimated that the voltage which was drawn (48VDC) would have dissolved any whiskers before they could have shorted the circuit.
The team reviewed the Westinghouse failure analysis for the SSPS logic and noted that the Zener diodes for the universal cards have a high failure probability to short. Based on an engineering review conducted in 2005, the licensee identified these cards as being critical and having potential to trip the reactor or cause spurious transients. The potential age degradation component of this failure is applicable to older diodes on all cards identified as providing critical functions. Failures of diodes in similar applications at the site could affect both trains of the SI system and both units. The licensee planned corrective actions to implement a life cycle management plan for similar logic cards in order to increase reliability.
To determine the immediate extent of condition, the licensee tested all SSPS cards on Unit 2 (trains A and B). Of the total 90 logic boards tested, 47 required repairs and retest. Nine boards, that were identified to contain out-of-spec components, have a function that may affect plant equipment. Of 6510 components tested, 103 were determined to be degraded, and one failed. Based on the licensees testing, none of the degraded components were identified as affecting the SSPS logic. The Zener diode which caused this event was the only component discovered failed. As an immediate corrective action, the licensee replaced the degraded components and retested the cards.
Corrective Actions In addition to the licensee testing all SSPS cards on Unit 2 (A and B) as an immediate corrective action, the licensee finalized procedures 1/2-AP-0 to locally reset SI. This procedure was reviewed by the team; however, it was not verified through simulation by
the licensee.
As a long term corrective action, the licensee is in the process of implementing a replacement and refurbishment strategy for all printed circuit boards, including SSPS cards, Electro-Hydraulic Control cards for the main turbine, and rod control cards. The licensee also proposed expanding their engineering study to identify specific loops on logic cards which may identify failures that affect safety equipment not previously recognized. Additionally, the licensee plans to test all Unit 1 SSPS cards during the upcoming refueling outage.
.8 Licensees Testing and Evaluation of the Maintenance and Card Replacement Activities (Charter Item i.)
a. Inspection Scope The special inspection team was tasked to review and assess the licensees testing and evaluation of the maintenance and card replacement activities in the SSPS. The team reviewed applicable plant drawings, operations and engineering logs. The team conducted interviews with I&C technicians, operators, and maintenance managers. The team reviewed the plant technical manuals and troubleshooting guidance for SSPS.
The team also reviewed the licensees interim root cause report.
b. Findings and Observations No findings of significance were identified.
Based on review of past testing records, the team concluded that the licensees program was unable to detect card or component degradation. The licensee did not have a preventative maintenance program or procedure, rather, relied upon the surveillance testing (to verify operability) and system redundancy to ensure availability and reliability.
The licensee had a KIMKA automated printed circuit board tester for approximately one year. This tester can detect failed or degraded components on the circuit cards. After the event, the licensee used the tester to identify the failed Zener diode, as well as two other degraded components on the card.
The team observed that the licensee initiated an engineering review in 2005 to identify logic cards susceptible to failure which could cause inadvertent transients and/or spurious safety actuations. The review identified 144 SSPS cards from both units (including the universal card A-313 which caused the June 29 spurious SI), that should be evaluated further through a formalized preventative program in order to improve system reliability. The team did not find any evidence that any action was taken to address the susceptibility after this engineering review.
Based upon card failures in other systems during 2006 and 2007, the licensees corrective action program identified short term corrective actions that were initiated to review and implement a replacement and refurbishment program for the SSPS cards.
The licensees review was completed by March 2007, prior to a Unit 2 refueling outage.
The implementation of the program was to be completed within 90 days (June 2007).
However, there was no evidence that this program was implemented by the scheduled
due date. The inspectors also noted the surveillance procedures for the SSPS cards were not changed. In addition, the team noted that there was no formal maintenance program to identify, trend, or predict failure of degraded components in the SSPS system.
The team identified a URI for the lack of an effective program to detect and correct degradations in the Unit 2 SSPS logic cards. This item will remain unresolved pending review and evaluation of the licensees final root cause report, engineering testing data, and diode failure analysis. This URI is identified as 05000339/2007009-02: Lack of an Effective Program to Detect and Correct Degradations in Unit 2 SSPS Cards.
.9 Emergency Plan Implementation (Charter Item j.)
a. Inspection Scope The special inspection team was tasked to determine and assess the licensees implementation of the emergency action level recommendations for appropriate emergency classification for the event on June 29, 2007. The team reviewed the licensees emergency plan, the applicable logs, and associated licensee procedures.
The team also discussed the event with operators and members of the licensees staff and management.
b. Findings and Observations No findings of significance were identified.
After a review of the documentation and event sequence, the team concluded that the licensee classified the event correctly, i.e., no emergency classification was warranted.
A criterion for a Notification of Unusual Event declaration was a forced downpower or temperature decrease for a greater than 10 gpm identified RCS leak. From the beginning of the event until PORV 2455C lifted and subsequent pressurizer relief tank rupture disc ruptured, the operating crew had no indications of a RCS leak inside containment, e.g. no suspected leak. The operating crew was aware that a spurious B SI had occurred and that the increase in RCS inventory had caused a solid condition, which resulted in RCS inventory being released through PORV 2455C and the ruptured rupture disc. The reactor had already tripped and the temperature decrease was due to a spurious SI and not a suspected RCS leak. The licensee did not have a concern of losing coolant inventory and being unable to make-up lost inventory.
4OA6 Meetings, including Exit On July 26, 2007, the special inspection team leader presented the inspection results to Mr. D. Stoddard and other members of the staff. The licensee acknowledged the findings. The team asked the licensee whether any of the material examined during the inspection should be considered proprietary. No proprietary information was identified.
ATTACHMENT: SUPPLEMENTAL INFORMATION
SUPPLEMENTAL INFORMATION KEY POINTS OF CONTACT Licensee Personnel S. Hughes, Manager, Nuclear Operations P. Kemp, Supervisor, Station Licensing J. Kirkpatrick, Manager, Nuclear Maintenance L. Lane, Plant Manager G. Lear, Manager, Organizational Effectiveness T. Maddy, Manager, Nuclear Protection Services R. Klearman, Corrective Action Program C. McClain, Manager, Nuclear Training F. Mladen, Manager, Nuclear Site Services T. Huber, Director Nuclear Site Engineering J. Rayman, Nuclear Emergency Preparedness M. Sartain, Director, Nuclear Safety and Licensing J. Scott, Supervisor, Nuclear Training (operations)
D. Stoddard, Site Vice President M. Walker, Engineer ITEMS OPENED, CLOSED AND DISCUSSED Opened 05000339/2007009-01 URI Lack of Procedural Guidance During Spurious Safety Injection and Procedure Usage Flexibility Acceptability (Section 4OA3.6)05000339/2007009-02 URI Lack of an Effective Program to Detect and Correct Degradations in Unit 2 SSPS Cards (Section 4OA3.8)
Closed None.
Discussed None.
Attachment 1
A1-2 LIST OF DOCUMENTS REVIEWED Procedures 2-PT-36.1A, Train A Reactor Protection and ESF Logic Actuation Logic Test, Unit 2, Rev 52 2-PT-36.5.2, Reactor Protection and ESF Train B Slave Relay Time-Response Test, Unit 2, Rev. 18 Completed Work Orders 00600883-01, Tin Whisker Inspection of SSPS Cards, completed 26 October 2005 00600884-01, Tin Whisker Inspection of SSPS Cards, completed 26 October 2005 00780785-01, Safeguards Output Card (Train B), approved 29 June 2007 Drawings NA-DW-1046F57, Solid State Protection System: Universal Board Schematic Diagram Units 1
& 2, Sheet 1 of 1, Rev 0 NA-DW-1082H41, Solid State Protection System: Safety Injection Units 1 & 2, Sheet 20 of 29, Rev. 5 NA-DW-1082H41, Solid State Protection System: Feedwater Flow Units 1 & 2, Sheet 21 of 29, Rev. 4 NA-DW-1082H41, Solid State Protection System: Output Relays Units 1 & 2, Sheet 27 of 29, Rev. 11 NA-DW-6057D48, Solid State Protection System: Safeguards Output Board Schematic Diagram Units 1 & 2, Sheet 1 of 1, Rev 0 12050-ESK-6DV, Elementary Diagram 480 V Circuits Motor Operated Valves Sheet 44 02-SI-MOV-2865B, C, & 2867A North Anna Power Station Unit 2, Rev. 18 Training Manuals Nuclear Control Room Operator Development Program, North Anna Power Station, NCRODP-52 Safety Injection System, Distribution Number 33, 9 April 2007 Nuclear Control Room Operator Development Program, North Anna Power Station, NCRODP-77 Reactor Protection System, Distribution Number 24, 9 April 2007 Technical Manuals North Anna Power Station, Solid State Protection System, Westinghouse Electric Corp. Energy Systems Nuclear Services, 59-W893-00085, Rev. 11 Miscellaneous ET-N-07-0063, Testing of Solid State Protection System PC Boards Using the KIMKA Tester, 7/19/07 Attachment 1
A1-3 LIST OF ACRONYMS EF Degrees Fahrenheit AP Abnormal Procedure EDG Emergency Diesel Generator EOP Emergency Operating Procedure I&C Instrumentation and Control MCR Main Control Room MOV Motor Operated Valve NRC Nuclear Regulatory Commission PORV Power Operated Relief Valve PORV 2455C Pressurizer PORV, 2-RC-PCV-2455C RCS Reactor Coolant System SI Safety Injection SIT Special Inspection Team SSPS Solid State Protection System URI Unresolved Item VAC Volts alternating current VDC Volts direct current Attachment 1
July 3, 2007 MEMORANDUM TO: Shakur Walker, Team Leader Special Inspection Team FROM: William D. Travers, Regional Administrator /RA/
SUBJECT: SPECIAL INSPECTION TEAM CHARTER A Special Inspection Team (SIT) has been established for North Anna to inspect and assess the facts surrounding the Unit 2 spurious actuation of the 'B' train of safety injection (SI) and reactor trip. The team composition is as follows:
Team Leader: Shakur Walker, RII Team Members: Frank Ehrhardt, RII James Reece, RII The objectives of the inspection are to: (1) review the facts surrounding the Unit 2 spurious actuation of the 'B' train of safety injection and reactor trip on June 29, 2007; (2) assess the licensees response and investigation of the event; (3) identify any generic issues associated with the event; and (4) conduct an independent extent of condition review.
For the period during which you are leading this inspection and documenting the results, you will report directly to me. The guidance in Inspection Procedure 93812, Special Inspection and Management Directive 8.3, NRC Incident Investigation Procedures, applies to your inspection.
If you have any questions regarding the objectives of the enclosed charter, contact Charles A. Casto at (404) 562-4500.
Docket Nos.: 50-338, 50-339 License Nos.: NPF-4, NPF-7 Enclosure: SIT Charter cc w/encl:
L. Reyes, EDO S. Campbell, EDO W. Kane, DEDO CONTACT: Eugene F. Guthrie, DRP/RII 404-562-4662 Attachment 2
A2-2 SPECIAL INSPECTION TEAM (SIT) CHARTER UNIT 2 SPURIOUS SAFETY INJECTION ACTUATION AND REACTOR TRIP Basis for the Formation of the SIT -On 06/29/07 at approximately 5:52 p.m., Unit 2 received a spurious 'B' train safety injection (SI). This spurious 'B' train SI resulted in a Unit 2 reactor trip.
The single train SI resulted in ECCS flow to the reactor coolant system (RCS). The A train of SI was manually initiated per station emergency operating procedures. Subsequently, the A train SI was secured and reset however, the 'B' train of SI could not be reset. Continued injection from the B train SI resulted in RCS inventory increasing resulting in multiple actuations of the pressurizer power operated relief valves (PORV) to limit RCS system pressure. RCS inventory from the PORVs discharged to the pressurizer relief tank, which overfilled the tank and the rupture disc ruptured. Water from the rupture disc was released to the containment sump. Through local, manual actions the licensee was subsequently successful in securing injection from the B train of safety injection, and reactor coolant system pressure and level control was then maintained by normal charging and letdown. During the 'B'
train safety injection a main turbine and reactor trip occurred; auxiliary feedwater pumps started; a containment phase 'A' isolation occurred, ECCS pump actuation occurred; and emergency diesel generators started.
The cause of the spurious actuation on the 'B' train SI was determined, by the licensee, to be caused by a card failure in the SSPS system. The licensee determined that this card failure caused system voltage to reduce sufficiently to spuriously actuate portions of the ECCS systems.
Objectives of the SIT - The objectives of the inspection are to: (1) review the facts surrounding the Unit 2 spurious actuation of the 'B' train of safety injection and reactor trip on June 29, 2007; (2) assess the licensees response and investigation of the event; (3) identify any generic issues associated with the event; and (4) conduct an independent extent of condition review.
To accomplish these objectives, the following will be performed:
a. Develop a complete sequence of events related to the event.
b. Verify and assess the correct operation of equipment that actuated during the event.
c. Verify and assess the Licensees post trip review and investigations were adequate.
d. Review proper operation and design basis function of the power operated relief valves which cycled during the event on June 29,2007. Verify and assess the licensees inspection of the material condition of the power operated relief valves.
e. Identify and evaluate the effectiveness of the immediate actions taken by the licensee in response to this event including use of the emergency operating procedures related to starting the A train safety injection and the inability to reset B train safety injection.
f. Evaluate the adequacy of the procedures used by the operators to respond to the event.
Attachment 2
A2-3 g. Determine the cause, common cause potential, extent of condition, and corrective actions associated with the Unit 2, SSPS card failure.
h. Determine if there are any generic implications associated with this event. Promptly communicate any potential generic issues to regional management I. Review and assess the licensees testing and evaluation of the maintenance and card replacement activities in the SSPS.
j. Determine and assess the licensees implementation of the emergency action level recommendations for appropriate emergency classification on Unit 2 during the event on June 29, 2007.
k. Document the inspection findings and conclusions in an inspection report within 30 days of the inspection.
Attachment 2