ML20133G372
| ML20133G372 | |
| Person / Time | |
|---|---|
| Site: | Salem  |
| Issue date: | 01/13/1997 |
| From: | Scott(Ois) Morris NRC OFFICE OF INSPECTION & ENFORCEMENT (IE REGION I) |
| To: | Wenzinger E NRC OFFICE OF INSPECTION & ENFORCEMENT (IE REGION I) |
| Shared Package | |
| ML20133F923 | List:
|
| References | |
| FOIA-96-351 NUDOCS 9701160019 | |
| Download: ML20133G372 (36) | |
Text
.._ _ _ _ _ _ _. _
O i
4 MEMORANDUM FOR:
Ed Wenzinger, Chief Projects Branch 2 THROUGH:
John White, Chief Projects Section 2A FROM:
Scott Morris, Reactor Engineer Projects Section 2A i
SUBTECT:
COMMON RO(yr CAUSES OF RECENT SIGNIFICANT EVENTS AT SALEM GENERATING STATION Enclosed as Attachment 1 is a summary of recent significant events at Salem which, when viewed in the aggregate, indicate a continuing problem in the licensee's management i
4 organization.
Upon a review of this document, several recurring themes are present:
.I i
o Lack of aggressive management oversight of plant activities lack of aggressiveness to assure adequate corrective action implementation j
o o
Inadequate root ceuse analysis of events o
Slow identification and evaluation of degraded plant conditions o
Lack of procedural compliance 1
When pressed for explanation or resolution, the response on the part of the licensee is often is the same. For example:
4 e
o They believe that their programs are on improving trends g ha o
They are committed to excellence and plant betterment s:isco 4
o They have improved the quality of their procedures j
I o
They are dedicated to better training of their employees o
They have taken' steps to improve management oversight 3
In light of the continuing events at the facility, the effectiveness of these stated enhancements is in question.
1 9
/
Smu Mo s i
t I
l 1
)
9701160019 970113 PDR FotA 0'NEILL96-351 PDR
i-i
(
Public Service l
f Electric and Gas l
Company 2
l MANAGEMENT ASSESSMENT OF THE APRIL 7,1994 REACTOR TRIP SALEM I
GENERATING STATION 5
5._
l i
3
~
t l
MANAGEMENT ASSESShENT OF THE 1
APRIL 7,1994 REACTOR TRIP f
AGENDA i
l INTRODUCTION S. Miltenberger l
SEOUENCE OF EVENTS J. Hagan, EVENTS ANALYSIS J. Hagan CORRECTIVE ACTIONS J. Hagan INDEPENDENT ASSESSMENT OF EVENT J. Hagan CLOSING REMARIG S. Miltenberger o
e MANAGEMENT ASSESSMENT OF THE 4
APRIL 7,1994 REACTOR TRIP SEQUENCE OF EVENTS Initial conditions, April 7,1994 e Reactor power at 75%
e Rod control system in manual
- 12A circulating water pump out-of-service i
10:16 to 10:43 e Power reduction due to excessive marsh grass affecting circulating water system 4
e Operator manually inserting control rods to support load reduction 10:44 to 10:49 e Plant power at 80 MWE (about 7% reactor power)
I e Operators preparing to take main turbine off-line g
Qi
- Primary system temperature goes too low and operator manually withdraws control rods to restore h 'g temperature e Reactor power increases from 7% to 25%
10:49
- Automatic reactor trip on power range high flux low setpoint (25% power) l e Automatic safety injection on high steam flow with low primary system temperature (Tavg). Only protection Train A actuates 94 MMS-3
i MANAGEMENT ASSESSMENT OF THE APRIL 7,1994 REACTOR TRIP SEQUENCE OF EVENTS 10:49 to 11:05 e Emergency Operating Procedures entered e Unusual event declared - pfj 11:05 to 11:26 e Pressurizer Power Operated Relief Valves (PORV) automatically open on high pressure J
e Pressurizer is filling to a solid condition
/ [g e Operators identify that safety injection reset criteria are met. Safety injection is reset, Emergency Core Cooling j
Pumps are stopped and system realignments are begun.
7 e A steam generator safety valve opens resulting in primary system cooldown and depressurization 11:26 e Operators manually re-initiate safety injection due to rapid primary system pressure decrease e Automatic safety injection occurs just before operator action due to low pressurizer pressure (protection Train B) 11:49 e Pressurizer PORVs cycle to maintain primary system pressure e Pressurizer relief tank rupture disc operates due to flow from the pressurizer PORVs 94mm5-2
)
MANAGEMENT ASSESSMENT OF THE l
APRIL 7,1994 REACTOR TRIP i
SEQUENCE OF EVENTS i
I i
11:49 to 1:16 l
- Operators identify that safety injection reset criteria are met.
l Safety injection is reset, Emergency Core Cooling Pumps
]
are stopped e Operators stabilizing plant conditions with pressurizer solid j
1:16
- A " Precautionary Standby" alert declared to ensure proper j
technical support personnel are available i
4:30 1
e Operators reestablish a steam space in the pressurizer 5:15 s Emergency operating procedures were exited and a normal plant cooldown initiated 8:20
- Alert terminated. Precautionary technical support personnel were released. Plant cooldown continued.
l April 8,1994 01:06 e Plant entered hot shutdown (Mode 4) j 11:24 l
- Plant entered cold shutdown (Mode 5)
I
(
4 eH p
j MANAGEhENT ASSESSMENT OF THE f
{
APRIL 7,1994 REACTOR TRIP p
f }j EVENTS ANALYSIS '
k
/lf J
Initial load reduction to reactor trip l
- Circulating water pumps tripping due to marsh grass
- Dedicated team at circulating water structure j
experiences difficulty due to quantity of grass
- Operators reduce power in preparation for j
taking main turbine off-line
- Shift supervisor directs transfer of electrical i
l busses
- Operator has primary temperature trending l
down but does not communicate this to shift j
supervisor j
- Control rod withdrawal to restore primary l
temperature results in an automatic reactor i,
trip l
- Plant systems function as designed in response to the reactor trip i
Root cause
- Control Operator withdrew control rods at a higher rate than was required
- Inadequate command and control
I i
l MANAGEMENT ASSESSMENT OF THE APRIL 7,1994 REACTOR TRIP i
EVENTS ANALYSIS 4
First Safety Injection 1
i
- Immediately following the reactor trip a safety injection occurred j
- Main turbine stop valve closure generates a pressure pulse in j
the main steam lines
)
- Main steam flow transmitters unexpectedly respond to the j
short duration pressure pulse 1
l 8 Operators enter the emergency operating procedures
- Operators recognize only Train A actuation j
- Single train actuation results in additional component verification and positioning per procedures j
- Pressurizer going solid, PORVs operating as designed to l
control primary system pressure
- Operators verify plant conditions and reset safety l
injection allowing the securing of Emergency Core j
Cooling System equipment
- Primary system temperature is increasing due to residual j
heat Root cause
- Operator error allowed a low primary system temperature. This coincident with a false short duration high steam flow signal generated the safety injection.
- The false high steam flow signal was due to a steam flow transmitter design vulnerability.
unus4
I.
b -
i l
MANAGEMENT ASSESSMENT OF THE l
APRIL 7,1994 REACTOR TRIP EVENTS ANALYSIS i
J Second safety iniection e Primary system temperature increase results in secondary system pressure increase j
j
- Operators do not adequately communicate this with j
j each other tr j
e Steam generator safety valve operates to control j
l secondary system pressure
,y/
l e Atmospheric Relief Valves (MS-10) did not open at
/
/'
j their setpoint jo p
l
- Operator does not take manual control as trained i
e Second safety injection results from low pressurizer pressure e Pressurizer PORVs operate as designed to control primary system pressure e Pressurizer relief tank rupture disc functions as designed Root cause e Personnel performance
- Less than adequate crew communications (primary system temperature increase and effect on secondary system)
/
k[/
- Operator not taking manual control of MS-10 e Design of the MS-10 automatic control system $ / /) )
h 94MM5-7
MANAGEMENT ASSESSMENT OF THE APRIL 7,1994 REACTOR TRIP i
EVENTS ANALYSIS i
i y/
Conclusions l
- Initiating event was a rapid power decrease
. p+ /
4 j
caused by unusually high marsh grass in the fjo
/
e Plant protection systems operated as river
$l l
e Overall the plant performed as designed i
- Based on extensive testing results 4jl l
- No major equipment malfunctions j
c'J/
- Both protection trains would have actuated on a b i
valid input signal j/
- Rod control system operated per design f
e Reactor trip and second safety injection were i
caused by personnel error i
- First safety injection was caused by a steam l
flow transmitter design vulnerability and personnel error controlling primary system i
l temperature j
e Enhancements to low power operator training j
were identified i
i l.
94 MMS-8
l MANAGEMENT ASSESSMENT OF THE APRIL 7,1994 REACTOR TRIP EVENTS ANALYSIS i
e Plant procedures
- Emergency Operating Procedures worked well
- Areas of improvement were identified in procedures used for rapid power reduction i
and low power operation
- At times control room command and control was not up to expected standards - strong command and control was exhibitied after the safety injection
- More timely correction of the MS-10 valve automatic controls would have reduced reliance on operator action
- Pressurizer PORVs operated reliably to perform their design function but did show evidence of higher than expected wear
- Declaration of " Precautionary Alert" was a conservative measure by the plant staff I
i I
j MANAGEMENT ASSESSMFNr OF THE i
APRIL 7,1994 REACTOR TRIP CORRECTIVE ACTIONS j
I
- Equipment l
- Replaced pressurizer power operated relief I
i valve internals
- Installed modification to improve automatic p V i
Q control of MS-10 valves
- Installed modification to dampen steam flow u
Y transmitters sensitivity to pressure pulses
- Modifications to the circulating water travelling (/
4 screens are in-progress 3
1 1
^
1 i
i I
MANAGEMENT ASSESSMENT OF THE APRIL 7,1994 REACTOR TRIP CORRECTIVE ACTIONS i
i l
- Personnel / Training 1
- Individuals whose performance was less than j
expected will be provided additional training i
and evaluation l
- Conducted additional simulator training sessions for all operating crews to reinforce:
A Low power operation A Solid plant operation j
A Command and Control / Communications l
A Resource management - 3'# sliad MCO 6 g '6 ce l
A Operator actions following an automatic i
safety injection
- Reinforced and clarified management expectations to all operating crews #
y k) 9{had I
b i
F 1
i k
l 94MM5-%
i MANAGEMENT ASSESSMENT OF THE APRIL 7,1994 REACTOR TRIP CORRECTIVE ACTIONS
- Procedure
- Enhanced normal operating procedures for rapid power reductions and low power operation
- Revised abnormal operating procedures to include minimum condenser vacuum and circulators in-service for manual turbine trip
- Evaluating emergency operating procedure network to support establishment of a pressurizer steam space
- Management expectations were clarified within operating procedures
- All procedural changes were reinforced via training 94MMt9c
l o
i.
W f
I
{
MANAGEMENT ASSESSNENT OF THE p(#g//
i APRIL 7,1994 REACTOR TRIP l
INDEPENDENT ASSESSMENT OF EVENT
- Performed by Significant Event Response Team (SERT)
- SERT charter
)
- Independently assess event root causes l
- Evaluate plant response vs plant design
- Evaluate personnel and procedural performance
- Evaluate safety significance of event i
e SERT root causes were in agreement with line management determination i
e Overall the plant responded as designed
- SERT concurred with troubleshooting l
methodology and resulting evaluations
- Some individual and crew performance was less than expected g
g3 e Overall procedure use by the operators was ing) g*"j accordance with our training requirements vp
- Enhancements to normal and abnormal i
operating procedures were recommended i
l 94mm5 10 4
i l
MANAGEMENT ASSESShENT OF THE APRIL 7,1994 REACTOR TRIP INDEPENDENT ASSESSMENT OF EVENT l
l
- Safety Significance l
- No Condition II Safety Limits were exceeded
- Primary system temperature below the Technical Specification value was not of l
sufficient duration to cause safety implications
- There were no event induced fuel fail'ures
- Overall plant equipment operated as designed l
- All (3) Physical Plant Barriers (Fuel Cladding, RCS Pressure Boundary, Containment) performed as designed
- Event was bounded by Updated Final Safety Analysis Report (UFSAR)
- No abnormal radiation releases resulted from the Plant Trip i
/
}sg';;t /q j'f}/yll4/,
9 4 {e }i {fs y
?
l..-
1 l~
Docket Nos. 50-272; 50 311 i
License Nos. DPR-70; DPR-75 EA 94-124 I
Mr. Steven E. Miltenberger Vice President and Chief Nuclear Officer l
Public Service Electric and Gas Company P.O. Box 236 l
Hancocks Bridge, NJ 08038
Dear Mr. Miltenberger:
SUBJECT:
NOTICE OF VIOLATION AND PROPOSED IMPOSITION OF CIVIL PENALTY - $500,000 (Inspection Report Nos. 50-272/94-80; 50-311/94-80)
This letter refers to the NRC Augmented Inspection Team (AIT) inspection conducted on April 8-27, as well as a followup inspection conducted on May 1 - June 25,1994, at the Salem Nuclear Generating Stations, Hancocks Bridge, New Jersey. The inspection reports were sent to you on June 24 and July 1,1994, respectively. The AIT inspection was conducted to review the circumstances associated with an event which occurred at ticit 1 on April 7,1994, involving an automatic reactor shutdown and two automatic actuations of the safety injection system as a result of marsh grass intrusions at the circulating water intake structure. During the event, several other complicated incidents occurred, including problems with :;purious steam flow signals; failure of the atmospheric dump valves which caused safety relief valves to open; the pressurizer going solid due to the safety injections; the Pressurizer Operated Relief Valves opening numerous times to relieve system pressure; and the Pressurizer Relief Tank rupture discs rupturing due to being overpressurized.
Based on staff review of the AIT report, as well as the followup inspection, violations of NRC requirements were identified. The apparent violations were provided to you in a letter, dated July 6,1994. On July 28,1994, an open enforcement conference was conducted with and other members of your staff to discuss the violations, their causes, and your corrective actions.
PROPOSED ENFORCEMENT ACTION NOT FOICPUBLIC RFI FAf5 h0_UTAPPRdVAL OF THE DIRECTOR. OE p
OFFICIAL RECORD COPY a: PROP-SAL. alt 4 - ~
t
-m- - -
,--e
--.,r
l" i-Public Service Electric and Gas j
Company i
The event was initiated on April 7,1994 when the marsh grass intrusions at the circulating water intake structure caused clogging of the travelling screens, which led to a trip of the circulatory r
water pumps. Although the staffinitiated a power reduction (at a rate of between 1% and 8%
per minute), the Senior Nuclear Shift Supervisor left the control room shortly thereafter to bypass a vacuum permissive to restart a circulating water pump which had tripped, even though that activity was not authorized by procedure. In so doing, the SNSS abandoned his command
}
and control function during this short period.
Furthermore, while the SNSS was out of the control room, the NSS, who was in charge at the time, also abandoned his command and control function for a short period when he became directly involved in the withdrawal of control rods i
for the purpose of increasing reactor coolant temperature.
l
}
Subsequently, after the reactor and turbine tripped, a safety injection occurred because the sudden closure of the turbine stop valves caused a pressure wave that, coincident with the low j
reactor coolant temperature at the time, caused an automatic actuation of the safety injection l
system.
Afterwards, when the reactor coolant temperature increased (due to the rod
)
withdrawals) to a point where the atmospheric relief valves should have opened, the valves did not function, thereby causing a steam generator code safety valve to open, resulting in a rapid j
decrease in reactor temperature and reactor pressure and another safety injection occurring. This l
safety injection led to the pressurizer going solid, the PORVs opening, and the eventual rupture j
. of the pressurizer relief tanks rupture discs. These incidents all provided complications to the transient that could and should have been avoided.
i The related violations are described in the enclosed Notice of Violation and Proposed Imposition l
of Civil Penalties. The three most significant violations are described in Section I of the Notice i
and involve (1) the failure to identify and correct significant conditions adverse to quality at the j
facility related to spurious steam flow signals, and the atmospheric relief valves, resulting in l
unnecessary safety injections during the transient; (2) the failure by the SNSS and NSS to exercise appropriate command and control of the operations staff and the reactor during j
l transient; and (3) the failure by management to ensure adequate procedures at the facility, nell as adherence to those procedures by the operations staff.
With respect to the first violation, your staff was aware of the previous occurrences of spurious high steam flow signals during prior reactor trips at the facility. However, the cause of the condition was not identified and corrected. Rather, As a result,
. In j
addition, your staff was also aware in that problems existed with the atmospheric dump 1
valves as a result of improper implementation of a design modification in 1977. However, this
]
condition was not corrected. As a result, I
i O
CIA
- COPY, a: PROP-SAL. alt
?
?
j l
j Public Service Electric and Gas
^
Company i
With respect to the second violation, the SNSS left the control room during the loss of t
circulating water transient to override a circulator pump protective interlock (vacuum permissive) so as to restart a pump which had tripped, and in do this activity, relinquished the command function in the midst of a significant plant transient. During his absence, operators caused i
reactor coolant temperature to decrease below the minimum temperature for criticality, resulting in In addition, while the SNSS was absent from the control room, the nuclear shift supervisor (NSS), designated as responsible for the control room command function, assumed the duties of a reactor operator for a short period by performing control rod movements. As a result, for the period of time the NSS was manipulating the controls, no individual was responsible for the control room command and control function.
With respect to the third violation, procedures were not adequate to assist the operations staff with several evolutions during the transient, including the maximum rates for reducing power i
during transient conditions, recovery and control of the safety injection system, and recovery l
from a solid pressurizer condition. In addition, the SNSS willfully violated a procedure for when he i
These violations demonstrate that management provided inadequate direction to the staff at the facility, and the operator errors, non-conservative operational decisions, and deficient command and control that resulted, led to the staff overly concerned with recovery of the secondary side of the plant at a time when reactor coolant system conditions were deteriorating.
i 4
The number and nature of these violations is particularly disturbing since this was the fourth significant event at the Salem facility since November 1991 when an overspeed condition caused i
catastrophic damage to your turbine. NRC AIT inspections were conducted for all four of these events, which represen't an extremely high number of such inspections for so short an interval.
I The recent history raises serious questions regarding the adequacy of management's commitment to appropriate root cause analysis of the problems at the Salem' facility, as well as the i
commitment to improving performance and precluding such events in the future. Given the l
significant programmatic deficiencies exemplified by these three violations, each of the violations is individually classified at Severity Level III in accordance with the " General Statement of Policy and Procedures for NRC Enforcement Actions," 10 CFR Part 2, Appendix C, (Enforcement Policy).
The NRC recognizes that subsequent to the identification of the violations, actions were taken to correct the violations and prevent recurrence. These actions, which were described at the enforcement conference, included, but were not limited to, (1) convening of a significant event response team to review the event and its ramifications; (2) oral reinforcement that all staff are expected to identify and participate in the correction 'of problems at the facility; (3) additional i
[0 RECORD FI_
a: PROP-SAL. alt
Public Service Electric and Gas Company training to individuals whose performance was less than expected, including simulator training sessions for all the operating crews; (4) issuance of an Information Directive to all shift personnel to reinforce and clarify management's expectations regarding command, control, and communications; (5) revision of certain procedures; and Nonetheless, to emphastze the importance of aggressive management of the Salem facilities to ensure that (1) problems, when they exist, are promptly identified and corrected, (2) operations supervision, particularly the Senior Reactor Operators, maintain appropriate command and control of the reactor at all times, particularly during transient conditions, and (3) management expectations are appropriately included in procedures, and staff are expected to adhere to those procedures, I have been authorized, after consultation with the Director, Office of Enforcement, and the Deputy Executive Director for Nuclear Reactor Regulation, Regional Operations, and Research, to issue the enclosed Notice of Violation and Proposed Imposition of Civil Penalties (Notice) in the cumulative amount of $500,000 for the violations set forth section I of the l
Notice.
The base civil penalty amount for each Severity I2 vel III violation is $50,000. Application of the escalation and mitigation factors for each of the three violations is as described in the enclosed escalation / mitigation analysis. After application of those factors, the three penalties have been respectively escalated by 250%, 200% and 250% to $175,000, $150,000, and
$175,000.
You are required to respond to this letter and should follow the instructions specified in the enclosed Notice when preparing your response. In your response, you should document the specific actions taken and any additional actions you plan to prevent recurrence. In your response, you should also describe the methods that you have established to measure the effectiveness of those corrective actions.
After reviewing your response to this Notice, including your proposed corrective actions and the results of future inspections, the NRC will determine whether further NRC enforcement action is necessary to ensure compliance with NRC regulatory requirements.
In accordance with 10 CFR 2.790 of the NRC's " Rules of Practice," a copy of this letter and its enclosure will be placed in the NRC Public Document Room.
C ORD~COPU a: PROP-SAL. alt i
I
Public Service Elatric and Gas Company The responses directed by this letter and the enclosed Notice are not subject to the clearance procedures of the Office of Management and Budget as required by the Paperwork Reduction Act of 1980, Pub. L. No.96-511.
I Sincerely,
)
Thomas T. Martin Regional Administrator
Enclosure:
- 1. Notice of Violation and Proposed Imposition of Civil Penalty
- 2. Escalation / Mitigation Analysis l
a: PROP-SAL. alt
Public Service Electric and Gas Company cc w/ encl:
J. Hagan, Vice President - Nuclear Operations S. LaBruna, Vice President - Nuclear Engineering C. Schaefer, External Operations - Nuclear, Delmarva Power & Light Co.
C. Vondra, General Manager - Salem Operations R. Hovey, General Manager - Hope Creek Operations F. Thomson, Manager - Nuclear Licensing and Regulation R. Swanson, General Manager - QA and Nuclear Safety Review J. Robb, Director, Joint Owner Affairs A. Tapert, Program Administrator R. Fryling, Jr., Esquire M. Wetterhahn, Esquire P. Curham, Manager, Joint Generation Department, Atlantic Electric Company Consumer Advocate, Office of Consumer Advocate W. Conklin, Public Safety Consultant, Lower Alloways Creek Township K. Abraham, PAO (2)
Public Document Room (PDR)
Local Public Document Room (LPDR)
Nuclear Safety Information Center (NSIC)
NRC Resident Inspector State of New Jersey i
\\
a: PROP-SAL.ait
- - ~
- _ - ~ _ -
Public Service Electric and Gas Company
)
DISTRIBUTION:
PDR j
SECY CA I
l JTaylor, EDO JMilhoan, DEDR JLieberman, OE TMartin, RI Stewis, OGC TMurley, NRR JRoe, NRR Enforcement Coordinators RI, RII, RIII, RIV, RV FIngram, GPA/PA BHayes, OI JSurmeier, SP DWilliams., OIG EJordan, AEOD OE:Chron OE:EA DCS JStone, NRR SDembek, NRR VMcCree, OEDO CMiller, NRR MShannon, ILPB RI:OE RI:DRP RI:DRP RI:DRSS DHolody/mjc JWhite EWenzinger RCooper 8/ /94 8/ /94 8/ /94 8/ /94 RI:DRS RI:RC RI:DRA RI:RA JWiggins KSmith WKane TMartin 8/ /94 8/ /94 8/ /94 8/ /94 OE DEDR JLieberman JMilhoan 8/ /94 8/ /94 WNO a: PROP-SAL.ait
ENCLOSURE NOTICE OF VIOLATION AND PROPOSED IMPOSITION OF CIVIL PENALTIES i
Public Service Electric and Gas Company Docket Nos: 50-272; 50-311 Salem Nuclear Generating Station License Nos: DPR-70; DPR-75 Units 1 & 2 EA 94-004 As a result of NRC review of the results of an Augmented Inspection Team inspection conducted on April 8-26, 1994, as well as a followup inspection conducted on May 1-June 25,1994, violations of NRC requirements were identified. In accordance with the." General Statement of Policy and Procedure for NRC Enforcement Actions," 10 CFR Part 2, Appendix C, the Nuclear Regulatory Commission proposes to impose civil penalties pursuant to Section 234 of the Ato.mic Energy Act of 1954, as amended (Act), 42 U.S.C. 2282, and 10 CFR 2.205. The particular violations and associated civil penalties are set forth below:
I.
Violations Aewewd Civil Penalties A.
10 CFR 50, Appendix B, Criterion XVI, Corrective Action, requires, in part, that licensees promptly identify significant conditions adverse to quality, determine their causes, and take corrective action to preclude recurrence.
Contrary to the above, prior to April 7,1994, the licensee did not promptly identify and correct two significant conditions adverse to quality, as evidenced by the following examples:
1.
During reactor / turbine trips on June 10,1989, July 11,1993, and February 10,1994, spurious high steam flow signals occurred; however, the licensee did not identify and correct the cause of this occurrence. As a result, a this condition recurred on April 7,1994, during a transient, led to an unnecessary Safety Injection actuation in response to a reactor trip; and 2.
In March 1977, the licensee modified the control system for the main steam atmospheric relief valves (MS-10s), and during implementation of this modification, deficiencies were introduced, namely, Since that time, the licensee failed to identify and correct this condition, even though opportunities existed to do so, namely, As a result, MS-10s failed to open in response to a reactor coolant system heatup and N OFpFI QP [
a: PROP-SAL. alt
Enclosure 2
repressurization following the initial safety injection actuation, thereby resulting in opening of the main steam safety valves in lieu of the MS-10s, and a second unnecessary safety injection actuation on April 7,1994 l
e This is a Severity Level III Violation (Supplement I)
Civil Penalty - $175,000 i
B. Technical Specification 6.1.2 requires that the Senior Nuclear Shift Supervisor j
(SNSS) or, during his absence from the control room, a designated individual, shall l
be responsible for the control room command function.
l 1
Technical Specification 6.8 requires written procedures be established, implemented j
and maintained covering the activities referenced in Appendix A of Regulatory Guide 1.33, which includes administrative procedures.
i PSE&G Administrative Procedure NC.NA-AP.ZZ-002(Q), Attachment 32, Shift f
l Management Responsibility for Station Operation, requires, in part, that the SNSS
)
{
shall remain free to survey and analyze all operating parameters, noting that intense i
involvement in any particular detail may run the risk of losing control and i
perspective of the overall operation.
l l
Contrary to the above, on April 7,1994, during a grass intrusion event at the
]
i circulating water system, the SNSS did not remain free to survey and analyze all operating parameters, and for a short period lost control and perspective of the l
overall operation, in that:
i 1.
The SNSS left the control room during the loss of circulating water transient to override a circulator pump protective interlock so as to restart a circulating water pump, and in do this activity, relinquished the command function in the midst of a significant plant transient. During his absence, operators caused reactor coolant temperature to decrease below the minimum temperature for criticality, resulting in
- and 4
i-2.
While the SNSS was absent from the control room, the nuclear shift supervisor (NSS), designated as responsible for the control room i
command function, assumed the duties of a reactor operator by performing j
control rod movements for a short period. As a result, for the period of time the NSS was manipulating the controls, no individual was responsible for the control room command function.
j i-l OEFICJA
)
a: PROP-SAL. alt I
4 b
Y Enclosure 3
3 This is a Severity level III Violation (Supplement I)
Civil Penalty - $150,000
)
C.
10 CFR Part 50, Appendix B, Criterion V, states, in part, activities affecting q'uality
]
shall be prescribed by documented instructions, procedures, or drawings, of a type i
appropriate to the circumstances and shall be accomplished in accordance with these mstructions, procedures, or drawings.
i Contrary to the above, on April 7,1994, activities affecting quality were either prescribed by procedures that were not appropriate to the circumstances or the activities were not accomplished in accordance with the procedures, as evidenced by the following i
examples:
i f
i 1.
Prior to the reactor trip, the Senior Nuclear Shift Supervisor, did not use station j
. procedure, NC.NA-AP.ZZ-0013(Q), Control of Temporary Modifications," when j
bypassing a vacuum permissive for station circulating water pump 12A in that he changed a switch postition to allow the pump to restart.
In addition, the procedure for coping with loss of condenser vacuum did not detail management expectations regarding restarting of circulators during loss of vacuum events.
i
]
2.
While experiencing the loss of condenser vacuum, a rapid power reduction was initiated from approximately 75 percent power, using procedure Sl.OP-IO ZZ-4 l
0004(Q), Power Operation.
The procedure was not adequate for the circumstances in that it did not provide management expectations as to the i
maximum rate of power reduction to be used during the transient, and it did not state the management expectation as to when operators should cease the effort to maintain plant operations and instead stabilize plant conditions by either a turbine or reactor trip. As a result, operators exceeded any shutdown rate included in their training, and allowed the reactor temperature to drop below the minimum temperature required for critical operations.
3.
During an initial actuation of the safety injection system, an emergency core cooling system partial train actuation occurred, namely,
, and operators implemented emergency operating procedure,1-EOP-TRIP-1. The procedure was not adequate in that guidance was not sufficient to allow operators to recover and control the safety injection system in a timely manner in that
. As a result, the plant pressurizer filled with water from the safety injection, significantly complicating the event because it challenged the reactor coolant pressure boundary.
f a: PROP-SAL. alt
Enclosure 4
4.
During the recovery from the solid pressurizer condition, emergency operating procedure, EOP-CFST-1, Coolant Inventory Status Tree, was not adequate in that the functional recovery procedure for high pressurizer level, FRCI-1, could not be used because of the requirement to secure all reactor coolant pumps to ensure that the Reactor Vessel level Indication System indicated greater than 100 percent full. As a result, the plant was recovered from the water solid condition without any procedural guidance, l
i.
This is a Severity level III Violation (Supplement I) l Civil Penalty - $175,000 II.
Violations not Aueuad a Civil Penalt_v A.
10 CFR 50.57 requires, in part, that emergency plans and procedures for event classification and notification of offsite authorities be implemented. Salem Emergency Plan and Event Classification Guide, Attachment 8, NRC Data Sheet, 2
requires that specified information regarding the event description be completed, approved, and provided to the designated communicator for transmission (to the NRC) within 60 minutes. The specified information includes systems affected, actuations and their initiating signals, causes, effect of event on plant, and actions taken or planned. It also requires that anything unusual or not understood be noted.
Contrary to the above, on April 7,1994, although the NRC was informed of an event at the facilityinvolving
, specified information was not communicated to the NRC within 60 minutes, as evidence by the following examples of information that was not provided:
1.
The apparent logic mismatch of the protection system and resultant unexpected (or lack of) operation of the (ECCS) flow path valves and the unexpected condition of the main steam and feedwater isolation systems; 2.
The cause of the reactor trip; 3.
The effect of the event on the plant (namely, the resultant filled
- i pressurizer or " solid" RCS condition); and, 4.
The operator plans to recover from the solid RCS condition.
This is a Severity Level IV violation (Supplement I).
i PROP-SAL. alt
Enclosure 5
l B.
10 CFR 50, Appendix B, Criterion VIII, Identification and Control of Materials, Parts, and Components, requires in part, that measures be established for the identification and control of parts and components. These measures must assure that identification of the item is maintained throughout installation and prevents the use of incorrect parts.
Contrary to the above, prior to April 7,1994, measures were not established for the configuration control of certain parts and components, as evidenced by the following examples:
1.
During the 1993 Unit 2 outage, power operated relief valve (PORV) internals made of 17-4PH stainless steel (original design material) were installed in valves 2PRI and 2PR2, rather than internals made of 420 stainless steel as recommended by the vendor, and approved by the licensee as design change replacement material.
2.
The post-trip investigation for the April 7,1994 event, identified that the installed summator module for the high steam flow setpoint did not have j
the proper identification. Specifically, This is a Severity Level IV violation (Supplement 1).
Pursuant to the provisions of 10 CFR 2.201, Public Service Electric and Gas Company (Licensee) is hereby required to submit a written statement or explanation to the Director, Office of Enforcement, U.S. Nuclear Regulatory Commission, within 30 days of the date of this Notice of Violation and Proposed Imposition of Civil Penalties (Notice). This reply should be clearly marked as a " Reply to a Notice of Violation" and should include for each alleged violation:
(1) admission or denial of the alleged violation, (2) the reasons for the violation if adadtted, and if denied, the reasons why, (3) the corrective steps that have been taken and the results achieved, (4) the corrective steps that will be taken to avoid further violations, and (5) the date when full compliance will be achieved. If an adequate reply is not received within the time specified in this Notice, an order or a demand for information may be issued to show cause why the license should not be modified, suspended, or revoked or why such other action as may be proper should not be taken. Consideration may be given to extending the response time for gc od cause shown. Under the authority of Section 182 of the Act,42 U.S.C. 2232, this response shall be submitted under oath or affirmation.
b RD@w a: PROP-SAL. alt
Enclosure 6
Within the same time as provided for the response required above under 10 CFR 2.201, the Licensee may pay the civil penalties by letter addressed to the Director, Office of Enforcement, U.S. Nuclear Regulatory Commission, with a check, draft, money order, or electronic transfer payable to the Treasurer of the United States in the cumulative amount of the civil penalties proposed above, or may protest imposition of the civil penalties in whole or in part, by a written answer addressed to the Director, Office of Enforcement, U.S. Nuclear Regulatory Commission.
Should the Licensee fail to answer within the time specified, an order imposing the civil penalties will be issued. Should the Licensee elect to file an answer in accordance with 10 CFR 2.205 protesting the civil penalties, in whole or in part, such answer should be clearly marked as an " Answer to a Notice of Violation" and may: (1) deny the violations listed in this Notice, in whole or in part, (2) demonstrate extenuating circumstances, (3) show error in this Notice, or (4) show other reasons why the penalties should not be imposed. In addition to protesting the civil penalties in whole or in part, such answer may request remission or mitigation of the penalties.
In requesting mitigation of the proposed penalties, the factors addressed in Section V.B of 10 CFR Part 2, Appendix C (1992), should be addressed. Any written answer in accordance with 10 CFR 2.205 should be set forth separately from the statement or explanation in reply pursuant i
to 10 CFR 2.201, but may incorporate parts of the 10 CFR 2.201 reply by specific reference (e.g., citing page and paragraph numbers) to avoid repetition. The attention of the Licensee is directed to the other provisions of 10 CFR 2.205, regarding the procedure for imposing civil penalties.
Upon failure to pay any civil penalties due which subsequently have been determined in accordance with the applicable provisions of 10 CFR 2.205, this matter may be referred to the Attorney General, and the penalties, unless compromised, remitted, or mitigated, may be collected by civil action pursuant to Section 234c of the Act,42 U.S.C. 2282(c).
The response noted above (Reply to Notice of Violation, letter with payment of civil penalties, and Answer to a Notice of Violation) should be addressed to: Director, Office of Enforcement, U.S. Nuclear Regulatory Commission, ATTN: Document Control Desk, Washington, D.C.
20555 with a copy to the Regional Administrator, U.S. Nuclear Regulatory Commission, Region I, 475 Allendale Road, King of Prussia, Pennsylvania 19406 and a copy to the Senior Resident Inspector, Salem Generating Station.
Dated at King of Prussia, Pennsylvania this day of August 1994 OERtMbh a: PROP-SAL. alt
" ESCALATION / MITIGATION ANALYSIS **
A.
FSC?h6n and Mitiration Factors for Appendix B. Criteria XVI. Violation 1.
Identification: +50%
The two examples of the violation were identified by the NRC during the staff review of the findings of the AIT inspection. Therefore, 50% escalation on this factor is warranted.
2.
Corrective Action: 0%
The licensee's corrective actions, as summarized in'the licensee presentation at the enforcement conference, although acceptable, were not considered sufficiently prompt and comprehensive to warrant mitigation because Therefore, no adjustment on this factor is warranted.
3.
Licensee Performance: +100%
Since November 1991, the NRC has conducted four AIT inspections to review four events at Salem. Although none of the events involved the degradation of safety related equipment, the events all presented operational challenges to that equipment, as noted in the most recent SALP report for the facility issued in October 1993. In addition, a $50,000 civil penalty was issued to the licensee in March 1994 for violations at the facility. Given this past history and the continuing failure of the licensee top take adequate corrective actions to prevent such occurrences,100%
escalation on this factor is warranted.
4.
Multiole Occurrences: +100%
Since there were two examples of this violation, bom of which contributed to complications during this event,100% escalation on this factor is warranted.
5.
Prior Oooortunity to Identifv: 0%
Although the licer.see had prior notice of the specific problems that went uncorrected, this factor was a consideration in the decision to classify the violation at Severity level III. Therefore, no adjustment on this factor is wananted.
6.
Duration: 0%
Although the failure to correct these conditions existed for an extended duration, this factor was also a consideration in the decision to classify the violation at Severity level III. Therefore, no adjustment on this factor is warranted.
A 2
B.
Fcen12 tion and Mitiention Factors for Command and Control Violation n
1.
Identification: +50%
The two examples of the violation were identified by the NRC during the staff review of the findings of the AIT inspection. Therefore,50% escalation on this factor is warranted.
2.
Corrective Action: 0%
The licensee's corrective actions, as summarized in the licensee presentation at the enforcement conference, although acceptable, were not considered sufficiently prompt and comprehensive to warrant mitigation because Therefore, no adjustment on this factor is warranted.
- 3. Licensee Performance: +50%
Given the poor overall licensee performance in the past few years, as noted in the
)
application of this factor for the first violation, escalation on this factor is again warranted. However, since none of the previous violations or incidents involved a command and control violation, the Region I believes that 50% escalation on this factor is warranted.
4.
Multiple Occurrences: +100%
Since there were two examples of this violation, both of which contributed to complications during this event,100% escalation on this factor is warranted.
i 5.
Prior Oooortunity to Identify: 0%
Not applicable 6.
Duration: 0%
Not applicable e
4
,e-e
~
1-4 i
3 i
C.
Escalation and Mitientian Factors for Procedures Violation 4
4 1.
Identification: +50%
The four examples of the violation were identified by the NRC during the staff review of the findings of the AIT inspection. Therefore,50% escalation on this factor is warranted.
2.
Corrective Action: 0%
The licensee's corrective actions, as summarized in the licensee presentation at the enforcement conference, although acceptable, were not considered sufficiently prompt and comprehensive to warrant mitigation because Therefore, no adjustment on this factor is warranted.
- 3. Licensee Performance: + 100%
Since November 1991, the NRC has conducted four AIT inspections to review four events at Salem. Although none of the events involved the degradation of safety related equipment, the events all presented operational challenges to that equipment, as noted in the most recent SALP report for the facility issued in October 1993. In addition, a $50,000 civil penalty was issued to the licensee in March 1994 for procedural violations at the facility. Given this past overall history in general, and the continuing procedural violations in particular,100% escalation on this factor is warranted.
4.
Multiple Occurrences: +100%
l Since there were four examples of this violation, all of which contributed to complications during this event,100% escalation on this factor is warranted.
5.
Prior Oooortunity to Identify: 0%
Not applicable 6.
Duration: 0%
Although the procedural inadequacies existed for an extended duration, this was a consideration in the decision to classify the violation at Severity Level IV, and iherefore, no escalation on this factor is warranted.
,+,er m
w
-. -. ~ -. -
- ~..
11/15/91 13:46 U.S N.R.C. REGION 1 KIN 002
- Not).14 91 14:54 HRC SALEM NJ Pe2 1
Eygntcg,,,, or Salem Uj)1,1,,,2 Turbine /Gengy,_mJy_ c Pr.911minar.y..Spuy.ent(e_of T
Failura - Navambar o. leal Initial conditione:
l Unit 2 returned to 100% power at 0:30 a.m.
following load reduction to 60% for soler magnetic dieturbance.
Operators were at the front standard performing monthly turbine trip testing that was recommended by Wootinghouse.
The operatorm had uompleted tanting und were preparing to exit the area.
The manual test lever was still in the test position. bypassing all trips appociated with the auto atop oil including the mechanical overspeed trip.
Tima lbte.n1 11:21:b4 a.m.
Reactor trip occurred due to turbine trip signal (T=0) low auto stop oil pressure) and remotor power above 50%.
Turbine stop valves indicated closed
(<90% open).
Turbine electro-hydrhuiio control (EHU) oystem emergency trip solenoid valve received a signal to open from the reactor protection system (RPG) due to a remotor trip.
Emergency trip solenoid vaive apparently failed to {
open to ensure a turbine trip.
Turbine stop valves, governor valves. reheat stop valves and intercept valves apparently closed se expected un n turbine trip due to loss of EHC fluid.
The reactor shutdown normally.
Operators entered emergency operating procedures.
i 11: 21:00 a.m.
Turbine trip signai (low auto stop oil pressure)
(T 1.0 seo.)
cleared.
EHC fluid repreneurized and turbine stop valvee. reheat stop and interoept valves were allowed to reopen.
Governor valves remained j
closed as designed per EHC circuitry.
11:22:31 a.m.
Generator output brechers opened em designed due (Tc27.5 sec.)
to turbine trip indication (e.g.,
stop valvea <90%
and timo delay).
Turbine was disconnected from the grid and the unit began to overspeed.
Turbine overaPeed protection controller (OPO) went a signal to redundant OPC solenoid valves to i' ant l
olose sovernor and intercopt valves to prevent i
OPC solenoide apparently failed to open to limit turbine speed to <3034.
Turbino governor valves apparently began to reopen due to EHC system shifting from a load controi to npeed control mode.
a
/
l 11/15/91 13:47 U.S N.R.C. REGION 1 KIN 003
' NOV 14 '91 14:54 NRC SALEM NJ P03 l
11:22:33 a.m.
Turbine stop valves indicated open (>90%).
(Ts.30 ene.)
Control room operator noted turbine console speed indication pegged high at 2500 RPH.
Senior shift supervisor noted OPC light on control panel was lit, indicating that un OPC logic initiation signal occurred.
11:22:47 a.m.
Turbine first stage pressure decreased to (T=O3 sec.1 approximstely 10% indicated turbine power.
This appeared to be an abnormaily tant depressurization (normally takes about 10 minutes on a renctor/ turbine trip) and may be indicative or a ateam firne path throush the turbine, Time Unknown Operatorm locally on turbine deok observed generator fire and turbine missiles. and sensed vibration and incrossing pitch noise.
Operator at turbine front standard stoped the testina.
manually tripped the turbine. and evocuated the OrCQ.
L1:23:06 som. Turbine stop valven indicated aloned.
Thin was (T=74 sec.)
indicative of a turbine trip mignal initiation, j
11:26 a.m.
Site fire brigade reported to scene.
Installed deluge, sprinkler and cardox aystema actuated.
Fire brigade assisted in fighting fire.
L1: 40 a.m.
Unusual Event declared due to fire in the protcoted area lasting more than to minutes.
12:10 p.m.
NRC notified via MNH phone 12:44 p.m.
Alert declarnd and then de-escalated due to turbine onning penetration.
1:30 p.m.
Two NRG residente arrived onsite.
1 2:41 p.m.
Unusual Event de-escalated when all. firon were fully extinguiched.
3:20 a.m.
Cold shutdown monteved November 11. 1991.
Ws
~
~
4
},
609 935 3741 NOO 21 '91 17:57 NRC SALEM NJ P01 jo.rhg'
. *NO Ct ) h K
i O PSKiiE=-
- ]'
j 80 Para Pass, Newark, NJ 071011201430 7000 MA1UNG ADDRESSs RO. Box 570. Newark NJ 0710t News Release L l
- r. w um s -4
/, q r -
s l
FOR IMMEDIATE RELEASE MEDIA CONTACT:
November 21, 1991 Neil Brown NR*
(201) 430-6017
- /),j '
4 3
INVESTOR CONTACT:
I arian smith AllG14 l
(201) 430-6564
- *NI C C WAff J
PSE&G SAlfE IT MISSED OPPORTUNITY MlrWDCc6 TO PREVENT SALEM 2 DAMAGs 74, g q
j Public Service Electric and Gas Company (PSEEG) said today TLi3 (Nov. 21) that it missed an opportunity to prevent a turbine j'
overspeed condition which caused significant equipment damage at its Salen 2 nuclear unit because of the misinterpretation of j
earlier test results.
}
The company also said today that Salen 2 would probably return to service about mid-summer, 1992, and that equipment t
repair costs could be in the range of 565 to $75 million, i
virtually all of which would be covered by insurance.
j-Robert J. Dougherty Jr., PSE6G senior vice president-electric, said the company discovered late yesterday it conducted tests in october which had indicated that certain turbine protection devices were inoperable. "Unfortunately, these results were interpreted an a test procedure inadequacy rather than an equipment problem," Dougherty said.
"While we recognise that information can be j
misinterpreted and equipment can fail in the operation of power plants, and while we purchase insurance to cover these i
situations," Dougherty said, "We are extremely disappointed that we did not use the information available to us to prevent this failure from occurring.
}
aIn addition to our efforts to return the unit to service as promptly as possible, we will address measures needed j
to strengthen our procedures and management processes to prevent such evente in the future."
a Dougherty noted that the company's ongoing l'
investigation revealed that identical components on the Salem 1 turbine were replaced in September 1990 because they failed to pass a routine check. Based on the salen 1 experience, the same components at the Salen 2 unit were to be replaced at its next j
refueling outage, which was scheduled to begin in January, 1992.
i (more)
/
', NOU 21 '91 17:58 HRC SALEM NJ P02 g
Dougherty seid "an additional review of industry experience has also, identified similar failures at other facilities." He said those events are being reviewed to determine if there are any common issues.
In assessing the equipment damage at Salen 2, Dougherty i
said, it appears that rebuilding or replacing the generator will take the longest time.
He said there are several options available which are being reviewed from financial and technical viewpoints. He added that the company is investigating the option of purchasing a new i
General Electric generator within the industry that is compatible with the sales design. He said the company is also considering the use of a spara Westinghouse generator purchased for use at Salen 1. He said the cost of the generator repair could range from $25 to $35 million.
Dougherty said the cost of repairing the turbine and condeneer could approach $40 million.
"Given what we know today, we believe that the systems can be rebuilt and the unit returned to service by mid-summer, 1992."
Dougherty said the detailed investigation to determine the root cause of the incident is continuing.
"Today (Nov. 21), va completed and are in the process of analysing the results of an engineered test to recreate the hydraciic and electric control status at the time of the Nov. 9 incident. While the detailed analysis is incomplete, we do know, as previously reported, that the regularly scheduled monthly turbine test was being conducted at the time of the incident.
During the Nov. 9 test, several turbine trip mechanisms were taken out of service by design so that they could be tested manually." (A atrip mechanism" is a system designed to automatically shut down the facility or its components.)
The' engineered test completed early today (Nov. 21) was designed to test additional trip mechanisms that provide a redundant layer of protection for the unit, Dougherty explained.
Preliminary test results indicate those devices did not operate as designed. A detailed PSEEG investigation, including a review j
by onsite NRC officials, is currently under way to determine the reason why.
During the Nov. 9 incident at Salam 2, the nuclear side of the unit was not affected. No radiation was released and the unit's reactor shut down safely and automatically.
(more)
--~.
.. -.. ~..
-. ~. -
.. ~. _. -. _. -. - -.
03/W95 10G2 U. 5 t l. P. C. PEGIC#3 1 Fill 00;
)
U.S. NUCLEAR REGULATORY COMMISSION i
c.
REGION I Report No. 50-311/91 41 License No. DPR-75 f
Licensee:
Pubhc Service Electric and Oas (PSE&O) Company P.O Box 236 J.
Hancocks Bridge, New Jersey 08038 I
Inspection of:
Salem Nuclear Generating Station, Unit-2 Hancocks Dridge, New Jersey Conducted:
November 10 through December 3,1991 Inspectors:
Thomas P. Johnson, Senior Resident Inspector, Salem / Hope Creek, Division of Reactor Projects (DRP), Region 1 (RI) 1 Stephen T. Barr, Resident Inspector, Salem, DRP, RI David M. Silk, Senior Operations Engineer, Division of RE: actor Safety (DRS), R1 Roy K. Mathew, Senior Engineering Specialist, DRS, RI n
John C. Tsao, Senior Engineer, Reactor and Plant Systems Branch.
Office of Nuclear Regulatory Research Steven R. Jones, Reactor Systems Engineer, Office of Nucicar Regulation Approvc<h 7
John Vlhite,'Chibf, Reactor Pro'jects Section 2A Date (Tea trader)
REPORT SCOPE The Augmented Inspection Team (AIT) reviewed the circumstances and determination of the causes of the November 9,1991 event involving the destruction of the Low Pressure Turbine and Main Ocnerator. Areas examined included Sequence of Events, Operator and Nuclear Plant System Performance, Management Performance, and Licensee Event Assessment Efforts.
D 1
[
'l ON-
/,/ ! l O hb y.
_ _ ~ _ _ _._~_.-
._~m_
sll 03/20/95 10:21 U.5 H.P.C. PEG!OH I ilt1 00:
I 4
i i
l TABLE OF CONTENTS p
1
,c i
R E P O R T S C O P E............................................
1 I
........ n TABLE OF CONTENTS 4
i i
1.0 I NTR ODU CTI ON....................................... I 1
j 1.1 Executive Summary..................................
1 i
1.1.1 Event Description Summary........................
l 1.1.2 AIT Assessment Summary......................... 3 4
l.2 Augmented Inspection Team Scope and Objective 2.0 TURBINE GENERATOR FAILURE EVENT...................... 5 1
2.1 Sequencc of Even ts................................
5 l-2.2 Turbine Control and Trip System Description.................. 5 8
2.3 Ev en t De scription...................................
2.4 P recursor Events...............................,...
10 3
2.4.1 Pertinent Events...............................
10 t
3 2.4.2 Other Evaluated Events..........................
12 3
- 2. 4. 3 A s ses s men t..................................
13 2.5 Generator and Electrical System Assessment..................
14 j
2.6 Turbine Surveillance Test and Maintenance Assessment...........
16 i
2.7 Turbine Generau:d Missile Hazard Assessment 18 18 5
2.7.1 General Description 2.7.2 Regulatory Basis for the Safety Evaluation of Missile Hazards 18 19 2.7.3 Turbine Missile Hazard Analysis j
2.7.4 Turbine Inspection Activitics......................,
20 20 l
2.7.5 Damage Assessment 2.7.6 A ssessment...............................,.. 21 l
k j
3.0 OPERATOR AND NUCLEAR PLANT SYSTEM PERFORMANCE.......
21 21 3.1 Operator Performance Assessment.....................
22 3.2 Nuclear Plant System Performance Assessment 23 3.3 Emergency Operating Procedure Assessment..................
3.4 Emergency Classification, Notification, and Event Reporting 24 j
Assessment 25 3.5 Fire Protection System Assessment........................
4.0 M AN AGEMENT PERFORM ANCE.........................
27 27 4.1 Management Response and Action.................
'1 J
11
---c--
..n,
l 0920/95 10:25 U.S N.P.C. PEGION 1 FIN 00J
}
.l 1
i
/
Table of Contents
/^
5.0 LICENSEE EVENT ASSESSMENT EFFORTS....................
28 5.1 Signi0 cant Event Response Team (SERT) 28 5.2 Summary of SERT Findings............................ 30 5.3 Assessment of SERT Efforts 31 6.0 PLANS AND SCHEDULES FOR REPAIR AND RESTORATION........
32 7.0 FINDINGS AND CONCLUSIONS............................
32-7.1 Personnel Performance............................... 32 7.2 Equipment Performance 33 7.3 Procedure Adequacy and Adherence....................
35 8.0 EX I T M EETI N G....................................... 36 APPENDICES APPENDIX A PRECURSOR EVENTS
SUMMARY
APPENDIX B SEQUENCE OF EVENTS
SUMMARY
APPENDIX C NRC AUGMENTED INSPECTION EXIT MEETING ATTENDEES APPENDIX D NRC AUGMENTED INSPECTION TEAM CIIARTER e-APPENDIX E SELECTED PHOTOGRAPHS APPENDIX F LICENSEE EVENT REPORT NO. 50-311/91-017 APPENDlX G LIST OF ACRONYMS FIGURES FIGURE I ELECTRO-HYDRAULIC / AUTO-STOP TURBINE CONTROL SYSTEM FIGURE 2 TURBINE STEAM FLOW DIAGRAM FIGURE 3 SALEM UNIT-2 TURBINE BUILDING OVERVIEW FIGURE 4 DEPICTION-TURDINE FRONT STANDARD FIGURE 5 500 kV SWITCHYARD DIAGRAM FIGURE 6 LOW PRESSURE TURBINE, CROSS-SECTION FIGURE 7 TURBINE FRAGMENT LOCATIONS, MAP 1 AND 2 O
111
, ~. _ -. -. - ~. - _.
... _...... -.-... _ - ~.
I
~
03/20'95 10:26 U. 5 N. P. C. PEG 1ON 1 V111 005 ACKNOWLEDGMENT The members of the Augmented Inspection Team acknowledge and appreciate the effort and assistance of the following individuals who contributed to the development and editing of this
- report, Barry C. Westreich, Reactor Engineer, DRP, R1 l
Isabel B. Moghissi, Reactor Engineer Intern, NRR Robert G. Schaaf, Reactor Engineer Intern, DRP, RI Kay L. Gallagher, Branch Secretary, DRP, RI l
n IV
032045 lo:26 U.5 N.P.f.,
FEGloh 1i'IU OoE i
DETAILS l.0 INTRODUCTION 1.1 Executive Summary 1.1.1 Event Description Summary On November 9,1991, the Salem Nuclear Generating Station, Unit 2 (Unit 2) was operating g
at 100% reactor power. At about 11:00 a.m., plant operators initiated a routine test procedure to verify the operability of the steam turbine automatic mechanical trip mechanisms. The test procedure involved the manipulation of mechanical tr g'
turbine Auto Stop Oli (AST) system, the primary turbine protection mechanism. By de the test procedure required the complete isolation of the AST system from any turbine g
control or trip function (including the mechanically-actuated turbine overspeed trip device) in "
order to prevent an actual turbine trip during testing of the mechanical devices.
W[d, 9 T-A redundant back-up system for turbine overspeed protection and emeigency trip functions AST' was assumed to be operational. The back-up system consists of three electrically actuated solenoid valves designed to provide redundant automatic control and trip of the turbine in t.n
~
OPC-20-1 and OPC-20-2) and to cause a turbine trip on a reactor trip (by reliance on the ci backup emerger.cy trip solenoid valve ET 20). The system diagram is shown in Figme 1.
+/L During the performance of the test, a momentary oil pressure perturbation (a pronounced decrease lasting about 1.5 seconds) occurred in the AST system. Though of shott duiation.
the momentary oil pressure decrease was sufficient to open the AST Interface Valve. The AST Interface Valve functioned to relieve the Emergency Trip Fluid (ETF) pressure from the pilot valves affecting operation of turbine steam admission valves, i.e., Stop Valves, Governor Valves. Reheat Stop Valves, and intercept Valves. Consequently, those valves closed and isolated steam flow to the high and low pressure turbines.
The oil pressure perturbation also resulted in the activation of three low AST pressure signals to the Reactor Protection System (RPS), in accordance with the design of the RPS logic, two out of three low AST pressure signals are considered as indicative of a turbine trip.
Consequently, the Reactor Trip Breakers opened to cause an immediate reactor plant trip.
Due to the test in progress, the primary turbine trip system (Auto Stop Oil) was isolated and incapable of providing turbine trip assurance. Necessarily, reliance was placed on the bact-up emergency turbme trip system involving solenoid valve ET-20.
By design, opening of the Reactor Trip Breakers caused ET-20 to be electrically energized.
The reactor trip also initiated a 30 second delay for opening the output breakers from the Main Generator. Though energized, the ET-20 solenoid valve failed to open,tu assure relief of ETF pressure to maintain the turbine steam admission valves closed.
o P 20/95 10:2~
U.S H.P.C.
PEG!on i 1.1H 007 2
m When the AST oil pressure returned to nGrmah aftcr & m7mentary perturbation, the AST Interface Valve closed (by design). Since the ED20 solenoid valve, though enegized, did not function, ETF pressure was returned to the pilot valves affecting fhe operation of the turbine steam admission valves and initiated re-opening of those valves. Depending on the I
actual configuration of the Stop Valves and Governor Valves, steam may have also been admitted to the turbine through the bypass valve associated with each Stop Valve.
While various possible individual steam admission valvepositions may have existed (including the possible positions of the bypass valves associated with each Stop Valve), the actual configuration apparently was sufficient to admit steam to the turbine at about the same time J
as the output breakers from the Main Generator opened. The disconnection of the main generator from the grid effectively removed allload resistance fmm the turbine-generator system. Consequently, as high energy steam was re-admitted to the turbine, the machine experienced an overspeed condition.
At the normal overspeed control setpoint (103% of the normal rated turbine speed of 1800 rpm), the OPC-20-1 and OPC-20-2 solenoid valves were electrically energized. However, the valves failed to open and relieve the ETF pressure that was maintaining the Governor and Intercept Valves open. Consequently, the turbine-generator unit continued to overspeed.
The overspeed condition, in which the turbine reached approximately 2900 rpm, caused O
several blades in the No. 22 low Pressure turbine section to separate from the rotor disc, penetrate the 1.25 inch thick steel turbine casing, and become projectiles from the turbine.
Since the Salem turbine generators are outside on the turbine building roof, the projectiles landed on the roof and the ground around the turbine building (See Figure 7). No nuclear safety systems were affected by the turbine projectiles.
The resulting eccentric motion of the rotor shaft apparently caused severe vibration at the Main Generator. Consequently, the generator's hydrogen seals failed and seal oil lines ruptured. Hydrogen gas (used for generator cooling) and seal oil (used to pressurize the generator hydrogen seals) were released and ignited. A fire erupted in the immediate area of the generator.
When the operators performing the turbine test recognized the situation (about 70 seconds after the reactor trip), they restored the AST system to normal. An operator also manually tripped the turbine to assure that the AST system functioned to open the Interface Valve and relieve the ETF pressure that was maintaining the steam admission valves open. The operator's actions resulted in finally isolating the turbine from further steam admission. The event duration was about 74 seconds.
In accordance with their emergency plan, the licensee declared the situation as an Unusual Event. The event was later briefly upgraded to an Alert until the licensee determined that turbine projectiles had not affected any safety-related system, All reactor plant systems operated normally, and the reactor was brought to a safe shutdown condition. The fire was gs
0220/95 10:23 U. S H. P. C.
PEGloN 1 1 !N OO?
y 3
extinguished within 20 minutes by a combination of automatically actuated fire suppression systems and rapid response from the on-site fire brigade.
Licensee management representatives immediately responded to the site and provided oversight, direction, and control of recovery efforts. Actions were initiated to comprehensively investigate the circumstances and determine causal factors. No significant personnel injuries occurred. NRC Resident Inspectors reported to the site to initiate evaluation of the event and the licensee's response. The Unusual Event was terminated in about three hours. Photographs depicting the damage are included in Appendix E.
1.1.2 AIT Assessment Sununary The proximate cause of the event was the failure of all of the back-up emergency and overspeed protection trip devices to function due to mechanical binding of the three solenoid QW valves (Parker-Hannifin, Part No. MRFN16MX0834, Westinghouse Style No. 822A848001).
awl 6)
The mechanical binding was a result of foreign debris and sludge in the two OPC solenoid t
g, vaives and foreign debris, rust, and corrosion in the ET-20 solenoid valve, cP **s.)
G Several contributing causes and precursor events were identified. The principal findings included the determination that there was no preventive maintenance performed on these valves since installation, and the periodic operational testing of the valves was insufficient to effectively verify the hydraulic performance of each device. Further, by design, the majority of the automatic turbine trip features were bypassed when the mechanical trip testing procedure was performed. In this configuration, the turbine trip capability is principally dependent on the proper functioning of a single back-up emergency turbine trip solenoid salve ET-20.
Potentially, this event was preventable, in a Licensee Event Report, the licensec committed Zd to replace the ET-20, OPC-20-1, and OPC-20-2 solenoid valves in Unit 2 after discovering on September 10,1990 that similar components in Unit I were defective. An opportunity was available in May 1991 to effect replacement. However, the work was deferred to the planned January 1992 refueling outage due to management decision that may have been caused by a deficiency in committment tracking. Additionally, on October 20,1991, operators and their supervisors permitted turbine startup without resolving a turbine system test discrepancy which indicated that the turbine overcpeed protection system was not functioning properly.
The licensec's actions subsequent to the event were effective and correct. The scactor and safety-related systems operated normally and functioned as designed, No radiological release occurred. No safety injection was required. The operators were well trained and qualified and effectively followed the Emergency Operating Procedures for a reactor trip. The reactor j
was stabilized and brought to a safe hot shutdown and then a cold shutdown condition l
without incident. The licensee correctly classified the event in accordance with the
,.s Emergency Classification Guide and made all of the required notifications and reports.
03GF95 10:29 U. S tl. P. C.
PEG 10tl 1 i!!!
003 4
r Senior management representatives responded immediately to the site and initiated actions to effectively organize, control, and direct event investigation and recovery efforts while keeping the NRC informed. These actions included protecting the' scene and configuration for review by the NRC and the licensee's Significant Event Response Team (SERT). The SERT was well trained and qualified and effectively analyzed the occurrence in accordance with well established and recognized event investigation techniques.
The site fire brigade was well trained and equipped. The fire brigade was effective in controlling the fire and mitigating further damage to the facility. All automatic fire suppression systems operated as designed. The flammable materials (hydrogen gas and seal oil) were effectively controlled and isolated to eliminate fuel flow to the fire.
1.2 Augmented Inspection Team Scope and Objective Upon being informed of the event on November 9,1991, the NRC Region 1 Regional Administrator and senior management from the Office of Nuclear Reactor Regulation 04RR) and the Office for Analysis and Evaluauon of Operational Data (AEOD) determined that an Augmented Inspection Team (AIT) :;hould be formed to review and evaluate the circumstances and significance of this occurrence. Accordingly, an AIT was selected, briefed, and dispatched to the site on November 10, 1991. Members of the AIT are identified on the Cover Sheet of this report.
O
~
The formal charter for the AIT was finalized on November 12,1991 (Appendix D). The charter directed the team to conduct an inspection and accomplish the following objectives:
Determine the specific circumstances, including the sequence of events.
Determine root cause(s), if possible, Determine if the operational testing of the turbine was a contributor to the event.
Evaluate the licensee's actions following the event.
Determine and evaluate the response of plant systems.
Determine the impact of the event on nuclear safety-related systems.
Review the licensee's analysis and efforts relative to root cause determination.
Review the vahdity of the original turbine missile bataid analysis.
Identify potential generic issues.
Document the results of the AIT review.
On November 10, 1991, the AIT arrived at the site. The AIT members initially met with senior licensee representatives and discussed the scope and objectives of the AIT. The licensee provided a briefing relative to the current status of their investigation and informed the AIT of all known and available information.
During the period between November 10 and December 3,1991, the team conducted an independent inspection, review, and evaluation of the circumstances and events associated with this occurrence. The effort consisted of: (1) direct observation of the licensce's
._ - - ~.. -
01 20'95 10:29 U. 5 ti. P. C. PEGIOt1 1 6 li t olo i
5 n
performance of recovery activities (including system testing and troubleshooting efforts), (2) review of pertinent records and documentation, (3) interviews with involved licensee -
personnel, (4) examination of equipmer.i, components, systems, and structures, (5) l independent analysis and interpretation of information and data, and (6) assessment of findings and conclusions.
On December 3,1991, the AIT conducted an Exit Meeting with the licensee. This meeting was open for observation by members of the public and media representatives. The AIT j
described the event and presented the findings and conclusions as described in this report.
2.0 TURBINE GENERATOR FAILURE EVENT 2.1 Sequence of Events
]
The AIT independently developed an event description and a detailed sequence of events.
l The AIT analyzed the sequence by developing a flow chart of the event, determining the associated causal factors, and evaluating previous system and procedural changes for their effect. The evaluation and analysis of the event was based on interviews with various 2
licensec personnel associated with the event, observation of licensee troubleshooting efforts, and a review oflicensee documents. Among the documents reviewed were logs and records, j
computer-generated operational sequence of events, Safety Parameter Display System (SPDS) data, control room chart recorder traces, load dispatcher alarm printouts, procedures, and ~
O operator logs. The AIT also reviewed the sequence of events developed by the licensee SERT in an effort to ensure that all available information was considered. A sequence of events summary is provided in Appendix B.
+
2.2 Turbine Control nnd Trip System Description i
i Unit 2 is equipped with a Westinghouse turbine assembly, a Westinghouse Electio-11ydraulic Control (EHC) system, and a General Electric generator and exciter. The turbine assembly consists of one double axial flow high pressure (HP) turbine and three identical double axial l
flow low pressure (LP) turbines. The shafts of the HP turbine, LP turbines, gerierator, and exciter are connected in series to form a single composite unit.
The purpose of the EHC system is: (1) to control the speed of the turbine-generator from i
startup to synchronization with the electrical grid, (2) to control the load of the generator from synchronization to 100% electrical load, (3) to limit turbine overspeed on partial load rejection, and (4) to effect rapid turbine shutdown (trip) to protect the turbine-generator unit.
The portion of the EHC system which effects emergency overspeed and trip control is depicted in Figure 1.
Four turbine Stop Valves (with associated bypass valves) and four Governor Valves respectively isolate and control the flow of steam to the HP turbine. The moisture-laden steam leaving the HP turbine is directed to six identical Moisture Separator Reheaters n
~f 03/20/95 10:30 U.S II.P.C. PEGloll 1 i111 011 i
6 (MSRs). The MSRs remove moisture and reheat the steam for additional use. The reheated steam is supplied to the three LP turbmes through a Reheat Stop Valve and an Intercept Valve from each MSR. Exhaust steam from the LP turbines is directed to the main condenser. A system diagram is depicted in Figure 2.
The EHC system uses high pressure hydraulic fluid to open the turbine steam admission valves (Stop Valves, Governor Valve, Reheat Stop Valves, and Intercept Valves). The Stop Valve bypass valves are air operated and are opened when the turbine-generator is latched (AST oil reset) and the respective Stop Valve is closed (less than 90% open). The EHC electronic Governor Valve controller varies Governor Valve position to control turbine generator speed and load. The Governor Valves are positioned by servo-actuators in the hydraulic fluid line in response to signals generated by the electronic Governor Valve controller. The Turbine Stop, Reheat Stop, and Intercept Valves are not controlled in any i
intermediate positions by the EHC system.
The hydraulic portion of the EHC system consists of three sub-systems: the high pressure EHC fluid system, the ernergency trip fluid (ETF) system, and the AST system. High pressure EHC fluid provides the motive force to position the steam admission valves. The EHC system also supplies high pressure hydraulic fluid to the ETF system through an orifice. ETF pressure controls the position of pilot operated dump valves which affect the operation of each steem admission valve actuator. less of ETF pressure causes the dump valves to open. With the dump valves open, the high pressure EHC fluid is drained from the valve actuator, causing the steam admission valves to close.
AST oil is supplied through an orifice from the turbine bearing lubrication system less of AST oil pressure initiates a turbine trip by allowing the Interface Valve to open. The AST oil pressure switches provide turbine trip signals to the reactor piutection system, generator, and EHC electronic Governor Valve controller on a loss of AST oil pressure. A turbine trip signal (2 out of 3 indications of low AST oil pressure) causes a reactor trip by opening the two reactor trip breakers (RTB-A and RTB-B) if reactor power is greater than 50%. To assure a turbine trip, opening the RTD-A and RTB-B energizes tmbine trip solenoid valves AST-20 and emergency trip solenoid ET-20, respectively.
The primary turbine trip system is the AST S,ystem. The AST System provides the following protective trips by relieving AST oil pressure to cause the Interface Valve to open and relieve ETF pressure:
Low bearing oil pressure Thrust bearing excessive movement Low condenser vacuum Mechanical overspeed, and Remote inputs to solenoid AST-20. The inputs include:
I 03 40/95 10:31 U.S N.P.C.
PEGION 1Ilu 011 l
l 7
i C-Manual from the control board, l
Reactor Trip Breaker " A",
Electrical overspeed,
)
High-high level in any Steam Generator, l
l Failure of DC power to EHC, j
l Low lube oil level, IAw EHC fluid level, Low EHC fluid pressure, Main generator cooling water failure, High vibration, Loss of main feedwater pumps, Generator reverse power (30 second delay on opening of output breaker), and Main output transformer protective relay.
A back up emergency trip system is provided by the ET-20 solenoid which, when energized, directly relieves ETF pressure to cause the closure of all steam admission valves. ET-20 is energized on the following inputs:
Manual trip from the control board, Reactor Trip Breaker "B",
High high level in any Steam Generator, and p
Lew Auto Stop Oil pressure.
The Overspeed Protection Control (OPC) energizes a pair of solenoids (OPC-20-1 and OPC-20-2) when 103% of normal speed is sensed. When energized, these solenoids open to effect the closure of the Governor Valves and Intercept Valves to arrest an overspeed condition.
On a reactor trip, generator output breakers are designed to open 30 seconds after a turbine trip. The electronic Governor valve controller is designed to maintain the Governor Valves closed after receiving a turbine trip signal (as a function of AST pressure switch, 63-3-AST).
In addition, another AST oil pressure switch is designed to provide a redundant turbine trip initiation signal to the ET 20 valve as a function of low AST oil pressure.
The turbine Front Standard contains the manual trip and test levers required to manipulate the AST system for testing of the mechanical trip functions (See Figure 4 and Appendix E).
By design, holding the test lever in the TEST position isolates the Interface Valve from any AST oil pressure perturbation by any of the five AST turbine trip devices. This feature allows testing of the turbine trips associated with the AST system without affecting turbine operation. While in this configuration, the AST system is prevented from reacting to any mechanical trip input, including the 13 remote trip inputs that would energize the AST-20 solenoid valve.
l Turbine overspeed protection is provided through redundant components. The EHC l
P clectronic Governor Valve controller normally maintains the turbine at the referenced speed of 1800 rpm. If a generator fault causes the generator output breakers to open under load.
l
(
an anticipatory overspeed trip is initiated through the ET 20 and AST-20 solenoids. Opening
. - - _. ~. _. -.. - - - - -.. -.
.. ~. - -
-. - - ~ -
03/20/95 10:32 U. S N. P.C. FEGICtl 1 i Ill or s
1 i
8 A
the generator output breakers, combined with significant pressure in the MSRs, also causes an anticipatory initiation of the OPC.
OPC is set to initiate at 103% overspeed. In addition to actuating the OPC-20-1 and OPC-I i
20-2 solenoids, the OPC initiation also maintains the Governor Valves closed through the F.HC electronic Governor Valvo controller, j
The AST mechanical overspeed trip is also set to actuate at 103% overspeed. While the trip function is normally reliable, as previously mentioned, the AST system is by-passed j
whenever the turbine Front Standard test lever is in the TEST position.
2.3 Event Description Salem Unit 2 returned to 100% reactor power at 6:30 a.m. on November 9,1991,' following a load reduction to 80% the previous day due to a solar magnetic disturbance alert. At about
~
l 11:00 a.m., two equipment operators (EOs) and the nuclear shift supervisor (NSS) commenced a monthly turbine mechanical trip operation test in accordance with OP III-1.3.7,
" Turbine Automatic Trip Mechanisms Operational Tests". While not a NRC required J
activity, the test is recommended by the turbine vendor (Westinghouse) to verify operability of the four mechanical turbine trip features, i.e., Overspeed Trip, Vacuum Trip, Thrust Bearing Trip. and low Bearing Oil Pressure Trip. The test was performed locally at the F'
turbine Front Standard, as shown in Figures 3 and 4.
The operators were in the process of completing the final trip test, law Bearing Oil Prcssure Trip. By procedure, the manual test lever was in the TEST position, which bypassed all mechanical trips associated with the AST system. The turbine generator was opeiating normally, and the EHC system used to control steam admission to the turbine was in a load control mode.
At 11:21:54 a.m., the reactor tripped due to an indicated turbine trip (caused by actuation of three low AST oil pressure switches) concurrent with reactor power greater than 50%. The indicated turbine trip was caused by a perturbation in AST oil pressure. The pertmbation in oil pressure did not affect the isolated portions of the AST system, but it did cause the Interface Valve to open. Subsequently, ETF was drained, the pilot valves opened, and all four of the turbine Stop Valves closed. Other turbine steam admission valves (Governor, Reheat Stop, and Intercept) probably also closed, however, this could not be confirmed.
The operators at the turbine Front Standard noted a loud ' thud" (which was probably the Stop Valves closing) and attempted to contact the control room for plant status. The operator continued to maintain the test lever in the TEST position. Consequently, the turbine trip inputs to the AST-20 solenoid remained bypassed.
At 11:21:55.5 a.m. (1.5 seconds after the trip), the AST oil pressure returned to normal.
Consequently, the Interface Valve closed and the Stop Valve bypass valves opened. The ET-
1 1
03/20/95 10:32 U.S N.R.C. PEGIOtt 1 1.111 gi; 9
(
r-20 solenoid valve, though energized, failed to function to maintain relief of ETF.
~
Subsequently, ETF repressurized and initiated the re-opening of the turbine steam admission valves and the consequent readmission of steam to the turbinc.
Note: During a later re-creation of the event, the licensoe learned that the 63-3.AST j
pressure switch failed to actuate to maintain the Governor Valves closed, as was expected. Analysis revealed that the actual pressurc setpoint of the switch had drifted to a value (39 psig) which was less than the expected design value (between 50 and 55 psig). Apparently, the momentary AST oil pressure
]
perturbation, while sufficient to open the Interface Valve (at about 50 psig),
i was not low enough to affect the 63-3-AST pressure switch.
4 i
in response to a reactor trip indication, the control room operators implemented the l
emergency operating procedures (EOPs) and proceeded to stabilize the reactor plant, j
Through the Sequence of Events recorder, the load dispatcher was informed of the plant trip.
At 11:22:21.8 a.m. (27.8 seconds after the trip), the generator output breakers (1-9 and 9-
- 10) opened, disconnecting the generator from the grid and removing the load froin the turbine generator. As a result of the readmission of steam to the unloaded turbine generator, j
it began to overspeed. At 103% of the normal turbine-generator speed, the OPC solenoid valves (OPC 20-1 and 20-2) were expected to actuate to arrest the overspeed condition by c
closing both the Governor and Intercept Valves. However, both of these solenoid valves i
l failed to function.
Note: By design, the EHC system is expected to automatically shift from load l
control to speed control mode when the generator output breakers open.
While this control mode shift may have occurred and possibly caused the Governor Valves to begin to close, the normal stroking speed was insufficient to effectively restrict the flow of steam to the turbine in time to ptevent the overspeed condition.
l At 11:22:33 (39 seconds after the trip), the Stop Valves indicated fully open.
t The operators at the Front Standard report that they heard a high pitched noise and rumbling sound, observed the generator on fire, and saw projectiles coming from the 22 Low Pressure i
turbine. Consequently, at 11:23:08 (74 seconds after the trip), the operator at the Front Standard repositioned the test lever to the normal position and physically verified that the i
manual trip lever was in the TRIP position before evacuating the area. The turbine Stop Valves immediately closed, and the turbine was finally isolated from main stcam.
During the overspeed event, the evidence indicates that the generator experienced extreme and severe vibration. The vibration was probably the result of the eccentric movement of the main shaft when the 22 Low Pressure turbine sustained damage and destruction. The l
c.
vibration at the generator was sufficient to physically move the 600 ton machine.
j j
-w
j 03 ;.0 4 s to:::
U.s u.P.C. FEG10tl i impl uit 10 Consequently, it is likely that the hydrogen seals became impaired. The evidence from the licensee's damage assessment supports the probability that the pressurized hydrogen gas in l
the generator (used for cooling) exited the machine through the impaired seals and ignited, resulting in a fire.
l The vibration was also severe enough to fracture and break the generator bearing seal oil supply line. Consequently, seal oil was ignited by the hydrogen fire. The oil continued to l
feed the fire until the oil supply pumps were later secured.
The installed automatic Dre suppression systems actuated, including the generator / exciter carbon dioxide system, the bearing deluge system, and the turbine building sprinkler system.
At 11:25 a.m., the site fire brigade responded to the scene and effected complete control and suppression of the file within about 20 minutes, j
i At 11:40 a.m., the licensee declared an Unusual Event in accordance with the Emergency Classification Guidelines (ECG) for a fire lasting more than ten minutes in the protected area. At 12:10 p.m., the licensee notified the NRC via the Emergency Notitication System (ENS). At 12:44 p.m., the licensee declared an Alert, as required by the ECG, for,
situations in which a potential existed for safety system impairment due to turbine missile j
generation. The licensee determined that safety systems were not affected by this event and de-escalated the classification to Unusual Event. At 2:45 p.m., the Unusual Event was terminated. These events are summalized in Appendix B.
2.4 Precursor Events 2.4.1 Pertinent Events The AIT reviewed potential industry precursor events that were pertinent to Unit 2 turbine generator failure. This included a review and assessment of the licensee's SERT review of precursor events. These events are summarized in Appendix A. The following precursor events were identined:
A.
Ginna Reactor Trip without Turbine Trip on April 6,1985 (LER 50-244/85-07)
The turbine failed to trip us acquired after a reactor trip on low steam generator level during stanup. The turbine was manually tripped locally. The emergency trip (ET-
- 20) solenoid valve failed to trip due to mechanical binding of the solenoid plunger.
l l
I
_____s j
0340/95 10:34 U.S II.P.C. PEGIOl4 1 Ilit ole 11 13.
Crystal River 3 Reactor Trip without Turbine Trip on February 23,1988 (LER 50-302/88-06)
The reactor tripped during a feedwater transient. Subsequently, the turbine failed to trip and could not be tripped from the control room. The operators isolated main steam to the turbine and manually tripped the turbine locally. The failure of the turbine to trip was caused by a faulty turbine trip solenoid, apparently due to an incorrect fit of a solenoid valve in the trip block.
l C.
Salem Unit 1 Reactor / Turbine Trip on August 31,1988 (LER 50-272/88-15)
A reactor trip and turbine trip occurred on low AST oil pressure during turbine mechanical trip testing. (The same turbine testing pmcedure in progress during the current event). Subsequent licensee investigation determined that a 1/32 inch pressure reducing orifice in the AST oil supply was probably clogged. The clogged orifice, in conjunction with the ongoing trip testing, allowed auto stop oil pressure to decrease to the turbine trip /rcactor trip setpoint. The orifices were immediately cleaned on Unit I and were periodically cleaned on both units during refueling outage periods. Though this was not an example of failure of a turbine trip solenoid, the event is pertinent due to the similarity of events.
D.
Salem Unit 1 Reactor Trip on September 10,1990 (LER 50-272/90-30)
A reactor trip, on low steam generator leyc1 occurred during a feedwater transient induced by an erroneous turbine overspeed signal. The signal was initiated by the turbine OPC during operator troubleshooting activities to isolate a steam leak on the high pressure turbine. Licensee investigation found that the OPC solenoid trip valves would not function due to mechanical binding. Both OPC solenoids (OPC 20-1 and 20-2) and the emergency trip solenoid (ET-20) valves were replaced on Unit 1. The LER indicated that the same solenoid valves in the Unit 2 turbine control system would be replaced during the next outage of sufficient duration. Though such an outage did occur during May 1991, the solenold valves were not replaced at that time.
Due to management decision and deficiency in commitment tracking, the nplacement of the solenoid valves was deferred to the planned refueling outage in January 1992.
E.
Ginna Reactor Trip without Turbine Trip on September 26,1990 (LER 50-244/90-012)
The reactor tripped when 1&C technicians were inspecting a turbine cabinet. The inspection activities resulted in low AST pressure. The turbine did not trip on the reactor trip signal due to a turbine emergency trip solenoid (ET-20) failure. The most probable root cause was mechanical binding of the solenoid pilot valve plunger and spool, similar to the previous 1985 Ginna event. The cause of the binding was believed to be due to foreign material in the solenoid plunger / spool area. The
0340<95 1o:35 U.S H.P.c. PEGION 1 1IN OLT
~
[
12 licensee reported replacing the ET-20 solenoid with one less susceptible to mechanical binding.
F.
Salem Unit 2 Turbine Generator Startup on October 20,1991 (no LER)
The Salem Unit 2 turbine generator was started up after Mode 2 operation at 0%
power. The unit opercted with the reactor critical for two days in order to reduce steam generator chloride concentration. The return to power seguired the licensee to perform activities in accordance with LOP-3,
- Integrated Operating Procedure, Hot Standby to Minimum lead". Step 5.33 of that procedure provided instructions to place the turbine on line in accordance with OP 111-1,3.1, " Turbine Generator Operation". Step 5.1.13 of that procedure specified a test of the OPC by verifying that the Intercept Valves close when the OPC test switch is turned to the TEST position.
Each of two licensed control room operators attempted this test once on October 20 1991. In both instances, the Intercept Valves failed to close, indicating a problem with the OPC system. To varying degrecs, the Unit Shift Supervisor, the Senior Shift Supervisor, and the Operations Engineer were aware of the problem. Apparently, none of the five individuals clearly understood the nature or implications of the test discrepancy. Accordingly, the test and procedure were not reviewed or verified by any of the supervisors, erroneous assumptions were made about the nature of the problem (i.e., that the test discrepancy was due to a procedural problem as opposed to au possible component or operational defect), and inadequate commimications and directions ensued. Consequently, the individuals continued to restan the turbine and return the unit to full power without sufficiently understanding or resolving the apparent defect with the OPC system. In this instance, the actions of the operators and supervisors were r.ot in conformance with the expected conduct and quality of opciations relative to discrepancy evaluation and resolution.
2.4.2 Other Evaluated Events The AIT also reviewed the following events as possible precursors; the team concluded that these events did not directly or indirectly affect or lead to the Unit 2 turbine generator failure.
A.
A Solar Magnetic Disturbance (SMD) alert occurred at 5:31 p.m. on November 8, 1991, and Salem Unit 2 power was reduced from 100% to 80% as required by procedurcs. The SMD alert was declared by the load dispatcher due to potential geomagnetic induced currents in the grid resulting from solar disturbances which could affect the main power transformers. The SMD alert was de-escalated, and Salem Unit 2 returned to full power at 6:30 a.m. on November 9,1991.
n
03 CO/93 10:36 U. S H. P. C FEGIOtt 1 i HI U1.
I3 m
B.
An increased main generator hydrogen gas consumption was noted on November 4, 1991. Hydrogen gas pressure is normally maintained about 70 psig. The unit had J
been using approximately 3000 standard cuble feet per day (SCFD) of hydrogen to j
compensate for known leakage through several hydrogen supply and vent valves and through the No.10 hydrogen seal. This leakage was being monitored by system engineering and operations personnel and was being processed by the seal oil system and/or vented to the atmosphere. Consequently, there was no increased risk for gas detonation.
On November 4,1991, the hydrogen leakage increased from 3000 to 5600 SCFD.
Scal oil differential pressure was raised from 1013 psig to approximately 15 psig.
Subsequently, the leakage returned to 3000 SCFD.
C.
The Salem Unit 2 main generator suffered two internal failures in 1983 and a major winding failure on October 4,1984 Following the October 1984 failure, the licensee clected to replace the installed Westinghouse generator with the General Electric generator that had been planned for Hope Creek Unit 2. The generator replacement project started in October 1984 and was completed in April 1985. The generator work was performed under design change package (DCP) 2EC-2011. No failures of this generator had been expetienced since installation.
F 2.4.3 Assessment PSE&G missed valuable opportunities to prevent the Salem Unit 2 turbine generator failure.
The Salem Unit 1 event of September 20,1990 identified failed turbine trip solenoid valves.
Insufficient priority and importance was assigned to the verification of operability and replacement of the solenoid valves at Salem Unit 2. Due to the failure to recognize and track the completion of the LER commitment, the licensee elected to defer replacement until the planned refueling outage in January 1992 in lieu of an earlier opportunity during a planned outage in May 1991.
During the Unit 2 turbine generator startup on October 20,1991, operators identified an apparent problem with the OPC system, which may have been an indicator of OPC solenoid valve failures. However, several operations personnel, including licensed operators, a shift supervisor, a senior shift supervisor, and a senior operations engineer failed to react appropriately to the problem by assuring proper resolution in accordance whh the normal conduct of operations.
The 1985 and 1990 Ginna, the 1988 Crystal River Unit 3, and the 1990 Salem Unit f events were all examples of events involving failed turbine trip solenoid valves that may have been poorly communicated or insufficiently regarded. Further, the NRC issued Generic Letter 91-15, " Operating Experience Feedback Report, Solenoid-Operated Valve Problems in U.S.
Reactors," and the associated NUREG-1275, Vol. 6, on September 18, 1991. This report identified several solenoid valve problems, including applications in turbine trip control
I 09 20'95 lo G U.S ttP.C. PEGIOtl1 1H1 011 14 n
systems. The Generic Letter did not require any speciDe response or action, but it did advise licensees to review the information and consider actions to avoid siinilar problems.
The AIT found no indication that the licensee had directed any attention or priority to assessing the implications of this information (relative to turbine control systems) as of the date of this occurrence.
2.5 Generator and Electrical System Assessment
Background
The electric power output of Salem Unit 2 is rated at 25 kV output voltage stepped up via the main generator transformer to the station 500 kV switchyard via outpt;t breakers 1-9 and 9-
- 10. Safety-related equipment are normally powered by the station power transformer from j
the 13 kV ring bus which is powered from the off-site 500 kV switchyard, as shown in Figure 5.
During power plant operation, the 4.16 kV group buses supplying balance of plant equipment are powered by an auxiliary power transformer which receives its power from the unit generator. When the unit generator trips, the power source is transferred to the station power transformer.
O The transient recorder which monitors electrical pammeters was out of service prior to this event. On November 8,1991, Unit 2 decreased power output from !!50 MWe to 900 MWe as a precautionary measure due to the SMD. On November 9,1991, all electrical systems were reported normal, and the power was returned to 100% reactor power (1150 MWe).
Prior to the event, all support systems and generatos piotective circuits were operating
- normally, Main Generator and System Resnonst The currently mstalled main generator at. Unit 2 is manufactured by General Electric. The following pertains:
Generator Rating - 1300 MVA,1800 RPM,60 HZ, stator voltage 25 kV,0.9 pf, 75 psig of Ilydrogen pressurc Exciter type - Alterrex excitation system Rating - 3500 KVA,4 poles,1800 RPM,0.95 pf,60 Hz The generator casing and the end shields at either end are of welded gas tight construction, supporung and enclosing the stationary armature winding and core, the rotating fictd, and the gas coolers. The principal cooling medium is hydrogen gas, which is contained within the frame and circulated by fans mounted at each end of rotating Deld. A separate deionized e,.
water cooling system is also provided for the armature winding. In order to prevent the
c1 20/95 10:37 U. 5 N. P. C. FEGIOli 1 i lil Gi 32 6.0 PLANS AND SCIIEDULES FOR REPAIR AND RESTORATION The licensee had originally planned to shut down Unit 2 on January 4,1992 for a normal refueling outage. The November 9,1991 forced outage required rescheduling of the planned outage activitics and the inclusion of the additional work required for the repair of the turbine, generator, and associated auxiliary facilities and equipment.
The generator work is considered the critical path element in the current outage. The licensec considered several options which included repair / rewinding of the existing generator, replacement with a similar General Electric unit, or replacement with a Westinghouse unit.
Subsequently, PSE&G elected to replace the generator with a similar General Electric unit.
The licensee has initiated disassembly of the turbine. Preliminary damage assessment reports j
indicate that the lip turbine sustained only minor impairment and will be repaired.
liowever, the licensee has initiated action to replace the thrco LP tmbines with an existing spare assembly on site.
Due to damaged incurred from turbine blades, the licensee plans to seplace over 2500 of the 11,000 tubes in the main condenser. Further, several turbine generator support and auxiliary systems, components, and structures will require repair or replacement. As of the time of this inspection, the licensee had not committed to a firm outage schedule.
7.0 FINDINGS AND CONCLUSIONS Based on independent assessment and review of available information, the AIT concluded that the proximate cause of this event was the failure of turbine control solenoid valves (OPC-20-1, OPC-20-2, and ET-20) to function as designed to prevent turbine overspeed, and effect and maintain closure of steam admission valves (Turbine Stop Valves, Governor Valves, Reheat-Stop Valves, and Intercept Valves) in the event of a reactor trip. The solenoid valves failed to function due to mechanical binding. Mechanical binding of the solenoids was caused by a combination of foreign material, sludge build-up, and general corrosion which prevented the functioning of the solenoids' internal components (i.e., spool pieces and pilot valve assemblies). Solenoid valve malfunction was not detected or corrected by the licensee as a result of ineffective surveillance test methods and lack of any preventive maintenance. The AIT determined the following findings and contributing causal factors relative to this event:
7.1 Personnel Performance Management communication and personnel understanding of the policy and expectation relative to conduct of operations involving procedure adherence, resolution of procedural and equipment problems, and quality of operations appears to be deficient in the specific case of the turbine startup on October 20,1991. Five licensed personnel, including operators and supervisors, failed to adequately resolve n
m.__ _ _ _ _ _ _. _
m._.___
OD&95 lo G U. 5 ti. P. C. PEGION 1 6 Ill 021 j
N test discrepancy involving the overspeed protection control system prior to returning I
the turbine to full operation. Consequently, an opportunity to prevent this event was i
missed. (Contributing Causal Factor)
All personnel actions following the initiation of the event, including management and operator performance were adequately accomplished and%ere correct and seasonable l
for the circumstances, a
Event classification was accurate. All notifications were performed in a timely j
manner. The event was accurately reported to the NRC in acco2 dance with regulatory requirements. Appropriate action was taken to account for personnel on-site.
Fire protection personnel were well trained, qualified, and effective in maintaining control of the fire scene, extinguishing the fire, controlling reflashes, searching for i
personnel who may have been injured, and assuring amelioration of hazards. Proper actions were taken to assure that the pctcatial for personnel injury was minimized.
i
!i The licensee failed to react in a timely manner to the Salem Unit I solenoid failures
~
by effectively verifying the operability of, or replacing the devices in Salem Unit 2 in accordance with an LER commitment. (Contributing Causal Factor) 4 The Unit Shift Supervisor's absence from the control room to assist in performance of i
(as opposed to supervising) the turbine test procedure, while not pmhibited, was imprudent. However, the individual's action in this regard was not a contributing factor to this event.
7.2 Equipment Performance Reactor Systems All reactor trip and protcctive systems functioned as designed. All reactor emergency l
j safety systems and features were available for use, but were not required for recovery. Area and effluent mdiological monitoring systems remained operable.
Nuclear safety-related systems were not affected by missiles generated from the de:,truction of the low pressure turbine.
Turbine Control Systems i
Though not conclusive, the information available indicates that the initial transient, i.e., low AST pressure indication to the RPS was most likely due to clogging of the
{}
supply pressure reducing orifice by foreign material (similar to the Unit 1. event reported in LER $0-272/88-0l$). However, the possibility remains that the operator i
j J
03/20/95 10:39 U.S H.P.C.
PEGIOtl i 1.111 UL 34 n
at the Front Standard may have inadvertently moved the test lever to momentarily perturb the AST oli pressure. (Contributing Causal Factor)
While ET 20, OPC-20-1, and OPC-20-2 were energized in accordance with design, none of the solenoids functioned hydraulically. The AST-20 solenoid was confirmed to be operable, but was by passed during the period of the event. (Contributing Causal Factor)
All of the AST pressure switches affecting the RPS logic operated as designed, but the 63-3 AST pressure switch (which is not part of the RPS) did not function as expected. The 63-3 AST pressure switch was set at 39 psig (approximately 10 to 15 psig less than the AST pressure switches affecting RPS). The 63-3 AST pressure switch is responsible to for re-referencing of the Governor valve controller fwm full-load to no-load when the turbine is expected to trip. Consequently, when the initial turbine trip signal occurred, the Governor Valve was not re-referenced to a no load situation. Instead of closing the Governor Valves for the no-load condition, the l
valves re-opened when hydraulic trip fluid repressurized in the EHC system.
(Contributing Causal Factor) 1 None of the solenoid valves were subjected to any PM program. The vendor did not prescribe any PM for the devices; consequently, a PM program was not initiated.
(Contributing Causal Factor)
The local turbine speed tachometer, which could have provided early indication to the operators at the Front Standard of an overspeed condition was not maintained operable since 1987. (Contributing Causal Factor)
The 63-3 AST switch was not subjected to any recurring calibration program.
(Contributing Causal Factor) i Upon release of the test lever, a final turbine trip was inserted automatically or by operator manual initiation. Based on the results of the licensec's troubleshooting test, the AST portion of the turbine trip protective systems functioned as designed.
Consequently, when the lever was returned, the system functioned as designed to effect closure of the steam admission valves, The EHC System performed as designed, with the exception of the component i
failures involving the ET and OPC solenoid valves, and the 63-3 AST pressure switch setpoint.
Surveillance and operational testing of turbine trip performance and overspeed did not specifically verify the proper hydraulic functioning of each solenoid valve, independently. (Contributing Causal Factor)
03/20/95 10:39 U. S H. F. C. PEGloh i n11.
UJ e
35
,~.
The periodic testing of the mechanical trip function effectively isolates 17 possible trip signals or inputs while the test is being performed; prior to performing the test, there is no verification that the back-up trip and overspeed systems are functional.
(Contributing Causal Factor)
Information (from internal and external experience) concerning previous component failures of turbine solenoid valves does not appear to have been generally regarded by the licensee as significant or of sufficient importance to warrant priority attention nnd corrective action. (Contributing Causal Factor)
Fire ProtectioJLSysicJn_j All automatic fire suppression systems operated as designed and were effective in providing initial control of the generator hydrogen and oil fire.
Electrical System Electrical system operation during this event was normal and as designed. No electrical safety systems were affected by this event. A review of pievious generator failuies revealed no direct correlation to this event.
C The generator destruction was due to the turbine overspeed. The consequent fire resulted from the escape and ignition of hydrogen gas (used for cooling) and seal vil from impaired hydrogen seals and fractured seal oil piping. The impairment of the hydrogen seals and seal oil piping was the consequence of the extreme and severe vibration sustained at the generator as a result of the turbine overspeed.
Following the reactor trip, all electrical relays, breakers, and generator protective devices performed as expected.
7.3 Procedure Adequacy and Adherence Following the event, the operators adhered to the requirements and directions
~
provided by the EOPs. The EOPs were sufficient to effect plant stabilization and assure safe cooldown and shutdown of the reactor.
The procedures that were established and implemented to verify the operability of the turbine overspeed control system, to meet the licensee's understanding of the requirements of TS 3.3.4, were not generally effective. Procedure SP(O) 4.3.2 adequately verified the operability of the turbine steam admission and contiol valves but did not sufficiently verify the operability of the overspeed control system.
(Contributing Causal Factor)
[
.. ~ _. _ _.__.____ _.._
r ' '
03/20'95 10:do U..S 11.P.C. PEGIOti 1 1.111 02J l
36
/
l The licensce's applicatbn of various Operating, and Instrument and Control Procedures to satisfy the channel calibration requirements of TS 3.3.4 is not well established. The procedures (OP 111-1.3.2,2PD-6.1.004, and OP 111-1.3.1) are used 1
to satisfy the TS requirements for the channel calibrations, but since the procedures are not dedicated TS surveillance procedures, and are considered as Category 11 procedures, a record of their performance is not always maintained. As a result, there is uncertainty, in some cases, as to the licensee conformance with these procedures. (Contributing Causal Factor)
The NRC Standard Review Plan, upon which Unit 2 was evaluated, generally assumes the availability of three diverse and redundant overspeed protection devices (OPC, mechanical, and emergency trip). In the case of Unit 2, two of those three (mechanical overspeed and electrical input to AST-20) are prevented from functioning whenever the AST system is under test. (Contributing Causal Factor)
On October 20,1991, certain licensed operators and superviscrs did not sufficiently adhere to the specifications of10P-3, " Integrated Operating Procedure-Hot Standby to i
Minimum tead", Step 5.33, which required the turbine to be operated in accordance with OP-Ill-1.3.1., " Turbine Generator Operation". OP-III-1.3.1., Step 5.1.13 specifies testing of the OPC by verifying that the Intercept Valves close when the i
OPC test switch is in the TEST position. When tested, the Intercept Valves did not close as was expected. Regardless, turbine-f,enerator startup was permitted without resolving this test discrepancy. (Contributing Causal Factor) 8.0 EXIT MEETING On December 3,1991, the NRC conducted an exit meeting that was open to the public and media representatives. The findings of the AIT were presented as described in this report.
Meeting attendees are identified in Appendix C.
.