ML042120012
| ML042120012 | |
| Person / Time | |
|---|---|
| Site: | Calvert Cliffs  (DPR-053, DPR-069) |
| Issue date: | 07/29/2004 |
| From: | Lanning W Division of Reactor Safety I |
| To: | Vanderheyden G Constellation Generation Group |
| References | |
| EA-04-110 IR-04-008 | |
| Download: ML042120012 (57) | |
See also: IR 05000317/2004008
Text
July 29, 2004
Mr. George Vanderheyden
Vice President - Calvert Cliffs Nuclear Power Plant
Constellation Generation Group, LLC
1650 Calvert Cliffs Parkway
Lusby, Maryland 20657-4702
SUBJECT:
NRC SPECIAL INSPECTION (SI) TEAM REPORT NO. 05000317/2004008 AND
05000318/2004008, AND PRELIMINARY WHITE FINDING - CALVERT CLIFFS
NUCLEAR GENERATING STATION
Dear Mr. Vanderheyden:
On May 14, 2004, the US Nuclear Regulatory Commission (NRC) completed a Special
Inspection at the Calvert Cliffs Nuclear Power Plant, Units 1 and 2. The enclosed report
documents the inspection findings which were discussed with you and other members of your
staff during an exit meeting on June 18, 2004.
This inspection examined activities conducted under your license as they relate to safety and
compliance with the Commissions rules and regulations and with the conditions of your license.
The team reviewed selected procedures and records, observed activities, and interviewed
personnel. In particular, the inspection reviewed event evaluations (including technical
analyses), root cause investigations, relevant performance history, and extent of condition to
assess the significance and potential consequences of issues related to both reactor trips on
January 23, 2004, (Unit 2) and March 20, 2004, (Unit 1).
The team concluded that the overall response of Constellation to the reactor trips on January
23, 2004, and March 20, 2004, were adequate, in that the plants were taken to a safe shutdown
condition. Nevertheless, the operators were challenged by equipment problems and
implementation of emergency operating procedures. Several of these issues were the result of
human performance issues. During the Unit 2 event, some operator actions, executing
emergency operating procedure 0, Post-trip Immediate Actions, were delayed due to past
operating practices.
This report documents one finding that appears to have low to moderate safety significance.
As described in Section 2.1 of this report, this finding involved a reactor regulating system
(RRS) relay that was not designed for the voltage conditions to which it was exposed and had
been in-place since the original construction of the facility. In addition, when Calvert Cliffs
implemented a modification to the turbine bypass valve and atmospheric dump valve control
system in 1992, they missed an opportunity to identify the inappropriate design. This condition
resulted in the relay failure in the RRS that prevented the system from properly regulating the
Mr. George Vanderheyden
2
reactor coolant temperature after the Unit 2 reactor trip on January 23, 2004. This resulted in a
safety injection actuation signal and steam generator isolation.
This finding was assessed using the reactor safety Significance Determination Process (SDP)
as a potentially safety significant finding that was preliminarily determined to be White for Unit 2
(i.e., a finding with some increased importance to safety, which may require additional NRC
inspection). The finding appears to have low to moderate safety significance because the
likelihood of core damage increased due to the loss of normal decay heat removal and loss of
low pressure feedwater supply. Based on analysis of the failed relay, the RRS condition would
have resulted in a similar uncontrolled cooldown, following a reactor trip any time during the
previous eight months.
We believe that we have sufficient information to make our final risk determination for the
performance issue regarding the RRS relay failure. However, before the NRC makes a final
decision on this matter, we are providing you an opportunity to either submit a written response
or to request a Regulatory Conference where you would be able to provide your perspectives
on the significance of the finding and the bases for your position. If you choose to request a
Regulatory Conference, we encourage you to submit your evaluation and any differences with
the NRC evaluation at least one week prior to the conference in an effort to make the
conference more efficient and effective. If a Regulatory Conference is held, it will be open for
public observation. The NRC will also issue a press release to announce the Regulatory
Conference.
Please contact Mr. Richard Conte at (610) 337-5183 within 10 business days of the date of this
letter to notify the NRC of your intentions. If we have not heard from you within 10 days, we will
continue with our significance determination and enforcement decision and you will be advised
by separate correspondence of the results of our deliberations on this matter.
Additionally, based on the results of this inspection, the team identified seven findings of very
low safety significance (Green). Five of these issues were determined to involve violations of
NRC requirements. However, because of their very low safety significance, and because they
have been entered into your corrective action program, the NRC is treating these issues as
non-cited violations, in accordance with Section VI.A.1 of the NRCs Enforcement Policy. If you
deny the non-cited violations noted in this report, you should provide a response with the basis
for your denial, within 30 days of the date of this inspection report, to the Nuclear Regulatory
Commission, ATTN: Document Control Desk, Washington, D.C. 20555-0001; with copies to the
Regional Administrator, Region I; the Director, Office of Enforcement, United States Nuclear
Regulatory Commission, Washington, D.C. 20555-0001; and the NRC Resident Inspector at
the Calvert Cliffs facility.
In accordance with 10 CFR 2.390 of the NRC's "Rules of Practice," a copy of this letter and its
enclosures will be available electronically for public inspection in the NRC Public Document
Room or from the Publicly Available Records (PARS) component of NRCs document system
(ADAMS). ADAMS is accessible from the NRC Web site at http://www.nrc.gov/reading-rm/
adams.html (the Public Electronic Reading Room).
Mr. George Vanderheyden
3
If you have any questions, please contact Mr. Conte at (610) 337-5183.
Sincerely,
/R.V. Crlenjak for:
Wayne D. Lanning, Director
Division of Reactor Safety
Docket Nos:
50-317, 50-318
Enclosure:
Inspection Report 05000317/2004008 and 05000318/2004008
w/Attachments: Supplemental Information
Attachments:
A. Supplemental Information
B. Special Inspection Team Charter
C. Unit 2 Sequence of Events
cc w/encl:
M. J. Wallace, President, Constellation Generation
J. M. Heffley, Senior Vice President and Chief Nuclear Officer
President, Calvert County Board of Commissioners
J. M. Petro, Esquire, Constellation Energy Group, Inc.
J. E. Silberg, Esquire, Shaw, Pittman, Potts and Trowbridge
Director, Nuclear Regulatory Matters
R. McLean, Manager, Nuclear Programs
K. Burger, Esquire, Maryland Peoples Counsel
State of Maryland (2)
Mr. George Vanderheyden
4
Distribution w/encl:
H. Miller, RA/J. Wiggins, DRA (1)
OE Mail
F. Congel, OE
S. Figueroa, OE
C. Miller, RI EDO Coordinator
R. Laufer, NRR
R. Clark/P. Tam, PM, NRR (Backup)
J. Trapp, DRP
N. Perry, DRP
M. Giles, SRI - Calvert Cliffs
J. OHara, DRP - RI - Calvert Cliffs
K. Farrar, ORA
D. Holody, ORA
R. Urban, ORA
G. Matakas, ORA
Region I Docket Room (with concurrences)
W. Lanning, DRS
R. Crlenjak, DRS
R. Conte, DRS
A. Blamey, DRP
DOCUMENT NAME: C:\\ORPCheckout\\FileNET\\ML042120012.wpd
After declaring this document An Official Agency Record it will be released to the Public.
To receive a copy of this document, indicate in the box: "C" = Copy without attachment/enclosure "E" = Copy with attachment/enclosure "N" = No copy
OFFICE
RI/DRP
RI/DRS
RI/DRS
RI/DRP
RI/ORA
NAME
ABlamey
RConte
WSchmidt
JTrapp
DHolody
DATE
07/22/04
07/22/04
07/27/04
07/27/04
07/27/04
OFFICE
RI/DRS
NAME
WLanning(RVCfor)
DATE
08/02/04
OFFICIAL RECORD COPY
Enclosure
U.S. NUCLEAR REGULATORY COMMISSION
REGION I
Docket Nos:
50-317, 50-318
License Nos:
Report Nos:
05000317/2004008 and 05000318/2004008
Licensee:
Constellation Generation Group, LLC
Facility:
Calvert Cliffs Nuclear Power Plant, Unit 1 and Unit 2
Location:
1650 Calvert Cliffs Parkway
Lusby, MD 20657-4702
Dates:
February 16, 2004 - May 14, 2004
Inspectors:
A. Blamey, Team Leader
S. Barr, Senior Operations Engineer
G. Cobey, Senior Risk Analyst
M. Giles, Senior Resident Inspector
A. Passerelli, Reactor Engineer
J. Richmond, Resident Inspector
W. Schmidt, Senior Risk Analyst
H. Williams, Operations Engineer
Approved by:
Wayne D. Lanning, Director
Division of Reactor Safety
Enclosure
ii
TABLE OF CONTENTS
SUMMARY OF FINDINGS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iv
1.0
Description of Events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1
Event Summaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
Unit 2 January 23, 2004, Excessive Steam Demand Event . . . . . . . . . . . . . . . . 1
Unit 1 March 20, 2004, 11 Steam Generator Low Level Event . . . . . . . . . . . . . . 2
2.0
Equipment Failures and Root Causes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2.1
Unit 2 Reactor Regulating System Quick Open Circuit
. . . . . . . . . . . . . . . . . . . 4
1.
Failure to Adequately Implement a Modification Design Review of the
Reactor Regulating System Quick Open Circuit . . . . . . . . . . . . . . . . . . . 4
2.
Administrative Procedure for Control of Maintenance Activities . . . . . . . 9
3.
Maintenance Rule Classification & Monitoring of Quick Open Function
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.2
Unit 2 SIAS Actuation Signal Failure to Reset from Control Room . . . . . . . . . . 10
1.
Failure to Adequately Implement Modification Work Instructions for
Wiring Terminations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.
Previous Corrective Actions for Compression Style Terminations . . . . 12
3.
Control of Testing Scope and Acceptance Criteria . . . . . . . . . . . . . . . . 13
2.3
Test Records for Safety Related Work Not Retained by Document Control . . . 13
2.4
Unit 1 Digital Feedwater Design Deficiency . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
3.0
Human Factors and Procedural Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
3.1
Failure to Properly Implement Station Emergency Operating Procedures . . . . 17
3.2
Failure to Have Procedures Required by Regulatory Guide 1.33 . . . . . . . . . . . 20
3.3
Simulator Fidelity Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
3.4
Failure to Comply with Station Work Control Procedures . . . . . . . . . . . . . . . . . 22
4.0
Emergency Preparedness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
4.1
Failure to Recognize and Report an Unusual Event During the January 23, 2004,
Unit 2 Reactor Trip . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
5.0
Cross Cutting Aspects of Findings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
6.0
Generic Issues
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
7.0
Risk Significance of the January and March 2004 Events . . . . . . . . . . . . . . . . . . . . . 27
8.0
Overall Adequacy of Licensee Response . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
9.0
Exit Meeting Summary
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
KEY POINTS OF CONTACT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-1
LIST OF ITEMS OPENED, CLOSED AND DISCUSSED . . . . . . . . . . . . . . . . . . . . . . . . . . . A-1
LIST OF DOCUMENTS REVIEWED
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-3
LIST OF ACRONYMS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-9
SPECIAL INSPECTION TEAM CHARTER . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B-1
Enclosure
iii
REVISED SPECIAL INSPECTION TEAM CHARTER . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B2-1
UNIT 2 SEQUENCE OF EVENTS
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . C-1
Enclosure
iv
SUMMARY OF FINDINGS
IR 05000317/2004-008, 05000318/2004-008; 02/16/04-02/20/04, 03/22/04-03/23/04, 05/10/04-
05/14/04; Calvert Cliffs Nuclear Power Plant, Units 1 and 2; Special Inspection Team.
The inspection was conducted by four regional inspectors, two resident inspectors, and two
regional senior reactor analysts. One finding, assessed as Preliminary White on Unit 2, and
seven other Green findings were identified. The significance of most findings is indicated by
their color (Green, White, Yellow, Red) using IMC 0609, Significance Determination Process
(SDP). Findings for which the SDP does not apply may be Green or be assigned a severity
level after NRC management review. The NRCs program for overseeing the safe operation of
commercial nuclear power reactors is described in NUREG-1649, Reactor Oversight Process,
Revision 3, dated July 2000.
A.
NRC Identified and Self-Revealing Findings
Cornerstone: Mitigating Systems
Preliminary White. A self-revealing event identified a finding of low to moderate safety
significance, because Calvert Cliffs Nuclear Power Plant (CCNPP) did not perform a
modification design review, as required by station procedures. Following a Unit 2
reactor trip on January 23, 2004, the atmospheric dump valves and turbine bypass
valves automatically Quick Opened, as designed. However, the Quick Open signal did
not clear when the reactor coolant temperature dropped below the Quick Open setpoint,
because of a reactor regulating system relay failure. As a result, an uncontrolled
cooldown of the reactor coolant system occurred, which in turn caused a loss of the
normal heat removal system.
This finding was more than minor because it was considered to be a precursor to a
more significant event. A Significance Determination Process Phase-3 risk analysis
determined that this finding was of low to moderate safety significance, based on the
change in core damage frequency. (Section 2.1.b.1)
Green. A self-revealing event identified a non-cited violation of very low safety
significance of Technical Specification 5.4.1, because CCNPP did not adequately
implement modification work instructions. As a result, during a plant event, recovery
actions were delayed because operators were unable to reset the "B" channel of the
safety injection actuation signal (SIAS) system from the control room.
This finding was more than minor because the SIAS system was returned to service,
following modification work, and subsequently became unable to perform its function,
similar to example 5.b in NRC Inspection Manual 0612 Appendix E. This finding had
very low safety significance because the finding did not represent an actual loss of a
safety function.
A contributing cause of this finding was related to the Human Performance cross-cutting
area because maintenance technicians did not adequately implement written work
instructions. (Section 2.2.b.1)
Enclosure
v
Green. The inspectors identified a non-cited violation of very low safety significance of
10 CFR 50 Appendix B, Criterion XVII, "Quality Assurance Records," because CCNPP
did not retain records of test results. From 1999 to March 2004, CCNPP did not retain
wiring verification point-to-point test records for modifications of safety-related circuits.
As a result, after the records are transferred to Records Management, verification of the
work performed cannot be done.
This finding was more than minor because the failure to retain the required records was
not an isolated example, and the records were irretrievably lost, similar to example 1.b in
NRC Inspection Manual 0612 Appendix E. This finding was not suitable for a
Significance Determination Process evaluation, but was reviewed by NRC management
and determined to be of very low safety significance.
A contributing cause of this finding was related to the Human Performance cross-cutting
area because station personnel did not adequately implement written instructions in a
safety-related procedure. (Section 2.3)
Green. A self-revealing finding of very low safety significance was identified because
CCNPP failed to perform an adequate design review which resulted in reduced reliability
of the digital feedwater system during a plant event on March 20, 2004.
This finding was more than minor because it effected the design control attributes of the
Initiating Events cornerstone. Incorrectly specifying the design voltage resulted in
reduced reliability of the digital feedwater control system which increased the likelihood
of an event that upset plant stability during power operation. This finding was of very
low safety significance, because one of two turbine driven feedwater pumps and one of
three condensate and condensate booster pumps remained operable during the Unit 1
March 20, 2004, event. (Section 2.4)
Green. The inspectors identified a non-cited violation of CCNPP Technical Specification
5.4.1.b because the operating crew did not properly implement station emergency
operating procedures during the Unit 2 reactor trip reactor shutdown on January 23,
2004.
The finding was more than minor because it affected the Initiating Events Cornerstone
in that the failures to follow station procedures complicated the plants post-trip response
and the ability of the operators to restore normal plant conditions. This finding had very
low safety significance because the finding did not represent an actual loss of a safety
function, and was not potentially risk significant due to an external initiating event.
A contributing cause of the finding was related to the Human Performance cross-cutting
area because licensed operators did not properly implement station emergency
operating procedures. (Section 3.1)
Green. The inspectors identified a non-cited violation of CCNPP Technical Specification
5.4.1.a because CCNPP did not have a procedure (off-normal) for the failure of the
reactor regulating system (RRS) as required by Regulatory Guide 1.33.
Enclosure
vi
This finding was more than minor because if the operators had switched to the alternate
channel of RRS, after the failure of the RRS relay in the X channel, the atmospheric
dump valves (ADVs) and turbine bypass valves (TBVs) would have properly controlled
reactor temperature and terminated the uncontrolled cooldown. This finding had very
low safety significance because the finding did not represent an actual loss of a safety
function, and was not potentially risk significant due to an external initiating event.
(Section 3.2)
Green. A self-revealing event identified a finding in that CCNPP did not follow
procedural requirements in their risk assessment and control of the work on March 20,
2004, which resulted in an unanticipated reactor trip. Specifically, the provisions and
controls of procedures NO-1-100, Conduct of Operations, NO-1-117, Integrated Risk
Management, and MN-1-100, Conduct of Maintenance, were not followed.
This finding was more than minor because the failures to follow station procedures
affected the Initiating Events cornerstone in that the failure to properly risk-classify and
control the work in the control room on March 20 lead to the reactor trip. This finding
had very low safety significance because the finding did not represent an actual loss of
a safety function, and was not potentially risk significant due to an external initiating
event.
A contributing cause of the finding was related to the Human Performance cross-cutting
area because CCNPP managers and staff did not properly implement station
operations, risk management, and maintenance procedures. (Section 3.4)
Green. The inspectors identified a non-cited violation of 10CFR50.54(q) because the
operating crew did not properly recognize plant conditions commensurate with an
Unusual Event in accordance with the emergency plan and implementing procedures
during the Unit 2 reactor trip on January 23, 2004.
This finding was more than minor because it effected the response organization
performance attribute of the Emergency Preparedness Cornerstone in that failure to
properly recognize plant conditions commensurate with an Unusual Event classification.
This finding was of very low safety significance, because it involved an implementation
problem during an actual event and the CCNPP staff failed to identify the Unusual Event
in the post trip review.
This finding is related to the Human Performance cross-cutting area because the
operating crew did not properly recognize plant conditions commensurate with an
Unusual Event in accordance with the emergency plan during a Unit 2 excessive steam
demand event on January 23, 2004. (Section 4.1)
Enclosure
Report Details
1.0
Description of Events
1.1
Event Summaries
January 23, 2004, Excessive Steam Demand Event (Unit 2)
On January 23, 2004, Calvert Cliffs Unit 2 was operating at 100% power. At 3:26 p.m.,
the 22 steam generator feed pump (SGFP) unexpectedly tripped off. Operators
attempted to reset and restart the 22 SGFP three times but were unable to reset the
pump trip. Without this pump operators were unable to maintain water level in the
steam generators, and they monitored water level approaching the trip criteria. Just
prior to the operators initiating a manual reactor trip, the automatic trip set point was
reached, and the unit experienced a reactor trip at 3:27 p.m. The operators entered
emergency operating procedure (EOP) - 0, Post-Trip Immediate Actions. When the
reactor tripped, the reactor regulating system controlled the removal of stored energy in
the reactor coolant system (RCS) and the secondary system with the quick-open signal
which opened the turbine bypass valves (TBVs) and atmospheric dump valves (ADVs).
The quick open signal is designed to have the TBVs and ADVs initially fully open, then
modulate, to automatically control RCS temperature at 532 degrees Fahrenheit. In this
event however, the TBVs and ADVs remained full open, causing a rapid overcooling and
depressurization of the RCS.
Due to lowering steam generator water levels, an Auxiliary Feedwater Actuation Signal
(AFAS) occurred at 3:28 p.m., and the 21 and 23 Auxiliary Feedwater (AFW) pumps
started and provided water to the steam generators as designed. At 3:28 p.m. RCS
pressure decreased to 1740 psia, causing a Safety Injection Actuation Signal (SIAS).
Equipment which started as a result of the SIAS included the 2A and 2B emergency
diesel generators (EDGs), the high pressure safety injection pumps, the containment
spray pumps and the low pressure safety injection pumps. RCS pressure remained
high enough that the charging pumps were the only source of injection into the RCS.
The SIAS signal also stopped RCS letdown and de-energized the back-up pressurizer
heaters. The operators stopped two reactor coolant pumps (RCPs) per the procedure
requirements following receipt of a SIAS signal.
At 3:28 p.m., a Steam Generator Isolation Signal (SGIS) was received when steam
generator pressure decreased below the setpoint, causing the main steam isolation
valves (MSIVs) to shut. This isolated steam flow through the TBVs. At approximately
nine minutes after the reactor trip, the operators transferred control of the ADVs to the
Auxiliary Shutdown Panel to remove the "quick open" signal which was still present to
the ADVs. Once ADV control was transferred to the Auxiliary Shutdown Panel, the
ADVs closed and the cause of the overcooling and depressurization event was
terminated. RCS pressure was 1523 psia and the RCS temperature was 486 degrees
Fahrenheit. Pressurizer level was below the bottom of the indicating range.
2
Enclosure
Due to post-trip reactor decay heat, the plant parameters began to restore to normal
post-trip values. At 3:39 p.m., pressurizer level instrumentation began to indicate a level
increase as expected.
At 3:55 p.m. the operators transitioned to EOP-1, Reactor Trip. With a SIAS signal still
present, all charging pumps were running and RCS letdown was isolated. The
operators took manual control of pressurizer main spray to control rising RCS pressure.
RCS pressure was stabilized at 2318 psia at approximately 3:56 p.m. At 4:01 p.m., the
operators removed 22 and 23 charging pumps from service because pressurizer level
was still increasing (264-inches). At 4:06 p.m., 21 charging pump was stopped. All
injection had been stopped at this point. At 4:08 p.m., the operators stopped the RCS
heat-up at 515 degrees Fahrenheit by modulating the ADVs, and RCS pressure began
to decrease uncontrollably.
The operators attempted to reset the SIAS signal to allow restoration of plant systems.
Of specific interest to the crew were the back-up pressurizer heaters needed to raise
RCS pressure to normal value and the RCS letdown control valves needed to lower
pressurizer level to the normal operating band. At 4:17 p.m., SIAS channel "A" was
reset from the control room. SIAS channel "B" would not reset from the control room.
The operators were able to reset the SIAS "B" signal at the Engineered Safety Features
Actuation System (ESFAS) cabinet located in the cable spreading room at 4:27 p.m.
At 4:45 p.m., RCS pressure was at 1782 psia, and the operators began heating the RCS
to return parameters to within the normal post-trip temperature band of 525-535 degrees
Fahrenheit. Back-up pressurizer heaters, RCS letdown, and 21 charging pump were
returned to service. Due to the large volume of subcooled water that had been added to
the pressurizer, RCS pressure continued to slowly decrease despite having all
pressurizer heaters in service. At 5:18 p.m., a second SIAS actuation was received at
approximately 1750 psia. At this point, RCS pressure stabilized at approximately 1745
psia for the next 30 minutes. To restore pressurizer heater capacity the operators
blocked and reset SIAS, and pressurizer pressure began to recover to its expected
value.
March 20, 2004, No. 11 Steam Generator Low Level Event (Unit 1)
During the Unit 2 refueling outage in April 2003, CCNPP identified a design deficiency
related to varistors installed in the feedwater regulating valve and feedwater regulating
bypass valve digital feedwater indicators. The same deficiency existed on Unit 1, in that
if a ground occurred on the 1Y09 or 1Y10 bus the potential existed to lose all AC power
to the Unit 1 digital feedwater control system. CCNPP planned for the removal of these
varistors and considered different compensatory measures to be used until that
replacement. Operations management did not want to use an interim fix based on the
risk involved in their implementation and decided the varistor vulnerability could be
controlled by limiting the work done on the 1Y09 and 1Y10 buses. Calvert Cliffs records
indicated that the initial work control strategy was effective in that work was prevented
on the buses through approximately August 2003; however, soon thereafter work on the
3
Enclosure
1Y09 and 1Y10 buses again became routine with no additional controls placed on the
work.
On March 20, 2004, maintenance technicians performed work on 1-ER-101, a 500KV
chart recorder in the control room. As part of that work, at 1:19 p.m., maintenance
technicians were installing the recorder into panel 1C29. As the recorder was being
installed, a power lead to the recorder became pinched between the recorder and its
case, and shorted to ground. This resulted in a large bang in the control room, as well
as a ground on 1Y09.
The technicians notified the control room operators of this issue, and the operators
reviewed their indications and controls for the operating units, and noted no immediate
concerns. The only noted abnormality at that time was that the 12 steam generator
digital feedwater back-up central processing unit had re-booted. Later review of saved
data also showed that the feedwater regulating valve controller had also re-booted.
Unknown to the operators, due to the ground on bus 1Y09 phase C, a potential of 208
volts existed from line to ground on phases A and B of 1Y09 and their associated
components. Due to the known design issue with the Dixson digital feedwater
indicators, this potential caused the Dixsons to attempt to reduce the voltage back to
120VAC through the varistors. The varistors tried to limit the voltage for approximately
19 minutes. At 1:39 p.m., due to the ground that still existed on 1Y09, the varistors
failed, causing a line to neutral fault. This caused the fuses in the digital feedwater
controls to open for the 1Y09 power feed. During this time period, the operators noticed
the flashing of digital feedwater components. The digital feedwater electrical power
feeds are designed to be protected by the use of an automatic bus transfer (ABT) switch
which shifts the input power source from 1Y09 (primary) to 1Y10 (backup) if a fault
appears on 1Y09. The ABT shifted from 1Y09 supply to the 1Y10 supply, but due to the
short existing down stream via the varistors, the fuses for the 1Y10 feed to the digital
feedwater controls also opened.
This resulted in the loss of the 11 digital feedwater ABT bus, which in turn resulted in
the deenergization of the 11 main and 12 backup CPUs for digital feedwater. The 11
FRV positioner selector solenoid, 1-SV-1111B failed to electrically shift to the A position,
because of mechanical binding in the solenoid valve. This resulted in a loss of signal to
the 11 FRV, which immediately began closing. Steam generator water levels dropped
quickly, resulting in a reactor trip at 1:40 p.m. Approximately eight seconds prior to the
trip, the operators had shifted 12 FRV to manual. Additionally during the last 15
seconds of the event, 11 SGFP tripped on high discharge pressure as a result of the 11
FRV closure.
Immediately after the trip the TBVs opened as a result of the quick open signal. They
shut when the quick open signal was clear but did not reopen to modulate TBV flow and
control RCS temperature. The operators placed the TBV controller in manual, but the
valves still remained closed. Since the TBVs were not operating, the ADV controller
was placed in manual and modulated to control the cool down. During the same time,
12 steam generator feed pump was still reacting to the loss of 11 steam generator feed
4
Enclosure
pump, increasing in speed and discharge flow. At approximately 20 seconds post trip,
12 FRV shut due to high 12 steam generator water levels, and 12 steam generator feed
pump tripped on high discharge pressure. When steam generator water levels
decreased to the AFAS setpoint, the system actuated and re-initiated feed to the steam
generators.
2.0
Equipment Failures and Root Causes
2.1
Unit 2 Reactor Regulating System Quick Open Circuit
a.
Inspection Scope
Following a Unit 2 reactor trip on January 23, 2004, the reactor regulating system (RRS)
Quick Open signal did not clear when the reactor coolant temperature dropped below
the setpoint. As a result, an uncontrolled cooldown of the reactor coolant system
occurred.
The inspectors reviewed the design of the RRS, ADVs, and TBVs, and interviewed plant
personnel to independently determine what occurred and evaluate the initiating causal
factors. The inspectors also reviewed the material history and maintenance activities
associated with the RRS system. The inspectors assessed CCNPPs root cause
analysis and corrective actions to evaluate the adequacy of CCNPPs conclusions and
actions.
b.
Findings
One self-revealing preliminary white finding and two inspector observations are
documented in this section. The observations were minor issues that were related to
the human performance cross-cutting area.
1.
Failure to Adequately Implement a Modification Design Review of the Reactor
Regulating System Quick Open Circuit
Introduction. A self-revealing event identified a Preliminary White finding because
CCNPP did not perform a modification design review, as required by station procedures.
A preliminary risk analysis determined the finding to be of low to moderate safety
significance, because the likelihood of core damage had increased due to an RRS relay
failure. Following a reactor trip on January 23, 2004, the ADVs and TBVs automatically
Quick Opened, as designed. However, the Quick Open signal did not clear when the
reactor coolant temperature dropped below the Quick Open setpoint, because of the
RRS relay failure. As a result, an uncontrolled cooldown of the reactor coolant system
occurred, which in turn caused a loss of the normal heat removal system.
Description. Following a reactor trip, the ADVs and TBVs automatically Quick Opened,
as designed. However, the Quick Open signal did not clear when the reactor coolant
temperature dropped below the setpoint of 557 degrees T-Avg. As a result, an
uncontrolled cooldown of the reactor coolant system occurred, which in turn initiated
5
Enclosure
SGIS and SIAS actuations. The SGIS actuation resulted in a main steam isolation valve
closure, which disabled the main feedwater pumps (high pressure feedwater supply)
and the TBVs, and caused a loss of the normal heat removal system.
CCNPP subsequently determined that the Quick Open signal failed to reset because an
RRS X-channel K-7 relay contact failed to open when the relay was de-energized. The
K-7 relay had last functioned properly, following a reactor trip, on May 28, 2003.
CCNPP determined that the degraded relay contacts would probably have failed to open
next time that the relay de-energized, following the May reactor trip. The inspectors
concluded that the degraded RRS relay left the plant susceptible to an over-cooling
event from May 28, 2003 to January 23, 2004.
The failed relay, along with two similar relays, were sent to an independent laboratory
for failure modes and effects analysis. The laboratory analysis report identified that the
failed relay contacts had extensive burning and pitting, consistent with electrical welding
of the contacts. In addition, the report stated that arcing had occurred, as evidenced by
burn marks inside the relay case, and that flash-over had deposited soot on one
adjacent contact. The laboratory concluded that the failure was due to "burning and/or
welding" of the contacts, but could not determine whether (1) inductive-kick, (2)
excessive load, or (3) a one-time event initiated the contact burning that lead to the
contact failure.
The CCNPP Root Cause Analysis Report (RCAR), "Failure of Atmospheric Dump Valve
Quick Open Override Relay (K-7)," concluded that the root cause was a poor design
practice during original plant construction and a subsequent 1992 modification (FCR 85-
0068). The RCAR conclusion was based on the following facts and reasoning:
The relay contacts were not rated for the actual circuit voltage; rated for 29 VDC,
but installed in a 125 VDC circuit, and therefore failed prematurely.
The relay (Allied Controls model MHJLO-12A) was a system interface device
between the RRS system (Combustion Engineering design scope) and the ADV
and TBV valve actuator 125 VDC circuit (Bechtel design scope).
Laboratory failure analysis attributed the failure to an over-current event.
The contacts had not reasonably reached end-of-life, and the steady state load
current was low, compared to the contact current rating (no excessive load).
Other than the failed (welded closed) contact pair, the relay was in good
mechanical and electrical condition.
In 1992, the ADV and TBV valve actuator circuits were modified (FCR 85-0068).
A second load was added into the circuit controlled by the K-7 relay contacts.
The contact ratings for the K-7 relay were not checked to verify that they were
adequate for the additional load.
The RRS modification FCR 85-0068 specified specific design reviews and analysis
requirements. The inspectors determined the specified requirements were not
adequately performed, in that a required electrical analysis for the added loading on the
existing ADV and TBV control circuit did not verify the K-7 contact ratings.
6
Enclosure
CCNPPs interim corrective actions replaced all K-7 relays in both Unit 1 and Unit 2, and
performed targeted reviews, based on function and risk, of similar system interfaces to
identify other underrated relay contacts. The interim action to install new K-7 relays
reduced the likelihood of a premature contact failure, until a plant modification could be
performed to eliminate the design deficiency. No other underrated relay contacts were
identified. CCNPPs longer term corrective actions included a modification to the K-7
circuit to restore proper contact ratings for the circuit.
The inspectors review of CCNPPs RCAR analysis identified several weaknesses:
The RCAR did not consider the third failure possibility that was identified in the
laboratory analysis report, as a "one-time event" which could have initiated the
contact burning that lead to the contact failure. Such a failure mode could have
resulted from a maintenance error during testing activities.
The RCAR did not discuss the testing and examination results of the other three
K-7 relays, which had operated under similar conditions and length of service
time (both Unit-1 K-7 relays and the second Unit-2 K-7 relay). Those relays were
bench tested for contact resistence; two relays appeared to have adequately low
contact resistence. The relays were also opened for an internal contact visual
examination. The inspectors examined those relay contacts and noted evidence
of excessive contact pitting on two relays.
The RCAR did not discuss apparent contradictions in contact rating information
provided by an Allied Controls relay applications engineer (original equipment
manufacturer), and an independent root cause review conducted by an outside
organization.
Overall, the inspectors concluded that while the CCNPP root cause determination was
not thorough, the proposed corrective actions appeared reasonable.
Old Design Issue Considerations
In the original plant design (1977), the Combustion Engineering RRS design provided
the K-7 relays as interface devices for a control signal output to the Bechtel designed
ADV and TBV valve control system. The inspectors concluded that the K-7 relay
contacts were underrated for their application since initial plant construction and startup.
NRC MC 0305, "Operating Reactor Assessment Program," Section 04.07 defines an
"Old Design Issue" as a finding that involved a past design-related problem in an
engineering analysis or installation of plant equipment (e.g., a modification), that does
not reflect a performance deficiency associated with an existing program or procedure.
MC 0305 section 06.06(a) provides guidance for the treatment of Old Design Issues,
and states that the NRC may refrain from considering safety significant findings if the
Old Design Issue satisfies the following criteria:
7
Enclosure
Licensee Identified.
Not likely to have been previously identified by on-going licensee efforts.
Self-revealing issues are not considered to be licensee identified. In addition, CCNPP
had a prior opportunity to identify this issue, during a 1992 modification to the ADV and
TBV control system. Therefore, because this design-related finding did not satisfy the
above criteria, it is not considered to be an Old Design Issue and is being treated similar
to any other inspection finding, in accordance with MC 0305-06.06(a). This guidance is
consistent with Section VII.B.3 of the NRC Enforcement Policy.
Analysis. This finding was a performance deficiency because CCNPP did not perform
an adequate design review, as required by station procedures, during a 1992
modification to the Quick Open circuit. Traditional enforcement does not apply because
the issue did not have any actual safety consequences or potential for impacting the
NRCs regulatory function and was not the result of any willful violation of NRC
requirements or CCNPP procedures. This finding affected the Mitigating Systems
cornerstone objective because it negatively impacted systems that are used to respond
to initiating events to prevent core damage. This finding was more than minor because
it was considered to be a precursor to a more significant event. If auxiliary feedwater
(AFW) had not maintained steam generator level, once-through cooling would have
been necessary to remove reactor decay heat. If once-through cooling and alternate
feed had both failed, the sequence could have proceeded to core damage.
The finding was evaluated in accordance with IMC 0609, Appendix A, "Significance
Determination of Reactor Inspection Findings for At-Power Situations," using Phase 1,
Phase 2, and Phase 3 significance determination process (SDP) analyze. The Phase 1
screening determined that a Phase 2 evaluation was required, because the finding
represented an actual loss of a risk significant function of a non-Technical Specification
equipment train that was classified as Maintenance Rule high safety significant, for
greater than 24 hours2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br />.
A fault exposure time of 240 days was used for non-ATWS initiating events. On May
28, 2003, a reactor trip occurred, and the RRS quick open circuit and ADVs/TBVs
functioned properly. Based on licensee review, the K-7 relay contacts would have failed
to open on the next relay actuation, following the May reactor trip. There were no other
demands or tests which would have demonstrated whether the quick open function was
operational from May 28, until the reactor trip on January 23, 2004, when the RRS relay
failure was identified by a self revealing event.
A fault exposure time of 28 days was used for ATWS initiating events. During the early
part of an operating cycle, the RCS negative temperature coefficient is not be large
enough (provide sufficient negative reactivity) to allow emergency boration (EB) to
shutdown the unit given an ATWS. Based on licensee information this condition would
last for 7 weeks if ADVs/TBVs operate properly and 11 weeks if ADVs/TBVs failed open.
From a Phase 2 perspective this would equate to a 4 week exposure time where EB
would not have functioned following startup from the May 28, 2003 trip.
8
Enclosure
The internal events Phase 2 analysis, for CDF and LERF, was conducted in
accordance with IMC 0609 Appendix A, using the Risk-informed Inspection Notebook
for Calvert Cliffs Nuclear Power Plant Units 1 and 2, revision 1, dated July 2, 2002 and
with Draft IMC 0609, Appendix H, Containment Integrity SDP, respectively. From a
Phase 2 perspective, the finding had low - moderate CDF safety significance and very
low LERF safety significance.
The Region I senior reactor analyst (SRA) conducted a Phase 3 Risk Assessment, to
refine the Phase 2 analysis and to incorporate external events for both CDF and
LERF. The analysis used an update Calvert Cliffs SPAR model, Rev 3i, dated
November 2001. The assumptions used were that a plant transient would result in an
over-cooling event which would have caused SGIS and SIAS actuations. A SGIS
actuation would close the MSIVs resulting in the loss of the main feedwater pumps as a
high pressure feedwater supply and the inability to remove decay heat with the TBVs.
The SIAS actuation would isolate the turbine building service water cooling system,
resulting in the loss of the CBPs as a low pressure feedwater supply and in a loss of
redundancy in instrument air supplies and in TDAFW room cooling.
The Phase 3 analysis determined that the finding represented low to moderate (WHITE)
CDF safety significance for internal and external initiating events. The internal events
analysis resulted in a CDF of approximately 2E-6 for the 240 day exposure period.
The dominant core damage sequence was a transients with successful reactor trip
followed by a loss of steam generator cooling and failure to initiate once through cooling
(Feed and Bleed). An ATWS was the second dominant sequence, given the increased
time that emergency boration would not be sufficient to shutdown the reactor with the
TBVs/ADVs failing open. The finding represented a very low LERF safety significance
because there were no SGTR sequences identified as part of the SPAR analysis. The
SRA reviewed the licensees risk assessment relative to external events, finding that
both fire and seismically induced transients contributed to the total CDF increase, but
not to a sufficient extent to increase the total risk above low - moderate risk significance.
Using similar assumptions to those used in the Phase 3 analysis CCNPP, using an
integrated internal and external initiating events PRA model, estimated the CDF safety
significance at approximately 7E-6 for the 240 days.
Enforcement. There were no violations of NRC regulatory requirements because the
reactor regulating system, atmospheric dump valves, and turbine bypass valves were
not safety-related. CCNPP entered this finding into their corrective action program as
IR4-025-059. (FIN 50-318/2004008-01, Failure to Adequately Implement
Modification Design Review of the Reactor Regulating System Quick Open
Circuit.)
2.
Administrative Procedure for Control of Maintenance Activities
The inspectors identified that CCNPP did not implement written procedures for the
control and revision of in-process maintenance work instructions. During the Unit 2
2003 refuel outage, MO 2-2002-01126 performed PM Checklist IPM56001, "Functional
9
Enclosure
Test of ADV and TBV Quick Open." That functional test was new, and had not
previously been performed. Extensive pen-and-ink changes were made to the written
work instructions, without utilizing the formal PM revision process, as required by MN-
10-102 section 5.1, "PM Change Process." MN-10-102 section 5.2.B(d) limited pen-
and-ink changes to "Administrative or Editorial" changes. The inspectors determined
that the changes made were technical in nature, in that they changed the method of
performing the test. The inspectors did not identify any additional similar examples, and
concluded that this appeared to be an isolated example of this behavior.
Technical Specification 5.4.1 required, in part, that written procedures shall be
established and implemented as recommended in NRC Regulatory Guide (RG) 1.33
Appendix A. RG 1.33 Appendix A, section 1.d, "Administrative Procedures," required
written procedures for procedure adherence and temporary changes. The inspectors
determined that this was a minor violation of regulatory requirements, because the
failure to adequately review and approve the Checklist procedure change did not, in this
instance, impact the final test results. CCNPP entered this issue into their corrective
action program as IR4-036-977.
3.
Maintenance Rule Classification & Monitoring of Quick Open Function
The Maintenance Rule (MR) basis document stated that the RRS control system (quick
open signal & modulating control signal to ADVs & TBVs) was classified as MR high
safety significant. The TBV and ADV valve control functions were scoped as part of the
main steam system. The TBVs were classified as a MR non-risk function. The ADVs
were classified as a MR high safety significant function. All functions were monitored for
reliability (functional failures) at the system level. Only the ADV function was monitored
for availability. The ADV and TBV functions were identified as "operate on demand"
(i.e., a standby function).
CCNPPs Loss of Normal Heat Removal analysis indicated that component failures
which resulted in a loss of the normal heat removal system were risk significant. The
inspectors noted that there was an apparent discrepancy between the MR classification
and the risk significance, as determined by the stations probability risk assessments.
NRC Regulatory Guide 1.160, "Monitoring the Effectiveness of Maintenance," endorses
NUMARC 93-01, and provided guidance for MR implementation. Both documents
stated that high safety significant functions and standby low safety significant functions
should have performance criteria established to assure reliability and availability are
maintained. The inspectors concluded that the Quick Open function (RRS, ADVs, and
TBVs) should reasonably have been monitored for both functional failures and
availability, at the train level. The inspectors reviewed the systems material history and
concluded that, prior to the January 2004 reactor trip, the Quick Open function would not
reasonably have required goal setting and monitoring under the MR (a)(1) requirements.
The inspectors concluded that, in this instance, not monitoring the Quick Open function
for availability was a minor issue. CCNPP entered this issue into their corrective action
program as IR4-035-902.
10
Enclosure
2.2
Unit 2 SIAS Actuation Signal Failure to Reset from Control Room
a.
Inspection Scope
Following a reactor trip on January 23, 2004, during the plant recovery phase of the
event, the control room operators were unable to reset the "B" channel of SIAS.
The inspectors reviewed the SIAS system design, testing, and surveillance program
elements, and interviewed plant personnel to independently determine what occurred
and evaluate the initiating causal factors. The inspectors assessed CCNPPs cause
determination and corrective actions to evaluate the adequacy of CCNPPs conclusions
and actions.
The inspectors reviewed selected maintenance and modification activities associated
with the SIAS system. The inspectors assessed post-maintenance and modification test
adequacy by comparing the test methodology to the scope of work performed. In
addition, the inspectors evaluated the test acceptance criteria to verify whether the test
demonstrated that the tested components satisfied the applicable design requirements.
The inspectors reviewed the recorded test data to determine whether the acceptance
criteria were satisfied.
b.
Findings
One self-revealing green finding and two inspector observations are documented in this
section. The observations were minor issues that were related to the human
performance cross-cutting area.
1.
Failure to Adequately Implement Modification Work Instructions for Wiring Terminations
Introduction. A self-revealing finding of very low safety significance (Green) identified
that maintenance procedures had not been adequately implemented to terminate a
wiring connection during a modification in the SIAS "B" channel reset circuit, in April
2003. As a result, during a plant event, recovery actions were delayed because
operators were unable to reset the "B" channel SIAS actuation from the control room.
This finding was related to the Human Performance cross-cutting area.
Description. In March 2003, CCNPP performed a modification on the SIAS reset
circuitry. Post-modification testing (PMT) consisted of point-to-point wiring checks to
verify that the newly installed or modified circuits conformed to design requirements. No
operational or functional test of the SIAS reset function was performed. CCNPP
determined that the reset function was not specifically credited in any design basis event
and not identified as a safety function in the FSAR or Technical Specification Basis.
Therefore, CCNPP concluded that logic system functional testing was not required to be
performed on the SIAS reset circuits.
Following a reactor trip in January 2004, a SIAS actuation occurred, per plant design.
Approximately 49 minutes into the event, operators attempted to reset the SIAS
11
Enclosure
actuation signal, to allow recovery of pressurizer level and pressure control. However,
"B" channel of SIAS could not be reset from the control room. At approximately 60
minutes into the event, an operator reset the "B" channel SIAS modules individually in
the cable spreading room.
During post-event troubleshooting, a visual inspection in control room panel 2C10
identified a wire hanging free in the air. The loose wire was determined to have come
off of hand-switch 2HS-6901 terminal 7, which was part of the daisy chain circuit for the
SIAS "B" channel reset. CCNPP determined that the wire probably pulled off the hand-
switch terminal during one of two maintenance activities, performed in the same panel in
close proximity to the affected hand-switch (MOs 2200302348 and 2200302349, to
replace containment air cooler fan control hand-switches), in August or October 2003.
The inspectors reviewed the modification installation work order 2-2000-00544,
"Remove SIAS Contacts from Containment Purge Isolation and Hydrogen Purge." The
inspectors were unable to evaluate the adequacy of the PMT because CCNPP had not
retained the test record data (see 2.3 below). The inspectors were initially unable to
determine whether the wire had mistakenly not been terminated during the modification
installation process, or whether the wire had subsequently come loose during adjacent
work in the panel. However, the inspectors determined that the SIAS panel wiring was
safety related and seismic category-1. Therefore, the inspectors concluded that CCNPP
did not properly terminate a wire to hand-switch 2HS-6901 during a previous
modification, and the subsequent quality inspection process failed to identify the faulty
termination.
Analysis. This finding was a performance deficiency because written work instructions
were not adequately followed. Traditional enforcement does not apply because the
issue did not have any actual safety consequences or potential for impacting the NRCs
regulatory function and was not the result of any willful violation of NRC requirements or
CCNPP procedures. This finding was more than minor because the SIAS system was
returned to service, following modification work, and subsequently became unable to
perform its function as a result of the deficiency, similar to example 5.b in NRC
Inspection Manual 0612 Appendix E, "Examples of Minor Issues." This finding affected
the Mitigating Systems cornerstone objective to ensure availability, reliability, and
capability of mitigating systems, because it was associated with the cornerstone
attributes for human performance.
This finding was determined to have very low safety significance, and screened out as
Green, using the NRC Significance Determination Process (SDP) Phase-1 screening
worksheet for NRC MC 0609 Appendix A, "Reactor Inspection Findings for At-Power
Situations." This finding had very low safety significance because the finding did not
represent an actual loss of a safety function, and was not potentially risk significant due
to an external initiating event. CCNPP entered this finding into their corrective action
program as IR4-025-652 and IR4-028-080.
12
Enclosure
A contributing cause of the finding was related to the Human Performance cross-cutting
area because maintenance technicians did not adequately implement written work
instructions.
Enforcement. Technical Specification 5.4.1 required, in part, that written procedures
shall be established and implemented as recommended in NRC Regulatory Guide (RG)
1.33 Appendix A. RG 1.33 Appendix A, section 9.a, "Procedures for Performing
Maintenance," required pre-planned maintenance activities be performed in accordance
with written procedures for maintenance that can affect the performance of safety
related equipment. CCNPP modification work order 2-2000-00544 required, in part, that
wires were properly terminated, in accordance with E-406, "Installation Standard - Main
Control Board Wiring."
Contrary to the above, on January 23, 2004, a self-revealing event identified that on
March 20, 2003, CCNPP did not adequately implement work order 2-2000-00544
instruction steps to terminate wiring added by the modification. Because this violation
was of very low safety significance, and CCNPP entered this finding into their corrective
action program (IR4-025-652 and IR4-028-080), this violation is being treated as a
non-cited violation (NCV), consistent with Section VI.A of the NRC Enforcement Policy.
(NCV 05000318/2004008-02, Failure to Adequately Implement Modification Work
Instructions for Wiring Terminations.)
2.
Previous Corrective Actions for Compression Style Terminations
In 1997, Unit 1 had a reactor trip (LER 50-317/1997-009) which was a direct result from
an improper compression style termination, on a similar, but non-safety related, control
panel hand-switch. Calvert Cliffs root cause analysis report (CCER 9701) had a
corrective action (IR1-053-484) to evaluate the use of a different type of termination
(such as a crimped lug, under the switchs termination screw), to replace the
compression style termination currently used. CCNPP engineering subsequently
determined that the compression style termination was more reliable (ES 1998-00394).
CCNPP's review of the current issue also contains a corrective action to evaluate use of
a non-compression style termination. The inspectors concluded that CCNPP may have
missed an opportunity to have prevented this failure.
3.
Control of Testing Scope and Acceptance Criteria
The SIAS reset circuit PMT consisted of wiring continuity checks, performed by ME-001,
"Wiring Verification." No functional or operational test of the SIAS reset circuit was
performed. ME-001 required that a supervisor verify the list of "circuits or drawings" to
be checked, prior to the wiring continuity checks. The inspectors identified that the
supervisor's "circuit identification" was designated by drawing number. The inspectors
determined that the designated drawings contained numerous circuits, cables, and
schemes, many of which were not involved with the modification. The test points, for
the point-to-point continuity checks, were not pre-planned and were only required to be
performed on the portion of the circuit that was new or modified. The selection of the
13
Enclosure
test points was considered a skill-of-the-craft attribute. The inspectors concluded that
allowing the maintenance technicians to select the test points may not test all aspects of
the modification that are required to be tested. In addition, the inspectors identified that
the test procedure (ME-001) did not require supervisory review or approval of the test
results.
10 CFR 50 Appendix-B Criterion XI, "Test Control," required, in part, that testing
required to demonstrate that systems will perform satisfactorily shall be performed in
accordance with written procedures which incorporate requirements and acceptance
limits. In the instance of the SIAS reset circuit modification, the ME-001 wiring
verification test was the only PMT performed. The inspectors determined that ME-001
did not contain adequate written requirements or acceptance criteria, because the
testing scope was designated at the drawing level, not the individual circuit or wire level.
The inspectors determined that this was a minor violation of regulatory requirements,
because the inspectors did not identify any instance where this performance deficiency
had impacted the final test results. CCNPP entered this issue into their corrective action
program as IR4-023-641.
2.3
Test Records for Safety Related Work Not Retained by Document Control
a.
Scope
The inspectors reviewed selected post-maintenance and post-modification test records
to evaluate whether the retained quality assurance records were adequate to verify that
the associated tests demonstrated that safety related systems could perform their
intended functions, after completion of maintenance or modification activities.
b.
Findings
Introduction. The inspectors identified a non-cited violation of very low safety
significance (Green) because CCNPP did not retain records of test results, as required
by 10 CFR 50 Appendix B, Criterion XVII, "Quality Assurance Records." Specifically,
CCNPP did not retain wiring verification point-to-point test records for modifications of
safety related circuits. As a result, after the records were transferred to Records
Management, verification of the work performed could not be done. This finding was
related to the Human Performance cross-cutting area.
Description. In February 2004, the inspectors were unable to independently verify
whether post-modification testing (PMT) had been adequately performed for
modification work order 2-2000-00544, "Remove SIAS Contacts from Containment
Purge Isolation and Hydrogen Purge." CCNPP Records Management did not retain
individual point-to-point wiring verification documents, as required by ME-001, "Wiring
Verification." Without the required test records, there was no documentation to identify
which circuits, wires, or schemes had been checked, and no documentation to identify
from where-to-where that the continuity checks had been performed.
14
Enclosure
For the SIAS reset circuit modification, no additional operational testing of the SIAS
reset function was performed. The PMT relied entirely on the point-to-point wiring
verification tests to verify SIAS reset circuit functionality. ME-001 section 6.3 required
schematic drawings to be highlighted with the actual circuit locations where the point-to-
point wiring checks had been performed. ME-001 section 7 required that all highlighted
drawings and data sheets be attached to the initiating maintenance work order. ME-001
section 9 required that the records generated by the procedure were to be identified as
permanent and retained for the lifetime of the plant.
CCNPP retained quality assurance safety related paper records by converting them into
electronic records through an optical imaging process. The inspectors identified that in
1999, a revision to a CCNPP Records Management checklist excluded the ME-001
highlighted drawings as records which were required to be optically imaged. The
marked-up drawings were highlighted in color, but the imaging system could not image
color.
Analysis. This finding was a performance deficiency because written procedure
instructions were not implemented. Traditional enforcement does not apply because the
issue did not have any actual safety consequences or potential for impacting the NRCs
regulatory function and was not the result of any willful violation of NRC requirements or
CCNPP procedures. This finding was more than minor because the failure to retain the
required records was not an isolated example, and the records were irretrievably lost,
similar to example 1.b in NRC Inspection Manual 0612 Appendix E, "Examples of Minor
Issues."
This finding was not suitable for an NRC Significance Determination Process evaluation,
but was reviewed by NRC management and determined to be of very low safety
significance (Green). This finding would have been considered as potentially greater
than very low safety significance if the missing test records had been associated with
inadequate testing that had been linked to subsequent equipment failures which
resulted in an event of greater than very low safety significance. CCNPP entered this
finding into their corrective action program as IR4-025-931.
A contributing cause of this finding was related to the Human Performance cross-cutting
area because station personnel did not adequately implement written instructions in a
safety related procedure.
Enforcement. 10 CRF 50 Appendix B, Criterion XVII, "Quality Assurance Records,"
required, in part, that sufficient records shall be maintained to furnish evidence of
activities affecting quality, including results of tests. The CCNPP Quality Assurance
(QA) Policy, revision 57, stated that the QA Program satisfied the requirements of
American National Standards Institute (ANSI) N18.7-1976 for administrative controls
and quality assurance of safety related plant activities. ANSI N18.7 section 5.2.17,
"Inspections," required that records shall be kept in sufficient detail to permit adequate
confirmation of the inspection program. CCNPP safety related procedure ME-001,
"Wiring Verification," section 7 and 9, required all highlighted drawings and records
generated by the procedure to be retained for the life of the plant. Contrary to the
15
Enclosure
above, on March 26, 2004, the inspectors identified that CCNPP had not retained test
records, as required by ME-001, since approximately 1999.
Because this violation was of very low safety significance, and CCNPP entered this
finding into their corrective action program (IR4-025-931), this violation is being treated
as a non-cited violation (NCV), consistent with Section VI.A of the NRC Enforcement
Policy. (NCV 05000318/2004008-03, Test Records for Safety Related Work Not
Retained by Document Control)
2.4
Unit 1 Digital Feedwater Design Deficiency
a.
Inspection Scope
The inspectors reviewed the design deficiency in the digital feedwater system and its
effect on the loss of feedwater to the 11 steam generator which resulted in a Unit 1
reactor trip on March 20, 2004. The inspectors reviewed the digital feedwater control
system design, interviewed plant personnel to independently determine what occurred
and evaluated the initiating causal factors. The inspectors assessed CCNPPs cause
determination and corrective actions to evaluate the adequacy of CCNPPs conclusions
and actions.
b.
Findings
Introduction. A self-revealing event identified a finding because CCNPP failed to
perform an adequate design review, as required by station procedures. This resulted in
reduced reliability of the digital feedwater system and subsequently caused the Unit 1
reactor trip on March 20, 2004.
Description. On March 20, 2004, during maintenance on the 1-ER-101, 500 KV Bus
Voltage chart recorder technicians created a short circuit on the C phase of
instrument bus 1Y09. Instrument bus 1Y09 is a three phase ungrounded AC system
and since this is a three-phase ungrounded system the voltage on the other phases (A
& B) increased. The normal AC power for the 11 steam generator digital feedwater
control system is supplied by the B phase from instrument bus 1Y09. The increase in
voltage on the B phase resulted in metal oxide varistor (varistor) failures in the digital
feedwater position indicators for the feedwater regulating valve. These failed varistors
acted as a short circuit and opened a fuse in the normal power supply to the digital
feedwater control processor. The power supply automatically transferred to the backup
AC power supply from instrument bus 1Y10, but since the failed varistors were still in the
circuit, the fuse from the backup power supply opened which removed both the normal
and backup AC power to the 11 steam generator digital feedwater control system. The
digital feedwater regulating valve failed to automatically transfer to DC control, which
resulted in closing the regulating valve, reducing the level in 11 Steam generator
resulting in a subsequent reactor trip.
16
Enclosure
Calvert Cliffs had previously determined that the design of the digital feedwater position
indicators for the feedwater regulating valve and bypass valves were not designed for
the higher voltages that a three-phase ungrounded system could sustain with one phase
shorted. Calvert Cliffs failed to correctly specify the power supply voltages on the digital
feedwater position indicators as required by ES-021, Design Input Requirements
Preparations. The design input requirement evaluation concluded that the power supply
for the position indicators, which include the varistors, were approximately 115 VAC.
However, since the power supply is a three-phase ungrounded system the varistors
could be expected to experience voltages as high as 208 VAC during a fault condition.
Therefore, under the conditions experienced, the varistors failed, which resulted in
reduced reliability of the feedwater digital control system and resulted in the reactor trip.
Analysis. This finding was a performance deficiency because CCNPP did not perform
an adequate design review, as required by station procedures ES-020, Impact Screens
for the Engineering Service Process, and ES-021, Design Input Requirements
Preparation. Calvert Cliffs did not properly identify and evaluate critical aspects of the
electrical power supply. Traditional enforcement does not apply because the issue did
not have any actual safety consequences or potential for impacting the NRCs
regulatory function and was not the result of any willful violation of NRC requirements or
CCNPP procedures. This finding affected the Initiating Events cornerstone objective
because it contributed to increasing both the likelihood of a reactor trip and the
likelihood that mitigating equipment or functions will not be available. This finding was
more than minor because it effects the design control attributes of the Initiating Events
cornerstone. Incorrectly specifying the design voltage resulted in reducing the reliability
of the digital feedwater control system which increased the likelihood of an event that
upset plant stability during power operation.
The finding was determined to be of very low safety significance (Green), using the NRC
Significance Determination Process (SDP). The Phase-1 screening determined that a
Phase-2 evaluation was required because the finding affected both the Initiating Event
and Mitigating Systems Cornerstones. In the Phase 2 risk analysis, the Transient and
Loss of Component Cooling Water (LOCCW) initiating events were reviewed, as
specified the Calvert Cliffs Plant Specific Risk Notebook. For Transients, the initiating
event frequency was increased by one order of magnitude, because the digital
feedwater control issue increased the likelihood of a loss of feedwater. For both the
Transient and LOCCW SDP worksheets, the mitigation credit for the power conversion
system (PCS) was decreased by one order of magnitude, because the feedwater
system was less redundant with only one feed path available. The dominant core
damage sequence was a transient, with successful reactor trip, followed by a loss of
steam generator cooling and failure to initiate once through cooling (Feed and Bleed).
The Phase-2 analysis determined that this finding was of very low safety significance
(Green), because the finding did not affect the auxiliary feedwater system, and one of
two turbine driven feedwater pumps remained available.
Enforcement. There were no violation of NRC regulatory requirements because the
affected equipment was not safety-related. Calvert Cliffs entered this finding into their
corrective action program as IR200400168. (FIN 50-317/2004008-04, Failure to
17
Enclosure
Adequately Implement a Modification Design Review of the Digital Feedwater
Control System)
3.0
Human Factors and Procedural Issues
3.1
Failure to Properly Implement Station Emergency Operating Procedures
a. Inspection Scope
The inspectors reviewed the licensed operator performance following the Unit 2 reactor
trip on January 23, 2004. Specifically, the inspectors looked at operator use of the
emergency operating procedures (EOPs) and previous related training provided to the
operators, and compared operator actions to procedural requirements.
b. Findings
Introduction. The inspectors identified a Green finding because CCNPP did not follow
procedural requirements in their implementation of EOP-0, Post-trip Immediate
Actions, and EOP-1, Reactor Trip. The operators mis-diagnosed plant conditions in
EOP-0 and incorrectly proceeded to EOP-1 rather than EOP-4, Excess Steam Demand
Event. Further, once in EOP-1, the operators failed to comply with the direction of that
procedure. These failures to follow station procedures complicated the plants post-trip
response and the ability of the operators to restore normal plant conditions.
18
Enclosure
Description
Procedure Implementation
Following the Unit 2 reactor trip on January 23, the control room operators entered
procedure EOP-0. This procedure is organized around critical safety functions which
must be satisfied when a reactor trip occurs, to ensure that the plant is placed in a
stable, safe condition or that the plant is configured to further respond to a continuing
casualty. When the ADVs and TBVs remained open following the trip, RCS
temperature, pressurizer pressure and pressurizer level deviated from the acceptance
values in EOP-0. While in EOP-0, operators attempted to recover these parameters to
the values expected after a normal reactor trip. As part of that effort, the operators
implemented steps of EOP-4 out of sequence in order to attempt to restore RCS
parameter values. Following the SIAS, the safety injection compounded these out-of-
sequence operator actions, and the pressurizer was overfilled with a large mass of cold
water. RCS pressure was then dominated by this large volume of cold water, not the
smaller-than-usual steam vapor bubble. The pressurizer bubble slowly lost its energy to
the colder water volume, and pressure began to decrease, leading to the second SIAS.
In addition, the operators also incorrectly exited EOP-0 to go to EOP-1 when both the
RCS Pressure/Inventory Safety Function and the Core/RCS Heat Removal Safety
Function were not met. The diagnostic flowchart in EOP-0 directs the use of EOP-1
only when all safety functions are met. In this case, with two safety functions not met,
EOP-0 directs the implementation of EOP-4. However, the operators should have been
able to correctly enter EOP-4 even after entering EOP-1. Procedure EOP-1 requires
that the diagnosis of an uncomplicated trip is correct by verifying the Safety Function
Status Checks Intermediate Acceptance Criteria are satisfied. If any parameter does
not satisfy the Intermediate Acceptance Criteria, and cannot be readily returned to within
the Acceptance Criteria, the operator should perform EOP-8, Functional Recovery
Procedure, or re-diagnose the event using the EOP-0 diagnostic flowchart and
implement the appropriate procedure. On January 23, 2004, the Unit 2 operators
identified that the RCS Pressure and Inventory parameters and the Core and RCS Heat
Removal parameters did not satisfy the safety function status checks for at least the first
12 intermediate checks, yet the operators remained in EOP-1 and did not re-diagnose
the event and did not implement the appropriate procedure, which in this case was
Also contributing to the improper use of the EOPs was the operators implementation of
the provisions of procedure NO-1-201, Calvert Cliffs Operating Manual. Section 5.1C
of that procedure allows deviation from controlling technical procedures to prevent
conditions directly adverse to personnel safety, plant safety, plant stability, or the safety
of the public. The procedure does not include a required condition that no other
approved procedure is available (as is required by 10CFR50.54(x) and ANSI Standard
N18.7/ANS 3.2-1976, Section 5.2.2). On January 23, operators used the provision of
NO-1-201 to deviate from EOP-0 and EOP-1 when in fact other procedures were
available to be used to mitigate this event.
19
Enclosure
Licensed Operator Training
Calvert Cliffs has increased the time allowed to execute EOP-0, to allow the operators to
concurrently implement procedure steps from other EOPs, without executing the entire
EOP. Calvert Cliffs allows this practice while in EOP-0, so that key plant parameters
can be restored to normal operating bands. This philosophy resulted in the operators
performing actions using knowledge-based skills as opposed to procedure-base skills
during high stress condition. This practice significantly increased the potential for
operator errors, and in the case of the January 23, 2004 event, it resulted in improper
transitions in the EOP procedures.
Analysis. This finding was a performance deficiency because the operating crew did not
follow procedural requirements in their implementation of EOP-0, Post-trip Immediate
Actions, and EOP-1, Reactor Trip. Traditional enforcement does not apply because
the issue did not have any actual safety consequences or potential for impacting the
NRCs regulatory function and was not the result of any willful violation of NRC
requirements or CCNPP procedures. The finding is greater than minor because it
affected the Human Performance attribute of the Mitigating Systems cornerstone
objective. This finding was determined to have very low safety significance, and
screened out as Green, using the NRC Significance Determination Process (SDP)
Phase-1 screening worksheet for NRC MC 0609 Appendix A, "Reactor Inspection
Findings for At-Power Situations." This finding had very low safety significance because
the finding did not represent an actual loss of a safety function, and was not potentially
risk significant due to an external initiating event. CCNPP entered this finding into their
corrective action program as IR4-025-167.
A contributing cause of the finding was related to the Human Performance cross-cutting
area because licensed operators did not properly implement station emergency
operating procedures.
Enforcement. Calvert Cliffs Technical Specification 5.4.1.b, states that written
procedures shall be established, implemented, and maintained covering the emergency
operating procedures required to implement the requirements of NUREG-0737. EOP-0
requires, when safety functions are not met following a reactor trip, the transition to the
appropriate optimal recovery procedure EOP. EOP-1 requires, when Safety Function
Status Checks Intermediate Acceptance Criteria are not satisfied, the return to EOP-0 or
the use of EOP-8.
Contrary to the above, on January 23, 2004, licensed operators at Calvert Cliffs Unit 2
improperly implemented EOP-0, Post-trip Immediate Actions, and EOP-1, Reactor Trip.
With several safety functions still not satisfied, the operators improperly went to EOP-1
from EOP-0 instead of proceeding to EOP-4 as required by EOP-0. Further, with Safety
Function Status Checks Intermediate Acceptance Criteria not satisfied, the operators
improperly remained in EOP-1 instead of returning to EOP-0 to re-diagnose the event.
Because this violation was of very low safety significance, and CCNPP entered this
finding into their corrective action program (IR4-025-167), this violation is being treated
20
Enclosure
as a non-cited violation (NCV), consistent with Section VI.A of the NRC Enforcement
Policy. (NCV 05000318/2004008-05, Failure to Properly Implement Station
Emergency Operating Procedures)
3.2
Failure to Have Procedures Required by Regulatory Guide 1.33
a.
Inspection Scope
The inspectors reviewed CCNPP procedures following the Unit 2 reactor trip on January
23, 2004. Specifically, the inspectors looked at procedures available to operators for the
use of the Reactor Regulating System (RRS) and for contingency measures for use
upon that systems failure.
b.
Findings
Introduction: The inspectors identified a Green finding because CCNPP did not have
any procedural guidance for the failure of the RRS during their implementation of EOP-
0, Post-trip Immediate Actions. Upon the reactor trip and the subsequent failure of the
K7 relay in the X channel of the RRS, if the operators had switched to the alternate Y
channel of RRS, the ADVs and TBVs would have properly controlled the RCS
temperature and terminated the uncontrolled cooldown event.
Description: Calvert Cliffs Unit 2 did not have procedures specifically designed to
combat the malfunction of the RRS. Although the station had procedural direction on
how to change RRS channels, that direction was contained in the precautions of
operating instruction OI-7, Reactor Regulating System, rather than an alarm response
procedure or abnormal operating procedure used to combat a malfunction. The RRS is
designed to provide control functions for the steam dump valves (TBVs and ADVs) for
quick opening to remove stored energy upon a reactor trip, using RCS T-ave and
turbine trip as initiating signals. This quick open feature provides for automatic control
of RCS T-ave and pressurizer pressure following a reactor trip. On January 23, when
the RRS failed and incorrectly kept the TBVs and ADVs open, Unit 2 operators did not
have proper procedural guidance direct their response to the RRS malfunction event,
forcing them to improperly use parts of different EOPs to combat the resulting off-
normal RCS T-ave and pressurizer pressure and level parameters.
Analysis: This finding was a performance deficiency because the CCNPP is required to
have a written procedure to combat the malfunction of a pressure control system.
Traditional enforcement does not apply because the issue did not have any actual safety
consequences or potential for impacting the NRCs regulatory function and was not the
result of any willful violation of NRC requirements or CCNPP procedures. The finding is
greater than minor because it affected the Procedure Quality attribute of the Mitigating
Systems cornerstone objective. This finding was determined to have very low safety
significance, and screened out as Green, using the NRC Significance Determination
Process (SDP) Phase-1 screening worksheet for NRC MC 0609 Appendix A, "Reactor
Inspection Findings for At-Power Situations." This finding had very low safety
21
Enclosure
significance because the finding did not represent an actual loss of a safety function,
and was not potentially risk significant due to an external initiating event. CCNPP
entered this finding into their corrective action program as IR4-018-361.
Enforcement: Calvert Cliffs Technical Specification 5.4.1.a, states that written
procedures shall be established, implemented, and maintained covering the applicable
procedures recommended in Regulatory Guide 1.33, Revision 2, Appendix A, February
1978. That Regulatory Guide specifies procedures for abnormal, offnormal, or alarm
conditions and procedures for combating emergencies and other significant events,
including the malfunction of a pressure control system.
Contrary to the above, on January 23, 2004, licensed operators at Calvert Cliffs Unit 2
did not have a specific procedure to respond to the abnormal condition created by the
malfunction of the RRS, a pressure control system. Because this violation was of very
low safety significance, and CCNPP entered this finding into their corrective action
program (IR4-018-361), this violation is being treated as a non-cited violation (NCV),
consistent with Section VI.A of the NRC Enforcement Policy. (NCV 05000318/2004008-
06, Failure to Have Procedures Required by Regulatory Guide 1.33)
3.3
Simulator Fidelity Issues
a.
Inspection Scope
Through operator and training department interviews, and through licensee document
review, the inspectors reviewed the licensed facilitys control room simulator and its
ability to accurately reproduce the events of January 23, 2004. Specifically, the
inspectors looked at the simulator model replication of the RCS pressure and
temperature transients observed that day by the control room operators.
b.
Findings
No findings of significance were identified.
Observations
Calvert Cliffs identified a difference between the plants RCS pressure and pressurizer
temperature parameter response when the Unit 2, January 23 reactor trip was replicated
on the plant specific simulator. Calvert Cliffs validates plant trips on their simulator to
verify simulator fidelity. The validation of the January 23 reactor event, including the
reactor trip and post-trip equipment and operator actions, revealed substantial
differences in actual and predicted RCS pressure and temperature during the initial
depressurization and subsequent repressurization. Specifically, after the TBVs and
ADVs were shut, a plant heat-up occurred, and pressurizer level and RCS pressure
increased in both the plant and the simulator. Operator action was required in the plant
to stop the RCS pressure increase, but no operator action was required in the simulator
to stop and control the RCS pressure. This contributed to the operators not fully
22
Enclosure
comprehending the behavior of pressurizer parameters, particularly the pressurizer
bubble-water mass energy interface.
Calvert Cliffs identified the cause of the fidelity issue as deficiencies in the original
simulator software model that was installed in 1985. The licensee identified that the
discrepancy between plant and simulator responses was due to three simulator model
coefficient deficiencies. The Calvert Cliffs plant specific simulator did not correctly
replicate RCS pressure and pressurizer temperature for this excess steam demand
event. The differences between the actual plant response and the response previously
experienced by the operators in their simulator training confused the operators and
complicated the recovery of plant parameters following the Unit 2 reactor trip. However,
enforcement action was not pursued due to the apparent cause of the issue being an
original design issue, the identification of which was beyond the scope of expected and
required testing.
3.4
Failure to Comply with Station Work Control Procedures
a.
Inspection Scope
The inspectors reviewed CCNPP work control practices and decision-making processes
which led up to the reactor trip on March 20, 2004, at Unit 1. Specifically, the inspectors
interviewed licensed operators, managers, and maintenance staff who had been
involved in the decisions to defer the replacement of vulnerable components in the
digital feedwater system, those who had been involved in the scheduling of electrical
work during the week of March 14, and those who were immediately involved on March
20. The inspectors reviewed CCNPP work control procedures and compared CCNPP
actions against those procedural requirements and expectations.
b.
Findings
Introduction. This self-revealing event identified a Green finding because CCNPP did
not follow procedural requirements in their risk assessment and control of the work that
was performed March 20, 2004, on Unit 1. Specifically, the provisions and controls of
procedures NO-1-100, Conduct of Operations, NO-1-117, Integrated Risk
Management, and MN-1-100, Conduct of Maintenance, were not followed. These
failures to follow station procedures impacted the Initiating Events cornerstone in that
the failure to properly classify and control the work in the control room on March 20 lead
to an unanticipated reactor trip.
Description. In February 2004, a maintenance order was generated to repair the 500KV
Bus Voltage Recorder 1ER-101 because the chart paper was not advancing. This piece
of equipment is owned by the Generation Protections and Test Unit (GPTU), and that
group was scheduled to be onsite March 20 to perform other site work. Based on that
scheduling, the work on the recorder was placed on that weeks work schedule as
emergent work. Calvert Cliffs records and NRC interviews of CCNPP staff show that
the NO-1-117 work control/risk assessment process was implemented during the week
of 3/14, yet a number of barriers failed or were bypassed:
23
Enclosure
1)
No one involved in the scheduling or performance of the 1ER-101 recorder work
recognized that the work involved the 1Y09 bus, and that the work should have
gotten additional management attention, per the prior management decision
made last year. The exact nature of the intended 1Y09/10 work prohibition or
control was not clear due to the lack of documentation.
2)
No one involved in the scheduling or performance of the 1ER-101 recorder work
recognized that the work involved a trip sensitive area as defined in NO-1-100.
Therefore, the additional preparation and oversight required by NO-1-100 were
not implemented. The inspector also identified that the current revision of NO-1-
100 directs the requirements of MN-1-124, Conduct of Integrated Work
Management, be followed for work in trip sensitive areas, yet those
requirements in essence had been relocated to NO-1-117 without a change to
the NO-1-100 procedure direction.
3)
The risk assessments performed by the responsible group supervisor (RGS) and
outage work control (OWC) in accordance with NO-1-117 incorrectly rated the
1ER-101 work as low risk. Due to the work being performed in a trip sensitive
area and due to the previously identified vulnerability of the feedwater control
system to work involving the 1Y09 bus, this work should have been rated as
medium risk. The additional controls associated with a higher risk assessment
were not implemented.
4)
The actual 1ER-101 work was not performed in accordance with the
expectations of MN-1-100, in that the pre-job briefing committed to using STAR,
supervisory oversight, and peer checks as defenses to prevent errors. The
failure to implement these practices directly led to the pinching of the power
supply wire during the reinstallation of the 1ER-101 and the creation of the
ground which initiated the reactor trip event.
When work was done on the 1ER-101 recorder, a piece of equipment powered by the
1Y09 bus, the lack of increased oversight and control of the work allowed a ground to be
created, and all AC power to the Unit 1 digital feedwater control system was lost. This
failure, combined with a latent failure in the DC power control system, led to an actual
partial loss of feedwater event and a Unit 1 reactor trip.
Analysis. This finding was a performance deficiency because CCNPP did not properly
risk-assess the work planned for March 20 nor did they put in place the proper
precautions for the work, as required by station procedures. Traditional enforcement
does not apply because the issue did not have any actual safety consequences or
potential for impacting the NRCs regulatory function and was not the result of any willful
violation of NRC requirements or CCNPP procedures. This finding was more than
minor because the combination of procedural non-compliances was considered to be a
precursor to a more significant event, and it affected the Significance Determination
Process Initiating Event Cornerstone by being a transient initiator contributor (i.e.,
directly led to an unplanned reactor trip). The finding screened to Green because the
procedural non-compliances did not contribute to a LOCA initiator, contribute to the
24
Enclosure
likelihood mitigation equipment would not be available, or increase the likelihood of a fire
or flood.
A contributing cause of the finding was related to the Human Performance cross-cutting
area because CCNPP managers and staff did not properly implement station
operations, risk management, and maintenance procedures.
Enforcement. No violation of regulatory requirements occurred. The inspectors
determined that the finding did not represent a noncompliance because it occurred on
non-safety-related secondary plant equipment. CCNPP entered this finding into their
corrective action program as IR4-028-774. (FIN 50-317/2004008-07 Failure to Comply
with Station Work Control Procedures)
4.0
4.1
Failure to Recognize and Report an Unusual Event During the January 23, 2004, Unit 2
a.
Inspection Scope
The inspectors reviewed the CCNPP implementation of the emergency plan during the
January 23, 2004, Unit 2 reactor trip. The inspectors conducted interviews with licensed
senior reactor operators (SRO), reactor operators (RO) and station support staff. The
inspector reviewed station procedures and industry guidance and then compared the
operators performance to station procedures and industry guidance documents.
b.
Findings
Introduction. The inspectors identified a non-cited violation of very low safety
significance (Green) because the CCNPP operating staff and post trip review did not
recognize that plant conditions required an emergency level classification (Unusual
Event) in accordance with station procedures. As a result, CCNPP did not correctly
implement the station emergency plan as required by 10CFR50.54(q). This finding was
related to the Human Performance cross-cutting area.
Description. During the Unit 2 reactor trip and subsequent failure of the Reactor
Regulating System, the ADVs and the TBVs failed open for approximately nine minutes.
This resulted in an uncontrolled RCS cooldown which emptied the pressurizer, resulted
in an automatic SIAS actuation, and a SGIS isolation. This excessive cooldown event
was partially terminated by the SGIS isolation which isolated the TBVs and was finally
terminated when the operators regained local control of the ADVs.
During this event the operators identified that the open TBVs and ADVs were not
responding as designed. However, when evaluating plant conditions, the operators did
not correctly conclude that the cause of the conditions (failed open TBVs and ADVs)
were unexplained. Since the operators did not understand why the valves had failed to
25
Enclosure
reclose they should have concluded that the condition was unexplained. This would
have resulted in a determination that plant conditions met the entry conditions for EOP-
4, Excessive Steam Demand Event, and required an Unusual Event classification.
The CCNPP emergence action level (EAL) classification matrix specified that EOP-4,
Excess Steam Demand Event is Implemented, met the criteria for an Unusual Event.
Unexplained lowering of one or both steam generator pressure are entry conditions into
EOP-4, Excessive Steam Demand Event. These plant conditions were present, and
Station procedure NO-1-201, Calvert Cliffs Operating Manual, requires classification
of an event when the plant condition meets the emergency classification criteria.
Therefore, the licensed operators should have reasonablely identified that an Unusual
Event classification was required.
During the post-trip review, CCNPP concluded that an Unusual Event declaration was
not required because EOP-4, Excessive Steam Demand Event, entry conditions were
not met. The review concluded that the source of the excessive steam demand was the
failed open TBVs and ADVs, and because the cause of the excessive steam demand
was known, the entry condition for EOP-4 was not met. Therefore, this review
concluded that EOP-4 was not entered and no Unusual Event declaration was required.
The post trip review incorrectly concluded that the cause of the failed open TBV and
ADVs was explained. The failed open TBVs and ADVs were not responding as
designed and met the entry conditions for EOP-4. Therefore, these plant conditions
were commensurate with an Unusual Event. During the week of January 17, 2004, the
inspectors discussed the Unusual Event classification with CCNPP. On May 28, 2004,
CCNPP completed a 1-hour report in accordance with 10 CFR 50.72(a)(1)(i) for this
event.
Analysis. The inspectors determined that this finding was a performance deficiency
because CCNPP operating staff did not recognize plant conditions commensurate with
an Unusual Event and therefore, the plant operating staff did not declare an Unusual
Event nor did the post-trip review identify the Unusual Event. Traditional enforcement
does not apply because the issue did not have any actual safety consequences or
potential for impacting the NRCs regulatory function and was not the result of any willful
violation of NRC requirements or CCNPP procedures. This finding was more than
minor because it effects the response organization performance attribute of the
Emergency Preparedness Cornerstone in that failure to recognize plant conditions
indicative of an Unusual Event resulted in not identifying the Unusual Event. The finding
was assessed using MC 0609, Appendix B, Emergency Preparedness Significant
Determination Process, sheet 2, Actual Event Implementation Problem. The finding
was determined to be of very low safety significance (Green) because the operators
failed to identify Unusual Event conditions during an actual plant event.
This finding is related to the Human Performance cross-cutting area because reactor
operators and the Unit 2 post trip review did not recognize plant conditions
commensurate with an Unusual Event and did not report the Unusual Event prior to
prompting by the NRC.
26
Enclosure
Enforcement. This was a violation of 10CFR50.54(q) which states in part that licensees
shall follow their emergency plans. The CCNPP Emergency Response Plan, Section
4.0, Emergency Measures, states "Emergency Response Plan Implementing
Procedures contain procedures and guidance for accident assessment and emergency
classification." ERPIP-3.0, Immediate Actions, Attachment 2 entitled "Emergency
Classification" directs operators to evaluate plant conditions against EAL Criteria.
Specifically, EAL QU5 states the conditions for an unusual event: "EOP-4, Excess
Steam Demand Event is implemented." NO-1-201, "Calvert Cliffs Operating Manual"
states "During EOP-0, should it become apparent that an EAL condition is met, the
classification should begin right away without waiting for the next EOP to be
implemented."
Contrary to the above, the operating crew, in response to the January 23, 2004, reactor
trip event, did not recognize plant conditions that were commensurate with an Unusual
Event. The CCNPP post-event review did not identify that an Unusual Event should
have been identified based on plant conditions. This violation has been entered in
CCNPP corrective action program as IR4-023-606 and is being treated as a non-cited
violation (NCV), consistent with Section VI.A of the NRC Enforcement Policy. (NCV
05000318/2004008-08, Failure to Recognize an Unusual Event During the Unit 2
5.0
Cross Cutting Aspects of Findings
Section 2.2 describes a finding where maintenance technicians did not adequately
implement written work instructions. As a result, during a plant event, recovery actions
were delayed because operators were unable to reset the "B" channel SIAS actuation
from the control room.
Section 2.3 describes a finding where station personnel did not adequately implement
written instructions in a safety related procedure to maintain records that can be used to
determine the scope of work performed on safety related components. The failure to
retain the required records, as required by ME-001, since approximately 1999 resulted
in the records being irretrievably lost
Section 3.1 describes a finding where licensed operators did not properly implement
station emergency operating procedures. These actions resulted in the operators
performing actions using knowledge-based information as opposed to skill-based
information. This resulted in the operators selecting what actions should be performed
out of station procedures based on plant indications, under high stress conditions which
significantly increased the potential for operator errors, and in the case of the January
23, 2004 event, it resulted in improper transitions in the EOP procedures.
Section 3.4 describes a finding where CCNPP managers and staff did not properly
implement station operations, risk management, and maintenance procedures. This
resulted in a precursor to Unit 1 reactor trip on March 20, 2004.
27
Enclosure
Section 4.1 describes a finding where licensed operators and the post-trip review failed
to recognize plant conditions commensurate with an Unusual Event during the January
23, 2004, Unit 2 excessive steam demand event.
6.0
Generic Issues
During this inspection, no significant issues were identified requiring the issuance of
generic communications to the nuclear industry.
7.0
Risk Significance of the January and March 2004 Events
The team conducted an initiating event assessments and concluded that each event
resulted in a moderate risk significance conditional core damage probability (CCDP)
(between E-6 and E-5 per event). These risk assessments were conducted using the
NRCs standardized plant analysis risk (SPAR) model for Calvert Cliffs. The model was
updated to reflect the licensees operating experience and procedures. The licensee
also performed an initiating event assessments for these events and reached similar
conclusions.
Unit 2 January Reactor Trip
The team concluded that this event resulted in a conditional core damage probability
(CCDP) in the mid E-6 range. The following assumptions were used:
A general plant transient occurred due to a partial loss of feedwater.
The failure of the K-7 relay in the reactor regulating system resulted in the
turbine bypass valves and the atmospheric dump valves remaining full open and
not modulating to control reactor plant parameters, which resulted in an
uncontrolled cooldown of the reactor coolant system. As a result, steam
generator isolation and safety injection actuation signals were generated to
mitigate the event.
The dominant accident sequences for this transient event were: 1) failure of steam
generator cooling and Failure of once through core cooling; 2) failure of the reactor
protection system to shutdown the reactor and failure to limit reactor coolant system
pressure; and 3) failure of the reactor coolant pump seals and failure of high pressure
recirculation.
Unit 1 March Reactor Trip
The team concluded that this event resulted in a CCDP in the low E-6 range. The
following assumptions were used:
A general plant transient occurred due to a partial loss of feedwater. The main
feedwater flow path to SG11 was unavailable due to the digital feedwater control
system failure.
A-1
Enclosure
Attachment
Turbine bypass valves fast opened but failed to stay open to control pressure.
This resulted in the need to use the atmospheric dump valves.
The dominant accident sequences for this transient event were: 1) failure of steam
generator cooling and Failure of once through core cooling; 2) failure of the reactor
coolant pump seals and failure of high pressure recirculation; and 3) failure of the
reactor protection system to shutdown the reactor and failure to limit reactor coolant
system pressure.
8.0
Overall Adequacy of Licensee Response
The team concluded that the overall response of CCNPP to the January 23, 2004, Unit
2 reactor trip and the March 20, 2004, Unit 1 reactor trip were adequate because the
plants were taken to a safe shutdown condition. However, during both events the
operators were challenged by equipment malfunctions that were the result of less than
adequate design review and maintenance work. In addition, the operators had
problems implementing emergency operating procedures during the Unit 2 reactor trip.
The Unit 1 trip was the result of not correctly implementing work management control
procedures. CCNPP had an opportunity in both cases to previously address the issues.
CCNPPs corrective actions regarding equipment and procedures for this event have
been appropriate. However, the CCNPP identification and reporting of the Unusual
Event condition was delayed and had to be prompted by the NRC. These events show
the need for improvements in the human performance areas.
9.0
Exit Meeting Summary
The NRC presented the results of this special inspection to Mr. George Vanderheyden,
and other members of CCNPP management on June 18, 2004, via conference call.
Calvert Cliffs management acknowledged the findings presented. No proprietary
information was identified.
ATTACHMENT A
SUPPLEMENTAL INFORMATION
KEY POINTS OF CONTACT
Licensee Personnel:
T. Roberts, Supervisor - Electrical & Control Systems
M. McMahon, Supervisor - FIN Team
C. Yoder, Engineer - Electrical & Control Design
R. Stark, Senior Engineer - Electrical & Control Design
H. Winters, System Engineer
D. Lenker, Supervisor - Electrical and I&C Design
J. Kilpatrick, Senior Engineer - 50.59 Program
H. Daman, General Supervisor - Electrical and I&C Maintenance
A-2
Attachment
R. Simmons, Supervisor - I&C Maintenance
S. Collins, System Engineer
E. Roach, Lead Assessor
NRC Personnel:
M. Giles, Senior Resident Inspector - Calvert Cliffs
R. Fuhrmeister, Senior Reactor Inspector
A. Della Greca, Senior Reactor Inspector
E. McKenna, NRR
LIST OF ITEMS OPENED, CLOSED AND DISCUSSED
Opened
Failure to Adequately Implement Modification Design
Review of the Reactor Regulating System Quick Open
Circuit (Section 2.1.b.1)
Closed
Failure to Adequately Implement Modification Work
Instructions for Wiring Terminations (Section 2.2.b.1)05000318/2004008-03
Test Records for Safety Related Work Not Retained by
Document Control (Section 2.3)05000317/2004008-04
Failure to Adequately Implement a Modification Design
Review of the Digital Feedwater Control System (Section
Failure to Properly Implement Station Emergency
Operating Procedures (Section 3.1)05000318/2004008-06
Failure to Have Procedures Required by Regulatory Guide
1.33 (Section 3.2)05000317/2004008-07
Failure to Comply with Station Work Control Procedures
(Section 3.4)05000318/2004008-08
Failure to Recognize an Unusual Event During the Unit 2
Reactor Trip (Section 4.1)
A-3
Attachment
LIST OF DOCUMENTS REVIEWED
Section 2.1 Reactor Regulating System Quick Open Circuit Failure
Issue Reports
IR4-025-059
IR4-028-151
Work Orders
MO 2-2004-00291
Design Bases
CCNPP Quality Assurance Policy, revision 57
FSAR Section 7.4.1, "Reactor Regulating System"
Procedures
Checklist IPM56001, "Functional Testing of ADV and TBV Quick Open"
Drawings
86924SH0001X, "Unit 2 RRS Channel-X Schematic"
86924SH0001Y, "Unit 2 RRS Channel-Y Schematic"
86924SH0002X, "Unit 2 RRS Channel-X Schematic"
86924SH0002Y, "Unit 2 RRS Channel-Y Schematic"
63069, "Turbine Steam Dump and Bypass Control Schematic"
12017-0101, "RRS Block Diagram"
12132-0050, "RRS Reactor Program Unit Calculator Function and Wiring Diagram"
86-922-E, "RRS Test Panel Schematic Wiring Diagram"
Other Documents
Modification Package FCR 85-0068
System Descriptions, System 56 and 83A
System Health Report, System 56 and 83A
Maintenance Rule Scoping Document, System 56 and 83A
Main Steam System Risk Significant Components Report, revision 0, dated 03/19/1998
NUMARC 93-01, revision 2, "Monitoring the Effectiveness of Maintenance"
Section 2.2 SIAS Actuation Signal Failure to Reset from Control Room
Issue Reports
IR4-025-652, "ESFAS SIAS "B" Would Not Reset from the Control Room"
IR4-028-080, "Signal for SIAS Reset is Not Tested from Control Room"
IR4-022-941, "Unexpected Second SIAS Actuation"
A-4
Attachment
Work Orders
MO 2200000544, "Remove SIAS Contacts from Ctmt Purge Iso & H2 Purge"
MO 2200400304, "Check for Open Switch Contacts"
MO 2200302348, "Replace 23 CAC Hand Switch 2HS5301"
MO 2200302349, "Replace 24 CAC Hand Switch 2HS5302"
Design Bases
E-406, "Installation Standard - Main Control Board Wiring"
NRC Generic Letter 1996-01, "Testing of Safety Related Logic Circuits"
RG 1.187, "Guidance for Implementation of 10 CFR 50.59"
Procedures
ME-001, revision 0, "Wiring Verification"
STP-M-220B-2, "Engineered Safety Features Actuation System Channel ZE Functional Test"
PR-1-101, revision 20, "Preparation and Control of Technical Procedures"
EN-1-102, revision 8, "10 CFR 50.59 / 72.48 Reviews"
ES-017, revision 5, "10 CFR 50.59 Reviews"
Drawings
63076SH0042, "Schematic Diagram for Containment Vent & Hydrogen Purge"
63059, "Schematic Diagram for ESFAS"
63059A, "Schematic Diagram for ESFAS"
87310SH00002, "Wiring Diagram for Panel 2C10"
Other Documents
System Description No. 048, revision 2, "Engineered Safety Features Actuation System"
System Health Report for Systems 48 and 52
NEI 1996-07, revision 1, "Guidelines for 10 CFR 50.59 Implementation"
Section 2.3 Safety Related Test Records Not Retained
Issue Reports
IR4-025-652, "ESFAS SIAS "B" Would Not Reset from the Control Room"
IR4-028-080, "Signal for SIAS Reset is Not Tested from Control Room"
Work Orders
MO 2200000544, "Remove SIAS Contacts from Ctmt Purge Iso & H2 Purge"
MO 2200400304, "Check for Open Switch Contacts"
Design Bases
E-406, "Installation Standard - Main Control Board Wiring"
CCNPP Quality Assurance Policy, revision 57
ANSI N18.7-1976, "Administrative Controls and Quality Assurance for the Operational Phase of
Nuclear Power Plants"
Procedures
A-5
Attachment
ME-001, revision 0, "Wiring Verification"
Drawings
61036, Rev. 62, "Schematic Diagram 208/120V Instrumentation Buses 11 & 12 Unit #1"for
Containment Vent & Hydrogen Purge"
63059, "Schematic Diagram for ESFAS"
63059A, "Schematic Diagram for ESFAS"
87310SH00002, "Wiring Diagram for Panel 2C10"
Section 2.4 Unit 1 Digital Feedwater Design Deficiency
Issue Reports
IR200400168, March 20, 2004, Unit 1 Reactor Trip During Scheduled Maintenance
Procedures
ES-020, Rev. 10, Impact Screens for the engineering Service Process
ES-021, Rev. 04, Design Input Requirements (DIR) Preparation
EN-1-100, Rev. 16, Engineering Services Process Overview
EN-1-101, Rev. 06, Design Change and Modification Implementation
Analysis
ESP-199602497, Rev. 01, dated 11/27/2001
ESP-200001092. Rev. 03, dated 8/27/01
Drawings
61036, Rev. 62, "Schematic Diagram 208/120V Instrumentation Buses 11 & 12 Unit #1"
Simplified Drawing - Digital Feedwater Power Distribution System
Simplified Drawing - Digital Feedwater Power Scheme, Unit One Post 2002 RFO
Section 3.1
Failure to Properly Implement Station Emergency Operating Procedures
Issue Reports
IR200400053 23 Steam Generator Feed Pump Trip Resulting in Unit 2 Plant Trip, Root Causal
Analysis (Issue Report IR4-028-786)
IR200400056, Unplanned SIAS Actuation Following Unit 2 Trip, Root Causal Analysis (Issue
Report IR4-025-167)
IR4-025-059, Unit 2 Atmospheric Dump Valves Appear to Have Erroneous Open Signal From
RRS Channel X
IR4-025-165, -166 & -167, POSRC Recommendations During 1/24/04 Post-Trip Review
Meeting
IR4-028-786, Unit 2 Trip: Loss of 22 SGFP Resulted in Low SG Level Trip
A-6
Attachment
Procedures
AOP-7K, Overcooling Event in Mode One or Two (Unit 2), Rev. 2
AOP-7K, Overcooling Event in Mode One or Two, Basis Document (Unit 1 & 2), Rev. 1
EOP-0, Post-Trip Immediate Actions (Unit Two), Rev. 8
EOP-0, Post-Trip Immediate Actions Technical Basis Document, Rev. 14
EOP-1, Reactor Trip (Unit Two), Rev. 12
EOP-4, Excess Steam Demand Event (Unit Two), Rev. 15
NO-1-100, Conduct of Operations, Rev. 23
NO-1-111, Post-Trip Review, Rev. 6
NO-1-200, Control of Shift Activities, Rev. 29
NO-1-201, Calvert Cliffs Operating Manual, Rev. 15
Training Materials
Lesson Plan LOI-201-0-9, EOP-0 Post-Trip Immediate Actions Scope and Basis for the
Licensed Operator Initial Training Program
Lesson Plan LOR-63-1-02, ESFAS for the Licensed Operator Requalification Program
Lesson Plan LOR-201-0, 4-S-0-2, EOP-0, -1, and -4 Simulator Exercises for the Licensed
Operator Training Program
Lesson Plan LOR-201-0-8-02, EOP Basis Review for the Licensed Operator Training Program
Lesson Plan LOR-348-1-04, Calvert Cliffs Operating Experience, With Thermodynamic Review,
Unit 2 Trip 1/23/04
Other Documents
Unit 2 Control Room Operator Logs for January 23, 2004
Unit 2 Equipment Control Logs for January 23, 2004
Section 3.2
Failure to Have Procedures Required by Regulatory Guide 1.33
Design Bases
Regulatory Guide 1.33, Quality Assurance Program Requirements (Operation)
Calvert Cliffs - Unit 2 Technical Specifications
Procedures
OI-7, Reactor Regulating System, Rev. 9
NO-1-100, Conduct of Operations, Rev. 23
Training Materials
Reactor Regulating System Description No. 56, Rev. 0
LOI-58-1-13, Reactor Regulating System for the License Operator Initial Training Program
Simulator Operating Examinations for the Licensed Operator Training Program (various)
A-7
Attachment
Section 3.3
Simulator Fidelity Issues
Issue Reports
IR200400066, Simulator Fidelity Deficiency Root Causal Analysis (Issue Report IR4-020-078)
Procedures
AOP-7K, Overcooling Event in Mode One or Two (Unit 2), Rev. 2
Training Materials
Simulator Requal Session V, Scenario 03-03, Rapid Downpower with Expeditious Return to Full
Power
Lesson Plan LOR-348-1-04, Calvert Cliffs Operating Experience, With Thermodynamic Review,
Unit 2 Trip 1/23/04
Section 3.4
Failure to Comply with Station Work Control Procedures
Issue Reports
IR200400168, March 20 Unit 1 Reactor Trip Root Causal Analysis (Issue Report IR4-028-774)
IR4-000-887, Potential for Loss of AC Power to SG Level Control if Ground Occurs on 1Y09/10
IR4-028-847, 500KV Bus Voltage Recorder (1ER-101) Chart Paper Does Not Advance
Maintenance Orders
MO 1200400687, Replace 500KV Sensitive Voltage Recorder 1ER-101
Procedures
MN-1-100, Conduct of Maintenance, Rev. 22
MN-1-124, Conduct of Integrated Work Management, Rev. 6
NO-1-100, Conduct of Operations, Rev. 23
NO-1-117, Integrated Risk Management, Rev. 11
NO-1-200, Control of Shift Activities, Rev. 29
NO-1-201, Calvert Cliffs Operating Manual, Rev. 15
PR-1-103, Use of Procedures, Rev. 4
Section 4.0 Emergency Preparedness
Issue Reports
IR4-023-606, Event 4092, Reactor Trip on 1/23/04 Met the Conditions of ERPRP UE Due to
Excessive Steam Demand for 9 Minutes Following the Trip.
Procedures:
NO-1-201, Calvert Cliffs Operating Manual
EOP-4, Rev. 15, Excess Steam Demand Event
EOP-4 Basis Document
Calvert Cliffs EAL Technical Basis Manual, Rev. 10
A-8
Attachment
Other:
Timely Declaration of an Emergency Class, Implementation of an
Emergency Plan, and Emergency Notification.
Failure of Licensed Senior Operators to Classify Emergency
Events Properly.
A-9
LIST OF ACRONYMS
ABT
Automatic Bus Transfer
ADV
Atmospheric Dump Valve
Auxiliary Feedwater Actuation System
ANSI
American National Standards Institute
Alarm Response Card
Apparent Violation
Condensate Booster Pump
Conditional Core Damage Probability
Calvert Cliffs Nuclear Power Plant
Core Damage Frequency
CR
Condition Report
Control Room Supervisor
Condensate Storage Tank
Delta Core Damage Frequency
Emergency Operating Procedure
Feedwater Regulating Valve
Final Safety Analysis Report
GPTU
Generation Protection and Test Unit
LER
Licensee Event Report
Maintenance Rule
Non-Cited Violation
Non-Licensed Operator
NRC
Nuclear Regulatory Commission
OWC
Outage Work Control
Post Modification Test
Plant Operations Review Committee
Quality Assurance
RCAR
Root Cause Analysis Report
[NRC] Regulatory Guide
RGS
Responsible Group Supervisor
Reactor Regulating System
Steam Generator Feedwater Pump
[NRC] Significance Determination Process
SGIS
Steam Generator Isolation Signal
Safety Injection Actuation Signal
Stop, Think, Act, Review
TS
Technical Specification
Unusual Event
Updated Final Safety Analysis Report
B-1
ATTACHMENT B
SPECIAL INSPECTION TEAM CHARTER
February 5, 2004
MEMORANDUM TO:
Richard J. Conte, Team Manager
Division of Reactor Safety
Alan J. Blamey, Team Leader
Special Inspection
FROM:
Wayne D. Lanning, Director
Division of Reactor Safety
SUBJECT:
SPECIAL INSPECTION CHARTER - CALVERT CLIFFS
NUCLEAR POWER PLANT UNIT 2
A special inspection has been established to inspect and assess the automatic reactor trip that
occurred at Calvert Cliffs Unit 2 on January 23, 2004. The special inspection will be conducted
onsite during the week of February 17, 2004, to allow Calvert Cliffs to complete the root cause
analysis for this event. The apparent cause review was completed prior to restart of the unit on
January 25, 2004. The team will include:
Manager:
Richard J. Conte, Chief, Operational Safety Branch
Leader:
Alan Blamey, Senior Operations Engineer
Members:
John Richmond, Resident Inspector at Susquehanna
Steve Barr, Operations Engineer
Herb Williams, Operations Engineer
Mark Giles, Senior Resident Inspector at Calvert Cliffs - Part Time
Eugene Cobey, Senior Reactor Analyst - Part Time
Unit 2 was operating at 100% power when the 22 steam generator feed pump spuriously
tripped. Operators were unable to reset and restart the pump, so they manually tripped the
reactor. Just prior to the manual trip, the reactor automatically tripped due to low steam
generator water level. The steam dump valves (atmospheric and condenser) fast opened as
designed, but did not modulate to properly control RCS temperature. Operators had difficulty
stabilizing RCS pressure such that a safety injection actuation signal occurred a second time
during the first two hours after the reactor trip. Additionally, operators were unable to reset the
safety injection actuation signal from the main control board, and some of the pressurizer back-
up heaters were previously removed from service due to a leaking pressurizer spray valve, a
known problem. The basis for this inspection is to independently evaluate equipment and
human performance, and to assess Constellations root cause evaluation and corrective
actions.
B-2
This special inspection was initiated in accordance with NRC Inspection Procedure 71153
Event Follow-up and NRC Management Directive 8.3, NRC Incident Investigation Program.
The decision to perform this special inspection was based on the initial risk assessment
coupled with the various complications that occurred following the trip. The inspection will be
performed in accordance with the guidance of NRC Inspection Procedure 93812, Special
Inspection, and the inspection report will be issued within 30 days following the exit meeting for
the inspection. If you have any questions regarding the objectives of the attached charter,
please contact me at (610) 337-5191.
Attachment:
Special Inspection Charter
Distribution:
H. Miller, RA/J. Wiggins, DRA
J. Trapp, DRP
M. Giles, DRP
N. Perry, DRP
E. Cobey, DRS
J. Jolicoeur, RI EDO Coordinator
R. Laufer, NRR
R. Clark/P. Tam, PM, NRR (Backup)
W. Lanning, DRS
R. Crlenjak, DRS
B. Holian, DRP
D. Screnci, ORA
B-3
Special Inspection Charter
Calvert Cliffs Nuclear Power Plant Unit 2
Automatic Reactor Trip - With Equipment and Potential Human
Performance Problems
The objectives of the inspection are to verify the facts and assess the issues surrounding the
automatic reactor trip that occurred at Calvert Cliffs Nuclear Power Plant Unit 2 on January 23,
2004. Specifically the inspection should:
1.
Independently evaluate the equipment and human performance issues to assess the
adequacy of the scope of Constellations investigation. This evaluation will:
Assess the adequacy of Constellations investigation and root cause evaluation
of the circumstances surrounding the cause of the automatic reactor trip and the
post trip response from the perspective of the equipment performance and
human performance.
Assess the adequacy of Constellations plans for corrective actions and extent of
condition review for the equipment and human performance issues.
2.
Independently evaluate the quality of operator response, and their implementation of
procedures, including Emergency Operating Procedures.
3.
Assess the adequacy of testing activities, prior to the event, to verify equipment
operability after maintenance or modification activities.
4.
Assess the adequacy of programs (i.e., workarounds, configuration control, corrective
action program) to address known equipment issues.
5.
Independently evaluate the risk significance of the event.
6.
Assess the effectiveness of related simulator and training issues.
7.
Document the inspection findings and conclusions in a special inspection report in
accordance with Inspection Procedure 93812 within 30 days of the exit meeting for the
inspection.
B2-1
ATTACHMENT B2
REVISED SPECIAL INSPECTION TEAM CHARTER
April 13, 2004
MEMORANDUM TO:
Richard J. Conte, Team Manager
Division of Reactor Safety
Alan J. Blamey, Team Leader
Special Inspection
FROM:
Wayne D. Lanning, Director
Division of Reactor Safety
SUBJECT:
SPECIAL INSPECTION CHARTER - CALVERT CLIFFS
NUCLEAR POWER PLANT UNIT 2 - SUPPLEMENTAL TO
INCLUDE UNIT 1 TRIP
A special inspection had been established to inspect and assess the automatic reactor trip that
occurred at Calvert Cliffs Unit 2 on January 23, 2004. As a result of a comparable conditional
core damage probability, and similar deterministic factors related to operator performance, the
Calvert Cliffs Unit 1 reactor trip of March 20, 2004, is now included in your teams review.
The special inspection will be conducted onsite on or about May 10, 2004, on a not to interfere
basis with the Unit 1 startup from a refueling outage scheduled to start April 9, 2004. This is to
allow Constellation Energy to complete the root cause analysis for Calvert Cliffs Unit 1
March 20, 2004, event. The apparent cause review was completed prior to restart of the unit on
March 22, 2004. The team remains the same per the previous charter but implementation
should be with less resources and less scope. For example since the post trip response for
Unit 1 was less severe than Unit 2, the main focus of the Unit 1 review will be the 20 minutes
before the trip when digital feedwater controls adversely responded to the loss of an instrument
bus.
Unit 1 was operating at 100% power when technicians inadvertently grounded an instrument
when reinstalling an instrument recorder. The grounding effected digital steam generator
feedwater control system, eventually causing a loss of feedwater and an almost simultaneous
automatic and manual reactor trips. Operators were unable to manually control the turbine
bypass valves and the main condenser was lost for 10 hours1.157407e-4 days <br />0.00278 hours <br />1.653439e-5 weeks <br />3.805e-6 months <br />. Just prior to the trip signal, the
operators dealt with diagnosing the effect of the ground on digital feedwater control. No safety
injection signals occurred and the post trip response was relatively benign, but there is a
question on how procedures were implemented in the 20 minutes prior to the trip. The risk for
this event increased due to the loss of the main condenser. The basis for this inspection is to
independently evaluate equipment and human performance, and to assess Calvert Cliffs root
cause evaluation and corrective actions.
B2-2
This special inspection was initiated in accordance with NRC Inspection Procedure 71153
Event Follow-up and NRC Management Directive 8.3, NRC Incident Investigation Program.
The decision to perform this special inspection was based on the initial risk assessment
coupled with our knowledge of preliminary performance deficiencies for the Unit 2 trip of
January 2004 and some uncertainty on how procedures were implemented in the 20 minutes
prior to the Unit 1 trip. The inspection will be performed in accordance with the guidance of
NRC Inspection Procedure 93812, Special Inspection, and the inspection report will be issued
within 30 days following the exit meeting for the inspection. If you have any questions
regarding the objectives of the attached charter, please contact me at (610) 337-5126.
Attachment:
Special Inspection Charter
Distribution:
H. Miller, RA/J. Wiggins, DRA
J. Trapp, DRP
M. Giles, DRP
N. Perry, DRP
E. Cobey, DRS
J. Jolicoeur, RI EDO Coordinator
R. Laufer, NRR
R. Clark/P. Tam, PM, NRR (Backup)
W. Lanning, DRS
R. Crlenjak, DRS
B. Holian, DRP
D. Screnci, ORA
R. Conte, DRS
A. Blamey, DRP
B2-3
Special Inspection Charter - Supplemental
Calvert Cliffs Nuclear Power Plant Unit 1
Automatic Reactor Trip - With Equipment and Potential Human
Performance Problems
The objectives of the inspection are to verify the facts and assess the issues surrounding the
manual and automatic reactor trip that occurred at Constellations Calvert Cliffs Nuclear Power
Plant Unit 1 on March 20, 2004. Specifically, the inspection should:
1.
Independently evaluate the equipment and human performance issues to assess the
adequacy of the scope of Constellations investigation. This evaluation will:
Assess the adequacy of Constellations investigation and root cause evaluation
of the circumstances surrounding the cause of the automatic reactor trip and the
post trip response from the perspective of the equipment performance and
human performance.
Assess the adequacy of Constellations plans for corrective actions and extent of
condition review for the equipment and human performance issues.
2.
Independently evaluate the quality of operator response, and their implementation of
procedures, including the hierarchal implementation of Emergency and Abnormal
Operating Procedures along with Alarm Response Procedures.
3.
Assess the adequacy of maintenance activities, prior to the event, to verify if an
adequate assessment of trip potential and risk was conducted prior to the instrument
recorder work.
4.
Independently evaluate the risk significance of the event.
5.
Assess the effectiveness of related training issues.
6.
Document the inspection findings and conclusions in a special inspection report in
accordance with Inspection Procedure 93812 within 30 days of the exit meeting for the
inspection.
C-1
Attachment
ATTACHMENT C
UNIT 2 SEQUENCE OF EVENTS
Unit 2 January 23, 2004 Excess Steam Demand Event
15:26.02
Initial Conditions
100% Reactor Power. 24 CWP secured for planned maintenance. RTCBs 1&5
open due to problems experienced earlier in the day during the performance of
an IM STP Reactor Reg System selected to Channel X.
15:26.37
22 SGFP Trips (With direction from the CRS, the CRO attempts multiple resets
of the 22 SGFP per plant stabilizing actions IAW AOP-3G. None of the resets
are successful and the CRS orders a manual reactor trip when S/G Low Level
Pre-Trips are received (coincident with -40 S/G levels per narrow range level
indication).)
15:27.48
RPS Steam Generator Low Level Channel A & D Trip. RTCBs 2, 3, 4, 6, 7, 8
open. RPS manual reactor trip from 1C05 due to action of RO.
15:28.20
ADVs and TBVs are not responding as designed as they are still full open and
RCS average temperature is well below 557oF.
15:28.26
All pressurizer backup and proportional heater banks automatically secure due to
pressurizer level falling below 101". The RO places all heater hand switches in
OFF shortly afterwards.
15:28.34
AFAS B actuation. ESFAS SIAS A & B actuation.
15:28.52
2B EDG, 21 & 22 LPSI pumps, 21 & 22 CS pumps, 21 HPSI pump all start.
15:28.53
22 Component cooling pump starts, 23 HPSI pump all start.
15:28.54
21 & 22 Boric acid pumps, 21/22/23 IRU, 24 CAC Fan all start.
15:28.57
ESFAS SGIS A & B Actuation.
15:28.59
Letdown secured. 2A & 2B EDG start.
15:29.00
21 & 22 MSIVs shut (with the MSIVs shut due to the SGIS actuation, the TBVs
are no longer contributing to the excess steam demand event. For approximately
the next seven minutes the RCS continues to cooldown at a rate of
approximately 160oF/hr.
15:29.13
Pressurizer level goes off-scale low.
C-2
Attachment
15:32.15
21B & 22A RCP secured in accordance with RCP Trip Strategy for SIAS
actuation.
15:37.00
The Quick Open Dump Signal from RRS is removed from both ADVs when the
TBO shifts the hand transfer valves in the 45 switchgear room to align ADV
control to 2C43. Over the next 32 minutes, an RCS heatup at approximately
57oF/hr takes place until RCS cold leg temperatures are restored to 515oF.
15:39.50
Pressurizer level returns to scale
15:47.30
The operating crew reduces AFW flow to each S/G from 300gpm to 150gpm.
Summary of EOP-O, Post Trip Immediate Actions:
Safety Function Status
Reactivity Control - Complete
Vital Auxiliaries - Complete
RCS Pressure and Inventory Control - Not Met
Core and RCS Heat Removal - Not Met
Containment Environment - Complete
Rad Levels External to Containment - Complete
Safety System Actuations
AFAS - Verified
SIAS - Verified
SGIS - Verified
15:55.00
EOP-1, Reactor Trip, is implemented from EOP-0. Upon entry, the crew
recognizes the high RCS pressure and the rapidly rising pressurizer level and
prepares to take stabilizing actions.
15:56.00
The RO takes manual control of the Main Spray Controller, 2HIC100, (which has
been greatly reduced due to only having one RCP operating in the spray line
loops) and places the output at approximately 30-35% to stop the RCS pressure
rise at 2335 psia. Subsequent minor manual Main Spray Controller
manipulations results in a stable RCS pressure at around 2318psia. Note - the
main spray valves, 1CV100E and 1CV100F, did not start to open until 2300psia
(based on a pressurizer controller setpoint of 2250 psia).
15:58.00
Due to the insurge from the RCS heatup, along with approximately 4100 gallons
of injection from the Charging system, Pressurizer level has reached ~210 and
the Pressurizer temperature has reached a minimum value of 514°F (saturation
for 771 psia).
15:59.00
The Pressurizer insurge continues as full Charging is still present at 128 GPM
and the 57°F/hr RCS heatup continues. At this point, due to the large volume of
cold water in the Pressurizer and the lack of full heater capability, RCS
pressure begins to rapidly drop from ~2318 to ~1800 psia over the next 22
minutes.
C-3
Attachment
16:01.46
22 & 23 charging pump are secured (H/S placed in PTL).
16:05.00
Based on Operator recall, the Main Spray Controller, 2HIC100, output signal is
lowered from 30 - 35% to approximately -2% (although 2HIC100 can be driven
to an output as low as -20%, an output of 0% should represent a signal at which
both Main Spray valves are full shut).
16:06.50
21 Charging pump is secured.
16:08.00
The RCS heatup is temporarily secured per the operating crews decision to hold
RCS cold leg temperature at 515oF.
16:09.00
Based on Operator recall, both Pressurizer Proportional Heaters are returned to
AUTO and Backup Heaters 22 and 24 are placed in ON. Backup Heater 24 only
has a capacity of 225 KW (normal capacity is 300 KW) due to a previous CMF
that had one bank of heaters removed from service. Backup Heaters 21 and 23
can not be returned to service at this time due to the active SIAS signals.
16:17.28
SIAS A is reset remotely from the Control Room. SIAS B can not be reset from
the Control Room due to a problem with the reset pushbutton.
16:27.36
SIAS B is reset locally from the Cable Spreading Room.
16:33.35
21 Charging Pump is started per OI-2A in an effort to restore Letdown to restore
Pressurizer level. For approximately the next five minutes, the Operating Crew
attempts to restore Letdown, but problems associated with the Control Room
position indication for one of the Letdown isolation valves, 2-CV-516, delays the
successful restoration.
16:38.50
21 Charging Pump is secured when the Operating Crew believes that the
Letdown isolation valve, 2-CV-516, is not opening when attempts are made
using the hand switch.
16:39.00
Based on Operator recall, Pressurizer Backup Heaters 21 and 23 are restored
and placed in ON now that SIAS has been reset and both heater breakers have
been closed locally.
16:45.30
A second heatup of the RCS at approximately 35°F/hr is commenced to return
RCS cold leg temperatures to the EOP-1 acceptable range of 525 - 535°F. The
heatup and resulting Pressurizer insurge contributes to RCS pressure lowering
from ~1800 psia to ~1750 psia over the next 30 minutes. The combination of
Letdown and the RCS heatup result in the RCS Pressure lowering to 1750 psia
and a second SIAS actuation.
16:48.29
21 Charging pump is started per OI-2A in a second effort to restore Letdown to
restore pressurizer level.
C-4
Attachment
16:48.40
Letdown is successfully placed in service and raised to approximately 105gpm
over the next nine minutes.
16:57.23
Letdown is maintained between 100 & 115gpm until about 17:14.34.
17:04.00
Per CRS/SM direction, the RO lowers the Main Spray Controller, 2HIC100,
output signal to -20% (lowest possible output signal) to ensure that the Main
Spray valves are fully closed in an attempt to minimize any leakby on the valves.
17:14.34
Letdown flow is reduced to ~70 GPM as the Operating Crew recognizes that
RCS pressure is steadily lowering and re-approaching the SIAS setpoint.
17:18.01
ESFAS SIAS B actuation (lose capability to use pressurizer backup heater 23).
17:18.02
ESFAS SIAS A actuation (lose capability to use pressurizer backup heater 21).
17:20.53
21 charging pump is secured.
17:49.00
After using procedure guidance from EOP-4 and blocking SIAS, the Operating
Crew resets SIAS A remotely from the Control Room. The decision to block and
reset SIAS is made in order to recover full Pressurizer heater capability in an
attempt to restore RCS pressure which has remained between 1750 and 1780
psia for the previous 50 to 60 minutes.
17:53.29
SIAS B is reset locally from the Cable Spreading Room.
17:58.00
Based on Operator recall, Pressurizer Backup Heaters 21 and 23 are restored
and placed in ON now that SIAS has again been reset and both heater breakers
have been closed locally. The Operating Crew now has full Pressurizer heater
output. The Operating Crew decides to not attempt to reinitiate Charging and
Letdown until RCS pressure reaches 2100 psia in order to assure that another
RCS depressurization does not occur.
18:22.00
Based on Operator recall, the Main Spray Controller, 2HIC100, is returned to
automatic control.
18:25.00
SGIS is reset using guidance from EOP-3.
18:29.20
AFAS A & B are reset in accordance with OI-32B.
18:32.29
21 Charging pump is started in preparation for restoring letdown.
18:33.35
Charging and Letdown is restored in attempt to return Pressurizer level to the
EOP-1 acceptable band of 130 to 180. Letdown is established at approximately
45 - 50 GPM.
19:26.00
The Operating Crew exits EOP-1 and implements OP-2 and OP-4.
C-5
Attachment
19:30.00
The 21B and 22A RCPs are restarted in accordance with OI-1A. The 21 AFW
pump is secured.
19:50.00
Both MSIVs are reopened in accordance with OP-2.
19:55.00
Secured 21 AFW pump.
20.00.00
RCS parameters have reached normal post-trip levels and are considered
steady state.