ML20217Q021
| ML20217Q021 | |
| Person / Time | |
|---|---|
| Site: | Pilgrim |
| Issue date: | 08/15/1997 |
| From: | Conte R NRC OFFICE OF INSPECTION & ENFORCEMENT (IE REGION I) |
| To: | Boulette E BOSTON EDISON CO. |
| References | |
| 50-293-97-02, 50-293-97-2, NUDOCS 9708290112 | |
| Download: ML20217Q021 (3) | |
See also: IR 05000293/1997002
Text
9
k
,
August 15,1997
E. Thomas Boulette, PhD
Senior Vice President - Nuclear
Boston Edison Company
Pilgrim Nuclear Power Station
600 Rocky Hill Road
Plymouth, Massachusetts 02360-5599
SUBJECT:
INSPECTION REPORT 50-293/97-02
Dear Mr. Boulette:
This letter refers to your June 20,1997 and July 11,1997 letters which responded to a
Notice of Violation concerning two examples of inadequate electrical design control as
documented :n NRC Inspection Report 50 293/97-02, dated May 22,1997.
Thank you for informing us of your corrective actions. We have reviewed this matter in
accordance with NRC Inspection Manual Chapter 92903, " Follow-Up - Engineering." Your
corrective actions taken and planned seem appropriate to address the design control
inadequacies experienced with the 480/120 voltage regulating transformers and MO-130153,
RCIC r.ystem full flow test valve. The corrective actions will be inspected during future
inspections.
We do note that the unexpected shutdown of the voltage regulating
transformers, which were installed in 1992, occurred previously due to improper control of
breaker trip settings. Both inadvertent shutdowns of the voltage regulating transformers
placed the plant outside of the plant's design basis and revaaled not only inadequacies with
electrical design control, but also inadequacies with pre-operational testing.
In light of your response and corrective actions on programmatic control of vendor interfaces
being weak, the NRC plans to conduct an increased inspection focus at your facility. This
increase would include a review of the licensing and design bases information.
Your continued cooperation with us is appreciated.
Sincerely,
Original Signed By:
i
Richard J. Conte, Chief
Projects Branch 8
3
Division of Reactor Projects
i
i
Docket No. 50-293
License No. DPR-35
oGq0E8
ffh.h.lf lf
.v
l f'- Ol
9708290112 970815
ADOCK 05000293
G
-
)
.-. __ _
_
__.
_. ._ _ _ _ _ __ __ _ _
_
__ _
_ _ .. _ __
.
,
E. Thomas' Boulette, PhD
2
.
I
,
cc w/o cv of Licensee's Resoonse Letter
L. Olivier, Vice President - Nuclear and Station Director
T. Sullivan, Plant Department Manager
N. Desmond, Regulatory Relations
D. Tarantino, Nuclear Information Manager
cc w/cv of Licensee's Resoonse Letter:
<
R. Hallisey,' Department of Public Health, Commonwealth of Massachusetts
The Honorable Therese Murray
The Honorable Joseph Gallitano
B. Abbanat, Department of Public Utilities
Chairman, Plymouth Board of Selectmen
Chairman, Duxbury Board of Selectmen
Chairman, Nuclear Matters Committee
Plymouth Civil Defense Director
P. Gromer, Massachusetts Secretary of Energy Resources
J. Miller, Senior issues Manager
i
J. Fleming
A. Nogee, MASSPIRG
Office of the Commissioner, Massachusetts Department of Environmental Quality
Engineering
Office of the Attorney General, Commonwealth of Massachusetts
- T. Rapone, Massachusetts Executive Office of Public Safety
Chairman, Citizens Urging Responsible Energy
Commonwealth of Massachusetts, SLO Designee
,
4
$
e
e
. - .
- e-
' .
,
_
E. Thomas Boulette, PhD
3
'
Distribution w/cv of Licensee's Resoonse:
Region i Docket Room (with concurrences)
PUBLIC
Nuclear Safety Information Center (NSIC)
'
NRC Resident Inspector
R. Conte, DRP
M. Conner, DRP
C. O'Daniell, DRP
P. Milano, NRR
A. Wang, NRR
W.' Dean, OEDO
R. Correia, NRR
F. Talbot, NRR
'
DOCDESK
' Inspection Program Branch, NRR (IPAS)
DOCUMENT NAME: g:\\ branch 8\\pg\\nov.res
T2 vece4ve e sepy of this document,inacate in the boa: 'C' = Copy without attachinent/ enclosure
- E' = Copy with attachment / enclosure
'N' = No copy
' OFFICE.
Al/DRP
/.
/
l
, ,
NAME
RConte . /ff)/
DATE
08/14/97
7-
08/ /97
08/ /97
08/ /97
08/
17
OFFICIAL RECORD COPY
l
.
______________ _ __ _ _ _ -.
,-4
.
,
'
e
.
so
= -~.
Pdgren Nuclear Power station
. Rocky Hdi Road
Ptymouth. Massachusetts 02360
July 11,- 1997
BECo Ltr. 2.97.073
Senior Vice PresKlent - Nuclear
- U.S. Nuclear Regulatory Commission
Document Control Desk
Washington, D.C. 20555
License DRP-35
Docket 50 293
SUBJECT:
SUPPLEMENTAL REPLY TO NOTICE OF VIOLATION (REFERENCE NRC
INSPECTION REPORT NO. 50-293/97-02)
Enclosed is Boston Edison Cornpany's supplemental reply to the Notice of Violation contained in
the subject inspectior report.
In the initial reply to the Notice of Violation (97-02-02), documented in BECo letter 2.97.065
dated June 20,1997, we committed to submit a supplemental response to address vendor
interface issues by July 11,1997. This letter describes the status of actions taken or ongoing to
address ver. dor interface issues.
The following commitment is made in this letter:
Develop criteria that identifies categories of vendor interface (e.g., specialty skills, proprietary
.
information used for analyses or calculations, atypical or prototype design) that will require
'
additional controis. Create a checklist or matrix for determining additional controls when
special vendor oversight / interface is required. This commitment will be completed by
,
December 31,1997.
Please do not hesitate to contact me if there are any questions regarding the enclosed reply,
tLL
E.T. Boulette, PhD
KRD/dmc/297073
Enclosure 1: Supplemental Reply to Notice of Violation
i
Co203eay aCy._
l
.
_
- - . _ - .
.
.
. - .
.
-. . . ... . . -
- -._.
. -
.
.
4.-
.,
.
,
.
cc:
Mr. Alan B. Wang, Project Manager
,.
Project Directorate 1-3
Office of Nuclear Reactor Regulation
- Mail Stop: - OWF1482 -
U. S. Nuclear Regulatory Commission
1 White Flint North
11555 Rockville Pike -
Rockville, MD 20852
U.S. Nuclear Regulatory Commission
Region l '
.475 Allendale Road
King of Prussia, PA 19406 -
Senior Resident inspector
d
.
4
1
%
2
.
,
- - - - . , ,
,
. . - ,
, , ,
,-r-
,
-
-
--
m
.
i
- -
.
-
4
Boston Edison
Docket No. 50-293
Pilgrim
License No. DPR-35
Enclosure 1
Supplemental Reply to Notice of Violation
The cover letter of inspection Report 97-02 states, in part, "... additional problems with the
interface / oversight of vendor activities were observed during RFO #11 including rework of the
ECCS suction strainer slip joint clearances and the physical damage to the main steam line plugs
that occurred when lowering the steam separator into the vessel. Both examples involved over-
reliance on vendor communications. Also, continuing problems were experienced with RWCU
pump mechanical seals. The adequacy of the interface with vendors was documented in NRC
Inspection Report No. 50-293/95-22."
Adeauacy of Interface with Vendors Documented in IR 95-22/lFI 95-22-02
In NRC Inspection Report 95-22, an inspector follow item, IFl 95-22-02, was opened conceming
vendor services. The inspector considered the area of root cause analysis of supplier / vendor-
related problem reports to be a potential program weakness, and ownership of related corrective
action items (related to vendor controls) regarding measures to preclude problem recurrence was
not clear.
Since inspection Report 95-22 was issued, significant changes were made in the Problem Report
(PR) program to strengthen the root cause analysis process. Fundamental to the change is the
classification of a problem report into one of two levels, significant conditions adverse to quality
(SCAQ) or non-SCAQ.
A SCAO level problem undergoes a detailed root cause analysis. The group-level manager
responsible for the analysis approves the cause and corrective action to preclude recurrence.
Corrective actions to preclude recurrence must meet 3 criteria to be considered valid:
The problem would not have occurred had the noted cause not been present.
.
The problem will not recur due to the same cause if the noted cause is corrected or
.
eliminated.
Correction or elimination of the noted cause will prevent recurrence of similar conditions
which could occur due to the same cause.
In addition to group-level manager approval, a root cause analysis and its corrective actions must
be approved by the Corrective Action Review Board (CARB). The CARB reviews all PR root
cause analyses for technical content, accuracy, adequacy of all corrective actions, and
administrative completeness / acceptability.
If a vendor-related problem is designated as a SCAQ, then the cause and corrective action
undergoes the analysis and approval process. Therefore, ownership of corrective action items
related to vendor controls regarding measures to preclude recurrence is clear and within the
framework of the corrective action process.
3
.
- . .- --
. - ---- . -- - - . - . . - - -.._...._.-.- - - - --
O_-4
3
.
,
.
-
!
Discussion
.
Historical vendor interface issues including these examples served as a basis for developing
- j
preliminary criteria regarding vendor oversight / interface:
'
-* - ECCS Suction Strainer Slip Joint .
t
Main Steam Line Plugs
e
e~ - Regulating Transformers -
. . Fuel Shutdown Margin
-
There are two general types of vendors / contractor support: augmentation and specialty. The .
-
L work force must be augmented for maintenance, projects and engineering because of the '
.
increased workload and shift coverage during outages. - In these cases, we are able to monitor -
performance and its effectiveness because we possess the expertise ourselves.- This has not -
proved to be an area of significant concem relative to vendor interface.
.
There is, also, heavy reliance on contractors who possess, for example: specialty skills and
. knowledge, proprietary information used for analyses or calculations, unique modifications and/or
,
' new designs (prototype).
~
j
,
~
-
_.
.
ln some cases, our oversight did nM identify the critical design and implementation criteria. As a -
result of this insight, preliminary attributes have been identified regarding the issue of vendor
oversight and vendor interface.- These attributes, although preliminary, serve to determine -
.
I
'
whether or not a vendor is a specialty contractor. The attributes include:
-
i
- . - - Is the task routine or a one time task
i.
. . Does the vendor possess specialty knowledge and skills ?
is the vendor's information proprietary ?
e
. Is time a factor ?
Once these attributes are developed, we plan to create a matrix that flags extra measures or .
precautions to be taken depending on the attributes of the specialty service being considered.
Preliminary examples include:
,
Require third party review
e
Independent verification
.
e-
Enhanced management oversight of the implementation
.
.
Augmented quality oversight
. --
identify critical design and implementation criteria-
e
,
n(
. Augmented quality inspection of vendor design process
e
- Also, a common cause analysis is being conducted for problems identified during RFO #11.
When this letter was prepared,' the common cause analysis had not been completed. The -
results of the common cause analysis will be reviewed regarding vendor oversight / interface to
detemiine if additional adjustments to the identified corrective actions are necessary.
I
Summarv
,
-
- In recognition of vendor interface / oversight as an area for improvement, we are planning the
'
following action independent of on-going root cause or common cause analyses:
'
c
i Develop criteria that identifies categories of vendor interface _(e.g., specialty skills, proprietary
information used for analyses or calculations, atypical or prototype design) that will require
,
. additional controls.- Create a checklist or matrix of additional controls when special vendor -
oversight / interface is required;
.
<
P
4
"
,
-..
A-
$
w-
ee
,,my-tr
29'v'tf=mW'W9'
Y
d'-~TW'd*-T'--'
Y
1
V
't'er'a
w
44-
Decu*
-
-D'ac'F'
-
J-
T't-m-2r
u-=
t
'9--r--'se
w-h'--*-
-
_ _ - _ _ _ _ _
.
_
..
.
.
..
e
(.
.
In addition,-we will complete the following as part of our established corrective action process:
i
Review the results of the RFO #11 common cause analysis regarding vendor
e
oversight / interface issues. The focus of the review is to determine if additional actions or
adjustments are necessary for vendor oversight / interface.
Complete the root cause analysis for the ECCS suction strainer slip joint clearances and
.
implement corrective actions identified.
,
5
.
.
. _ - - _
_
_ _ _
. _ . _ _ - _ - _ - _ _ _ _ _ _ _ _ _ _ - _ _ _ _ _ _
__.
..
..
..
.
. - -
,
1.-
. . .
..
somon sm
Pdgern Nuclear Power station
Rocky Hdt Road -
Plyrnouth, Massachusetts 02360
L T. M ue, M
June 20,1997
senior vce President - Nuclear
g
U. S. Nuclear Regulatory Commission
Document Control Desk
Washington, DC 20555
License DPR 35
Docket 50-293
SUBJECT:
REPLY TO NOTICE OF VIOLATION (REFERENCE NRC INSPECTION REPORT
NO,50-293/97 02)
Enclosed is Bosten Edison Company's reply to the Notice of Violation contained in the subject
inspection report.
The following commitments are made in this letter:
Specification E-15A and regulating transformers documentation (vendor manual V-1184) will
.
be revised to reflect PDC 97-11 as part of the routine modification close-out process by
August 31,1997,
The electrical engineering department will update the design guide to enhance the Equipment
.
Codes and Standards section by June 30,1997,
The specification procedure will be revised to require identification of all new component
.
automatic functions by July 31,1997,
The. human performance aspects of the cause of the deficiency in specification E15A which
.
were addressed via the corrective action process (PR 97.9245) will be discussed in the
electrical engineering department training by August 31,1997.
The vendor interface issues associated with this violation will be addressed in our -
.
supplemental response due by July 11,1997.
The NRC requested inspection report (IR 97-02) that Boston Edison staff specifically address
. actions planned to strengthen the oversight and interface of vendor activities. This response will
be included in a supplement to this violation response by July 11,1997.
4D%2(,03l$ ?lyp
. . . .
- _ - _ .
.
. _
._
_.
_
_ . _ . .
__
_. _._._
.
.
.
,
.
.
9
Please do not hesitate to contact me if there are any questions regarding th enclosed reply.
!
,A J
.
.
--
.' T. Boulette, PhD
KRD/dmc/vio97-02
Enclosure 1: Reply to Notice of Violation
cc:
Mr. Alan Wang, Project Manager
Project Directorate 1-3
Office of Nuclear Reactor
'
Mall Stop: OWF 1482
U. S. Nuclear Regulatory Commission
1 White Flint North 11555
Rockville Pike Rockville, MD 20852
U. S. Nuclear Regulatory Commission
Region
475 Allendale Road
King of Prussia, PA 19406
Senior NRC Resident inspector
Pilgrim Nuclear Power Station
2
.
. , _ .
,
-
.n.m
--
.
__
.
.
.
,
4.
4
e
Boston Edison
Docket No. 50-293
Pilgrim
License No. DPR-35
Enclosure 1
Reply to Notice of Violation
'As a result of an inspection conducted from March 3 to April 28,1997, the following violation of
NRC requirements was identified (VIO 50-293/97-02-01). In accorda'nce with the NRC
Enforcement Policy (60 FR 34381; June 30,1995), the violation is described below:
ViolatioA
10CFR Part 50, Appendix B, Criterion Ill, Design Control, requires that applicable regulatory
requirements and design basis for those structures, systems, and components to which this
appendix applies are correctly translated into specifications, drawings, procedures, and
instructions.
1.
Contrary to the above, the function of the 480/120 voltage regulating transformers was not
correctly translated into specifications which became evident during a significant storm on
April 1,1997, when the running SSW and RBCCW pumps tripped and isolations of the
RWCU system and RBIS occurred with the resultant start of the SBGT system. The
transformers were procured and installed by engineering specification E-51 A and PDC 91-
51 A and automatically tumed off due to an unknown vendor installed undervoltage design
feature.
2.
Contrary to the above, the function of MO-130153 (RCIC system full flow test valve) was
not correctly translated into drawings when the valve functioned in the seal in close mode
on April 17,1997, causing an inadvertent overspeed/ trip and increased unavailability time
of the RCIC system turbine. Electrical design drawing mig 27 did not reflect an earlier
modification made at an indeterminate date which converted MO-1301-53 to jog open/ jog
close.
The above failures adversely affected the performance of safety related equipment and
unnecessarily challenged the operators.
This is a Severity Level IV violation (Supplement IV)."
3
.
.
,
.
.
,
.
REASON FOR THE VIOLATION
Example No. 1 - 480/120 Voltaae Reoulatina Transformers
The reason for the violation cited as example No.1 was that purchase specification (E15A) did
not specify operation during voltage transients. A contributing cause was a vendor-installed
undervoltage trip which came with the delivered transformers without the knowledge of Boston
Edison. This feature in the delivered product was not captured in the equipment documentation.
Discussion:
-
The regulating transformers were installed in 1992 via design modifications per PDC 9159A and
PDC 9159B and associated safety evaluation nos. 2664,2685, and 2706. Transformers X55,
X56, X57, and X58 were designed te regulate input voltages of 480 volts +/- 20% (i.e.,576 384
volts) and provide regulated output voltages of 120VAC +/ 4%. Each regulating transformer
contains a programmable microprr,cessor control unit (MCU) that senses input voltage and
selects the proper voltago t::p to provide the regulated 120 volt output voltage. The MCUs were
programmed by the manufacturer to shut down the regulating transformer if loput voltage was
outside the transformer's input voltage range of 576 - 384 volts. The input voltage range of +/-
20% was based on the worst case voltage dips that would be seen during design basis accidents
concurrent with a loss of offsite power. Transient voltages below 384VAC due to 345KV grid
disturbances were not considered.
During a severe storm on April 1,1997, safeguard panels Y3 and Y4 :ost power due to the
automatic shut down of voltage regulating transformers X55 and X56. After each shut down, the
operators took manual action to open and reclose the input breakers to the transformers that
reset the transformers and restored power to Y3 and Y4, At the same time, regulating
transformer X58, which powers the "B' train of the post accident sampling system (PASS), shut
down and was reset in the same fashion. Transformer X57 was tagged out of service for
maintenance during the storm.
The transformers shut down due to a transient undervoltage condition. This condition was
identified when a review of the 480VAC voltage level showed a voltage drop to approximately
350VAC during the storm.
The design documentation (vendor manual, etc.) provided by the manufacturer and supplier of
the transfomiers did not identify an automatic shut down feature if input voltage was less than
380 volts. The lack of the identification of the feature is significant because an automatic shut
down due to input voltages from zero volta to 384 volts would require a manual reset of the
transfomier versus a designed automatic reset if input voltage was zero volts. Specification E15A
did not request that the transformers continue operation or prohibit shutdown during voltages
outside the design regulation range.
The cause for the automatic shut downs of the regulating transformers was a deficiency in
specification E15A which did not address the effects of 480 volt transients of less than 384 volts.
The apparent cause of the deficiency was an unintentional cognitive error made by the utility
electrical engineer who prepared the specification. Nuclear engineering procedure 3.08,
' Specifications and Reports", govems the preparation and issuance of specifications. There were
no unusual electrical engineering work location characteristics that contributed to the error. The
4
- -
-
-
-
.
-
-
-
-
-
.
-
,
,
,-
.
,
.
human performance aspects of the cause of the deficiency in specification E15A was addressed
via the corrective action process (PR 97.9245) which concluded that the problem report would be
discussed in the electrical engineering department training.
,
Example No.2 - MO 130153 wirino discrepancy
. The reason for the violation cited as example No. 2 was lack of design control at a previous time
in plant operation due to elementary and connection tirawings not reflecting a change that made
MO-130153 a jog close valve.
.
Discussion:
During RCIC testing on April 17,1997, MO 1301-53 operated as a seal-in close valve instead of
operating as a jog valve in both open/close directions. A new bucket for MO-130153 was
installed under PDC 93-38 during RFO #11. The new bucket was wired in accordance with the
existing connection and elementary drawings per the design basis. These documents referenced
MO-130153 as a jog open/ seal-in close valve. The wiring in the old bucket was then
investigated. It was found that a wiring change made to an auxiliary relay in the bucket changed
the function from a seal in close valve to a jog close valve. This change was not captured in
previous drawing revisions. A search of the maintenance request (MR) database also revealed
this change was not captured on a MR. It is, therefore, concluded that the change was made at
cn indeterminate date possibly dating back to pre 1972 startup testing.
The undocumented wiring change did not result in a failure of valve MO-130153.
CORRECTIVE STEPS TAKEN AND RESULTS ACHIEVED
Example No.1
The microprocessor control units (MCUs) for the four regulating transformers were modified via a
design change (PDC 97-11) on April 12,1997. The modification disabled the undervoltage and
cvervoltage shut down functions. This allows the transformers to operate in the unregulated
mode when the input voltage is outside the design range. Post work tests were performed to
ensure the regulating transformers would continue to operate during simulated undervoltage and
overvoltage conditions. During the tests, the highest and lowest transformer taps were selected
without a shut down of the respective regulating transformer. Based on the satisfactory results of
this testing, the regulating transformers were declared operable.
Problem report, PR 97.1658, was written to document that the safety evaluation 2664 performed
to allow the installation of the regulating transformers did not evaluate the consequences of an
undervoltage transient shut down. The problem report evaluation concluded that the
undervoltage shutdown was not considered because the vendor had not notified Boston Edison
about the transformer's undervoltage trip, and specification E15A did not ensure conti.1ued
operation of the transformers during voltage transients. The evaluation also concluded that the
procedures to write specifications and safety evaluations are adequate; however, the
specification procedure will be enhanced to require identification of all new component automatic
functions. The electrical engineering department design guide will be also be updated to
enhance the Equipment Codes and Standards section.
5
e
_
.
.
.
--
_
._ _ . . _ _ _ . _ _ _ _ _ . _ _ _ _ _ . . _ _ _ _ _ _ _ _ . _ _ .
..
.
,
.-
.
..
Example No. 2
An engineering design change (FRN 93-38-21) was issued on April 18,1997, to change valve
- MO 130153 to a jog open/close valve. The valve was then stroked (jogged in the open and
-
close directions) with satisfactory results. The document changed the closing circuit from a seal-
'
in type circuit to a jog type circuit. The affected drawings will be revised as part of the routine
modification close-out process. With this change, the control circuitry for MO 130153 functions
as a jog circuit in the open and close directions. The change did not affect the automatic closing
function of valve MO 130153.
'
A review was conducted of the other breakers replaced under PDC 93 38 focusing on those that
power motor operated valves that contain a jog control switch circuit. The review concluded the
buckets were wired in accordance with design, and the undocumented wiring change was an
isolated instance that could date back to initial start-up testing (circa 1972). Current procedures
and work practices require approved design changes for wiring changes and documenting wiring
,
discrepancies in accordance with the problem report process.
- CORRECTIVE STEPS THAT WILL BE TAKEN TO AVOID FURTHER VIOLATIONS
Example No.1
Specification E-15A and regulating transformers documentation (vendor manual V-1184) will be
revised to reflect PDC 97-11 as part of the routine modification close-out process by August 31,
,
1997.
d
!
The specification procedure will be enhanced to require identification of all new component
automatic functions by July 31,1997.
The electrical engineering department will update the design guide to enhance the Electrical
Codes and Standards and section by June 30,1997.
.
l
The human performance aspects of the cause of the deficiency in specification E15A, which were
addressed via the corrective action process (PR 97,9245), will be discussed in the electrical
engineering department training by August 31,1997.
The vendor interface issues associated with this violation will be addressed in our supplemental
3
response due by July 11,1997.
Example No. 2
Our current procedures and work practices require approved design changes for wiring changes
and documenting wiring discrepancies in accordance with the problem report process. No further
changes are required to prevent recurrence.
6
.
- .
.
. - -
- .
. - - . .
-- -
- -
-
- ,-
_
..
._.
. _ _ _ . . _
_
,
.-
.
,
- . .s_-
-
..
+
. DATE WHEN FULL COMPLIANCE WILL BE ACHIEVED
Example No.-1-
. Full compliance will be achieved by August 31,1997, following implementation of the last open
'
corrective action. These commitments are being tracked under RC 97.0023.
Example No. 2
,
Full compliance was achieved when MO-130153 was rewired and tested on April 18,1997.
REFERENCES
1)
LER 97-007 * Safeguards Buses De energized and Losses of Off site Power during -
. Severe Storm while Shut Down", Boston Edison Letter #2,97.050 dated May 1,1997.
,
2)
LER 97 009 "RCIC System inoperable due toTurbine Overspeed Trip during Surveillance",
Boston Edison Letter #2.97.056 dated May 19,1997.
.-,
,
,
l
i
,
7