ML20211C885
| ML20211C885 | |
| Person / Time | |
|---|---|
| Site: | Crystal River  |
| Issue date: | 01/15/1987 |
| From: | Belisle G, Jackson L, Runyan M NRC OFFICE OF INSPECTION & ENFORCEMENT (IE REGION II) |
| To: | |
| Shared Package | |
| ML20211C810 | List: |
| References | |
| 50-302-86-39, NUDOCS 8702200250 | |
| Download: ML20211C885 (22) | |
See also: IR 05000302/1986039
Text
- _ - - _ _ _ _ - _ _ - _ - _ _ -_-- --_
.
.
gMaio UNITED STATES
' NUCLEAR REGULATORY COMMISSION
f 'o
"
[
g
p
j. 101 MARIETTA STREET, N.W.
REGloN li
- i ' ATLANTA, GEORGI A 30323
~C
,o s.
- ...+ -
r
.
.
,
Report No;: -50-302/86-39 .
o
c .
-
,-
Licensee: ' Florida Power Corporation
- 3201 34th Street, South .
- St. Petersburg, FL 33733
'
Docket No.: 50-302 License No.: DPR-72
Facility Name: Crystal River 3 .
Inspect. ion Conducted: November 17-21 and December 1-5, 1986
Inspectors: k T.Ld.Emd
M. F. Runyan
I/16 [8 7
Date signed
G
h T d (/ .wM I s!B7
L. H. Jackson g Dite ' Signed
Accompanying Personnel: C. G. Walenga, IE
G. A. Belisle, Region II
M. C. Shannon, Region II
Approved by: h V T (). [
G. 'A. Belisle, SectiortfChief
8!I5!B7
D&te Signed
Division of Reactor Safety
SUMMARY
Scope: This special, announced inspection was conducted in the areas of licensee
action on previous enforcement matters, quality assurance effectiveness, and
licensee action on previously identified inspection findings.
Results: One violation and one deviation were identified.
1
070220g A
g goD PDR
02
G l
~
l
_
'
'
-
~
n_-A________ ._
,
REPORT DETAILS
1. Persons Contacted
Licensee Employees
- J. Alberdi, Manager, Nuclear Site Support
J. Andrews, Nuclear Plant Engineer
K. Baker, Manager, Engineering Assurance
W. Bandhauer, Assistant Nuclear Plant Operations Manager
- P. Breedlove, Nuclear Records Management Supervisor
- R. Bright, Manager, Nuclear Licensing
T. Catchpole, Senior Quality Auditor (Site)
M. Collins, Nuclear Safety and Reliability Superintendent
D. Conkle, Nuclear Project Engineer
D. Cook, Nuclear Plant Engineer
C. Doyel, Supervisor, Nuclear Engineering
K. Dyer, Engineering Aide
P. Ezzell, Nuclear Compliance Specialist
J. Gibson, Nuclear Technical Specification Coordinator
B. Gutherman, Quality Programs
D. Harper, Licensing
V. Hernandez, Senior Nuclear Quality Assurance (QA) Specialist
M. Jones, Nuclear Modifications Specialist
- R. Jones, Nuclear Modifications Specialist
M. Kirk, Nuclear Operations Technical Advisor
D. Kurtz, Supervisor, Quality Audits
- K Lancaster, Manager, Site QA
M. Lord, Nuclear Engineer
- M. Mann, Nuclear Compliance Specialist
P. McKee, Director, Nuclear Plant Operations
L. Moffatt, Nuclear Safety Supervisor
R. Murgatroyd, Nuclear Maintenance Superintendent
H. Pinney, Quality Programs
F. Quinn, Instrumentation and Control (I&C) Engineer
- V. Roppel, Manager, Technical Support
- W. Rossfield, Nuclear Compliance Manager
0. Salute, Nuclear Compliance Specialist
D. Shook, Manager, Nuclear Electrical /I&C Engineer
- P. Small, Maintenance Department Coordinator
F. Sullivan, Nuclear Electrical /I&C Engineering Supervisor
C. Tillman, Manager, Nuclear Operations Material Control
K. Vogel, Electrical Engineer
- E. Welch, Manager, Nuclear Electrical /I&C Engineering .
- G. Westafer, Director, Quality Programs
- K. Wilson, Manager, Site Nuclear Licensing
H. Wollesen, Nuclear Maintenance Specialist
Other licensee employees contacted included technicians and office
personnel.
- Attended exit interview
]
- _ _ . _ _ _ _ _ -
_ _ .. . . . _ _ . _ .
'
. . .
4
Z
. 2
.
4
- 2. Exit Interview
'
The-inspection scope and findings were summarized on December 5; 1986, with
'
these persons indicated in paragraph 1 above. The inspector described the
F areas inspected and discussed in detail the ~ inspection findings. No
,_
_ dissenting comments were received from the licensee,
f Violation: Failure to Perform an Adequate "O CFR 50.59 Review and .
Failure to Perform Post-Modificstion Testing,
Paragraph 5.b. (1)
~
Deviation: FSAR Requirements Are Not Being det in'That RM-L4 Has
Been Inoperable Since September 1979, Paragraph 5.b. (2)
,
The licensee did not identify as proprietary any of the materials prov'ided .
to or reviewed by the inspectors during this inspection.
'
- 3. Licensee Action on Previous Enforcement Matters (92702)
a. (Closed) Severity Level IV Proposed Violation 302/86-01-01: Inadequate
- Verification of Design Change.
The licensee's response dated April 4,1986,. denied .this violation.
The licensee included information to support their denial. The
violation was for failure to recognize the need for a bypass switch
l proposed for valves MUV-58 and MUV-73 during the design review process.
- The inspector reviewed' the response letter and discussed this item in
- detail with the engineering department manager and concurs that failure
- to add the bypass switch in the original design is not a violation.
- - The worst case failure is that MUV-58 and MUV-73 would be in their
- emergency safeguards (ES) actuation position and would thus cause 1
l borated- water to be injected into the reactor coolant system (RCS).
l The operator has sufficient controls to reduce the injection rate to a
[ low level and this would cause a slow shutdown o'f the plant if no other
i operator action was taken. Without the bypass switch, these valves
,
would be in the ES actuate position so they do not compromise plant
i safety.
i b. (Closed) Severity Level V Violation 302/85-07-02, Inadequate Procedure
'
for Retention of Records Required by Technical Specifications.
The licensee's response dated July 16, 1985, was considered acceptable
by Region II. The inspector reviewed AI-1100, Retention of Plant
l Operating Records, Revision 15, dated Jaauary 11, 1985. The inspector
i
also reviewed Revision 16 of this procedure. Revision 16 was approved
j by the Nuclear Plant Manager on August 14, 1985. This arocedure
contains FSAR Section 1.7.1.17 record requirements which includes
!
controls for training records and those records required to be retained
[
by Technical Specifications.
i
.
i
'
.
3
Enclosure 1 of this procedure specifically requires that restart
package from trip (RPFT) records and Unplanned Operating Event. Reports
(U0ER) be maintained for five years.
The inspector concluded that the licensee had corrected the previous
problem and developed corrective actions to preclude recurrence of
similar problems. Corrective actions stated in the licensee response
have been implemented.
c. (Closed) Unresolved Item 302/85-07-01: Post-Trip Review Procedure
Comments.
The inspector reviewed OP-210, Reactor Startup, Revision 21. This
procedure requires (Step 6.1.4.1) that for a reactor startup following
a reactor trip certain procedural controls have to be completed. This
step requires a sign-off (initials) in the procedure. The inspector
reviewed AI-704, Reactor Trip Review and Analysis, Revision 0. This
procedure establishes administrative requirements following transient
events for implementation of a Post-Trip Review program.
d. (Closed) Unresolved Item 302/85-15-05: Commitment Tracking.
This item contained the following parts:
(1) Part 21 Followup Commitment
(2) Post Accident Sampling System (PASS)
(3) NUREG-0578 Item 2.1.6.a Commitment
(4) Commitment System
During inspection 50-302/85-38, parts (1) and (3) were closed and their
closure is documented in that report. Parts (2) and (4) could not be
closed due to outstanding licensee audit findings from Audits OP 247
and 259 awaiting corrective action completion. During discussions with
licensee quality programs personnel, the inspector was informed that
those audit findings had been resolved and that the audits had been
closed.
4. Unresolved Items
Unresolved items were not identified during the inspection.
5. Quality Assurance Effectiveness
The objective of this inspection was to assess quality assurance effective-
ness. For this report, quality assurance effectiveness is defined as the
ability of the licensee to prevent, identify, and correct their own
problems. The term quality assurance effectiveness is used in this
application, but it is not meant to be limited to the licersee's Quality
Assurance Department. It is the total sum of all efforts to achieve quality
results.
.
4
This was a performance-based rather than compliance-based inspection.
Instead of verifying compliance to programmatic requirements, the principle
effort was to determine whether the results quality assurance was designed
to accomplish were actually achieved. The following sources of information
were used to base an objective measure of quality assurance effectiveness:
Nuclear Operations Performance Mcnitoring Reports
Monthly Synopsis of Operating Experience Reports
Nuclear Operations Monthly Reports
Quality Programs Trend Analysis Reports
In addition, the inspectors reviewed trending indicators tracked by various
groups and any other information deemed pertinent to the overall evaluation
of quality performance.
The inspection effort was divided into'the following areas:
Operations and Maintenance
Design Control
Licensing and Commitments
Quality Assurance Department
Each area is addressed separately in this report. Included in this
assessment is an evaluation of licensee actions to correct situations where
'
performance has not met stated goals or where trends have been adverse.
a. Operations and Maintenance
Within the functional areas of operations and maintenance, the
inspector analyzed trending indicators from various sources and
gathered other pertinent information upon which to base an evaluation
of quality assurance effectiveness.
Forced outage rate, the percentage of time planned for electrical
generation that the unit was unavailable due to forced events, is one
of the better indicators of quality assurance effectiveness. Many
quality related activities have a direct impact on the forced outage
rate, such as preventive maintenance, corrective maintenance, planning
management, training, procurement, material availability, surveillance
testing, and design control.
The 1986 forced outage is heavily skewed by a five-month outage to
repair reactor coolant pumps, which renders this indicator misleading
as an overall quality indicator. Forced outage for the first three
quarters of 1986 was 62.2%. The unit operated continuously from the
startup following the reactor coolant pump outage until November 12,
1986, when a dropped control rod precipitated a forced outage which was
still ongoing as of December 5, 1986. Over the past three years,
forced outage has been 20.2%, just over the goal of 20%.
m
.. . _ _ _ _ _ _ - _ _ _ _
.
5
The number of times the reactor trips inadvertently is another good
indicator of quality. The inspector expanded the study period to
include the last year of power operations which extends back to
mid-1985. During this time period, a total of nine unplanned reactor
trips occurred, as summarized below:
Date Cause
1. 8/20/85 Main turbine trip tu isolate steam leak
Licensee Event Report ((LER) 85-15)
2. 8/20/85 High RCS pressure due to faulty feedwater
control (LER 85-16)
3. 10/9/85 Manual trip after main steam isolation valves
close on failed transmitter (LER 85-20)
4. 10/26/85 Operator action, carrying out trip procedures
when a trip had not occurred, failed inverter
(LER 85-23)
5. 11/8/85 High RCS pressure following feedwater booster
pump trip (LER 85-25)
6. 11/22/85 Operator actions following feedwater trans-
ient, high RCS pressure (LER 85-26)
7. 12/3/85 Loss of two reactor coolant pumps due to
electrical bus failure (LER 85-28)
8. 12/7/85 Operator actions, procedural noncompliance
causes reactor protection trip logic spurious
trip (LER 85-30)
9. 1/1/86 Automatic trip following reactor coolant pump
shaft failure (LER 86-01)
During a period of 4h months, the reactor tripped nine times, which
appears to reflect poor quality assurance effectiveness. However,
following the reactor coolant pump outage, the unit operated for 4
months without a reactor trip.
Therefore, assessing the 4h month periods before and after the reactor
coolant pump outage, plant quality assurance effectiveness as measured
by reactor trips went from very poor to excellent.
There seems to be several reasons to explain the sudden turnaround.
The Reactor Trip Reduction Task Force (RTRTF) was established in F ch
1985, to pursue the goal of reducing reactor trip frequency at Cry.tal
River. Members were chosen from the Site Nuclear Operations and
Nuclear Engineering and Licensing staffs. Later, a representative from
Quality Assurance was added. The group meets approximately once a
month to review root causes of past trips, track ongoing corrective
actions, and review industry-wide information pertaining to trip
reduction. The inspector reviewed minutes from the May 6,1986, and
September 22, 1986, meetings. Within the minutes, action items and
modification lists were tracked.
A. _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ . _ _ _ _ _ _ _ . _ . _ _ _ _
- -_ - ____ _
~
.
6
One of the items tested by this group was a modification to improve the
reliability of the Emergency Feedwater Initiation and Control (EFIC)
System. This modification was the main factor in reducing trip
frequency and is discussed later in this section.
The inspector reviewed the status of several important items tracked by
the RTRTF and concluded that corrective actions were being aggressively
pursued. The overall effort of this group appears effective.
The inspector chose the following six reactor trip LERs to review the
status of committed corrective actions:
85-20
85-23
85-25
85-26
85-28
85-30
Corrective action was either complete or proceeding satisfactorily for
all items with one exception. In LER 85-26, the licensee committed to
perform an engineering evaluation to determine the adequacy of the
design of a failed relay which had previously resulted in the failure
of a pilot operated relief valve (PORV) to open on demand. The LER
report date was December 23, 1985. The commitment date was extended to
October 30, 1986 and then later extended to December 12, 1986.
Considering the safety-significance of this item, this appears to be an
isolated case of untimely corrective action.
The licensee performs a detailed post-trip analysis in a document
called an Unplanned Operating Event Report (UDER). The U0ER is used
for all reactor ' trips and other significant operating events. The
inspector reviewed the following U0ERs:
U0ER No. Trip Date Title
U0ER 85-08 11-8-85 Feedwater Booster Pump Trip
U0ER 85-10 11-22-85 Main Feedwater Upset During
Shutdown
These two reactor trips involved circumstances where the licensee
admitted that inadequate corrective actions for the first trip led
directly to the second trip. The November 22, 1985, trip involved
operator actions which may have been precluded had the training
following the November 8, 1985, trip stressed the importance of
allowing the EFIC system to automatically control once through steam
generator (OTSG) levels upon actuation.
__
.
7
Otherwise, the two U0ERs reviewed appeared to be conscientious and
comprehensive efforts to find and correct the root cause of the
problem. Safety considerations and assessment conclusions displayed a
broad-based perspective.
Another indicator of quality assurance effectiveness is the frequency-
of unplanned safety system actuations. The trending indicator for this
subject tracks unplanned actuations of high pressure injection, low
pressure injection, core flood, and ES bus undervoltage. Zero actua-
tions of these systems were reported for the calendar year beginning
with the fourth quarter of 1985. However, a number of unplanned
actuations of the emergency diesel generators (EDG) and the EFIC system
have occurred since 1985 which shed light on the licensee's quality
assurance effectiveness. The inspector reviewed the following
incidents.
LER Description
85-02 Automatic start of Emergency Diesel
Generator (EDG)
85-05 Unplanned actuation of an EDG
85-06 Unplanned actuation of an EDG
85-07 Unplanned actuation of an EDG
85-08 Unplanned actuation of Low Pessure
Injection (LPI)
85-12 Unplanned actuation of Emergency Feedwater
85-14 Unplanned actuation of Emergency Feedwater
85-19 Unplanned actuation of an EDG
85-20 Unplanned actuation of Emergency Feedwater
85-26 EFIC Actuation
85-27 EFIC Actuation
85-31 EFIC Actuation
86-01 EFIC Actuation
86-05 EDG Actuation
86-08 EFIC Actuation
The EDG actuations were caused primarily by personnel errors. There
were no repeat scenarios. Corrective actions appeared suitable to the
incidents. Other causes were material in nature and corrective actions
included the replacement of all four air start solenoid air filters
with filters designed for a higher pressure. The sum of these
corrective actions appears to have been effective.
EFIC actuations occurred until after the installation of a time delay
on the low OTSG level actuation. Many of the unplanned actuations of
this system were caused by a pressure spike (as when the turbine valves
shut) that gave spurious indication of low OTSG 1evel. The 2 second
time delay should eliminate this problem. In addition to these changes
plant procedures were modified where they were found deficient. Other
modifications to address flow rate limitations and power supply
_ __J
__
.
8
reliability are planned. The total sum of these efforts have appar-
ently now been proven effective. However, the attainment of relia-
bility of this system appeared to have taken an excessive amount of
time. Corrective action philosophy for some of the problems encoun-
tered early (after system installation approximately two years ago)
appeared to be overly specific and not all encompassing.
The inspector reviewed the circumstances which occurred on November 12,
1986, while the plant was being shut down to repair a dropped rod and a
leaking primary relief valve. A feed pump steam' supply perturbation
resulted in a low OTSG 1evel and an EFIC actuation. During the
emergency feedwater actuation, the emergency feed pump tripped on
overspeed. Crystal River has experienced problems in .the past with
overspeed trips of the Terry turbines used in the emergency feed pumps.
The overspeed condition which occurred in the past resulted from a
leaky steam supply valve which caused the pump shaft to idle and
allowed the initial burst of steam following actuation to raise the
pump speed to the overspeed setpoint. The November 12, 1986, EFW pump
overspeed trip was not related to the previous scenario. For unknown
reasons, a bypass warmup valve was shut which allowed the buildup of
water in the piping. When the actuation signal was received and the
steam supply valve opened, the water provided too much torque to the
pump shaft and an overspeed trip resulted. The licensee could not
explain why the normally throttled bypass valve (ASV-167) was found
shut. For corrective action, SP-300-40, Operations Daily Surveillance
Log was revised to include verification that ASV-167 was not~ closed and
that there was a wisp of steam in the tailpipe. This action should
prevent this event from recurring.
The licensee's dealings with unplanned safety system actuations
indicates a good quality attitude. Though corrective actions have not
always been prompt, they have been thorough and aggressive. The plant
is meeting its goal in this area.
Safety system unavailability, or the percentage of time a safety system
is unavailable, provides another measure of quality assurance effec-
tiveness. The licensee is currently developing this trending indicator
for Crystal River. Therefore, this information was not available for
inspection with the exception of the diesel generators. Seven quarters
of data had been compiled for both diesel generators, as follows:
Hours Out of Service (known and unknown)
1-85 0 18.1
2-85 73.7 2152
3-35 206 0
~4-85 56 23
1-86 18 98
2-86 16 22
3-86 15 24
-- _ _ _ _ _ _ _ _ _ _ _ - _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
'
.
9
Goals have not yet been established for diesel generator availability.
However, the' above data appears to establish a positive trend,
suggesting that the licensee's recent efforts to improve diesel
generator availability have been effective.
Given the lack of trending indicator data, the inspector attempted
to establish a data base for safety system unavailability by reviewing
information provided by the licensee's Nuclear Plant Reliability Data
System (NPRDS) and Machine History Trending Program. The following
safety systems were chosen for this review:
High Pressure Injection
Decay Heat Removal / Low Pressure Injection
Decay Heat Removal / Core Flood
Emergency Feedwater
NPRDS information for the past 12 months indicated a low number of
failures for these systems. In many cases, however, the cause of the
failure could not be determined or a root cause was not determined.
Definitive information was not provided concerning the amount of time
the system was unavailable for service. The Machine History for these
systems delineated each work request written against the system as
referenced to the tag number of the failed component. This tabulation
provided ambiguous information concerning the reliability of the High
Pressure Injection (HPI) System since HPI constitutes only a subset of
the Makeup System, under which the information is compiled. The
licensee was unable, short of a very time-consuming process, to
indicate which work requests were associated with the HPI system.
Under the licensee's Preventive Maintenance (PM) Trending Program,
PM-150, semi-annually, the 20 tag numbers and the 5 systems for which
the most work requests were written are reviewed to identify adverse
trends. The inspector reviewed PM-150 results dated April 1 and
September 22, 1986. In both cases, no trends were identified. There
is probably some connection between the lack of findings and the
nondescript method whereby the information is compiled. The Machine
History Trending Program appears to be underutilized as a tool for
identifying root cause failure modes, adverse trends, and vulnerable
systems. Used in its current form, it is providing little information
to improve the reliability of safety systems. The inspector was
informed that the Engineering Department periodically reviews informa-
tion provided by this system though this activity was not inspected.
The quality assurance effectiveness to ensure high availability of
safety systems is indeterminable but will be brought to light when this
parameter is tracked as a trending indicator. At that time, licensee
attention in this area should increase and it can be reasonably assumed
that the utilization and ef:'ectiveness of the NPRDS and Machine History
Trending Systems will be enhanced.
.
. . __- \
_-_- _ _ _ _ - _ _ - _ _ _ _ - _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
.
.
10
Another measure of quality assurance effectiveness is the licensee's
management of known deficiencies within the plant. Expeditious
corrective action displays a conscientious attitude toward quality.
One way to measure this is by examining the backlog of work requests.
The licensee defines a backlog work request as one that exceeds three
months age. In the past six months, the backlog was decreased from
approximately 350 to approximately 230, though an increasing trend
exists over the past two months. The inspector reviewed a printout
of all open work requests as of December 3,1986. Each work request
is categorized as either being ready for working or on hold for parts,
engineering, or operational considerations. Very few work requests
were on hold for parts which indicates good storeroom availability.
Work requests on hold were mainly due to engineering reasons and these
mostly had field problem reports associated with them. Only eight work
requests greater than 18 months old were designated as ready for working,
each of which were most likely only recently released from hold. Each
work request was prioritized as follows:
l
l (1) Technical Specification, Security, Personnel or
l Equipment Safety
(2) Schedule
(3) Outage
(4) Unknown, auditing review
The oldest category 1 work request was written February 12, 1986, and
was the only outstanding Category 1 work request written before July
1986. The licensee has apparently placed proper emphasis on closing
out work requests of safety significance. Other areas where the
licensee is meeting their goals include the percentage of maintenance
work requests greater than three months old (20 percent versus
<30 percent goal) and the ratio of highest priority to total non-outage
corrective maintenance work requests (17 percent versus <20 percent
goal). In all, the licensee appears to have positive management
control of the corrective maintenance system.
The inspector also reviewed the backlog for category 1 and 2 (Technical
Specification and important to safety) instrument calibrations. This
trending indicator is tracked by the licensee's PM-200 Effectiveness
Data Program. The number of category 1 and 2 instruments backlogged
peaked at about 45 in May 1986, but has been reduced to near zero since
August 1986. The supervisor in charge of this area indicated that the
backlog should be kept near zero in the future. Licensee actions in
this area are commendable and consistent with a high quality attitude.
l
.
.
- - - - __-____-___________-_______________ ._____.
_ _ _ _ _ _ _ _ _ _ _ _ - _ - _ _ _ _ _ . _ _ _
'
.
11
The inspector selected several performance indicators in the area of
preventive maintenance for review. Routine maintenance performance,.or
the percentage of all preventive maintenance surveillances scheduled
during the period that were not performed within the assigned grace
period, is tracked in the Performance Monitoring Report. The licensee
established a goal of two percent of PMs exceeding the grace period.
During the past six months through September, this goal was achieved in
four of the months including the last three. The licensee has also
established the goal of expending twice as many manhours on preventive ,
maintenance as corrective maintenance, or a ratio of 2 to 1. In the
September 30, 1986, Performance Monitoring Report, this ratio was
reported as 3.55. However, the inspector received the manhour totals
and determined that the true ratio was 0.28. Corrective maintenance
for the first three quarters of 1986, incurred 124,251 manhours whereas
preventive maintenance (which includes surveillance testing) comprised
35,025 manhours. The large disparity between the actual ratio and the
stated goal most likely results from two factors. The lengthy outage
for the first half of the year involved a large amount of corrective
maintenance. Also, it appears that the licensee's preventive mainten-
ance program has not been fully developed to the extent where
predictive and preventive measures are taken before corrective measures
become necessary. There is some indication that the licensee is
prepared to upgrade their preventive maintenance program. A new ten
year maintenance program is under development which will include some
of the Japanese maintenance philosophy of scheduled predictive
(overhaul) maintenance planning. Additional preventive maintenance
items will be added as well as a periodic effectiveness review of the
preventive maintenance program. These efforts should improve the
preventive to corrective maintenance ratio and will likely reduce the
total number of manhours expended on maintenance as a whole.
The general assessment in the area of operations and maintenance is
that quality assurance effectiveness has been average. The trend over
the past two years has been positive and the current attitude and
philosophy of licensee management personnel is consistent with goed
quality assurance practices,
b. Design Control
In the functional area of design control, the inspector reviewed and
analyzed various management indicators and program details in order to
base an evaluation of quality assurance effectiveness.
Temporary and permanent modifications are tracked using the Nuclear
Modifications File System. The system is capable of providing status
for open or closed items for temporary and permanent modifications.
The system is used to provide the status of modification packages and
the associated field change notices. For future inspection purposes,
the system is also capable of providing the status for all design
changes made to a specific plant system. A present weakness is the
inability to retrieve those design packages that were cancelled. This
is presently being addressed by the engineering staff.
.. .
. .
..
_ _ _ _ _ _ _ _ _ _ _ _
. - . _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ - _ _ _ . _ _
'
,. .
1
l
12
r
There is an effort to reduce the total number of design packages and
specifically to reduce the number of older design packages, some of
which date back to 1977. The FPC Nuclear Operations Quarterly Report
for June and September 1986, indicated that the total number of
modification packages outstanding was reduced by 306, from 870 in June
to 564 in September. The inspector reviewed the cancelled packages to
insure that modifications were being properly dispositioned with
adecuate justification and that needed or important to safety modifica-
tions were not being cancelled. An in-depth review of 17 of the
cancalled modifications indicates an acceptable program.
The inspector reviewed modification packages to assure proper docu-
mentation. There has been a recent effort to update older modification
packages with design reviews, design input requirements, and various
evaluation forms. Recently, completed modifications were very detailed
and it appeared that the engineering design effort improved in the
documentation area.
Field Change Notices (FCNs) were reviewed for indication of problems in
the modification package due to improper engineering input. FCNs for
several safety system modifications were reviewed to veri fy that
initial designs were adequate and that FCNs received the same level of
review as the original modification package. There appeared to be
problems in this area due to the number of FCNs associated with work
packages generated prior to 1985. For this reason, the inspector
focused attention on recently completed modification packages. It
appears that the general type of FCN is a minor change to the design
package. Many of the FCNs are generated during the walkdown phase of
the modification, which is done prior to starting work on the design
package. No FCNs were found that would indicate an engineering error
such as out of date schematics, inadequate design input data, or
inattention to detail. FPC management appears to have reached an .
acceptable level of control for processing design packages. It was
also noted that outside engineering assistance is only being supplied
from Gilbert Associates and Babcock and Wilcox (B&W) due to previous
problems with other engineering groups.
l
The inspector reviewed the program for updating modified schematics and
system diagrams. Prints and drawings usec by control room personnel
are required to be updated prior to turning a system over to opera-
tions. Selected system modifications were reviewed and it appears that
control room required prints are updated within four weeks of the
original request, which is procedurally submitted two weeks prior to
the design package implementation. Interim drawings and field changes
to drawings are maintained by the engineering group until final
revisions are received. Nothing was found to indicate that unrevised
drawings were used for design input. The system for tracking drawing
revisions appears weak and could be improved by initiating a more
through review of outstanding drawing revisions.
.
13
'Nonconformance Operating Reports (NCORs) directed at engineering were
reviewed in detail. Eight items were identified and three of these
were original design problems identified by the Gilbert engineering
department. The other items indicate that engineering could stress
more attention to detail in design package preparation and review. The
NCORs were not of type and quantity to indicate substantial design
program problems.
A new tracking system is now in place to track problems within the
engineering organization. It has input from QA audit findings, NRC
violations, INPO concerns, and internal engineering concerns.
Interviews with licensee personnel by the inspector indicate that the
system will be able to trend specific problems so that adequate
corrective action can be taken.
Temporary modifications, known as T-MARS, temporary modification
approval records, are normally initiated and implemented at the plant
site level. These modification packages have unique problems not
identified in the permanent modification packages.
Until recently, design reviews and design input requirements were not
included in the T-MAR packages, even though this program requires the
same level of engineering input as the permanent modifications program.
The inspector also observed that the authorization forms for extending
a T-MAR had not been completed as required by procedure. The four
oldest T-MARS still in effect were reviewed and found to have had this
deficiency.
There is concern that T-MARS are installed in the plant for extended
lengths of time. Five T-MARS presently installed were older than two
years. T-MARS also have an abnormally long expiration date, before
action is required to be taken.
Within this area, one violation and one deviation were identified and
are discussed in the following paragraphs:
(1) Failure to perform an adequate 10 CFR 50.59 review and failure to
perform post maintenance testing.
10 CFR 50.59 basically states that the licensee can modify systems
if the licensee determines that the modification does not involve
an unreviewed safety question. Records shall be maintained of the
safety evaluation which includes a detail of the basis used for
the determination.
ANSI 18.7-1976, Section 5.2.19.3 requires that tests shall be
performed following plant maintenance or modifications to confirm
that any change does not reduce the safety of operations. Also,
Section 5.2.6 requires that when equipment is ready to be returned
to service that operating personnel shall verify and document the
equipment's functional acceptability.
>
.
14
Crystal River 3 was in the process of heating up on 6/18/86, in
preparation for scheduled startup on 6/20/86. On the evening of
6/18/86, T-MAR T86-06-20-01 and work request 80209 were generated
in response to improper channel "C" T-HOT indication, RC-4A-TE3.
Technicians found that "C" T-HOT loop resistance temperature
detector had become inadvertently grounded and would require
containment entry for resolution of the prcblem.
T-MAR T86-06-20-01 was initiated to remove the installed circuit
ground in order to prevent the inadvertent ground from causing
false readings in the T-HOT channel. The justification section of
the safety evaluation makes the following statements, "The normal
function of the circuit can be performed without any ground" and
"Since the circuit will remain grounded, the same electrical
configuration exists". The engineering instructions, reference
drawing D8042350, Rev. G, and Bailey Instructions E92-346 and
E92-351 from Instruction Manual 206 Volume 1. These references do
not address removal or transfer of the system ground and do not
contain sufficient detail for determination by electrical
engineering that removal or transfer of system ground does not
affect system operation. Engineering personnel have verified that
a schematic for Rosemount linear bridge circuit is not contained
in the B&W instruction manual and would have to be obtained from
the original manufacturer, Rosemount.
The FSAR assumes that the worst case channel error is il F and
surveillance procedure SP-110 requires a bistable trip range of
=.5 F for a reactor trip function, in order to meet technical
specification requirements. The safety evaluation analysis that
was performed cannot guarantee the accuracy required by this
circuit.
The T-MAR T86-06-20-01 engineering instructions require that the
normal ground "must be re-introduced so the calibration out
voltage of the test circuit will be read correctly". The safety
evaluation has already stated that the system ground is not
required for the normal function of the circuit. This also brings
forth the point that the system is being functionally tested in
other than its normal configuration.
After completion of troubleshooting and completion of the T-MAR,
the channel was not functionally tested as required by ANSI
N18.7-1976. Plant surveillance procedure SP-100, Sections 7.7.2.3
and 7.7.2.4 require that a work request be initiated to trouble-
shoot an inoperable channel and that the applicable section of
SP-110 be completed to verify proper operation. A review of
completed SP-110 procedures and work request 80209 indicate that
the post maintenance testing was not performed as required.
.
15
The inspector also noted that T-MAR T86-08-11-01, which modifies
the T COLD bridge circuit RC-5B-TT3, may also have a deficient
safety evaluation because a similar schematic is required for
circuit analysis. This circuit is used for feedwater control and
is not considered safety related.
The licensee failed to perform an adequate 10 CFR 50.59 review
that would assure that the modification would not involve an
unreviewed safety question. The licensee also failed to perform
post maintenance testing of the affected channel. These items are
identified as violation 302/86-39-01, failure to perform an
adequate 10 CFR 50.59 review and failure to perform post mainten-
ance testing.
(2) FSAR Requirements Are Not Being Met in That RM-L4 Has Been
Inoperable Since September 1979.
The Crystal River FSAR Section 11.4.1., radiation monitoring
design basis states that the radiation monitoring system measures
radioactivity at selected plant locations in order to verify
compliance with 10 CFR 20. The spent fuel cooling water monitor
RM-L4 is provided to detect any radioactivity released in the
spent fuel storage area.
T-MAR 79-08-78 removed RM-L4 from service on 9/5/79 and it also
states that until RM-L4 is relocated, it is inoperative. The
licensee stated that daily grab samples were taken on the spent
fuel pool in order to compensate for removal of the continuous
monitor.
The T-MAR package also had various problems that are detailed in
the following subsections.
'
(1) The 10 CFR 50.59 review was not included in the T-MAR package
and was assumed to have been lost.
(2) The Design Input Record and the Design Data Sheets were not
added to the T-MAR package until 10/16/86.
(3) The T-MAR action completion date was not properly extended as
i required by procedure.
The plant position as stated by the licensee to the inspector was
that a request would be made to change the FSAR to remove the
RM-L4 requirement. Adequate management attention was not given to
this item.
'
.
.16
The FSAR commitment was that RM-L4 would be operational. The
radiation monitor has been out of service since September 1979
and this is a deviation from FSAR requirements. This item is
identified as Deviation 302/86-39-02.
The overall assessment of the design control area is that the total
program is average. In the areas of documentation, implementation,
trending, corrective action and procedural control, the modification
program appears to be well above average. The temporary modifications
program and the backlog of extremely old modifications are below
average. The problems associated with inadequate safety evaluations
also make.this area suspect and should be addressed by FPC management.
c. Licensing and Commitments
Within the functional area of licensing and commitments, which included
noncomforming operations reports (NCORs), LERs, and Special Reports,
the inspector analyzed trending indicators from various sources and
gathered other partinent information upon which to base an evaluation
of quality assurance effectiveness.
Problems identified via operational events are documented and evaluated
on NCORs. These NCORs precipitate into Licensee Event Reports (LERs)
when required by 10 CFR 50.72. The program controlling NCORs and LERs
is governed by compliance procedure CP-111, Documenting, Reporting, and
Reviewing Nonconforming Operations Reports, Revision 34. NCORs are
initially reviewed by the Nuclear Shift Supervisor for restrictions
imposed by Technical Specifications, identifying immediate notification
requirements, determining NCOR validity, and providing reportability
when required by 10 CFR 50.72 and 10 CFR 20.403, etc. The Nuclear
Safety Supervisor is then responsible for assuring the following:
(1) Investigation to determine the cause of the event or condition
when required.
(2) Designation of adequate corrective actions.
(3) Determination of the NRC and Plant Manager review requirements for
the Event Evaluation Summary.
(4) Evaluation and verification of corrective action assignment.
(5) Preparation of LERs and Special Reports.
The Nuclear Compliance Department is responsible for tracking,
coordinating, and documenting NCORs and corrective action on the
Compliance Tracking System.
.
.
17
The inspector reviewed a summary log of all NCORs written from
January 1 through November 15, 1986. The total NCORs issued as of
November 15, 1986, was 199. From these 199 NCORs, a total of 20 LERs
had been generated. A log of the total number of LERs written during
1985 (33) was reviewed by the inspector. This review focused on
repetitive LERs. Cases in point were LERs 85-02, 06, 07, and 19 which
documented problems with the diesel generators. LER 85-02 is repeti-
tive in that this is the third failure of a drain petcock on air start
solenoid air filter EGFL-4, in the "B" Emergency Diesel Generator air
start system.
LER 85-006 documented an automatic actuation of the "A" Engineered
Safeguards (ES) 4160 VAC Bus Undervoltage Protective Relaying which was
caused when electricians were preparing cables for termination. The
conductors were energized causing the conductors to short, thereby
blowing the fuses, and causing the start. LER 85-007 documents an
actuation of the "A" Emergency Diesel Generator automatic start '
circuit. The cause of this actuation was the result of a fuse failure.
The cause of the failure was investigated and was indeterminable. LER
85-007 also confirms that six previous starts have occurred.
LERs document significant problems with personnel errors. LER 85-008
documents an operator error which caused an Emergency Diesel Generator
start. The operator was counseled in the proper control of switching
devices. LER 85-009 documents failure to recognize the need to perform
a surveillance. LER 85-001 documents an operator error in that "A"
Emergency Diesel Generator was out of service and the operability of
the sump pumps in the tunnel containing the DC control feeds to the 230
KV switchgear and the correct breaker alignment for the remaining power
sources were required to be verified once every eight hours by
Technical Specifications. Again, this documents personnel error. LER
85-018 documents a personnel error in that the Nuclear Shift Supervisor
was not notified of a seismic trigger located on the crane ring girder
being out of service. Personnel error problems and failure to follow
i procedure at FPC is the cause of 24 NRC identified violations issued in
1985 and 17 in 1986. A review of NCORs and LERs for this same time
period also documents personnel errors and failure to follow procedures
as a problem at FPC. Failure to initiate adequate correction action to
preclude repetition of personnel errors and failure to follow proce-
dures is perceived to be a singificant weakness in the implementation
of an adequate QA program.
,
The inspector reviewed a random sample of commitments which FPC has
made to the NRC. These commitments are tracked on a computer program,
N0D-09, Nuclear Operations Commitment System (NOCS). The procedure
provides adequate guidance to perform tracking of commitment and
l necessary guidance to preclude deletion of previously identified
i
commitments. However, this procedure does not document actual dates of
implementation, nor does it document dates when commitments will be
implen.ented. Cases in point are commitments made in response to LER
83-025 which require administrative controls to be upgraded to better
address criteria for cable tray loadings. This commitment has only
been partially completed. Therefore, this system appears weak. t
, -. - --- _ _- =- . _ - _ . _ - _ _ _ - , - _
.
.. . .
- _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ - _ _
,
.
!
18
-
d. Quality Assurance Department
The purpose of this portion of the inspection was to assess the
effectiveness of the Quality Programs Department (QPD) to prevent,
identify, and correct problems. To accomplish this, QPD's nonconfor-
mance trending program, audit and surveillance program, and associated
overview activities were reviewed. Interviews were also conducted with
responsible upper management. Both strengths and weaknesses were
identified in all areas.
The Quality Programs Department has implemented a tracking and trending
program. This pragram has been evolving, and a modification to improve
its trend reporting capabilities was recently completed. To evaluate
the effectiveness of QPD's trend program, QPD's trend analysis reports
for the fourth quarter of 1985 through the third quarter of 1986 and
the CR-3 Nuclear Operations Quarterly Report for the third quarter of
1986 were reviewed.
It was evident that significant resources are being used to generate
trend information. This reflects positively on management's commitment
to quality. As experience with the trending program has been gained,
the reports have begun to provide more meaningful information to
management.
Areas for improvement also were identified in QPD's trend program.
These improvements include the need to develop a trending approach that
evaluates the significance of reported deficiencies, provide for more
in depth analyses, and provide for more responsive action as trends are
identified.
These recommendations for improvement are based on the fact that most
of the quarterly trend reports represented superficial analysis of
trends. Specifically, the reports emphasized numbers of deficiencies,
either open or closed, and their relationship to a moving average.
There did not appear to be a method of connecting the significance of
the deficiencies to the data provided or performing any analyses to
determine significant plant trends of a technical or operational
nature. With the exception of the trend report for the third quarter
of 1986, the trend reports appear to have not provided management with
information that would enhance safe plant operations.
The trend report for the third quarter of 1986 provides more signifi-
cant analytical information. The analyses in this report appear to be
the direct result of the modifications made to the trending / tracking
computer program. While the analyses represented a better grasp of
analytical techniques, there appears to be room for improvement in
following up on potential problem areas after they are identified by
the trending program. This could be accomplished through more
aggressive management attention.
' ' ' ' - '
_ _ _ - _ _ _ _ _ __-- -
- - _ _ _ _ _ _ - _ _ _ _ _ _ _ _ _ _ - - _ _ _
._ _
.
19
QPD's audit program was reviewed to assess its ability to prevent,
identify and correct problems of a significant technical nature.
Emphasis was placed on audits that were conducted in the areas of
design, modifications, and corrective action. The review included an
assessment of QPD's corrective action follow up capabilities, an
assessment of the qualifications of the audit staff, and an evaluation
of the involvement of overview organizations in the audit and correc-
tive action process.
An initial evaluation was made of the qualification of the audit staff
and other auditors that contributed to the audits being reviewed. The
audit staff was composed of experienced programmatic QA personnel who
had little actual design and operational expertise. This apparent
staffing weakness was mitigated by the use of technical consultants on
the various audits that were reviewed.
The inspector reviewed design and modifications audits conducted during
the last 18 months. Specifically, audit QPD-2/2 (modifications) and
QPD-279 (design control) were reviewed. The review of these audits
showed that the audit program was evolving from a strictly compliance-
to procedures audit function to one that had a mix of compliance and
'
actual work performance. QPD currently plans to revise their audit
practices for 1987 to ensure that audits are more technically oriented.
These audits will emphasize selective detailed review of specific plant
activities. This audit process has neither been approved by upper
management nor implemented; the effectiveness of the audit program
could not be evaluated. These changes of audit focus are considered to
be strengths of the program.
Associated with the audit program is the FPC Nuclear General Review
Committee (NGRC) subcommittee on audits. This subcommittee provides an
NGRC member to be a liaison for each audit. Discussion with the
subcommittee chairman revealed that future NGRC subcommittee plans are
to have the audit team leader report, in person, the audit results to
the subcommittee. Past practice was to have the liaison member
complete a checklist evaluating the audit. The current and proposed
NGRC activity is considered a strength.
QPD's ability to correct and prevent problems from recurring was
evaluated through an analysis of nonconforming items and corrective
action audit reports associated LERs and NCORs, and QPD's followup on
their audit findings.
Discussions were held with the Director, Quality Programs and the
Supervisor, Quality Audits on corrective action follow up. The
inspector also reviewed Corrective Action audits QP-271, QP-283, and
QP-290; associated LERs and NCORs that were reviewed as part of the
audit activities; and the followup to the audit findings of audits
QP-272 and QP-279. The review indicated that QPD is not always
effectively performing corrective action followup. In many instances,
. . .
._ .. _
--
.
20
the effectiveness of the corrective action taken to prevent recurrences
was not normally being assessed and the QPD did not appear to be
aggressive in performing a review to determine the extent and implica-
tions of an identified deficiency on current and past plant activities.
For example, the review of the three corrective action audits indicated
that LERs and NCORs were not reviewed to determine if the corrective-
action taken prevented recurrence of the problem or if the event was
the precursor of a wider problem. Generally, the audits appeared to
only evaluate compliance to the documented program.
Discussions with QPD management revealed an understanding of the
principles of in-depth corrective action follow up. Even so, there
appeared to be some reluctance on the part of QPD to accept a more
active and aggressive role.
The formation of a Management Review Committee to review the status of
NRC, INPO and QPD commitments was considered a strength as it provided
for better communications between the various groups.
The overall evaluation of the quality assurance organization's current
ability to prevent, identify, and correct problems is that the
organization is average. Some potential strengths that were identified
remain to be effectively implemented and with appropriate management
attention, identified weaknesses can be corrected.
Overall Conclusion
The objective of this inspection was to assess quality assurance effective-
ness. After reviewing the four areas inspected and various observations,
the inspection team concluded that the overall quality assurance effective-
ness is average with respect to other licensees inspected. In general,
weaknesses were balanced with strengths though the overall trend is toward
improving performance. This perceived trend is reflected in the trending
indicator data and appears to be the result of increased management
attention in areas of poor performance.
6. Licensee Actions in Previously Identified Inspection Findings (92701)
a. (Closed) Inspector Followup Item 302/85-07-03: Revision to Procedure
Control and Document Retention Program (PCDR).
The inspector reviewed Nuclear Quality Assurance Practice, Instruc-
tions, and Procedures Requirement Code: PCDR. An errata sheet was
added to this document dated August 30, 1985, adding 10 CFR 50,
Appendix B, Criterion VI and the Final Safety Evaluation Report Section
,
1.7.1.6 to Section 3, Basis. This section includes the documents which
'
form the basis for this practice. The inspector also reviewed AI-404,
Review of Technical Information, Revision 4 and N00-6, Technical
l Information Program, Revision 1.
t
_
21
AI-404 establishes methods which ensure consistent thorough documented
reviews of information pertaining to safety-related components
installed at Crystal River. Information reviewed will include but is
not limi',ed to vendor manuals, vendor manual changes, and vendor
letters. N00-6 establishes methods of handling and processing
technical information to assure current and accurate information is
provided at Crystal River 3. This technical information includes
vendor supplied engineeririg and equipment information such as drawings,
correspondence, manuals, and similar technical documents and changes
thereto.
b. (Closed) Inspector Followsp Item 302/86-01-02: Revision of Program
Controls for Rework Process.
The inspector reviewed M0P-405, Job Completion, Walkdown, Turnover, and
Nonconforming Item Reporting, . Revision 8. Section 4.2.6.7.a, now
includes requirements that exceptions to dispositioned rework require
Nuclear Engineering approval. This appreval is documented by Field
Change Notices (FCNs), modification approval records (MARS), Work
Package, Work Request, etc.
c. (Closed) Inspector Followup Item 302/86-01-03: Completion and Closecut
of Appendix R Fire Detection Installation.
The inspector reviewed an FPC QA Record Transmittal Sheet signed by the
Nuclear Modifications Supervisor on April 1, 1986. This sheet
documented transmittal of MAR 82-10-19-10, FCN 1 and 2, and Work
Requests Nos. 51295, 60569, 62122, 65041, 66963, 68699, 69107, and
76961 to the Records Management Section for their inclusion into the
Plant Quality file. The inspector verified that applicable cable pull
and cable termination cards were included in the completed MAR
documentation. The inspector was unable to verify that anchor bolt
installation sheets were included in the MAR documentation since these
records were apparently lost. A letter has been included in the MAR
documentation stating that these documents were lost. The loss of
anchor bolt installation sheets was documented on engineering change
notice (ECN) 2679 dated March 4, 1986.
l
t
.- __
- - -
_ _ _ _ _
_
D
21
AI-404 establisnes methods which ensure consistent thorough documented
reviews of information pertaining to safety-related components
installed at Crystal River. Infcrmation reviewed will include but is
not limited to vendor manuals, vendor manual changes, and vendor
letters. N00-6 establishes methods of handling and processing
technical information to assure current and accurate information is
provided at Crystal River 3. This technical information includes
vendor supplied engineering and equipment information such as drawings,
correspondence, manuals, and similar technical documents and changes
thereto.
b. (Closed) Inspector Followup Item 302/86-01-02: Revision of Program
Controls for Rework Process.
The inspector reviewed MOP-405, Job Completion, Walkdown, Turnover, and
Nonconforming Item Reporting, Revision 8. Section 4.2.6.7.a, now
includes requirements that exceptions to dispositioned rework require
Nuclear Engineering approval. This approval is documented by Field
Change Notices (FCNs), modification approval records (MARS), Work
Package, Work Request, etc.
c. (Closed) Inspector Followup Item 302/86-01-03: Completion and Closeout
of Appendix R Fire Detection Installation.
The inspector reviewed an FPC QA Record Transmittal Sheet signed by the
Nuclear Modifications Supervisor on April 1, 1986. This sheet
documented transmittal of RAR 82-10-19-10, FCN 1 and 2, and Work
Requests Nos. 51295, 60569, 62122, 65041, 66963, 68699, 69107, and
76961 to the Records Management Section for their inclusion into the
Plant Quality file. The inspector verified that applicable cable pull
and cable termination cards were included in the completed MAR
documentation. The in:pector was unable to verify that anchor bolt
installation sheets were included in the MAR documentation since these
records were apparently lost. A letter has been included in the MAR
documentation stating thao these documents were lost. The loss of
dnchor bolt installation sheets was documented on engineering change
notice (ECN) 2679 dated March 4, 1986
i
1
t
- - - - -
I