IR 05000315/1998016
| ML17335A250 | |
| Person / Time | |
|---|---|
| Site: | Cook  |
| Issue date: | 09/23/1998 |
| From: | NRC OFFICE OF INSPECTION & ENFORCEMENT (IE REGION III) |
| To: | |
| Shared Package | |
| ML17335A249 | List: |
| References | |
| 50-315-98-16, 50-316-98-16, NUDOCS 9810020295 | |
| Download: ML17335A250 (41) | |
Text
~
~
C U.S. NUCLEAR REGULATORYCOMMISSION
REGION III
Docket Nos:
License Nos:
50-315; 50-316 DPR-58; DPR-74 Report No:
50-315/98016(DRP); 50-316/98016(DRP)
Licensee:
Indiana and Michigan Power 500 Circle Drive Buchanan, Ml49107-1395 Facility:
Location:
Donald C. Cook Nuclear Generating Plant 1 Cook Place Bridgman, Ml 49106 Dates:
July 17 through August 27, 1998 Inspectors:
B. L. Bartlett, Senior Resident Inspector B. J. Fuller, Resident Inspector J. D. Maynen, Resident Inspector M. Holmberg, Reactor Inspector Approved by:
B. L. Burgess, Chief Reactor Projects Branch 6 98i0020295 980'723 PDR ADOCK 050003i5
EXECUTIVESUMMARY D. C. Cook Units 1 and 2 NRC Inspection Report 50-315/98016(DRP); 50-316/98016(DRP)
This inspection included aspects of. licensee operations, maintenance, engineering, and plant support. The report covers a 5-week period of resident inspection and includes the follow-up to issues identified during previous inspection reports.
~Oerations A shift Technical Advisor demonstrated a good questioning attitude when he challenged the adequacy of compensatory measures contained in a surveillance procedure during a surveillance test. The compensatory measures provided guidance to prevent plant operators from removing both trains of the component cooling water system from service.
(Section 01.1a)
A review of the cause for a higher than expected value for calculated Pressurizer delta-T determined that, following the removal from service of the west Component Cooling Water (CCW) pump and the placement of the east CCW pump into operation, CCW system flowincreased, resulting in lower Volume Control Tank temperature.
The review also determined that control room operators did not monitor plant parameters closely enough to detect the change in Pressurizer delta-t, resulting in exceeding procedural limits. Technical Specification limits were not exceeded.
(Section 01.1b)
During control room observations the inspectors determined that the licensee's contingency planning for the loss of the remaining train of Residual Heat Removal (RHR) did not clearly address the least risk significant course of action.
Because the contingency action was not clear, uncertainty existed among operators as to which action should be taken in the event the remaining train of decay heat removal was lost.
The remaining train of RHR was not lost and the licensee restored operability to both trains of RHR. (Section 01.2)
During a routine review of the control room caution tag logs, the inspectors identified a number of tags that were still hanging even though the issues listed on the tag had been resolved.
A quality assurance audit had previously identified a discrepancy between two procedures, one which required monthly reviews of the caution tag logs and one which excluded the review during outages.
The inspectors identified that the licensee's corrective action to eliminate the discrepancy was not implemented in a timely manner.
(Section 01.3)
The licensee had previously identified that their Corrective Action Program was ineffective. Reviews of current Condition Reports by NRC inspectors determined that additional changes to correct the ineffective program were required and were being performed by the licensee.
(Section 07.1)
~
~
Maintenance P
Overall, maintenance work was performed using approved work procedures and reflected good maintenance practices.
However, the licensee identified two maintenance errors:
a wrong valve was cut out on the Unit 2 main condenser and a maintenance test resulted in an electrical arc on the 2CD auxiliary transformer supplemental cooling system circuitry. These events challenged the adequacy of the corrective actions in progress to improve the work control program and indicated that room for improvement remains.
(Section M1.1)
While lining up the fire water tanks in preparation for a test, the suction source for all three fire pumps was isolated, resulting in the automatic start of all of the fire pumps.
The inspectors concluded that the auxiliary equipment operator was directed to use the engineering test procedure to align the fire water tanks, an activity forwhich the test procedure was not intended.
(Section M1.2)
The inspectors concluded that the work control process weaknesses identified in 1996 still exist. Past observations, recent observations, and licensee self-assessments indicated that the weaknesses have not been corrected.
The inspectors did not identify any specific examples where these weaknesses resulted in a risk significant issue.
However, the inspectors noted that the licensee's operations personnel were repeatedly challenged to ensure that safety margins were maintained.
(Section M1.3)
Ice project personnel identified shortcomings in the control of ice bagged for off-site storage, and prompt corrective actions were implemented.
Foreign material exclusion practices were improved to provide assurance that the ice would be of the required quality when loaded into the Unit 1 ice condenser.
The physical security of the ice during transport and storage was improved to maintain control over the ice during all phases of the work. (Section M1.4)
Repair crews showed good attention to detail during followup ice basket inspections which identified additional damage to the ice baskets.
The licensee took prompt action to determine and correct the causes of the original inspections which had missed some ice basket sections needing repair. (Section M1.5)
Inspection and repair work to restore damaged and out of specification components for the Unit 1 ice condenser appeared affective and comprehensive.
The decision to melt the ice in both ice condensers facilitated effective inspection and repair, which demonstrated the licensee's commitment to a quality repair effort for the ice condensers.
However, as a consequence of engineering and maintenance staff inexperience with this non-routine evolution (ice condenser melt out), water was entrained within the concrete ice bed subfloor of Unit 1. Lessons learned from Unit 1, enabled the licensee to implement actions that prevented substantive water intrusion into the Unit 2 floor. (Section M2.1)
En<ninee~ng
~
Licensee engineering personnel were involved in the assessments, repairs, and modifications to the ice condenser.
Extensive involvement of engineering was noted by the inspectors during their assessment of the ice condenser corrective actions.
(Section E1)
~e
~
During normal resident inspection activities, routine observations were conducted in the area of security and safeguards, fire protection, and health physics activities.
No discrepancies were note ~
~
~
~
Re ort Details Summa of Plant Status Unit 1 remained in Mode 5, Cold Shutdown, during this inspection period. The licensee presented the schedule for restart of Unit 1 to the NRC on July.9, 1998. The schedule indicates that proposed maintenance activities willextend into early November, followed by a 3 to 6-week heat-up and start-up period. Work on Unit 1 willtake precedence over Unit 2.
Unit 2 remained in Mode 5, Cold Shutdown, during this inspection period. The restart schedule for Unit 2 is not yet complete.
I. 0 erations
Conduct of Operations 01.1 General Comments 71707 The inspectors conducted frequent reviews of control room and in-plant operation of equipment during the extended outage of both reactor units. During observations of control room activities such as shift tumovers, operator response to annunciators, and equipment operations, the inspectors identified that control room operators monitored and operated equipment in a professional manner.
a.
During the performance of a surveillance on the Unit 2 West Component Cooling Water (CCW) pump, the Shift Technical Advisor (STA) questioned the adequacy of the compensatory measures contained in the surveillance test procedure.
The test procedure directed operators to close the discharge cross-tie valve and the heat exchanger outlet valve for the train not under test. The STA pointed out that ifthe train under test was being tested to demonstrate operability following maintenance, that both trains could be considered inoperable resulting in an inadvertent entry into Technical Specification (TS) 3.0.3. The surveillance procedure addressed the issue of cross-train operability; however, the STA was unsure of the adequacy of the proceduralized compensations.
A condition report (CR) was written (CR 98-4125) and a prompt reportability evaluation was begun.
The licensee's review was not complete at the end of the report period; consequently, it had not yet been determined whether or not both trains ever were or could have been inoperable despite the proceduralized compensations.
The STA's comments reflected a good questioning attitude.
b.
During the performance of a Unit 2 control room surveillance on August 17, 1998,,the Reactor Operator (RO) identified a higher than expected difference between the Volume Control Tank (VCT) temperature and the pressurizer liquid temperature.
The difference between the VCT and the pressurizer liquid temperature is calculated to determine
pressurizer delta - T. The pressurizer delta - T in this case was calculated at 315 degrees.
The licensee's administrative limitfor pressurizer delta - T was less than 300 degrees, and the TS requirement was less than 320 degrees.
During Mode 5 conditions, the pressurizer delta - T was normally maintained at approximately 287 degrees.
The difference was identified approximately 6 hours6.944444e-5 days <br />0.00167 hours <br />9.920635e-6 weeks <br />2.283e-6 months <br /> after switching the operating CCW pump from the west to the east pump.
The licensee's review confirmed that the pressurizer delta - T remained below the TS limit. The review also determined that:
~
due to the extended shutdown, there were low levels of reactor vessel decay heat,
~
the Unit 2 east CCW pump was started, and then the Unit 2 west CCW pump was shutdown for routine maintenance,
~
due to slight differences in fiowpaths and lower piping resistance, the change in CCW pumps resulted in a slightly higher flowrate to the letdown heat exchanger, and
~
although the increased flow rate was small, the heat input to the letdown heat exchanger from the reactor coolant system was also small due to the low levels of decay heat.
The higher flow rate resulted in a lower VCT temperature and resultant higher pressurizer delta - T.
During normal full power operations the effect of the higher CCW flow rate would not be significant due to the much higher heat input to the letdown heat exchanger, but the operators failed to recognize the difference in how the plant would respond white in Mode 5. Notwithstanding the difference in the plant's response to changes in the CCW system, the operators should have monitored their panels following the change of the CCW pumps.
The licensee's review determined that following the switch of the CCW pumps, appropriate monitoring of control room panels for changes in system temperatures by the control room operators was not performed.
01.2 Removal of East Residual Heat Train Outa e for Corrective Maintenance Unit 2 a.
Ins ection Sco e
1707 During routine control room observations the inspectors questioned the licensee's contingency plans for the loss of residual heat removal (RHR) capability during routine maintenance.
The inspectors reviewed the licensee's contingency plans, reviewed the work control process (additional detail on the work control process is located in the Maintenance section of this report), and followed the licensee's corrective actions.
b.
Observations and Findin s During system walk downs, a number of minor leaks were identified on the Unit 2 east RHR system.
The licensee decided to remove the Unit 2 east RHR train from service
and drain the system to repair the leakage.
The work was scheduled to begin on August 3, 1998.
On August 1, 1998, the licensee developed a contingency plan for responding to the loss of the Unit 2 west RHR train while the Unit 2 east RHR train was out of service.
The contingency plan required the operators to use the abnormal operating procedure, 02 Operations Head Procedure (OHP) 4022.017.001, "Loss of RHR Cooling," to start a reactor coolant pump (RCP) in order to restore forced reactor cooling in accordance with TS 3.4.1.3.
The contingency plan noted that starting an RCP would result in the reactor coolant system heating up above 200'F.
On August 3, 1998, the licensee removed the Unit 2 east RHR train from service.
The inspectors reviewed the contingency plan and questioned the licensee's preparedness to enter Mode 4. The inspectors determined that starting an RCP would result in violating several Mode 4 TS wh'en RCS temperature exceeded 200'F. The Operations Superintendent stopped the Unit 2 east RHR train work, and the system was restored to service pending a review of the NRC inspectors'omments and the contingency plan.
During the time that the Unit 2 east RHR train was out of service, the Unit 2 west RHR train was operating; therefore, a violation of technical specifications did not occur.
However, the inspectors determined that ifthe Unit 2 west RHR train had been lost during the Unit 2 east RHR work, and the operators followed the contingency plan as written, several violatons of TS would have occurred when Mode 4 was entered.
In the event of a loss of the remaining operating train of RHR the licensed operators would have been in the difficultcircumstance of deciding which TS to violate.
By complying with TS 3.4.1.3. to start a RCP, the operators would add energy to the RCS resulting in violations of several Mode 4 requirements.
By not operating a RCP the licensee would have violated TS 3.4.1.3. The inspectors also questioned several onshift operators regarding how long itwould take to reach the temperaure that would cause boiling in the RCS from present plant conditions following the loss of the remaining operating RHR train. None of the operators questioned knew the answer.
Information such as this would have helped plant personnel decide which TS was more risk significant given the current plant conditions.
The inspectors'nterviews of various licensed operators determined that some operators had decided that in the event of a loss of decay heat removal they would delay starting a RCP as they believed this was the more appropriate decision.
Yet, licensee management personnel had determined it was more appropriate to start a RCP and comply with TS 3.4.1.3. The failure to reach consensus regarding the appropriate contingency resulted in uncertainty as to which action would be completed to address the loss of decay heat removal.
In addition to the contingency plan problems identified by the inspectors, the licensee identified several other work planning issues associated with the Unit 2 east RHR work.
The following issues were identified:
~
The initial clearance boundaries (system isolation points) were inadequate for the scheduled work. This was identified during the licensee's normal clearance
review process (prior to the clearance order being placed into effect) by an assistant shift supervisor, and the clearance was corrected.
(CR 98-3804)
~
Five jobs orders were added to the system outage scope after the shutdown risk review had been performed by a shift technical advisor and therefore received no risk review. These jobs were identified by a shift technical advisor during a normal review of the Unit 2 east RHR system clearance (prior to the clearance order being placed into effect). (CR 98-3769)
~
On August 3, 1998, several cables had been removed from a Unit 2 containment temporary service penetration, breaking a foamed seal which was necessary for containment closure capability. Containment closure capability was established in preparation for the Unit 2 east RHR work. The removed cables were identified by operations personnel during a inspection of containment penetrations just prior to the scheduled start of the RHR outage.
(CR 98-3847)
~
On August 18, 1998, the licensee determined that institutional knowledge had been lost regarding an internal requirement to have the ice condenser available before entering mid-loop reactor vessel level conditions. The loss of RHR analysis while the RCS was drained to mid-loop conditions assumed that sufficient ice was in the ice condenser and that ice would be available for melting to assist with heat removal.
During this RHR outage there were no plans to enter a mid-loop level condition. (CR 98-4164)
While there was no actual threat to plant safety, the licensee's contingency planning did not address the least risk significant course of action in the event that the remaining operating train of RHR had failed. The failure to reach consensus regarding the appropriate contingency resulted in uncertainty as to which action control room operators would take to address the loss of decay heat removal.
Conclusions During control room observations the inspectors determined that the licensee's contingency planning for the loss of the remaining train of Residual Heat Removal (RHR) did not clearly address the least risk significant course of action. Because the contingency action was not clear, uncertainty existed among operators as to which action should be taken in the event the remaining train of decay heat removal was lost.
The remaining train of RHR was not lost and the licensee restored operability to both trains of RH Out of Date Caution Ta s Both Units lns ection Sco e
1707 During a routine review of caution tag logs in the control rooms, the inspectors identified several caution tags which appeared to no longer be appropriate.
The inspectors performed a more detailed review of the caution tag logs and interviewed personnel to determine ifrisk significant caution tags which were no longer appropriate were in the plant and to determine the root cause.
Observations and Findin s During a routine review of the caution tag logs on August 6, 1998, the inspectors identified several tags which appeared to be invalid. For example, Caution Tag 2-2976, dated March 10, 1997, stated that, "Auxhoist of Unit 2 polar crane is limited to 21 tons until hoist capacity is resolved with Whiting Corp." As the polar crane aux hoist was used for the Unit 2 refueling outage in the fall of 1997 to carry loads in excess of 21 tons, the inspectors presumed the tag was not valid.
The inspectors performed a cursory review of 116 caution tags (58 in each Unit) and identified others that appeared to be invalid. The inspectors questioned the on duty Shift Technical Advisor (STA) as to when an audit of the caution tag logs had last been performed as required by the procedure.
The STA responded that Condition Report (CR) 98-1 910 had been written stating that contrary to Plant Manager'
Instruction (PMI) 2110, Revision 23, Clearance Permit System, that the STAs were not performing a monthly audit of the caution tags.
. During an audit performed on May 5, 1998, by Plant Performance Assurance (PPA), the failure of the STAs to perform a monthly review of the caution tags had been identified.
Condition Report 98-1910 was written to document that the STAs had stopped
.
performing the monthly review during the extended forced outage.
The STAs had been given written guidance addressing the monthly reviews; however, it was in a separate, lower tier procedure.
Atthe time of this inspection the CR remained open, with open corrective action to revise PMI - 2110 and review the STA guidelines by September 30, 1998. The inspectors reviewed PMI -2110 on August 6, 1998, and determined that the procedure had not been revised.
The monthly STA review of caution tag logs was not being performed even though PMI - 2110 had not yet been revised.
Operations department management was informed on August 7, 1998, of the inspectors'indings related to the control of caution tags.
The licensee performed a root cause analysis of the failure to properly control caution tags and determine the reason for the failure to revise or followPMI -2110. Neither the inspectors'eview nor the licensee's review of the caution tag log identified any safety significant or risk significant conditions as a result of the out of date caution tags.
Members of the licensee's staff had recognized the need to change the proceduralized review of the caution tag logs monthly to exclude outages; however, the need to also revise PMI - 2110 was misse ~
~
This failure constitutes a violation of minor significance and is not subject to formal enforcement action.
The inspectors'eview of PMI -2110 determined that the procedure provided instructions to plant personnel on how to initiate a caution tag; however, there was no guidance or instructions regarding the removal of caution tags.
The licensee's root cause analysis also identified this procedural weakness and corrective actions to revise the procedure were initiated.
The inspectors also reviewed the other findings of the quality assurance audit which had identified the discrepancy between the PMI and the lower tier STA procedures.
The other audit findings were verified to have been corrected or to have corrective actions in progress.
No other examples of missed corrective actions to the audit were identified.
c.
Conclusions During a routine review of the control room caution tag logs, the inspectors identified a number of tags that were still hanging even though the issues listed on the tag had been resolved.
A quality assurance audit had previously identified a discrepancy between two procedures, one which required monthly reviews of the caution tag logs and one which excluded the review during outages.
The inspectors identified that the licensee's corrective action to eliminate the discrepancy was not implemented in a timely manner.
07.1 a.
-Quality Assurance in Operations Failures to Im lement Corrective Actions Both Units Ins ection Sco e 40500 During the followup to the licensee's failure to perform a monthly review of caution tags, the inspectors identified that the licensee's corrective actions had not been implemented and that the issue of invalid caution tags was not adequately addressed.
The inspectors performed additional reviews of selected licensee procedures, programs, and documents in order to more fullyassess the failure to implement corrective action.
Observations and Findin s In December of 1997, a licensee self-assessment
{PPA Audit PA-97-29) concluded "...
the implementation of the Corrective Action Program remains ineffective." The primary problems with the corrective action program, "... have been in the areas of Condition Report investigations, timeliness and the adequacy of preventive actions." The licensee also concluded that, "Program changes and personnel training, that were implemented during 1997, have failed to resolve these program deficiencies."
The failure to revise the PMI on caution tag logs discussed in Section 01.2, while minor, was an example of a continuing failure to resolve program deficiencies.
Other recent licensee identified examples include:
~
CR 98-4246 The licensee has a flowindicator on the cross-tie between units which allows the charging pumps from one unit to supply the charging header of the other unit.
Even though the cross-tie capability had never been used (except for initial testing) the TSs required that the flowindicator be calibrated once per 18 months.
During a review of the TS surveillance program as a part of the restart readiness review, the licensee identified that the flow indicator had not been calibrated since early 1993 and a CR was written on July 23, 1998.
On August 10, 1998, the licensee identified that the root cause analysis was required to be done that day and that the investigation had not yet begun.
Subsequently, a Licensee Event Report (LER) was issued within the time limits of 10 CFR 50.73, despite the loss in investigative time.
CR 98-4196 I
On August 19, 1997, the licensee identified that some preventive maintenance (PM) recurring tasks were being closed out without going through the required PM review process.
Part of the corrective action was to send a summary of changes to the PM program to the affected section heads.
On August 10, 1998, changes to the PM program were made but a summary of the changes to the PM program had not been sent to the affected section heads.
CR 98-4013 PPA Audit 98-20 identified that in four out of 63 jobs where test equipment was used, the job order was not referenced on the applicable issue ticket. This was important in order to ensure that iftest equipment was found to be out of calibration that all appropriate job activities could be identified. The audit also identified that this issue was repetitive and that it had been found in Audits 92-01, 94-15, and 96-14.
ln response to PPA Audit PA-97-29, and issues identified during the current extended shutdown, the licensee has implemented major changes to the corrective action program. Additional changes are being planned and implemented by the licensee.
These changes are being monitored by the NRC staff and willbe reviewed as part of the NRC Manual Chapter 0350 process.
The breakdown of the licensee's corrective action program and followup to the licensee's subsequent modifications is being tracked as Item 2 of the Case Specific Checklist attached to the July 30, 1998, letter to the licensee.
Pending the licensee's modifications to the corrective action program and the NRC's review of the effectiveness of the corrective action program, this item willremain open as an apparent violation (EEI 50-315/98016-01; 50-316/98016-01(DRP)).
c.
Conclusions
The licensee had previously identified that their Corrective Action Program was ineffective. Reviews of current Condition Reports by NRC inspectors determined that additional changes to corrective the ineffective program were required and were being performed by the licensee.
II. Maintenance M1 Conduct of Maintenance M1.1 General Comments Ins ection Sco e 62707 and 61726 Portions of the following maintenance job orders, action requests, and surveillance activities were observed or reviewed by the inspectors:
Job Order (JO) C44643, Replace flange fasteners on the Unit 1AB Diesel Generator (D/G) diesel driven lubricating oil pump JO C45487, Repair air leak on Unit 2 AB D/G ¹2 starting air compressor flexible hose JO R68838, Lubricate and clean the Unit 2 west Component Cooling Water pump motor JO C39794, Cut out and replace Unit 2 "C" Low Pressure Condenser, pressure instrument vent valve 2-TPS-432-IV
~
JO C39383, Perform setpoint change to 4 kV Buses 2A and 2B normal feed auxiliary transformer coil temperature controller 2-23-CT-AB 02 - Operations Head Procedure (OHP) 4030, Surveillance Test Procedure (STP).027AB, Revision 10, Unit 2 AB Diesel Generator Operability Test (Train B)
02-OHP 4030.STP.020W, Revision 5, Unit 2 West Component Cooling Water Loop Surveillance Test
Observations and Findin s The inspectors observed that, overall, the workers followed procedures and appropriately documented the required information. Significant observations are noted below:
b.1 Cut Out and Re lace Instrument Vent Valve 2-TPS-432-IV On August 18, 1998, the licensee identified that a wrong valve was cut out durin'g the performance of JO C39794.
Valve 2-TPS-421-IV, located near the 2B main condenser waterbox, was cut out instead of valve 2-TPS-432-IV, located near the 2C main condenser waterbox. The valves were installed in an identical configuration and properly labeled.
The worker immediately realized the error and notified his supervision.
The licensee investigated the event and determined that the cause of the event was personnel error in that the worker did not check the valve label prior to cutting the valve out. The worker was counseled, and an action request was written to replace the improperly cut out valve.
The inspectors reviewed the licensee's findings and interviewed the supervisor.
The inspectors did not identify any deficiencies in the licensee's findings for this event.
b.2 Perform Set oint Chan e to Tem erature Controller 2-23-CT-AB On August 19, 1998, maintenance personnel were performing a surveillance test of the temperature switch which controlled the 240 Vac supplemental cooling system for the Unit 2 CD auxiliary transformer (TR2CD). The test consisted of raising the temperature switch test inputs to the desired setpoints and verifying that the supplemental cooling system automatically started as designed.
When the setpoint to start the second group of four cooling fans was reached, an unexpected electrical arc occurred, and testing was immediately stopped.
No injuries were reported; however, due to the potential for personal injury from this type of event, the licensee started a root cause investigation.
On March 10, 1998, Action Request (AR) A158248 was written and indicated that a standing alarm was received in the control room indicating the voltage failure of a cooling motor fan on either the TR2CD or the 201CD reserve feed transformer (TR201CD). The AR also stated that neither TR2CD supplemental cooling fan groups could be operated.
On August 15, 1998, the AR was amended to state that an arc occurred on the southern bank of fans while attempting to swap cooling groups for a post-maintenance test. The inspectors did not identify any action in response to the AR which would prevent operation of the affected cooling group.
No caution tags or danger tags were placed at the TR2CD supplemental cooling system control panel which would have indicated a degraded condition to the maintenance personnel performing the test.
The inspectors discussed the electrical arc with the unit supervisor who authorized the maintenance test.
He stated that he was aware of the AR on the TR2CD fan, but he was not aware any restrictions on the operation of the TR2CD supplemental cooling system.
On August 19, 1998, after troubleshooting the TR2CD supplemental cooling circuitry, licensee instrumentation and control technicians identified a short in the control
circuitry downstream of the second cooling group breaker.
The licensee concluded that this short was the most likelysource of the August 19, 1998, electrical arc.
Conclusions Overall, maintenance work was performed using approved work procedures and reflected good maintenance practices.
However, the licensee identified two maintenance errors: a wrong valve was cut out on the Unit 2 main condenser and a maintenance test resulted in an electrical arc on the 2CD auxiliary transformer supplemental cooling system circuitry. These events challenged the adequacy of the corrective actions in progress to improve the work control program and indicated that room for improvement remains.
Closed Ins ection Follow-u Item 50-315/98012-03:
Review of licensee's root cause assessment for an event that involved the operation of all three fire pumps without a suction source.
On May 5, 1998, while lining up the fire system suction source in preparation for a routine surveillance, an auxiliary equipment operator (AEO)
inadvertently isolated both fire water tanks, resulting in all three fire pumps automatically starting.
Licensee management assigned a root cause investigation team to determine the sequence of events and the cause or causes leading to the inadvertent isolation of the fire pump suction source.
On July 31, 1998, the investigation team issued the final root cause analysis report.
tns ection Sco e 92902 The inspectors followed the team's investigation.
Documentation reviewed included the engineering test procedure and the final root cause analysis report.
~
- 12 Engineering Head Procedure (EHP) 4030 Administrative Technical Requirement (ATR).223.001, "Fire Pump Performance and Starting Sequence Tests," Revision 0
~
CR 98-1904, Fire protection system pumps were run for approximately 5 minutes without a suction supply Observations and Findin s The investigation team found that the inadvertent isolation of the fire pump suction was due to the Unit Supervisor (US) instructing the AEO to use the valve lineup in the engineering test procedure to align the fire pump system.
However, the test procedure objective was to verify the operability of the fire water pumps, not to align the system.
The licensee's team also identified numerous communications errors which contributed to the poor understanding by the control room team and AEO about what fire protection system configuration was required to perform the fire pump test.
Following this event, the licensee revised the operations and engineering briefing guides.
Licensee management required use of the appropriate briefing guide at every pre-job brief and stressed the use of system flowprints during briefings. The condition
report was reviewed with each operating shift, and the revised briefing guides were added to the operations training schedule.
Additionally, the engineering test procedure was revised to remove the valve lineup verification. The licensee planned to evaluate the effectiveness of these corrective actions after six months. The inspectors reviewed the corrective actions and concluded that the actions completed and planned appeared adequate to prevent recurrence.
10 CFR Part 50 Appendix B, Criterion V, requires, in part, that activities affecting quality shall be prescribed by documented instructions, procedures, or drawings, of a type appropriate to the circumstances and shall be accomplished in accordance with these instructions, procedures, or drawings. The licensee's operations department had written Procedure 12 OHP 4021.066.001, "Fire Protection System (Water) Operation," to provide instructions for aligning the fire water tanks.
However, the US instructed the AEO to perform the lineup in the engineering test procedure, an activity not covered by the test procedure objectives.
The inspectors concluded that using the test procedure to align the fire water tanks was a violation of 10 CFR Part 50, Appendix B, Criterion V, in
that an activity affecting quality was not accomplished in accordance with a procedure appropriate to the circumstances.
This non-repetitive, licensee-identified and corrected violation is being treated as a non-cited violation, consistent with Section VII.B.1 of the NRC Enforcement Policy (50-315/98016-02(DRP)).
Conclusions While lining up the fire water tanks in preparation for a test, the suction source for all three fire pumps was isolated, resulting in the automatic start of all of the fire pumps The inspectors concluded that, due to a personnel error, the auxiliary equipment operator was directed to use the engineering test procedure to align the fire water tanks, an activity for which the test procedure was not intended.
One non-cited violation was identified.
'
en Ins ection Follow-u Item 50-315/316-96007-01:
Work Control Process.
Ins ection Sco e 62707 I'nspection Report 50-315/316-96007 documented that the licensee experienced a number of events which revealed problems with the work control system.
The licensee identified an adverse pattern simultaneously with the NRC inspectors.
The inspectors observed the licensee's response and corrective actions to the work control weaknesses.
Observations and Findin s Since inspection Report 50-315/316-96007 was issued, problems with the work control system have continued to be identified by both the NRC inspectors and the licensee.
Significant obser vations are discussed below.
b.1 Past Observations Previous inspection reports have documented both NRC and licensee-identified problems with the licensee's work control process.
Inspection Report 50-315/316-96003 (the Integrated Performance Assessment)
documented that the cumbersome nature of the work control system did not facilitate effective control of the status of equipment.
Both the work control and the clearance control processes were cumbersome, imposing a burden on the unit supervisors.
Deficiencies in the work control process occasionally resulted in unnecessarily long equipment out-of-service times.
Deficiencies in the clearance control process had contributed to several past performance problems.
Although these processes were functional in controlling work they had several shortcomings:
~
The scheduling system did not easily support adjusting work activity schedules if difficulties arose with a job.
~
Unit supervisors'dministrative burden was increased with multiple work schedules containing similar information (e.g., parts of the "Plan Report" largely paralleled the "Surveillance Test Procedures Status Summary" ).
Inspection Report 50-315/316-97009 (the Operational Safety Team Inspection)
identified similar work control and clearance problems during a planned outage on the Unit 2 AB emergency diesel generator:
The clearance for MOVATStesting of 2-WMO-722 and 2-WMO-724 failed to isolate Essential Service Water (ESW). Isolating ESW was not stated in the initial clearance request to operations although itwas required.
There was no procedure for removing the 2AB Diesel Generator (D/G) inverter from service similar to the procedure used for the Unit 1 D/G inverters.
Inverter maintenance was delayed until a procedure was written.
The scheduling time line for the diesel maintenance did not accurately reflect the amount of Post Maintenance Testing (PMT). The shift supervisor commented that this burdened the control room crew with coordinating the testing.
b.2 Recent Observations The recent observations on the poor planning for the RHR maintenance outage were previously discussed in Section 01.2 b.3 Licensee Self-Ass'essments The inspectors discussed the above observations with the outage scheduling manager.
The manager acknowledged the observations and provided the inspectors with a copy of two recent performance assurance evaluations of the work control process:
On June 29, 1998, the licensee completed a performance assurance surveillance of the work control process which was documented in Performance Assurance Surveillance SURV-98-21. The surveillance report concluded that planning and scheduling processes for the work control process were inadequate.
On July 3, 1998, the licensee completed a performance assurance audit of the clearance permit system which was documented in Performance Assurance Audit PA-98-12. The auditors concluded that the implementation of the clearance permit system was effective; however, clearance requests were determined to be inadequate based on the number of returned requests.
c.
Conclusions The inspectors concluded that the work control process weaknesses identified in 1996 still remained.
Past observations, recent observations, and licensee self-assessments indicated that the weaknesses have not been corrected.
The inspectors did not identify any specific examples where these weaknesses resulted in a reduction in safety.
However, the inspectors noted that the licensee's operations personnel were repeatedly challenged to ensure that the safety margins for operating the plant were maintained.
M1A tce Stora e at Off-Site Facilit Unit 1 a.
lns ection Sco e 62707 The licensee identified that security and foreign material exclusion practices for ice stored at an off-site facilitycould potentially compromise the quality of ice designated for reloading into the Unit 1 ice condenser.
The inspectors observed the licensee's investigation, short term corrective actions, and long term corrective actions.
b.
Observations and Findin s The licensee manufactured borated ice for use in refilling the ice condenser baskets after the completion of basket repairs.
Ice was loaded into plastic bags which were then sealed to prevent foreign material entry. A security seal was also affixed to preclude the need to inspect every bag upon return to the plant from storage.
Loaded bags were placed in a refrigerated truck for transport to a cold storage facilitynear the plant.
The licensee identified that the method for closing the ice storage bags was not satisfactory.
Bagged ice was being stored stacked two deep in the refrigerator compartment, with each bag sitting atop a wooden pallet. The potential existed for foreign material from the pallet or pallet bottom to enter a bag that was not sealed properly. Additionally, ice project personnel identified that the security seal being applied to each bag was potentially susceptible to bypass ifitwere not installed tightly.
The licensee implemented corrective actions for.these deficiencies to provide assurance that the stored ice meets all requirements for quality. The corrective actions included:
A visual inspection of the exterior of all bagged ice in the storage location was made to identify those bags where the integrity of the closure or security seal was in doubt. Bags with unsatisfactory closures were segregated after screening and marked for an open bag inspection.
Each bag designated for opening was transported to an inspection point outside the refrigerator compartment, which was established as a foreign material exclusion zone (FMEZ). The bags were opened and inspected for evidence of foreign material entry and sampled for chemical analysis of the ice. Bags that passed the foreign material inspection and chemical analysis were resealed using an improved closure system and new security seal. The improved closure system consisted of a wide plastic cable tie wrap that allows for a much tighter seal of the bag opening. After cinching down the tie wrap, the security seal was installed in a way that would provide direct evidence of tampering with the bag opening.
Physical security for the ice while the ice is in a storage status was also increased.
Security officers spot checked the bag fillingprocess to ensure seals were being installed tightly. The licensee had sole access to the refrigerator compartment where ice is stored at the storage facility. Licensee security locks were installed on the refrigerator compartment door and on the door electrical control box. A licensee security officerwas at the storage facilitywhenever the storage room was opened and performed security searches of all personnel entering the compartment.
As each trailer arrived at the cold storage facility, the bags were inspected by security personnel to ensure that all seals were tight, before entry into the compartment.
The security plan required that when bagged ice was to be returned to the plant, the trailers coming from the cold storage facilitywould be searched before loading and then security personnel would seal the trailer.
The security plan required that each loaded trailer returning to the plant would receive the routine search given to any vehicle entering the protected area.
The bagged ice would be unloaded in the auxiliary building crane bay for loading into the ice transport equipment.
Each bag unloaded from the trailer would be inspected to ensure it had an intact security seal.
When the bag seal was removed and the bag was opened, a visual inspection of the interior of the bag would be done to verify that FME was maintained for the ice. The ice bag would then be dumped in the hopper of the ice transport equipment to be blown into the ice condenser baskets.
Conclusions Ice project personnel identified shortcomings in the control of ice bagged for off-site storage.
Prompt corrective actions were implemented.
Foreign material exclusion practices were improved to provide assurance that the ice would be of the required quality when loaded into the Unit 1 ice condenser.
Physical security of the ice during transport and storage was improved to maintain control over the ice during all phases of the work.
~
~
M1.5 Ice Basket Dama e Missedb Initial Camera Ins ection Unit1 Ins ection Sco e 62707 CR 98-3680 was initiated by licensee personnel to document basket damage Identified following the completion of ice basket inspections in Unit 1 ice condenser, Bays 13 - 24.
Additional damage to 16 baskets was identified by ice condenser repair personnel while removing damaged basket sections.
The inspectors observed the licensee's investigation, short term corrective actions, and long term corrective actions.
Observations and Findin s On August 27, 1998, the licensee had inspected bays 13 - 24 in Unit 1 ice condenser.
Initially,the work scope was to inspect all baskets, but after many damaged baskets were identified, the decision was made to remove and replace all row 3-7 baskets.
The inspection scope was modified to require inspection of Rows 1, 2, 8, and 9 baskets.
The inspections were performed on the inner (Rows 1, 2) and outer (Rows 8, 9) baskets in place in the ice condenser in order to preclude having to remove the baskets.
The inner and outer rows of baskets have interferences above them that make removal in 12 foot sections infeasible, leaving destructive removal as the only alternative.
Ice basket inspections were performed by Performance Assurance technicians using a video camera assembly and light lowered into each basket from the intermediate deck of the ice condenser.
The camera assembly consisted of two video cameras, one mounted vertically downward and one mounted at an oblique angle in the downward direction. Lighting was provided by drop lights lowered into the basket with the camera, and supplemented by a small flashlight taped to the camera housing. Technicians viewed the basket via a television monitor and logged all identified basket damage on data sheets.
The data sheets used for documenting the initial inspection listed no damage to those sections of the basket subsequently identified by the repair crew as damaged.
A total of 16 baskets were identified with damage not identified during initial inspection. The additional 16 damaged ice baskets were from a population of several hundred baskets that had been repaired as of the end date of this inspection period.
The Ice Condenser project production manager notified the inspectors when the condition was identified. Condition report investigation was focused on how the inspection effort could have missed the defects during the initial inspections.
The disposition of the defects showed that they most likely had been missed in the initial inspection, and were not because of damage after the initial inspection.
The production manager also discussed preliminary corrective actions, including the possible reinspection of baskets.
'I The inspectors discussed the examination effort with the Performance Assurance {PA)
supervisor and the nuclear engineering department representative on the ice condenser basket inspection team.
The PA supervisor stated that human performance was the prime cause in missing the defects.
The ice condenser production manager said that in discussing the inspections with some crews, inconsistency in using the cameras was identified. Some crews did not use the downward oriented'camera during the inspection, while most crews used both cameras.
The production manager stated that by not using the downward looking camera, some defects could be missed as the horizontal camera corkscrews down into the basket.
Additionally, while available lighting was judged as adequate for the inspections by most PA inspectors, some did request additional light.
The corrective actions for this deficiency included Reinspection, using an improved procedure and camera, of all baskets in Rows 1, 2, 8, and 9 of Bays 13 - 24 except basket sections scheduled to be replaced.
The baskets in Rows 3 through 7 were visually inspected after removal from the ice condenser and therefore did not require reinspection by video camera.
ice basket reinspection was video taped to provide a record of the inspection.
The video was reviewed by an independent inspector to verify that existing damage was identified by the reinspection.
The procedure for inspections was modified to require the use of both cameras during basket inspections.
The technique used by the inspectors was that the downward view was used for most of the basket inspection, supplemented by use of the oblique angle view for inspection of the stiffener and coupling rings and coupling screws.
The inspection video camera assembly was modified to add additional lighting with an integral dimmer capability. The video system was qualified for the reinspection in accordance with industry standards for visual inspections.
The inspection team was briefed before commencing inspection on the changes to the inspection technique and to reinforce the need for attention to detail.
c.
Conclusions Repair crews showed good attention to detail during followup ice basket inspections which identified additional damage to the ice baskets.
The licensee took prompt action to determine and correct the causes of the original inspections which had missed some ice basket sections needing repair.
M1.6'losed Ins ection Follow-u Item 50-315/316-96006-11:
Work Control System.
In February 1996, an NRC Integrated Performance Assessment Process team inspection identified several weaknesses in the licensee's work control process.
The weaknesses
'
included administrative burden on operations supervisors, clearance control problems, and poor human factoring. The inspectors concluded that these issues were similar to the issues discussed above in Section M1.3; therefore, the work control process willbe tracked under Inspection Follow-up Item 50-315/316-96007-01.
This item is closed.
Maintenance and Material Condition of Facilities and Equipment M2.1 Ice Condenser Re airs a.
Ins ection Sco e 62707 and 61726 The inspectors performed a general inspection of the Unit 1 and 2 ice condensers and observed work in progress for the restoration of the Unit 1 ice condenser including: ice basket segment removal, assembly, reinstallation and nondestructive examination of the bottom basket grid welds.
b.
Observations and Findin s b.1 General Condition Of The Ice Condensers The licensee completed ice melt of the Unit 1 and 2 ice condensers to allow unrestricted access to ice baskets for inspection and repair work. An ice delivery system built with parts from a grain auger system had been installed and tested to reload ice that had been stored off-site. For Unit 2, foreign material inspection and retrieval was in progress and the quantities and types of foreign materials were similar to that previously found in Unit 1. For Unit 1, the intermediate deck support structure for Bays 13 through 24 had been removed and the original ice baskets in Radial Rows 3 through 7 had been removed.
The licensee was in the process of in-situ inspection and repair of ice baskets in Radial Rows 1, 2, 8, and 9. Repairs and reconstruction of ice baskets in radial rows 3 through 7 occurred outside the ice condenser at the top of the steam generator enclosures in containment.
Outside containment the licensee was stripping insulation and installing new jacketing materials from the removed intermediate deck doors to restore the damaged doors to original condition. Work in progress outside containment included sandblasting, inspection and repair of ice basket bottom rim assemblies.
The inspectors considered that the licensee's decision to melt the ice in both ice condensers facilitated effective inspection and repair, which demonstrated a commitment to a quality repair effort. However, engineering and maintenance staff lacked experience with this non-routine evolution (ice condenser melt out), and consequently water was entrained within the concrete ice bed subfloor of Unit 1. Lessons learned from Unit 1, enabled the licensee to prevent substantive water intrusion into the Unit 2 floor.
b.2 Water Intrusion Into The Unit 1 Ice Bed Floor Melting of the ice within the ice condenser introduced water onto the lower ice bed concrete floorwear stab surface.
However, the licensee identified using specialized ultrasonic examination (UT) techniques that water was trapped within the concrete subfloor areas filledwith foam concrete.
The water had leaked through a faulty seal at the wear slab joint under the lower inlet door flashing and had accumulated in the
'
A y
~
'
copper foil lined foam concrete core material sandwiched within the concrete subfloor.
To drain the trapped water, the licensee driiled holes into the foam concrete in all but two bays and had analyzed water samples to evaluate its effects on the expanded foam concrete.
The drill had been stopped by steel shim material in two bays and the licensee was in the process of evaluating (CR 98-3532) the effects of re-freezing the lower floor area with trapped water in the foam concrete for these bays. To alleviate any potential future problems with water introduction into the subfloor the licensee had sealed seams and grout areas, and had prepared the concrete wear slab for a protective epoxy paint coating. The licensee had prevented significant intrusion of water into the floor of Unit 2 by resealing gaps in.the wear slab floorjoint under the lower inlet doors, prior to the Unit 2 ice melt.
b.3 Unit 1 tce Basket Restoration During the examination of the lower ice basket bottom assemblies, the licensee had identified cracked and missing attachment welds for the tower hold down bar assembly.
Additionally, the licensee had identified inadequate/missing welds in the lower lattice grid of this assembly on essentially all of the ice basket assemblies inspected thus far.
These conditions were identified and reported in Licensee Event Report (LER)98-032 as a condition outside the plant design basis (reported under 10 CFR 50.73 and to meet 10 CFR Part 21 requirements).
The inspectors estimated that 15 to 20 percent of a population of 200 basket hold down bar welds (4 welds per basket assembly) examined thus far had required weld repairs due to cracks or lack of fusion. To identify and correct these problems, the licensee removed the bottom assembly, performed
sandblasting and then performed a UT of the hold down bar weld and visual examination (VT) of all bottom assembly welds. The inspectors observed the UT process and considered it effective for identiTication of lack of fusion in the hold down bar filletweld. Additionally, the VToccurred in the fabrication shop, which afforded excellent illumination and access for an effective examination.
The deficient welds were recorded, removed and new welds made.
Following repair work the bottom rim assembly was re-galvanized.
This process appeared to be effective in restoration of the lower ice basket assembly to original design.
The fabrication of refurbished ice baskets and assembly of ice baskets for Radial Rows 3 through 7 occurred on top of the steam generator enclosures in upper containment.
This fabrication and assembly area provided adequate lighting and suitable environment for assembly operations.
No difficulties were encountered during the inspector's observations of cadmium plated Phillips head screw installation at the ice basket coupling joints. lce basket restoration and assembly for Rows 1, 2, 8, and 9 occurred within the ice condenser.
The lighting and working area established were sufficient for effective reassembly and the inspectors observed the maintenance personnel using a specially designed tray to capture screws and metal shavings during assembly that precluded introduction of foreign material into the ice baskets.
To identify baskets outside design limits, the licensee used an encircling mechanical collar as a go-no-go gage for verifying ice basket roundness.
To correct minor dents the licensee had constructed a special hydraulic tool to re-expand the ice baskets and restore the design ice basket roundness profile. The inspector reviewed the ice basket
'
serviceability requirements in Procedure 12 Construction Head Procedure (CHP) 5021.MCD.004 "Removal and Replacement of lce Condenser Ice Baskets," Revision 4. These serviceability requirements addressed dents, buckles, and tom ligaments, and appeared sufficiently detailed to ensure ice baskets were restored to design tolerances.
b.4 Non-seismic Gl col Pi in Su orts The licensee had identified multiple locations on glycol system piping supports which required modification to enable glycol piping to withstand seismic loads. The inspectors observed areas where the licensee had installed additional U-bolt type restraints to provide lateral pipe restraint. The licensee's initiative to identify and correct these seismically unqualified supports indicated a comprehensive effort for ice condenser restoration.
Conclusions Inspection and repair work to restore damaged and out of specification components for the Unit 1 ice condenser appeared effective and comprehensive.
The decision to melt the ice in both ice condensers facilitated effective inspection and repair, which demonstrated the licensee's commitment to a quality repair effort for the ice condensers.
However, as a consequence of engineering and maintenance staff inexperience with this non-routine evolution {ice condenser melt out), water was entrained within the concrete ice bed subfloor of Unit 1. Lessons learned from Unit 1, enabled the licensee to implement actions that prevented substantive water intrusion into the Unit 2 floor.
III. En ineerin E1 Conduct of Engineering a.
General Comments Licensee engineering personnel were involved in the assessments, repairs, and modifications to the ice condenser.
Extensive involvement of engineering was noted by the inspectors during their assessment of the ice condenser corrective actions.
Additional comments on engineering performance are discussed in Section M1.2, above.
IV. Plant Su ort R1 Conduct of Radiation Protection and Chemistry (71750)
During normal resident inspection activities, routine observations were conducted in the area of security and safeguards activities using Inspection Procedure 71750.
No discrepancies were noted.
'
s~
Conduct of Security and Safeguards Activities (71750)
During normal resident inspection activities, rbutine observations were conducted in the area of security and safeguards activities using Inspection Procedure 71750.
No discrepancies were noted.
F1 Control of Fire Protection Activities (71750)
During normal resident inspection activities, routine observations were conducted in the area of fire protection activities using Inspection Procedure 71750.
No discrepancies were noted.
X1 Exit Meeting The inspectors presented the inspection results to members of the licensee management at the conclusion of the inspection on August 27, 1998.
No proprietary information was identified by the licensee.
'
'
Licensee PARTIALLIST OF PERSONS CONTACTED
¹ J. Allard, Ice Project Production Supervisor
¹ K. Baker, Manager, Production Engineering
¹ J. Benedict, Performance Assurance
¹ J. Boesch, Maintenance Superintendent
¹ D. Cooper, Plant Manager
¹MB. Depuydt, Nuclear Licensing
¹ R. Eckstein, Chief Nuclear Engineer
¹ S. Farlow, l&C Engineering
¹ M. Finissi, Electrical and AuxiliarySystems Engineering
¹ D. Noble, Radiation Protection/ Chemistry Superintendent
¹ F. Pisarsky, Performance Engineering
¹ P. Russell, Plant Protection
¹ J. Tyler, Manager, Plant Protection and Emergency Preparedness
¹ L. Weber, Operations
¹ L'. Van Ginhoven, Materials Management
¹Denotes those present at the August 27, 1998, exit meeting.
'
4g v
~l
'
IP 37551 IP 40500 IP 61726 IP 62707 IP 71707 IP 71750 IP 92902 INSPECTION PROCEDURES USED On-site Engineering Effectiveness of Licensee Controls In Identifying, Resolving, and Preventing Problems Surveillance Observations Maintenance Observation Plant Operations Plant Support Activities Follow-up - Maintenance ITEMS OPENED, CLOSED, AND UPDATED OPENED 50-315/980'I6-01 50-316/98016-01 50-315/98016-02 CLOSED 50-315/96006-11 50-316/96006-11 EEI Programmatic breakdown in the area of corrective action NCV Use of engineering test procedure outside its intended activity causes inadvertent actuation of fire pumps IFI Work control system 50-315/98012-03 50-315/98016-01 ITEMS UPDATED IFI Review of licensee's root cause assessment of operating all three fire pumps without a suction source NCV Use of engineering test procedure outside its intended activity causes inadvertent actuation of fire pumps 50-315/96007-01 50-316/96007-01 IFI Work control process
'
AEO AEP AR ATR BOP bcc CC CCW CFR CR DCC D/G DRP DPR EDT EHP FMEZ IFI IR JO Ml MOV NCV NESW NRC NRR OHI OHP PA PC PMI PMP PPA PMT RHR STA STP US VT VCT LIST OF ACRONYMS AuxiliaryEquipment Operator American Electric Power Action Request Administrative Technical Requirement Balance of plant blind carbon copy carbon copy Component Cooling Water Code of Federal Regulations Condition Report Donald C. Cook Emergency Diesel Generator Division of Reactor Projects Demonstration Power Reactor Eastern Daylight Time Engineering Head Procedure Foreign Material Exclusion Zone Inspection Follow-up Item
Inspection Report
Job Order
Motor-Operated Valve
Non-Cited Violation
Non-Essential. Service Water
Nuclear Regulatory Commission
Nuclear Reactor Regulation
Operations Head Instruction
Operations Head Procedure
Performance Assurance
Protective Clothing
Plant Manager's Instruction
Plant Manager's Procedure
Plant Performance Assurance
Post Maintenance Testing
Shift Technical Advisor-
Sur veillance Test Procedure
Unit Supervisor
Visual Examination
Volume Control Tank
27