IR 05000261/1989020
| ML14176A769 | |
| Person / Time | |
|---|---|
| Site: | Robinson  |
| Issue date: | 09/14/1989 |
| From: | Kellogg P, Merschoff E NRC OFFICE OF INSPECTION & ENFORCEMENT (IE REGION II) |
| To: | |
| Shared Package | |
| ML14176A768 | List: |
| References | |
| 50-261-89-20, NUDOCS 8909270149 | |
| Download: ML14176A769 (14) | |
Text
S PA REGO&
UNITED STATES NUCLEAR REGULATORY COMMISSION
REGION II
101 MARIETTA STREET, ATLANTA, GEORGIA 30323 U. S. NUCLEAR REGULATORY COMMISSION
REGION II
AUGMENTED INSPECTION TEAM Report No.:
50-261/89-20 Licensee:
Carolina Power and Light Company P. 0. Box 1551
-
Raleigh, NC 27602 Docket No.:
50-261 License No.:
DPR-23 Facility Name: H. B. Robinson 2 Inspection Conducted Au st
-
Septe er 1, 1989 Team Leader:
Pa Date Signed Operational/Programs Section Team Members: R. Azua, Reactor Engineer, RIV B. Bonser, McGuire Project Engineer, RII W. Lefave, Plant Systems Engineer, NRR W. Levis, Brunswick Resident Inspector, RII R. Lo, Robinson Project Manager, NRR A oE.W. Mschoff, Deputy Director Date Signed Division of Reactor Safety Region II
0')
TABLE OF CONTENTS I. INTRODUCTION -
FORMATION AND INITIATION OF AUGMENTED INSPECTION TEAM (AIT) Background B. Formation of AIT C. AIT Charter - Initiation of Inspection D. Persons Contacted are listed in Appendix E. Abbreviations and acronyms are listed in Appendix II. BRIEF DESCRIPTION OF IDENTIFICATION OPPORTUNITIES III. SAFETY SYSTEM FUNCTIONAL INSPECTION Licensee Initiative B. Corrective Action Systems IV. DESIGN BASIS DOCUMENTATION REVIEW PROCESS Purpose of the DBD Process Identification and Resolution of AFW NPSH Discrepancy C. Conclusions V. OPERABILITY DEFINITION V COMMUNICATIONS VII. ROOT CAUSE DETERMINATION VIII.EXIT INTERVIEW
REPORT DETAILS I. INTRODUCTION -
FORMATION AND INITIATION OF AIT Background H.B. Robinson Unit 2 is a Westinghouse three loop pressurized water reactor. The Unit is located five miles North West of Hartsville, SC in Darlington Count Initial criticality was achieved in September 1970, and commercial operation began in March 197 On August 16, 1989, the licensee reported to the NRC that NPSHa was inadequate for all combinations of AFW pumps running and various CST level The SDAFW pump was declared inoperable and a seven day LCO was entered. On August 22, 1989, the licensee informed the NRC that inadequate NPSHa existed for two MDAFW pumps at rated flo On August 22, 198 the licensee reported that the unit was being shutdown due to NPSHa problems in the AFW syste Formation of Augmented Inspection Team On the morning of Thursday, August 24, 1989, the Regional Administrator, after further briefing by the Regional and Resident staff and consultation with senior NRC management, directed the dispatch of an Augmented Inspection Team headed by the Section Chief of the Region II Operational Programs Sectio The team included participation by NR C. AIT Charter -
Initiation of Inspection The Charter for the AIT was prepared on August 25, 1989, and the AIT members arrived at the Robinson site on August 28, 198 The special inspection commenced with an Entrance Meeting and briefing by licensee management at 1000 hours0.0116 days <br />0.278 hours <br />0.00165 weeks <br />3.805e-4 months <br /> on August 28, 198 The Charter for the AIT specified that the following tasks be completed: Develop and validate a detailed sequence of actions associated with the design basis reconstitution of the Auxiliary Feedwater System at the H. B. Robinson Plan This sequence should include the design process inputs, calculations, verification, findings, and disposition of deficiencie. Evaluate the significance of the NPSH deficiency on safety system performanc Review AFW events where the system response was not as expecte. Evaluate the accuracy, timeliness and effectiveness with which identified deficiencies were reported to Plant and Corporate management and their prioritization and disposition of eac Review the opportunities the licensee had to identify and correct this problem, i.e., post trip reviews, SSFI, and design basis documentation,. and determine why the problem was not fully scoped, root cause, and adequate timely corrective action implemente. Evaluate the licensee's actions taken including engineering interface to verify equipment operability when degraded system performance was eviden Determine what actions the licensee took regarding any problems identified with AFW performanc Persons contacted are listed in Appendix E. Abbreviations and Acronyms used are listed in Appendix II. BRIEF DESCRIPTION OF IDENTIFICATION OPPORTUNITIES In October 1986, the licensee commenced data collection for a self initiated SSFI on the AFW syste This information was collected by December 10, 198 On December 22, 1986, a reactor trip occurred during which a member of the SSFI team was in the control room and noted reduced AFW flow with all three pumps runnin This observation was turned over to the Technical Support Group for follow-u A special project was completed by ONS in January of 1987, at the request of Technical Suppor The results of this special project were that NPSHa was not the problem but additional testing should be performed to make an exhaustive conclusio This testing was incorporated into a plant problem identification report which was subsequently scheduled to be completed in 198 Additionally the SSFI identified a significant finding with respect to NPSH This finding was placed into the RAIL system for completion by the DBD process. At least two more plant trips occurred during which low flow conditions were noted in the AFW system with three pumps operatin Since this had been previously identified and turned over to Technical Support for resolution, operations considered it as a non-proble In July 1989, the licensee's NED completed a design hydraulic calculation which indicated NPSHa problems with the SDAFW pump. This calculation was forwarded to the Modifications and Projects Manager and Technical Support personnel on July 21, 198 The Technical Support discounted the calculation as being flawed and asked NED to perform the work agai On July 27, 1989, the ONS informed NED of actual test results collected on June 20, 1989, and attempted to explained the flaws in the original calculatio On August 16, 1989, NED provided Revision 1 of their calculation to the plan This was again reviewed both by Technical Support and ONS, the results were still considered to be non-conservativ On August 21, 1989, NED and ONS met to discuss the calculation and the actual plant dat Based on this review the AFW pumps were declared inoperabl III. SAFETY SYSTEM FUNCTIONAL INSPECTION Licensee Initiative Beginning in October 1987, the licensee conducted a self-initiated SSFI on the AFW system to assess the ability of the AFW system to perform its safety functio The assessment reviewed the functional areas of system design, testing, maintenance, training and operations. The areas were examined in accordance with functional area work plans which provided the necessary guidance for how the functional areas were to be reviewe The inspection was conducted from October 17, 1986, until December 10, 1986. The site was briefed on the team's findings on February 3, 198 A draft report was issued in March 1987, and the final report issued August 27, 198 The AFW SSFI report concluded that the operational readiness was 1'reasonably adequate."
There were, however, problems identified which had a "significant negative impact on operational readiness and require near-term attention."
The specific findings were classified by the team into one of three levels. The first of these classifications included those areas that represented the most significant concerns and which needed timely attentio The second classification included those areas for which increased attention to detail would contribute to a higher state of operational readines The third area was one in which no problems were found and strengths were identifie The specific findings and recommendations were summarized by functional area as an attachment to the SSFI repor In all, approximately 82 items were identified for actio Sixteen of the items were classified as significant concern These specific items were assigned as RAIL items by the Regulatory Compliance Organization on site for tracking and assignment of responsibilities to clos However, when the RAIL numbers were assigned, no priority system was used to ensure that those items which represented the most significant concerns were resolved first and in a timely manner as recommended by the repor In addition, since the RAIL items were not a Regulatory or other agency commitment the due dates could be extended by mutual agreement between the responsible individual assigned to the task and the responsible regulatory Compliance individual assigned to track the ite No management approval is required for an extension request for a RAIL item of this priorit There is also no limit to the number of times that the item can be extende Responsible managers do get a weekly printout of outstanding RAIL items in their are This document provides the RAIL number, initiating documentation, responsible individual, short description of the item, status of corrective action, due date, the number of times that the item has been extended, and the priorit The priority does not refer to the safety significance of the item but rather indicates whether or not a report is required to an outside agenc A Priority one RAIL item signifies the requirements to provide a report or response to an outside agency. The Priority two item is one which only HBR2 action is required and no report or response to an outside agency is necessary. A Priority three item is one which the closeout action is to be taken by an outside agency and no further HBR2 action is require The question of adequate NPSHa was addressed in Section 5.1.4 of the SSFI report beginning on page 6 The report states that the suction pressure criteria was met for each of the AFW pumps at the 35,000 gallon CST leve The report further notes that there was very little margin available for the steam driven pump and that although the case of simultaneous operation of all. three pumps was not investigated it would appear that if the flow rate were doubled the corresponding pressure drop would increase by a factor of four which would lead to NPSH problems on the steam driven pump before the 35,000 gallon CST level was me Some of the assumptions noted by the inspector in the calculations provided in the report include the following: degrees F water temperature 2. 6" pipe (approximately 1' of head added to compensate for some reducers gates valves and 4' inch pipe)
3. 35,000 gallons on CST corresponded to 4.7' pumps delivered rated flow (300 gpm for MDAFW Pump, 600 gpm for SDAFW Pump)
The deficiency regarding the NPSHa concern during three pump operation was classified as a significant concern in Section 3 of the report and assigned a RAIL number for tracking purposes and assignment of responsibility to clos RAIL item 87R779 was assigned to Technical Support for resolution on August 26, 198 The priority listed is two since no response was required to an outside agency. No due date is listed for resolution of the ite The item was updated on December 22, 1987, to reflect a due date of March 15, 198 In all, the item was extended six times with a present due date of September 26, 1989. The resolution of the item was ultimately turned over in August 1987, to NED for inclusion in the DBD for the AFW syste The SSFI team left the site on December 10, 198 On December 22, 1986 a member of the SSFI team was in the control room at the time of the Reactor trip and noted the degraded AFW flow conditions during the three pump operatio.I
- Il5 On December 29, 1986, a calculation was provided to a Technical Support member from an NSSS engineer (both of whom were also on the SSFI team)
which concluded that the steam driven pump would not operate correctly (NPSHa
< NPSHr)
if the CST level were below approximately 20 feet which corresponds to 76% or 145,000 gallons in the CS This information was not, included in the SSFI report which was issued after this dat A stronger statement in the report including this additional information would have underlined the significance of the proble The AFW SSFI was a missed opportunity to identify the NPSHa problems associated with the AFW pumps. Although the report lists the NPSHa concern as significant, the wording was not strong enough to indicate the potential of the proble The licensee's lack of aggressiveness in resolving non-regulatory commitment issues is considered to be a significant weaknes Of the 82 AFW SSFI items identified, 19 are still open with seven of these included in the significant categor Corrective Action Systems The inspector also reviewed the licensee's corrective action program as outlined in procedure PLP 26 R The program as defined in PLP 26 began in August 198 Since its inception 22 Significant Condition Reports were initiated in 1988 and 65 Significant Condition Reports were initiated thus far in 1989
.
Thirteen of the 22 SCRs from 1988 are still open with four of the 13 classified as nuclear safety related. Fifty-three of the 65 SCRs generated in 1989 are still open with 21 of the 53 classified as nuclear safety relate Plant management has recognized that the SCRs are not being closed in a timely manne One of the SCRs generated in 1989 involves the program itself to provide better methods for tracking and resolving identified significant condition Included in the closeout of this SCR will be the requirement to prioritize the SCRs such that those that are nuclear safety related receive attention firs IV. THE SYSTEM DESIGN BASIS DOCUMENT REVIEW PROCESS Purpose of the DBD Process The objective of the DBD process is to establish the current design basis and calculations/analyses of records applicable to HBR2 and control them for future us The DBD process is divided into three basic phases: a documentation collection phase, a validation phase, and a discrepancy resolution phas The first three DBDs at HBR2 were pilot project They included the Safety Injection System, AFW System, and Reactor Safeguards Syste The licensee defines the design basis for a system as a system's functional and regulatory requirements and the codes and standards of records that govern its design as modified by post operating license regulatory commitment System calculations/analyses of records demonstrate that a system complies with its design bases and dictates the system and component performance parameter The DBD process is intended. to verify that the system design basis, the calculations/analyses of record and the as-constructed system are consisten The HBR2 system DBDs are prepared in accordance with "Design Basis Reconstitution Project Guideline for Preparation of System Design Basis Documents."
This guideline provides background information on the original HBR2 design evolution and explains how available design information is to be evaluated and integrated into a set of DBDs in a common forma The HBR2 system DBD guideline contains several general and specific requirement These requirements include supplying sufficient information to satisfy the applicable design input requirements of ANSI N45.2.11; design basis information as defined in the general DBD guidelines; and reference to applicable FSAR transient analyses and other FSAR sections which define the systems functional requirement To assure incorporation of those requirements, the DBD follows a specific forma This format addresses system functional requirements, regulatory imposed design requirements, system design requirements, component design requirements and design margin To meet all the objectives of the guidelines the DBO generally includes only summary level information/statements with appropriate reference After the system DBD is approved for preliminary use, it is used as the inspection standard for field validation of plant procedures and hardwar The guideline states that a field validation will be performed of the system's critical design basis parameters that are contained in the system DB0 as they relate to the system hardware and its performance and configuration. The validation guideline was implemented by extracting design basis statements from the DBD for validatio The validation process for the AFW system reviewed 174 design basis statements, of these 117 were validated/exempted and one was determined to be out of scope. The remaining 56 items could not be validated and were referred to the Discrepancy Resolution Coordinator for follow-up action. These 56 items were grouped into
discrepancies and entered into the Discrepancy Resolution Information Progra A discrepancy resolution coordinator was in charge of getting resolution of the identified discrepancie Identification and Resolution of AFW NPSH Discrepancy On July 11, 1988, the site requested the DBD project to address concerns related to AFW performance capabilit The site was specifically concerned about the motor driven and steam driven AFW pump suction piping size and. the minimum NPSHa required for continuous multiple pump operation. This site concern resulted from plant investigation into items covered by the SSFI on the AFW system conducted in 1987 by the licensee's Corporate Nuclear Safety Grou The memo requested an analysis be performed during the DBD process to fully address the plant concern There was no priority attached to this memo, nor was the memo worded to indicate that anything other than routine work was involve On December 30, 1988, the preliminary DBD was approve It contained AFW system performance requirement These included the following system requirements: The head generated by the AFW pumps shall be sufficient to ensure that feedwater can be pumped into the SGs when the MS safety valves are dischargin. The relative pressures associated with. the various SG conditions that could exist when AFW is required must be considered when designing the system to supply the required flow to the SG. The minimum required flow to be delivered to the operable SGs against a SG pressure must be equivalent to the maximum accumulation pressure of the SG safety valves that must be opened to pass the decay heat loa.
Each AFW pump shall have sufficient head to deliver its rated capacity to the SGs at the safety valve set pressure (with three percent accumulation).
Following completion of the AFW system preliminary DBD and prior to beginning the validation phase it was identified that no calculations were found in the documentation search to support the AFW pump suction line sizin On March 20, 1989, implementation of the validation plan starte The licensee determined that validating the specific AFW system performance requirements in the DBD would require the preparation of an AFW system hydraulic calculatio An Interdiscipline Review Request (IRR DBD2-MAK-002)
was submitted to the NED Mechanical/Nuclear Unit requesting preparation of an AFW system hydraulic calculation. The IRR requested completion of the calculation by May 26, 1989, to support the AFW DBD validation schedul On July 20, 1989, the Mechanical/Nuclear Unit responded to IRR DBD2-MAK-002 with Revision 0 of the AFW system hydraulic calculatio This calculation determined that NPSHa could be a problem when all three AFW pumps are operating at a 1200 gpm flow rat To support pump operation at this flow rate, level in the CST would have to be greater than 76.7 percent. Below this level, flowrates would not be adequate to assure that the steam driven AFW pump would have sufficient NPS The following day, in accordance with procedure DBD.PII.1,
"Discrepancy Resolution", a SAP form was complete The SAP form provides a systematic approach to determine the prioritization of discrepancies and is used to determine the frequency of core damage based upon a particular initiating event (discrepancy).
The conclusion of the SAP was that:
"insufficient NPSHa to the SDAFW pump will result in a reduction of flexibility to supply secondary coolant." The PRA analysis determined that assuming failure of the SDAFW pump this discrepancy constituted a high risk potential to the frequency of core damage and required immediate evaluatio NED contacted the site and notified site management of a potential significant deficiency that could affect operabilit On August 16, 1989, NED completed Revision 1 of the AFW system hydraulic calculation. It included analyses of additional scenarios as requested by the site system enginee That same day, the site was notified again of an operability issue on the SDAFW pum C. Conclusions 1. There was not a common understanding of the DBD process and what it was to accomplish. In this case, the questions surrounding the AFW pump suction piping size and minimum NPSHa where not answered by the DBD process but resulted only in the generation of hydraulic calculations based on design dat The DBD process did not verify field data and operational experienc. There was a lack of design basis information on the AFW system which led to a poor comprehension of AFW system operatio Specifically with three pumps operating simultaneousl.
When NED recognized a significant deficiency with the AFW system and reported it to site managemen Site management failed to respond aggressively and accepted it as an old proble SYSTEM / COMPONENT DEFINITION OF OPERABILITY Operability is one of the issues that is being addressed as a result of the recent events at HBR2. The failure by plant personnel to attach any significant safety concerns to the reduced auxiliary feedwater flows, has raised questions as to how the plant identified operability problems and more importantly how the plant personnel define operabilit II
In response to the first question HBR2 utilizes several methods which include a surveillance test pro.gram, preventive maintenance program, post trip review program and a trip reduction assessment progra These programs were reviewed only with respect to the AFW syste The surveillance program for the AFW pumps.was found to be consisten The pumps were tested periodically as required, and the data obtained met the acceptance criteria set forth in the test. Based on the lack of any trending data, it is apparent from interviews that the data taken was not routinely correlated with any previous test data in an effort to verify that the pumps were operating in a consistent manne With respect to a preventive maintenance the licensee's program is performance based, which means that if a pump is performing its function, it is considered operable and thus it is not scheduled for maintenance. The MDAFW pumps had not been overhauled since the plant began operation approximately seventeen years ag The SPAFW pump had undergone some repair approximately ten years ago and was scheduled for maintenance during the upcoming planned outag The post trip review program and the trip reduction assessment program are both identified in the following two procedures; PLP-035,
"Plant Operating Experience Report, Revision 0," dated April 2, 1986, and PLP-036,"Trip Reduction Assessment Program (TRAP)",
Revision 0, dated April 8, 198 The purpose of these procedures is to identify those items or pieces of equipment that have failed to perform their function and have (a) caused a plant trip, (b) may cause a plant trip or (c) make it more difficult for the plant operators to recover from a plant tri These guidelines were found to 'be narrow in scope and have excluded the operator from identifying that piece of equipment that may be performing its function, thus not meeting the criteria set forth above, but operating below desig How the licensee defines operability is closely related to the paragraph described above. It has been noted that the operators determine that equipment that is found to perform its function is considered to be operable even though it may be operating in a degraded condition, i.e.,
operating below design. This was demonstrated during past events where the control room operators noted that the AFW pumps did not achieve their total design flow of 1200 GPM when all three pumps were operatin The operators determined that as long as the pumps produced a total flow in excess of 300 GPM (identified in plant FSAR), the pumps were operationa With this conclusion the operators never addressed this item in either the control room logs, the post trip review report or the TRAP, and thus was never identified as a possible concern. This definition of operability is shared by plant operators and other plant personne Their may cause the plant staff to be unaware of equipment degradation trends as long as a piece of equipment is performing its functio V COMMUNICATIONS Throughout this entire event, communications remained at the lowest levels in the organizatio The Tech. Support Manager and 0NS Manager were the highest levels that the December 22, 1986, reduced flow observation reached. The SSFI findings were transmitted to the manager of Technical Support for resolution. The Manager of Modification Projects received the NED notification of the hydraulic calculation problem and passed it on to Technical Support. The concerns of the reduced flow events and the more conservative calculations were carried out at the engineering level. This persisted until ONS became aware of the request for Technical Specifications and JCO preparation based on the less conservative Revision 1 calculatio When that concern was finally raised to the correct levels of plant management, aggressive action was carried out to define the problem and the necessary corrective action VII. ROOT CAUSES The AIT team considered there to be four root causes for the slow recognition of the AFW NPSH problem. These root causes are: Lack of Design information for the case of three pump operatio This lack of information coupled with the licensee's definition of operability led the operators/engineers to consider the AFW system operational and the DBD process to be in erro.
Lack of priority assigned to the licensee's internal SSFI finding The initial priority assigned to the SSFI findings was lost when they were entered into the RAIL system for actio. System Operability Attitud The licensee's attitude of equipment being operable if it functions when called on and passes its surveillance tests led to a false conclusion about the calculations performed by NED and ON.
Communications at too low a level without management involvemen Throughout this event, communications were carried out at the engineer to engineer level until the discussions concerning the JCO and Technical Specification changes were starte Management was unaware of the existence of more conservative calculation VIII. EXIT INTERVIEW The preliminary findings of this special inspection were discussed on September 1, 1989, with those persons indicated in Appendix No dissenting comments were receive APPENDIX A Persons Contacted Name Title Work Group J. Baruch Technical Representative Dresser Industries P. Bauer Engineer -
- G. Chappell Principal Engineer -
NED
- S. Clark Project Engineer On-Site NED C. Coffman Engineer, ONS S. Cracker Regulatory Compliance Senior Specialist
- R. Crook Senior Specialist - Regulatory Compliance
- J. Curley Director Regulatory Compliance CP&L
- C. Dietz Manager -
- J. Eads Project Engineer -
Nuclear Licensing NSD
- W. Flanagan Manager Outage/Modifications
- S. Griggs Technical Aide - Regulatory Compliance
- W. Gainey, J OPS Support Supervisor
- E. Harris, J Director, ONS J. Hauck Corporate Quality Assurance
- J. Kloosterman Project Engineer - Nuclear Licensing CP&L
- A. Lucas Manager -
NED
- W. Martin Principal Engineer -
BNP ONS A. McCauley Onsite Nuclear Safety C. Moon System Engineer - Tech Support
- R. Morgan Plant General Manager
- M. Page Manager Tech Support
- R. Pranity Principal Engineer -
Nuclear Licensing
- D. Quick Manager, Maintenance
- D. Stadler Onsite Licensing Engineer B. Waldsmith Reactor Operator
- R. Watson Senior Vice President Nuclear General CP&L
- K. Williams Senior Engineer -
Onsite NED
- H. Young Manager QA/QC NRC Personnel
- E. Adensam Project Director -
- L. Garner NRC Senior Resident Inspector
- K. Jury Resident Inspector
- E. Merschoff Deputy Director, Division of Reactor Safety, RH
- D. Verrelli Chief, Projects Branch 1, RII
Attended Exit Meeting
APPENDIX B ACRONYMS AND ABBREVIATIONS AFW AUXILIARY FEEDWATER ANSI AMERICAN NATIONAL STANDARD INSTITUTE AIT AUGMENTED INSPECTION TEAM CST CONDENSATE STORAGE TANK DBD DESIGN BASES DOCUMENT FSAR FINAL SAFETY ANALYSIS REPORT HBR2 H. B. ROBINSON UNIT 2 JCO JUSTIFICATION FOR CONTINUED OPERATION LCO LIMITING CONDITION FOR OPERATION MS MAIN STEAM MDAFW MOTOR DRIVEN AUXILIARY FEEDWATER NED NUCLEAR ENGINEERING DEPARTMENT NPSHa NET POSITIVE SUCTION HEAD available NPSHr NET POSITIVE SUCTION HEAD required by the pumps NRC NUCLEAR REGULATORY COMMISSION NSSS NUCLEAR STEAM SUPPLY SYSTEM PIR PROBLEM IDENTIFICATION REPORT PRA PROBABILISTIC RISK ASSESSMENT RAIL REGULATORY ACTION ITEM LIST SAP SCREENING AND PRIORITIZATION SCR SIGNIFICANT CONDITION REPORT SDAFW STEAM DRIVEN AUXILIARY FEEDWATER SG STEAM GENERATOR SSFI SAFETY SYSTEM FUNCTIONAL INSPECTION TRAP TRIP REDUCTION ASSESSMENT PROGRAM