3F0897-12, Responds to NRC Re Violations Noted in Insp Rept 50-302/97-07.Corrective actions:CP-111 Will Be Clarified to Include That Extensions Are Appropriate When Emergent Higher Priority Requirements Do Not Permit Completion of Analysis

Responds to NRC Re Violations Noted in Insp Rept 50-302/97-07.Corrective actions:CP-111 Will Be Clarified to Include That Extensions Are Appropriate When Emergent Higher Priority Requirements Do Not Permit Completion of Analysis

ML20217H277
Person / Time
Site:	Crystal River
Issue date:	08/05/1997
From:	Cowan J FLORIDA POWER CORP.
To:	NRC OFFICE OF INFORMATION RESOURCES MANAGEMENT (IRM)
References
3F0897-12, 3F897-12, 50-302-97-07, 50-302-97-7, NUDOCS 9708110061
Download: ML20217H277 (10)

Text

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ - _ _ _ _ _ - _ - _ _ __-_ _ _

Florida Power u

'c

% C Em "ra August 5,1997 3F0897-12 U.S. Nuclear Regulatory Commission Attn: Document Control Desk

Washington, D.C. 20555-0001 l

Subject:

Reply to Notice of Violation, NRC Inspection Report No. 50-302/97-07, i NRC to FPC letter,3N0797-03, dated July 7,1997 l

l Gentlemen:

(

In the subject letter, Florida Power Corporation (FPC) received Notices of Violation.

This correspondence provides our response to the violations.

Sincerely, John Paul Cowan Vice President Nuclear Froduction i i/

JPC/dwh Attachments xc: Regional Administratc ; ; ion 11 Senior Pesident inspector I2df NRR Project Manager 9708110061 970805 i,,

1 l

J

U. S. Nucle:r Regulatory Commission 3F0897-12

, Page.2 of 10 ATTACHMENT 1 FLORIDA POWER CORPORATION NRC INSPECTION REPORT NO. 50 302/97'07 REPLY TO NOTICES OF VIOLATION VIQLallON 50-302!97-07-01 Technical Specification 5.6.1.1 requires that written procedures shall be established, implemented, and maintained covering the applicable activities recommended in Regulatory Guide 1.33, Revision 2, Appendix A, February 1978. Regulatory Guide 1.33, Quality Assurance Program Requirements (Oporation), Appendix A, paragraph 1.d, includes administrative procedures for procedural adherence.

Licensee Compliance Procedure CP-111, Processing of Precursor Cards for Corrective Action Program, Revision 56, specified the time allowed for preparing the apparent cause summary and the corrective actions for a Grade "C" precursor card (PC). Steps 4.6 and 4.11 of the procedure provided for extensions to the time, as long as a record of the extension request was included in l the file for the PC. Step 3.2.5.1 of Procedure CP-111 stated that the Apparent Cause Evaluator l

(ACE) must be qualified, which included receiving apparent cause training and performing ACE functions at least yearly thereafter. Step 4.3.2.1.4 of Procedure CP-111 stated that the Precursor Card Screening Committee (PCSC) designate a qualified Root Cause Team (RCT) Leader for Grade "B" PCs. Step 4.3.2.1.1 of Procedure CP-111 stated that the PCSC is to grade each PC in accordance with Enclosure 3, PC Grading Guidance, of CP 111.

Contrary to the above, on May 9,1997, the inspectors identified the following examples where Procedure CP-111 was not followed:

The extensions for five of fifteen Grade "C" PCs reviewed (97-0039,0046,0697,0786 and 0867) were in violation of the procedural requirements in that the 20-days (total time) specified in CP-111 was exceeded and there was no record of any extension requests in the applicable PC file.

Individuals performing the ACE function for nine of the fifteen Grade "C" PCs were not qualified in that they had not received the apparent cause training. Also, for three of these nine PCs, neither the ACE nor the approving Responsible PC Manager had received the apparent cause training. Also, the inspectors identified one case where the RCT Leader for a Grade "B" PC (97-0052) was not fully quahfied to perform the function of a RCT Leader in that the individual had not received all the specified training.

For a number of the Grade "D" PCs examined (which included but was not limited to PCs 97-0624, 97-1540, 97-1671 and 97-1871), the guidance listed in Enclosure 3 of CP-111 indicated that the PCs should have been graded as a "C" in that these PCs met some portion of the guidance for grade "C" PCs.

U. S. Nucle:r Regul: tory Commusion 3F0897-12 :

• Page 3 of 10 ADMISSlQM.D.R DENIAL OF THE ALLEGED VIOLATION -

Florida Power Corporation accepts the violation.

REAS.ON FOR THE VIOLALT1QN The reason for the first violation example was failure to assign necessary resources for timely completion of required actions. As part of the CR 3 restart plan, special teams were formed to identify and document existing problems. This caused an increase in the number of Precursor '

Cards (PCs). Although information related to overdue cause investigations and corrective action steps was made available to appropriate line managers, the primary focus on identification of existing problems detracted from acting on overdue items in a timely manner. A contributing factor was management's expectations regarding the meeting of due dates. Emphasis is now placed on due dates; if established due dates cannot be met because of emergent higher priority requirements, then establishment of a revised due date is expected. Failure to meet a due date, either one initially established or one that is revised, is a performance issue and is dealt with accordingly.

j

-The reason for the second violation example was failure of management to ensure that individuals

. performing Root Cause Team Leader (RCTL) or Apparent Cause Evaluation (ACE) functions were properly qualified to perform their assigned task. The increase in the number of PCs resulted in management assigning individuals to perform ACE or RCTL functions in an attempt to control the backlog without verifying qualification requirements to pe: form these tasks. A contributing factor was unclear procedural guidance related to qualification requirernents for RCTLs in CP 111, or other documents, to ensure consistency of application.

The reason for the third violation example was improper interpretation of CP-111 for the grading of PCs. The PC Screening Committee (PCSC) classified the cited PCs as Grade "D" based on their being additional examples of an existing issue. The PCSC recognized that a root / apparent cause was already required for the existing issue and used the Grade "D" classification to avoid duplicated root / apparent cause effort.

CORRECTIVE STEPS THAT HAVE BEEN OR WILL BE TAKEN AND THE RESULTS ACHIEVEQ The Corrective Action Review Board (CARB) has reviewed the procedural steps associated with PC extensions CP-111 will be clarified to include that extensions are appropriate when emergent higher priority requirements do not permit completion of cause analysis or corrective action steps as initially scheduled. Further, the procedure will be enhanced to state the expectation that failure to meet an established. or appropriately revised, due date will be treated as a performanco issue.

The nine Grade "C" PC apparent cause evaluations and one Grade "B" PC root cause have been -

reviewed and signed by qualified apparent / root cause personnel.

i

- U. S. Nucl: r Regul: tory Co nmission 3F0897-12 Page 4 of 10 Closed PCs initiated under the graded process will be reviewed to identify whether other PCs exist that had non-quahfied personnel performing either root or apparent cause evaluations. Actions will be assigned to the responsible PC Manager to either have the root or apparent cause validated and signed by a qualified permn, or have a qualified person perform the required

- evaldlon. This action will be completed by Augcd 29,1997.

Thirty additional RCTLs were certified during the week of July 28,1997. This action is expected to improve the timeliness and quality of root cause determinations.

CQRRECTIVE STEPS THAT HAVE BEEN OR WILL BE TAKEN TO AVOID FUBIHEB VlOLALI1QMS I

CP-111 will be revised by August 29,1997, to clarify management expectations regarding PC l timeliness.

i The requirement for individuals signing as the RCTL or ACE to be properly qualified has been emphasized to PC managers.

CP-111 will be revised by August 29,1997, to clearly define the qualification requirements for the RCTL.

Nuclear Safety Assessment Team individuals responsible for the closure review of Grade *A," "B,"

and "C" PCs will periodically monitor the qualification of individuals performing the associated root / apparent causes.

.The PCSC has been made aware of the violation examples associated with improper PC grading.

Adherence to the procedural guidance for grading PCs has been emphasized, along with the expectation for conservative grading if: disagreement or indecision takes place during the screening process.

DATE WHEN FULL COMPLIANCE WILL BE ACHIEVED Florida Power Corporation is in full compliance.

VIOLATION 50-302/97-07-02 10 CFR 50, Appendix B, Criterion IX, Control of Special Processes, requires, in part, that measures shall be established to assure that processes such as nondestructive testing are controlled and accomplished by qualified personnel using qualified procedures in accordance with applicable codes, standards, specifications, criteria, and other special requirements.

1

U. S. Nuclear Reguiltory Commission 3F0897-12 Page.5 of 10 Contrary to the above, on May 7i 1997, the licensee failed to control and accomplish adequately planned nondestructive hydrostatic testing of the emergency feedwater system per Modification Approval Record (MAR) 96-10-02-01 and Maintenance Procedure (MP) 137, System Hydrostatic Pressure Testing, Revision 28, in that a briefing of Operations was neither required nor routinely expected; the addendum to clearance Engineering Change Order (ECO) 97-04-026 issued for the

, test did not incorporate requirements for protecting adjacent systems; the completed Maintenance Procedure 137, Enclosure 3, System Test Requirements, did not incorporate requirements for protecting adjacent systems; engineering personnel were not involved with the planned performance of the test; and the maintenance crew performing the test procedure did not recognize or understand the requirements for protecting adjacent systems.

ADMISS10tLOR_ DENIAL OF THE All.EGED VIOLATLQN Florida Power Corporation accepts the violation.

REASON FOR THE VIOLATION The reason for the violation was inadequate control of the project, which is required by Nuclear Operations Directive NOD 38, " Planning, Budgeting and Scheduling Project Controls."

Administrative controls and expectations were inadequate for ensuring that the Operations Department was involved in the pre-job briefing for the scheduled hydrostatic test. Pre-job briefings were procedurally accomplished in a number of ways. Some procedures reference Operations Instruction 01-14, " Operations Evolutions," or Administrative Instruction Al-500,

" Conduct of Operations," which identifies 01 14 for performing pre-job briefings. Other procedures simply date that a pre-job briefing will be performed. The format, documentation, and attendance requirements for the pre-job briefings varies. Additionally, hydrostatic tests were not routinely considered by maintenance personnel to be infrequently performed tests, which would requiro Operations Department involvement.

A contributing cause was failure of field engineering planners to adequately implement Maintenance Procedure MP-137, " System Hydrostatic Pressure Testing." MP-137, Enclosure 3, is completed by the field engineering planners and documents the hydrostatic test valve lineup.

Although the field engineering planners were aware of the requirement for protecting systems

. adjacent to the hydrostatic test boundary, actions to protect adjacent systems were not added to the valve lineup.

CORRECTIVE STEPS THAT HAVE BEEN TAKEN AND THE RESULTS ACHIEVID The Assistant Director of Nuclear Plant Maintenance held a stand down meeting with Site Maintenance Crew Supervision, Chiefs and Master Mechanics. Expectations related to project ownership and taking time to perform a thorough review of work scope and procedures prior to

- starting work was emphasized.

Maintenance Study Book Entry (MSBE) 97-05-03 was issued. This document required that Maintenance Shop Managers review the MP-137, Revision 29, changes with personnel under their cognizance.

U. S. Nuclear Regulatory Commission

- 3F0897-12

• Page 6 of 10 An Operatons Department night order addressing pre-job briefings and expectations was issued.

The importance of conducting pre-job briefings was discussed. Ensuring performance of proper pre job briefings by Operations Department individuals performing operations evolutions was emphasized. The use of Operations Instruction 01-14,

• Evolution Briefings," as a pre-job briefing

- guideline was reviewed.

Operations Study Book Entry (OSBE) 9707.04 was issued. This document informed operations personnel of the subject violation and the requirement to protect systems adjacent (interconnected) to a hydrostatic test boundary.

Nuclear Operations Engineering field engineering planners were briefed on the changes made to i- MP-137 through Revision 29. The requirement for incluomg provisions for venting or pressure monitoring downstream of hydrostatic test boundaries on the valve lineup form contained in Enclosure 3 to MP 137 was discussed.

l

i. QQRRECTIVE STEPS THAT HAVE BEEN OR WILL BE TAKEN TO AVOID FURTHER VIOLATIONS Administrative Instruction Al-100, " Facility Administrative Policies," was revised (Revision 20) to

! require single point accountability for complex tasks or multi-disciplined evolutions. This addition endorses the single point accountability requirements of NOD-38.

The Operations Department will develop and implement a site-wide procedure for governing pre-job briefings. This action will be completed by September 1,1997.

Maintenance Procedure MP-137, " System Hydrostatic Pressure Testing," was revised (Revision

29) to include additional guidance for the protection of systems adjacent to the hydrostatic test pressure boundary. A note was added under Prerequisites which stated, in part: "All valves in the boundary of the test and vent valves or monitoring methods outside the boundary shall be listed on Enclosure 3." This same note was added to the beginning of Enclosure 3, " System Test Requirements," which is prepared by engineering. Additionally, Section 4.2.2 was clarified to require that component / system piping not under test be vented or monitored during the test and recorded on Enclosure 3 before the actual test is started.

The prerequisite statement added to MP-137, Revision 29, was subsequently changed (Revision -

30) to specifically state that the protection of adjacent (interconnected) systems is a hydrostatic test requirement. This same information was added to the valve lineup form in Enclosure 3 to MP-137 as a constant reminder to operations personnel preparing Equipment Clearance Orders.

DATE WHEN FULL C.OMPLIANCE WILL BE ACHIEVED Florida Power Corporation is in full compliance.

l I

-_-__-_-_____N

x U. S. Nucle:r Regul tory Ccmmtsion 3F0897-12

'; Page 7 of 10 WOLATION 50-302/97-07-06

- 10 CFR 50, Appendix B, Criterion XVI, Corrective Action, requires, in part, that measures shall be established to assure that conditions- adverse to quality, such as failures, deficiencies, and deviations are promptly identified and corrected.

Contrary to the above, in the following three examples, the licensee failed to correct conditions adverse to quality promptly and adequately- ,

1. On April 17,- 1997, the security staff failed to understand adequately and develop guidance properly to address the multiple temporary losses of power to the central alarm station -

computers and intelligent multiplexer;

2. In December 1995, engineering failed to consider the extent of condition for the failure of a secondary alarm station uninterruptible power supply unit that was removed because of swollen batte ries due to old age;

3. Prior to and after the December 1995 failure of a secondary alarm station uninterruptible power supply unit, the licensee failed to establish a preventive maintenance program to assure all security uninterruptible power supply units would be maintained in a manner commensurate with their importance to maintaining security equipment in proper functioning order.

ADMISSION OR DENIAL OF THE ALLEGED WOLATION Florida Power Corporation accepts the violation.

REASON FOR THE VIOLATION The reason for violation Example 1 was failure of the Central Alarm Station (CAS) operators to ensure PCs were generated for failures of the CAS uninterruptible power supply (UPS). A contributing factor was a lack of understanding by the CAS operators of the function provided by the UPS. 'Also, hesitancy on the part of Security Department management to escalate PCs documenting problems with the CAS UPS led to a delay in correcting known equipment problems.

The reason for violation. Example 2 was inadequate implementation of the corrective action program and incomplete engineering evaluation. PC 95-2928 was generated on December 29, 1995, to document swolien batteries in the Secondary Alarm Station (SAS) UPS. -PC box

" Evaluate & Respond" was marked by the Shift Manager. A response to the PC was performed on November 27,1996. No extent of condition review or review for impact on s,imilar equipment was performed. Meanwhile, the SAS UPS had been removed by modification package MAR 88-03-04-25. Engineering personnel involved with addressing the PC failed to recognize a common

. failure mechanism that could affect the CAS UPS.

U. S. Nucizr Rrgul: tory Commission -

- 3F0897 ,

_'; Page 8 of 10

-The reason for violation' Example 3 (prior to_1995) was incomplete engineering evaluation.

Procedure MPP-107, " Turnover of Modifications to Nuclear Operations and MAR [ modification approval record) Closure Document identification," was in effect until 1994. Enclosure 10 to MPP-107 contained a Modification / Procedure Review form. M^R 86-05-12-01, Field Change Notice (FCN)- 3, initially -installed the_ CAS and SAS UPSs in 1989. At this time, the Nuclear Superintendent, Projects / Nuclear Projects Specialist (Maintenance Department) was responsible for the PM Program (PM 100, Preventative Maintenance Program). Enclosure 10 from M ~P-107 for this FCN did identify the Nuclear Superintendent, Projects / Nuclear Projects Specialist as being required to review the FCN. The FCN review concluded that no PM changes were required. A recent review of the vendor technical manual for the CAS and SAS UPS installed in 1989 revealed that the only recommended PM was to keep the aiea around the unit, especially the air inlet,- clean and relatively dust free. An evaluation for PM requirements beyond those recommended by the vendor (e.g., periodic battery replacement) was not performed.

The reason for violation Example 3 (after 1995) was the failure to perform an extent of condition or common mode failure review for the PC generated to document the CAS UPS failure.

Identification of a similar component should have resulted in actions to prevent a similar failure. In

-this case, the development of a PM for periodic replacement of the SAS UPS battery would have been appropriate.

CORRECTIVE STEPS THAT HAVE BEEN TAKEN AND THE RESULTS ACHIEVED Security Information Report (SIR) 11130 was issued on April 30,1997, and supplemented on May 1,1997. This SIR provided specialinstructions to the CAS operator concerning the function and operation oOhe UPS. Also included were steps to be taken in the event of a UPS malfunction.

A step was added to Queued Maintenance Task CS00000143 to replace batteries on the UPS

- units in the CAS (1) and SAS (2) and perform a visual inspection and cleaning every two years.

CORRECIIVE STEPS THAT HAVE BEEN OR WILL- BE TAKEN TO AVOID _ FURTHER VIOLAIlOMS Security personnel will receive additional training on equipment associated with their watch stations. Training will be completed by September 30,1997.

Through involvement with resolving this.vic;ation, Security management / personnel have been -

sensitized to the need to generate and escalate PCs associated with security hardware problems.

- The current corrective action program (CP _111) specifically addresses extent of condition reviews and reviews of similar components to determine if there is evidence of common mode failure.

These reviews should identify similar affected components and the need to create new PM tasks.

U. S. Nucletr Regulatory Commission 3F0897 -

• Page 9 of 10 Subsequent to this event, the PM Program transferred from the Maintenance Department to the Reliability Centered Maintenance group to devote diverse engineering resources for maintenance of the' PM Program. This group is responsible for review of MAR packages to ensure vendor component technical information and industry / engineering experience are included in determining

- the need for PM tasks.

DATE WHEN FULL COMPLIANCE WILL BE ACHIEVED Florida Power Corporation is in full compliance.

f-

.i

. i

E- ,

6*

l--

U. S. Nucle:r Regul tory Commission

! 3F089712 I a Page 10 of 10 ATTACHMENT 2 The following table contains a listing of commitments contained in this response:

Response Commitment Due Date Section Page 4 Closed PCs initiated under the graded process will be August 29,1997 reviewed to identify other examples of unqualified personnel performing either root or apparent cause evaluation / investigations. Actions will be assigned to the responsible PC Manager to either have the root or apparent cause validated and signed by a qualified person, or have a qualified person perform the required evaluation / investigation.

Page 4 CP 111 will be revised to clarify the procedural August 29,1997 requirement to extend PCs.-

Page 4 CP-111 will be revised to clearly define the August 29,1997 qualification requirements for the RCTL.

Page 4 NSAT individuals responsible for the closure review of Ongoing Grade A, B, and C investigations will periodically monitor the qualification of individuals performing the associated root / apparent causes.

Page 6 The - Operations Department will develop and September 1,1997 implement a site-wide procedure for- goveming pre-job briefings.

Page 8 Security personnel will receive additional training on September 30,1997 equipment associated with their watch stations.

b