ML20140E590

Forwards Enforcement Action Worksheet

ML20140E590
Person / Time
Site:	Saint Lucie
Issue date:	10/29/1996
From:	Davenport P NRC OFFICE OF INSPECTION & ENFORCEMENT (IE REGION II)
To:	Linda Watson NRC OFFICE OF INSPECTION & ENFORCEMENT (IE REGION II)
Shared Package
ML20140E502	List: ML20140E497 ML20140E508 ML20140E514 ML20140E519 ML20140E524 ML20140E530 ML20140E536 ML20140E541 ML20140E557 ML20140E573 ML20140E580 ML20140E590 ML20140E594 ML20140E620 ML20140E652 ML20140E659 ML20140E667
References
FOIA-96-485 NUDOCS 9704290042
Download: ML20140E590 (13)

Text

.

s From:

Patricia Davenport L((

ATP2.LJW2 4M77&A/ Q To:

f j

Date:

10/29/96 9:29am i

Subject:

Enforcement Action Worksheet Attached file you ask for.

J 9704290042 970423 PDR FOIA BINDER 96-485 PDR

i I

ENFORCEMENT ACTION WORKSHEET INADEQUATE DESIGN CONTROL PREPARED BY: John W. York DATE: October 28, 1996 i

NOTE: The Section Chief of the re;ponsible Division is responsible for preparation of this EAW and its distribution to attendees prior to an Enforcement Panel. The Section Chief shall also be responsible for providing the meeting location and telephone bridge number to attendees via e-mail [ENF.GRP. CFE. OEMAll. JXL. JRG, SHL. LFD: appropriate RII DRP. DRS: appropriate NRR NMSS).

A Notice of Violation (without "boilerplate") which includes the recommended severity level for the violation is required. Copies of applicable Technical Specifications or license conditions cited in the Notice or other reference material needed to evaluate the proposed enforcement action are required to be enclosed.

This Notice has been reviewed by the Branch Chief or Division Director and each violation includes the appropriate level of specificity as to how and when the requirement was violated.

Signature Facility: St. Lucie Unit (s): 1 and 2 Docket Nos: 50 335, 389 License Nos: DPR 67. NPF-16 Inspection Report No: 96 17 Inspection Dates: 10/7 11 and 10/15 18, 1996 Lead Inspector: John York 1.

Brief Summary of Inspection Findings:

[Always include a short statement of the regulatory concern / violation. Reference and attach draft NOV. Then, either summarize the inspection findings in this section or reference and attach sections of the inspection report. Inspectors are encouraged to utilize the Noncompliance Information Checklist provided in Enclosure 4 to ensure that the information gathered to support the violation is complete.]

The licensee replaced some safety related nuclear instrumentation drawers during the Unit 1 Outage.

The drawers were wired backwards because of incorrect drawings.

Part of the root cause identified the lack of a proper independent verification as a potential cause. This is a violation of 10 CFR 50 Appendix B Criterion III In examining the safety aspects of this event, one additional example of inadequate design verification was identified for BEACON on line core performance monitoring system.

In addition to the wiring problem for the drawers, the maintenance group connected the field cables for an NI backwards because the markings on the connectors were different than on the previous detectors. An NOV was written for failure to write a Condition Report (discrepancy report) and resolve this problem prior to installation of the detector.

See attached IR feeder and proposed NOV for details.

PROPOSED ENFORCEMENT ACTION NOT FOR PUBLIC DISCLOSURE WITHOUT THE APPROVAL OF THE DIRECTOR, OE

-.~

Y ENFORCEMENT ACTION WORKSHEET 3

2.

Analysis of Root Cause:

~

Lack of control and procedural adherence in the licensee's program for preparing and implementing Plant Change / Modifications (PC/Ms).

3.

Basis for Severity Level (Safety Significance):

[ Include example fnm the supplements, aggregation, repetitiveness, willfulness, etc.]

Aggregation of examples and application of Supplement I. C.7. a breakdown in the control of licensed activities involving two violations that are related that collectively represent a potentially significant lack of attention toward licensed activities.

The safety significance of reversing the detector inputs to the NIS drawers substantially reduced the safety margin between the TM/LP trip

.setpoint and the analysis limit even considering the increased TM/LP margin to the trip setpoint due to actual core operating conditions.

4.

Identify Previous Escalated Action Within 2 Years or 2 Inspections?

[by EA#, supplement, and Identification date.]

EA 96-249 - Inadequate 50.59 did not identify US0. 7/12/96 EA 96-040 - Boron Overdilution Event. Supplement 1. 1/22/96 EA 95-180 - Inoperable PORVs due to Inadequate PMT, Supplement 1. 8/4/95 5.

Identification Credit? No The miswired NI drawers were identified through an event (the failure to have the system respond properly), i. e. the analysis of the data by Reactor Engineering discovered the miswiring of the NI drawers but the error in the drawing should have been discovered in the design control process.

The design error associated with BEACON was identified through routine comparisons of actual plant data with predicted data. This error could have been discovered in the design control process.

Enter date Licensee was aware of issues requiring corrective action:

7/30/96 6.

Corrective Action Credit? Yes Brief summary of corrective actions:

In response to the issue, the licensee adopted corrective actions which included:

For immediate action the licensee prepared a change request for e

the modification package and channels A.C. and D were reconnected and testing was performed to verify proper NI response.

A root cause/self assessment and training meeting for the e

Engineering Department emphasizing importance of proper design PROPOSED ENFORCEMENT ACTION - NOT FOR PUBLIC DISCLOSURE WITHOUT THE APPROVAL OF THE DIRECTOR, OE

, ~.

's ENFORCEMENT ACTION WORKSHEET verification and importance of questioning attitude.

Tape was produced of this meeting for future engineering training.

Procedures (Engineering Quality Instructions) were revised to (1) e require all critical aspects be verified during the PC/M. (2) emphasize that the same level of verification is required for PC/Ms duplicated for the second unit. and (3) reinforce the 1

verification requirements for safety related drawings.

Walkdowns will be conducted (linear NIs) to revise any design e

documentation and tagging.

4 ASI targets will be established for future trending of ASI during e

power ascension.

Require cross-disciplinary reviews of design inputs e

Better documentation of assumptions in core design inputs and e

codes Explain application of corrective action credit:

4 Corrective action appears to be of appropriate scope.

7.

Candidate For Discretion? NO Explain basis for discretion consideration:

Since actual power conditions did not exceed trip setpoints, no l

escalation is warranted. Several examples of licensee's declining performance in engineering does not warrant mitigation.

i i

8.

Is A Predecisional Enforcement Conference Necessary? Yes Why:

1 To determine adequacy of licensee's proposed long-term corrective 1

actions regarding backward looks at modifications performed prior to the Unit 1 outage. This included discussions of other modifications that I

may not have been independently verified.

If yes, should OE or 0GC attend?

[ Enter Yes or No):

Should conference be closed? [ Enter Yes or No):

9.

Non Routine Issues / Additional Information:

9 PROPOSED ENFORCEMENT ACTIO" - NOT FOR PUBLIC DISCLOSURE WITHOUT THE APPROVAL OF THE DIRECTOR, OE l

1 i

0 ENFORCEMENT ACTION

-4 NORKSEEET 4

i 10.

This Action is Consistent With the Following Action (or Enforcement Guidance) Previously Issued:

[EICS to provide] [If inconsistent, include:]

4 Basis for Inconsistency With Previously Issued Actions (Guidance) 11.

. Regulatory Message:

Positive control must be established and maintained over the design process with particular emphasis on properly performing independent design verification.

12.

Recommended Enforcement Action:

j 1

SL III 13.

This Case Meets the Criteria for a Delegated Case. [EICs - Enter Yes or No]

14.

Should This Action Be Sent to OE For Full Review? [EICS - Enter Yes or No]

l If yes why:

l 15.

Regional Counsel Review [EICS to obtain]

No Legal Objection Dated:

16.

Exempt from Timeliness: [EICS]

Basis for Exemption:

Enforcement Coordinator:

DATE:

PROPOSED ENFORCEMENT ACTION - NOT FOR PUBLIC DISCLOSURE WITHOUT THE APPROVAL OF THE DIRECTOR, OE

o ENFORCEMENT ACTION WORKSHEET - ISSUES TO CONSIDER FO

' Problems categorized at Severity Level I or II.

a Case involves overexposure or release of radiological material in excess u

of NRC requirements.

Case involves particularly poor licensee performance.

a Information should be included to Case (may) involve willfulness.

address whether or not the region has had discussions with OI regarding o

the case, whether or not the matter has been formally referred to 01.

and whether or not 01 intends to initiate an investigation. A description, as applicable, of the facts and circumstances that address the aspects of negligence, careless disregard, willfulness, and/or i

management involvement should also be included, Current violation is directly repetitive of an earlier violation.

i a

Excessive duration of a problem resulted in a substantial increase in a

risk.

Licensee made a conscious decision to be in noncompliance in order to a

obtain an economic benefit.

a Cases involves the loss of a source.

(Note whether the licensee self-i identified and reported the loss to the NRC.)

Licensee's sustained performance has been particularly good, a

Discretion should be exercised by escalating or mitigating to ensure a

that the proposed civil Senalty reflects the NRC's concern regarding the violation at issue and tlat it conveys the appropriate message to the licensee.

Explain.

t t

PROPOSED ENFOACEMENT ACTION - NOT FOR PUBLIC DISCLOSURE WITHOU1' THE APPROVAL OF THE DIRECTOR, OE

i i

REFERENCE DOCUMENT CHECKLIST

[]

NRC Inspection Report or other documentation of the case:

NRC Inspection Report Nos.:

[]

Licensee reports:

[]

Applicable Tech Specs along with bases:

[]

Applicable license conditions

[]

Applicable licensee procedures or extracts l

[]

Copy of discrepant licensee documentation referred to in citations such as NRC; inspection record, or test results

[]

Extracts of pertinent FSAR or Updated FSAR sect'ons for citations i

involving 10 CFR 50.59 or systems operability

[]

Referenced DRDERS or Confirmation of Action Letters

[]

Current SALP report summary and applicable report sections

[]

Other miscellaneous documents (List):

I PROPOSED ENFORCEMENT ACTION NOT FOR PUBLIC DISCLOSURE WITHOUT THE APPROVAL OF THE DIRECTOR, OE

i NI INSPECTION ST. LUCIE-October 7-18. 1996 On July 30, 1996. St. Lucie Unit 1 was operating at approximately 100 %

power when reactor engineering was analyzing the data taken during power ascension and noted an anomaly in the results.

The data indicated three of the four excore linear detectors measured core power moving to the top of the core during power ascension. This was an unexpected 3henomena and aid not agree with the trend of the power moving to the

]ottom of the core indicated by RPS Channel B Linear Range Detector.

Control Channel #9 Linear Range Detector, and the BEACON Core Power Distribution Monitoring System.

Evaluation of the data collected indicated that RPS Channels A.C and D could have reversed (rolled) leads of the top and bottom chambers input to the RPS drawers'.

1 The modification performed during the outage associated with this problem was No. PC/M 009-195. During the outage, the licensee replaced the power range NI drawers for the Reactor Protection System (RPS) with new Gamma Metrics drawers. This modification combined the linear power range input to the RPS and the logarithmic wide range channel into a single drawer, i.e. reduced the number of drawers on Unit 1 from eight to four.

This modification increased the limits of the instruments range and replaced aging equipment.

Engineering' Verification-Root Cause A design error was responsible for the reverse connection (rolled leads) on four NI safety related drawers on Unit 1.

The Controlled Wiring Diagram (CWD) no. JPN-009-195-001/002 de)icted the upper Uncompensated Ion Chamber (UIC) connected to the lower JIC input at the NI drawer.

The root cause noted that the designer and the lead engineer interpreted conflicting information on the existing CWDs and made an assumption.

The independent verification may have caught this error had the process been properly performed. The drawings were prepared by the lead designer with input from the lead engineer. The drawings were then checked by a second designer who had no special knowledge of the NI design.

This check was essentially a drafting check. The drawings were then reviewed by the lead designer and then by the engineering supervisor.

Engineering Quality Instructions (01) 1.7. Design Input / Verification, dated July 5.1995, states in part that " Design verification is the process whereby a com; etent individual, who has remained independent of the design process, reviews the design inputs.... and design output to verify design adequacy. This independent review is provided to minimize the likelihood of design errors in items that are important to nuclear safety." Contrary to this requirement the first reviewer could not be considered as competent because he was not an engineer as required by PROPOSED ENFORCEMENT ACTION NOT FOR PUBLIC DISCLOSURE WITHOUT THE APPROVAL OF THE DIRECTOR, OE

2 011.7 and the lead engineer as the third reviewer could not be considered to have remained independent of this design project.

One of the action items to prevent recurrence was to check all the I&C and electrical PC/M to see if all the drawing approval signatures could qualify as independent verifiers. The licensee found three out of eight open modifications where this was a potential problem, two of these modifications were electrical and one was I&C. This therefore is not an isolated case.

This failure to perform independent verification according to procedure is identified as exam)le one of violation 50-335/96-17-XX. Failure to Control the Design 3rocess According to the Requirements of 10 CFR 50. Appendix B. Criterion III.

BEACON Core Power Distribution Monitoring System The licensee had installed BEACON during this refueling outage to replace the older IMPAX code used for in-core flux monitoring.

BEACON provided several significant improvements over IMPAX one being real-time flux profile monitoring. This improvement permitted reactor engineering to identify the NIS problem quickly and initiate prompt corrective actions.

During power operations, reactor engineering used BEACON to obtain the actual in-core flux profile. The actual in-core flux profile was then used to verify compliance with Technical Specifications and provide calibration information for the excore NIS drawers. As part of these routine surveillances, reactor engineering com) ares actual in-core flux 3rofile to the in-core flux profile predicted )y the core design code.

leactor engineering noted larger than normal errors between actual and predicted in-core flux profile.

Because BEACON used the same neutronics engine as used in the core design code, reactor engineering could not explain the error and notified the corporate core design engineers. As part of the process to resolve these errors, it was discovered that a simplifying assumption, used to overcome limitations of the IMPAX. was not accounted for in the original design of BEACON. This simplifying i

assumption was used because the licensee had changed the fuel design to incorporate a longer end cap to prevent debris induced fuel failures.

This longer end cap raised the overall core height by 2.64" causing an offset between detector midplane and actual core midplane.

The IMPAX code assumed detector midplane was along core midplane and could not accommodate the 2.64" offset. Therefore, the licensee, after discussion with the fuel vendor (Siemans), used this simplifying assumption to essentially lower the core midplane by 2.64" so that final design output would be referenced to detector midplane: not core midplane.

However, the engineer preparing the design inaut for BEACON was not aware of this simplifying assumption consequently 3EACON was referenced to core midplane resulting in an increased error between the core design predicted in-core flux profile and actual in-core flux profile.

PROPOSED ENFORCEMENT ACTION - NOT FOR PUBLIC DISCLOSURE WITHOUT THE APPROVAL OF THE DIRECTOR, OE

~

+

,e i

3-The licensee's root cause evaluation identified lack of cross-discipline review as the significant contributor to this design error.

The inspector concurred with the licensee's evaluation.

Engineering Quality Instructions (01) 17. Design Input / Verification, dated July 5,1995, states in part that " Design verification is the' process whereby a competent individual, who has remained ir* pendent of the design process. reviews the design inputs.... and design output to verify design adequacy. This independent review is' provided to minimize the likelihood of design errors in items that are important to nuclear safety." Contrary to this requirement, the design inputs were not

- adequately reviewed by a competent individual in that the core midplane offset was not. identified as a design input for BEACON. This failure to perform an adequate independent design review for the BEACON system is identified as example two of violation 50-335/96-17-XX, Failure to Control the Design Process According to the Requirements of 10 CFR 50, l

Appendix B. Criterion III The sa"ety significance of reversing the detector inputs to the NIS drawer; substantially reduced the safety margin between the TM/LP trip setpoint and the analysis limit even considering the increased TM/LP margin to the trip set)oint due to actual core operating conditions.

The safety impact of t1e failure to identify the core and detector midplane offset on TM/LP or LPD safety limits was minimal.

CONNECTOR SWAPS AT DETECTOR 6-CHANNEL B All four of the RPS Linear Range Detectors had the connectors reversed as previously discussed but the B channel unlike the other three channels was giving the correct data. At the same time that the drawers were being replaced on Unit 1, the detector. for channel B (detector no.

6) was being replaced as a maintenance activity.

During connection of the field cables, the connections were reversed for the upper and lower detection chambers, thereby causing the B channel to record properly.

The root cause for the swap of the cables was that the new detector had t

different labeling than the existing cables. The existing cables were labeled TOP SIG and B0T SIG, and the new detector had A and B.

The inspectors discussed this maintenance job with the I&C supervision who

- had supervised the latter part of this maintenance project.

Several opportunities were ) resented to the maintenance personnel, one when the detectors were checced out in the warehouse and a second time when this condition was noted in the field.

/

Maintenance personnel should have resolved the labeling problem by writing a Condition Report (CR) and having a formal resolution.

l Administrative Procedure No. 0006130. Condition Reports, rev. 4. dated L

March 22, 1996. Par. 8.1.1.A states in part that "Any individual who becomes aware of a problem or discrepant condition..

should initiate a PROPOSED ENFORCEMENT ACTION - NOT FOR PUBLIC DISCLOSURE WITHOUT THE APPROVAL OF THE DIRECTOR, OE 4

4 4

CR.. If doubt exists, a CR form should be initiated".

This failure to comply with the requirements of the administrative procedure is identified as violation 50-335/96-17-YY, Failure to Initiate a Condition Report for Labeling on Safety Related Detectors.

~

l 1

1

)

i i

)

PROPOSED ENFORCEMENT ACTION - NOT FOR PUBLIC DISCLOSURE WITHOUT THE APPROVAL OF THE DIRECTOR, OE

l

,4 1,

Violation 1 with two examples.

10 CFR 50 A)pendix B. " Quality Assurance Criteria for Nuclear Power Plants and ruel Reprocessing Plants " Criterion III requires, in aart, that... design control measures shall provide for verifying or clecking the adequacy of design, such as the 3erformance of design reviews...The verifying or checking process shall

)e aerformed by individuals or groups other than those who performed tie original design, but who may be from the same organization.

FPL Topical Quality Assurance Report. TOR 3.0, revision 11. " Design Control." Section 3.2.4 " Design Verification." stated, in part, " Design control measures shall be established to independently verify design input... Design verification shall be performed by technically qualified 2

individuals or groups other than those who performed the design.

Engineering Quality Instructions 1.7 " Design Input / Verification," rev.1, dated July 5.1995, states in part " Design verification is the process 4

whereby a competent individual, who has remained independent of the design process, reviews the design inputs.... and design output to verify design adequacy.

l Contrary to the above:

1.

Contrary to the above. on July 30, 1996. it was discovered that a design change (PC/M 009-195) was completed without an independent design verification by a competent individual.

Design change PC/M 009-195 to install new Gamma Metrics Nuclear Instrumentation i

drawers was completed by a lead designer and a lead engineer.

i

'This design change was independently verified by a second designer who had no special knowledge of the design.

A engineering supervisor approved the design.

Neither the second designer or engineering supervisor had remained independent of the design a

process.

2.

Contrary to the above, on July 30, 1996, it was discovered that an independent design review was not conducted for the installation 1

of a new core flux monitoring computer code BEACON.

During initial operation of BEACON it was found that the code did not 1

compensate for a core mid-plane offset created by a previous core modification. The engineer who prepared the design was not aware of the core mid-plane offset and the independent review of the new BEACON code did not identify this omission.

Violation 2 4

i Technical Specification 6.8. Procedures and Programs, paragraph 6.8.1 requires in part that written procedures recommended in Appendix A of Regulatory Guide 1.33 revision 2. February 1978 shall be established.

implemented..

PROPOSED ENFORCEMENT ACTION - NOT FOR PUBLIC DISCLOSURE WITHOUT THE APPROVAL OF THE DIRECTOR, OE a

l

~..

t e

2 Administrative Procedure No. 0006130. Condition Reports, revision 4.

. dated March 22, 1996. Paragraph 8.1.1.A states in part that "Any individual who becomes aware of a problem or discre) ant condition...

should initiate a CR.

If doubt exists, a CR form siould be initiated".

Contrary to the above on July 30, 1996. Instrument and Control technici'ans installing a plant design change (PC/M 009-15) did not initiate a condition report when they became aware of a discrepant condition concerning incorrectly marked cables. They continued to-install the modification and an error was made that resulted in cross-wiring of the nuclear instrumentation system.

l t

i i

l I

i PROPOSED ENFORCEMENT ACTION NOT FOR PUBLIC DISCLOSURE WITHOUT THE APPROVAL OF THE DIRECTOR, OE

i -

/

A)E'24 i

From:

Curtis Rapp, N(f k#c l To:

ATP2.ATB c

Date:

10/29/9611l04am 4

Subject:

Additinal VIO for St. Lucie Here is the Wordperfect version as you requested.

a i

i 4

j i

t 4

6 i

j l

2 i

1 I

h 9

mnox

t

)

The licensee also identified that BEACON was placed into service on Unit I without any benchmarking against IMPAX, the on-line core performance monitoring code BEACON was -

replacing. Instead, BEACON was installed on Unit 2 and benchmarked against CECORE, which did not require any modifications to accommodate the core midplane offset. Engineering Quality Instruction (QI) 3.7, Computer Software Control, revision 1, Section 5.4. requires that SQAl software shall be validated and verified (V&V'ed)in accordance with Section 5.6.

Section 5.6 states that new software shall be V&V'ed prior to use. V&V includes the use of test cases to ensure the new software produces correct results. Item 4 of Section 5.6 states that technical adequacy shall be determined by comparing the test case to results from alternative methods such as functionally equivalent and previously validated software. In the case of BEACON,IMPAX would have been functionally equivalent software. Benchmarking

]

BEACON against IMPAX may have identifed the design error concerning core midplane offset

-)

because the two codes would not have yielded the same results. Contrary to this requirement, BEACON was placed into service on Unit I without benchmarking against IMPAX. This is a Severity Level VI violation.

NOTE TO PANEL: This could be considered another example ofinadequie PMT as identified in EA 95-182. V&V is the post-mod acceptance test for software.

h J

i O

t

__