Text

Intervenor Exhibit I-MFP-81,consisting of 930301 Rev 00 to DCO-92-TN-NO55
	ML20059C666
Person / Time
Site:	Diablo Canyon
Issue date:	08/19/1993
From:	; AFFILIATION NOT ASSIGNED
To:	;
References
	CON-#194-14438 OLA-2-I-MFP-081, OLA-2-I-MFP-81, NUDOCS 9401060018
	Download: ML20059C666 (14)
	v • d • e

_

}3Y 60-278 323-

~

% ff Or.1GEph i

j_ - /W f;p-8/

NCR D arch 1, 1993

'93 Cf,T 23 P 6 :10

v r

HMQGEMENT

SUMMARY

Vibration data for STP P-6B, AFW Pump Testing, was potentially taken at the wrong location, which could have violated the requirements or ASME Section XI for inservice pump testing and Technical Specification 4.0.5.

This finding resulted in a Notice of Violation (NOV) from the NRC, issued on October 30, 1992 (ref. 5).

The root cause is that the sketch in the procedure was only a guideline and was not intended to provide exact directions to the operator.

The sketch in STP P-6B was corrected in Revision 24.

Corrective actions to prevent recurrence will include a revision to vibration testing STPs to clarify that an AR shall be written to evaluate the situation, if it is unclear where the vibration measurements are to be taken.

In addition, the corrective actions for another NCR (DCO QC-N045) will revise procedures AD1.ID2, AD1.ID3, and OM7.ID1 to provide additional guidance on when an AR is required for procedure revisions.

NUCt E AR REGut.ATORY CCW."!:"'N

'7NPh 9(

dhD_bh Of'ia

. L bc (f c A A S G MsiE h.No._f L&LW Dxt? N >

r~m

_ __ __ _.__ __ ta N rm t o(( _ _

weno t

' v kuinw

.n b

W 6

DaTE _

ou

- v. cess Rewter I d 0 t

9401060018 930819 J

PDR ADOCK 05000275 G

PDR 92NCRWP\\92TNN055.PSN Page 1

of 16

NCR DCO-92-TN-N055 Rev. 00 March 1, 1993 been performed with the incorrect diagram in the procedure (ref. 1).

During the week of September 28, 1992, an NRC inspector was onsite reviewing DCPP activities in the area of Inservice Inspection (ISI) and Inservice Testing (IST).

The inspector identified three issues:

one regarding a hydro examiner's certification (see NCR DCO-92-MM-N058), and two regarding STP P-6B (ref. 4).

The first issue regarding STP P-6B was the diagram error in the procedure.

This error calls into question the validity of the vibration data collected while the diagram was incorrect.

Even though the February 1992 rebaseline data is ' nown to have been taken at the correct location, the inspector felt that there was sufficient scatter in the subsequent data points such that the data could have been taken from the wrong spot.

Blue l

dots marking the exact locations were missing or painted over, in some cases.

He stated that he questioned several operators, who indicated that they would follow the procedure and take data where the diagram indicated.

(Operations i

conducted additional operator interviews; however, I

most of them were unable to recall the exact j

location where they take data.)

However, placing the vibration probe directly over the discharge of the pump is difficult, due to the configuration of the equipment (ref. 1).

In addition, the diagram is primarily intended to show eneral locations, w

not exact points (ref. 4).

The second issue regarding P-6B was that the error had not been documented on an AR, and no formal engineering evaluation of AFW pump operability was done.

However, a review of AD1.ID2 determined that an AR for the procedure revision was not required because the problem had been considered a

" typo."

On October 30, 1992, NRC Inspection Report 50-275/92-27 and 50-323/92-27 was issued, including two Level IV Notices of Violation (NOVs).

NOV "A"

was in regards to the examiner certification issue (see NCR DCO-92-MM-N058).

NOV "B" was in regards i

92NCRWP\\92TNN055.PSN Page 4

of 16 1

1

r c

NCR DCO-92-TN-N055 Rev. 00 March 1, 1993 to the two STP P-6B issues described above (ref.

5).

l A third issue discussed in the Inspection Report (but not strictly part of the NOV) was a weakness in test performance.

The missing or painted-over dots.were considered to potentially affect test i

repeatability, and in one 1ccation, the equipment configuration prevented proper positioning of the vibration probe directly over the blue dot (ref. 4 and ref. 2).

P D.

Inoperable Structures, Components, or Systems that Contributed to the Event:

None.

E.

Dates and Approximate Times for Major Occurrences:

1.

January 2, 1992:

Event date.

STP P-6B Revision 21 was issued with an incorrect diagram.

2.

September 25, 1992:

STP P-6B was Revision i

24 was issued for an unrelated reason, but correcting the error in the diagram.

3.

Sept. 28-Oct.

8, 1992:

Discovery date.

NRC inspector identified the error in the procedure.

4.

October 30, 1992:

The NRC issued a Level IV NOV for this finding.

F.

Other Systoms or Secondary Functions Affected:

None.

G.

Method of Discovery:

NRC personnel identified the discrepancy during an inspection in the area of Inservice Inspection and Testing.

92NCRWP\\92TNN055.PSN Page 5

of 16

NCR DCO-92-TN-N055 Rev. 00 March 1, 1993 H.

Operator Actions:

None.

I.

Safety System Responses:

None.

III.

Cause of the Event A.

Immediate Cause:

The immediate cause was that an AR was not initiated to identify and evaluate the consequences of the problem in STP P-6B.

B.

Determination of Cause:

Refer to Root Cause Analysis (ref. 7).

C.

Root Cause:

The root cause is that the sketch in the procedure was only a guideline and was not intended to provide exact directions to the operator.

D.

Contributory Cause:

1.

The system engineer did not formally document his review of the previous vibration data, because he felt that the error was outside the guidance in AD1.ID2 for writing hrs on procedure errors.

2.

Blue dots showing the proper locations for vibration measurements were considered operator aids only and were not programmatically controlled.

IV.

Analysis of the Event A.

Safety Analysis:

The operators that perform vibration measurements are trained to take readings on the bearing cap.

PG&E relies upon the skill-of-the-craft for these measurements, and the blue dots are only an aid to the operators.

Similarly, the diagram in STP P-6B 92NCRWP\\92TNN055.PSN Page 6

of 16 i

l

l l

NCR DCO-92-TN-N05c Rev. 00 March 1, 1993 is only meant to show general locations and the labelling of the data points (i.e., turbine outboard, turbine inboard,. pump inboard, and pump outboard locations are data points 1, 2,

3, and 4 in that order).

j As an example, the February 1992 performances of this monthly test were observed by the system engineer and the i3T coordinator to have been performed at the proper locations (ref. 2), even after the error was introduced into the procedure.

In another case, the probe was properly placed perpendicular to the surface, rather than improperly attempting to use an inaccessible blue dot.

The operators are not constrained to use locations only if marked by blue dots, and therefore did not immediate.'y consider that an AR was necessary for the missing or inaccessible blue dots.

Subsequent review of AP C-12 and AD1.ID2 has verified that no AR was necessary for the minor typo in the STP diagram.

Although not documented in a formal engineering evaluation, the data was reviewed by the system engineer and found to be:

consistently below alert levels comparable to Plant Manual Volume 9 baseline e

data.

If vibration levels had reached levels that required action to be taken, this vibration would also have been noted at the ir. correct location specified in the diagram.

In addition, tue error in the diagram has been corrected to minimize the l

potential for a similar misunderstanding in the future.

Therefore, the health and safety of the i

public was not adversely affected by this finding.

i B.

Reportability:

j t

1.

Reviewed under QAP-15.B and determined to be non-conforming in accordance with Section 2.1.3 as a problem that results or may result j

in an NRC Notice of Violation.

2.

Reviewed under 10 CFR 50.72 and 10 CFR 50.73 per NUREG 1022 and determined to be not reportable.

Any unexpected changes or excessive vibration would have been noted 1

l 92NCRWP\\92TNN055.PSN Page 7

of 16 4

1 1

. =., ~,

w

1 NCR DCO-92-TN-N055 Rev. 00 March 1, 1993 during the routine trending of test data (ref.

2); therefore, there were no inadequate or missed surveillances.

PG&E is required to submit a written response to the NOV within 30 days (see Investigative Action #3 below).

3.

Reviewed under 10 CFR Part 21 and determined that this problem will not require a 10 CFR 21 report, since (a) it is being evaluated under 10 CFR 50.72/73, and (b) it does not involve defects in vendor-supplied services / spare parts in stock.

4.

This problem will not be reported via an INPO i

Nuclear Network entry.

5.

Reviewed under 10 CFR 50.9 and determined to be not reportable since this event does not have a significant implication for public health and safety or common defense and security.

6.

Reviewed under the criteria of AP C-29 requiring the issue and approval of an OE and determined that an OE is not required.

V.

Corrective Actions A.

Immediate Corrective Actions:

1.

Revision 24 of STP P-6B, issued on September 25, 1992, corrected the error in the diagram.

2.

Sample test data taken at the incorrect location was within the normal data scatter.

However, the system engineer reviewed the previous months' data and found it to be consistently below alert and action levels.

3.

Pumps requiring vibration data were walked down and the surveillance test procedures were reviewed (see Investigative Action #5 (AE #05) below).

92NCRWP\\92TNN055.PSN Page 8

of 16

~

e

.~

n,-.>,-w

.a-_.._

._.s

..x.

I

?

4 NCR DCO-92-TN-N055 Rev. 00 March 1, 1993 B.

Investigative Actions:

1.

Investigate and document DCPP's position regarding ASME Code requirements for repeatability of pump vibration tests.

l RESPONSIBILITY:

D.

Spencer ECD:

COMPLT DEPARTMENT:

Plant Engineering Tracking AR:

A0283979, AE #01 2.

Discuss with plant management and determine PG&E's position on whether to dispute this NOV.

l RESPONSIBILITY:

C.

Groff ECD:

COMPLT DEPARTMENT:

Technical Services Tracking AR:

A0283979, AE #02 3.

Prepare and submit PG&E's reply to the NOV.

RESPONSIBILITY:

P.

Natividad ECD:

COMPLT DEPARTMENT:

Regulatory Compliance Tracking AR:

A0283979, AE #03 4.

Document the System Engineer's personal observations of vibration data collection per STP P-6B.

Did the operators appear to be referring to the diagram in the STP for placement of the probe?

Document the reasoning why the diagram error was not considered to be significant.

RESPONSIBILITY:

M.

Coward ECD:

COMPLT DEPARTMENT:

System. Engineering Tracking AR:

A0283979, AE #04 5.

Review other vibration STPs to verify that no l

others have incorrect diagrams.

l RESPONSIBILITY:

C.

Pendleton ECD:

COMPLT DEPARTMENT:

System Engineering Tracking AR:

A0283979, AE #05 6.

Perform a root cause analysis for this NCR.

l RESPONSIBILITY:

C.

Groff ECD:

COMPLT DEPARTMENT:

Technical Services Tracking AR:

A0283979, AE #06 92NCRWP\\92TNN055.PSN Page 9

of 16

i NCR DCO-92-TN-N055 Rev. 00 i

March 1, 1993 7.

Discuss with System Engineering and Operations and determine the method DCPP will use to control repeatability of vibration measurements (for example, programmatically-controlled blue dots?).

Determine the policy / program for whether all " operator aids" need to be programmatically controlled.

l RESPONSIBILITY:

D.

Spencer ECD:

COMPLT DEPARTMENT:

Plant Engineering Tracking AR:

A0283979, AE #07 t

C.

Corrective Actions to Prevent Recurrence:

In addition to the following corrective actions, the cor.ective actiors for NCR DCO QC-N045 will revise procedures AD1.ID2, AD1.ID3, and OM7.ID1 to provide additional guidance on when an AR is required for procedure revisions.

Those revisions will be tracked to completion in NCR DCO-92-QC-N045.

1.

Revise vibration testing STPs to instruct that i

if the blue dots are missing:

stop the test, write an AR to the System Engineer, and obtain guidance from the System Engineer on the proper location for the test probe prior to proceeding with the test.

RESPONSIBILITY:

D.

Spence'r ECD:

1/31/93 DEPARTMENT

Plant Engineering Tracking AR:

AG283979, AE #08 i

Outage Related? No 1

OE Related?

No NRC Commitment? No I

CMD Commitment? Yes D.

Prudent Actions (not required for NCR closure) 1.

In addition to the painters' training that will be performed as discussed in CHRON200466 (re: INPO SER 16-90 and OE 4604), please include information regarding the importance of replacing the blue dots on pumps, as directed by the applicable System Engineer.

RESPONSIBILITY:

V.

Smart DEPARTMENT:

GC Paint 92NCRWP\\92TNN055.PSN Page 10 of 16

.. ~ - - - - -

,..m

.y

_i,,.-.,

--m n,..

i NCR DCO-92-TN-N055 nev. 00 March 1, 1993 Tracking AR:

A0293919 2.

Revise or add to the System Engineering program procedure / checklist to include responsibilities for proper placement of blue dots for vibration measurements.

RESPONSIBILITY:

M.

Burgess DEPARTMENT:

System Engineering Tracking AR:

A0293933 VI.

Additional Information A.

Failed Components:

None.

B.

Previous Similar Events:

None.

A search of the STATUS database revealed several NCRs regarding NOVs received due to inservice testing issues.

However, none were due to a potentially misleading discrepancy in a diagram or drawing.

j C.

Operating Experience Review:

1.

NPRDS:

l Not applicable.

i 2.

NRC Information Notices, Bulletins, Generic Letters:

A search of the Operating Experience Assessment database under keywords PROCEDURE and DRAWING and PERFORMANCE did not reveal any items.

A search under keyword PAINT did not reveal any items related to testing or probe positioning, but did reveal NRC IE Notice 91-46 discussing painting-over of diesel generator fuel injectors (see Prudent Action

1 above).

3.

INPO SOERs and SERs:

A search of the Operating Experience Assessment database under keywords PROCEDURE 92NCRWP\\92TNN055.PSN Page 11 of 16 i

- -.. - 1

m r

NCR DCO-92-TN-N055 Rev. 00 i

March 1, 1993 and DRAWING and PERFORMANCE did not reveal any items.

A search under keyword PAINT did not reveal any items related to testing or probe positioning, but did reveal INPO SER 16-90 discussing paint binding of diesel generator metering rods.

(See Prudent Action #1 above.)

D.

Trend Code:

TN - A4 (Technical Services) - (Programmatic Deficiency)

E.

Corrective Action Tracking:

1.

The tracking action request is A0283979.

2.

Are the corrective actions outage related?

No.

F.

Footnotes and Special Comments:

None.

G.

References:

1 1.

Technical Specification 4.0.5 2.

Initiating Action Request A0280266 A0279061 A0279035 3.

STP P-6B, " Routine Surveillance Test of Steam-Driven Auxiliary Feedwater Pump" E-mail, "STP P-6B," from MGC2 dated 4.

October 8, 1992 E-mails dated October 8, 1992, October 9, 1992, and October 21, 1992 from RxTS to various, regarding NRC inspector exit meetings 5.

NRC Inspection Report 50-275/92-27 and S0-323/92-27, dated Oct. 30, 1992 6.

Nonlicensed Operator Requalification-Instructor Lesson Guide NLR9121, " Vibration Instrument Training" 92NCRWP\\92TNN055.PSN Page 12 of 16 m-r i*. -

- 'er m

w 1--

r

l 4

NCR DCO-92-TN-N055 Rev. 00 March 1, 1993 7.

Root cause analysis 8.

PG&E Letter No. DCL-92-262, dated November 25, 1992, " Reply to Notice of Violation in NRC Inspection Report 50-275/92-27 and 50-323/92-27" 9.

Memorandum from NOS'(E. Hamilton) to DCPP (W.

Crockett) dated December 9, 1992, in response to INPO SER 16-90 (SREF 92-114) and INPO OE 4604 (SREF 91-155)

(CHRON200466)

H.

TRG Meeting Minutes:

On November 5, 1992, the initial TRG convened and considered the following:

1.

The TRG reviewed the NOV.

There are three issues:

(1)' tests had been performed with the blue dots missing or in an inaccessible configuration, (2) the procedure diagram error indicating the incorrect location was i

discovered and corrected without an AR and formal evaluation, and (3) no AR had been initiated during previous tests when no blue dots were present.

However, the third-issue is not strictly identified as part of the violation.

l 2.

PG&E's position during the exit meetings was that the blue dots are just an aid to the operators, and we actually depend on operator training and skill-of-the-craft to position the probes properly.

In addition, no AR was written for the error in the STP because the arrow was still on the proper side of the pump (inboard vs. outboard).

Management may want to consider whether to dispute the NOV.

3.

The TRG reviewed the immediate corrective actions taken.

Investigative actions were assigned, and this TRG will reconvene on 11/13/92.

On November 19, 1992, the TRG reconvened and considered the following:

92NCRWP\\92TNN055.PSN Page 13 of 16

i s

i NCR DCO-92-TN-N055 Rev. 00 March 1, 1993 1

1.

The TRG reviewed the status of Investigative i

Actions #1-#5 above.

Plant management has i

decided to accept the violation, with the NOV response to discuss the minimal significance of the issue.

The AFW system engineer discussed his observations of proper testing and the discovery of the error during an unrelated procedure revision.

A review of other vibration STP diagrams was performed and i

they were determined to be adequate.

2.

The TRG reviewed a draft of the NOV response.

Blue dots are a separate issue and will not be discussed in the NOV responce; however, this TRG will investigate and tument PG&E's i

position on repeatability of vibration measurements.

Corrective actions credited in the NOV response are actually tracked by another NCR (DCO-92-QC-N045).

l 3.

Additional AEs were assigned to document the TRG's investigations (see Investigative Actions above).

This TRG will reconvene in approximately the first week of January

'93.

l On January 5, 1993, the TRG reconvened and considered the following:

1 1.

Blue dots snould De programmatically controlled, because they are how we ensure repeatability of vibration measurements.

STPs taking vibration r.easurements will be revised to stop the test, write an AR, and contact the system engineer for proper probe placement prior to continuing the test if the blue dots are missing.

I 2.

An INPO document referenced in AP E-55 discusses that " operator aids should be controlled," but E-55 is geared towards instructions posted on lamacoids, and there is some question whether we are even committed to the INPO document.

3.

The TRG proposed a corrective action to revise the System Engineering procedure to add the responsibility for knowing the proper location for blue dots / vibration measurements.

92NCRWP\\92TNN055.PSN Page 14 of 16 1

l

I NCR DCO-92-TN-N055 "av.

00 March 1, 1993 i

]

4.

The TRG discussed potential actions to prevent the blue dots from being painted over.

However, the TRG recognizes that a pump will be stripped before repainting, and ". hat the most feasible way to assure blue dots are repainted is when they are noted to be missing during the next monthly vibration STP.

These STPs are likely to catch missing blue dots more efficienu./ than the System Engineer (quarterly?) walkdowns.

1 5.

ML20059C666

Text

SUMMARY

References:

Navigation menu

Search