ML20238E568
| ML20238E568 | |
| Person / Time | |
|---|---|
| Issue date: | 12/31/1986 |
| From: | Heltemes C NRC OFFICE FOR ANALYSIS & EVALUATION OF OPERATIONAL DATA (AEOD) |
| To: | Stello V NRC OFFICE OF THE EXECUTIVE DIRECTOR FOR OPERATIONS (EDO) |
| Shared Package | |
| ML20238E398 | List:
|
| References | |
| FOIA-87-465 NUDOCS 8709150076 | |
| Download: ML20238E568 (6) | |
Text
- _ _ _ _ - _ _ - _ _ - _
pSo ug y
ge *
. UNITED STATES NUCLEAR REGULATORY COMMISSION vl
~{ .i WA$MNGTON, D. C. 20555
% / December 31, 1986 f
NOTE TO: Victor Stello, Jr. .
1
.w Executive Director for Operations '
FROM: C. J. Heltemes, Jr., Director Office for Analysis and Evaluation of Operational Data
SUBJECT:
THE AIiACHED RESPONSE TO BILL CONWAY AT INP0 In his letter to you, Bill Conway raises two issues. First, an objection to the use of contractors to review INPO programs; and second, a recommendation that the semiannual NPRDS report be terminated. I would like to coment on both of these issues.
As reflected in the enclosed response, our assessment program does not use a contractor to evaluate INP0 and has not changed at all. With the knowledge, and at least the initial consent of INPO, we have used EG&G to extract and process specific data; but the writing of the evaluation report, the drawing of conclusions and the presentation of the information are done by AE00. Thus, the NRC staff is evaluating this INPO program, not a contractor.
With regard to termination of the semiannual evaluation report, the requirement to prepare these reports originated with the Comission and we strongly recom-mend that these evaluations continue for the following reasons:
- 1. With the implementation of the LER rule in 1984, the NPRDS became the primary source of component and train level failure data. It appears unlikely that either the NRC or the industry will ever develop another failure data base to replace the NPRDS; the resources would be prohibi-tive.
- 2. Since the NRC is depending on the industry for data needed by the staff, there should be a formal and continuing basis for confidence that NPRDS data is of good quality (accurate and complete engineering and failure records) and is suitable for the various NRC needs for component and train failure data.
- 3. From an end user perspective, we continue to experience difficulty in using NPRDS data. User feedback indicates that there are problems of inconsistencies in the engineering reports, missing records and errors in the data, and insufficient narrative descriptions. In addition, the data may not support some end uses by design (e.g., not entering replacements in-kind for incipient failures or periodic preventative maintenance).
Some of these problems are to be expected of any data base system but some
)~c34 VW G709150076 870911 PDR FOIA QORDONB7-465 PDR
- l 4
' Victor Stello, Jr + ' 3-may significantly degrade the usefulness of the results and may not be obvious to the casual user. - The periodic evaluations allow us to monitor and report on such problems.
- 4. Perhaps most importantly, we are-not yet confident that the industry participation, in failure reporting, and the data base completeness, are at an acceptable level. As you know, our standard evaluation has been based '
on a comparison of the failures reported-in LERs against NPRDS data to dnermine completeness of the data base, and, consequently, INP0 has strongly emphasized to the industry the importance of reporting all such {
LER failures to the NPRDS. As a result, preferential reporting of failures i contained in LERs may be occurring which would skew our evaluation results ;
to the favorable side. The best check on this situation would be plant '
specific maintenance record comparisons to the NPRDS but the required resources would be terribly high. !
As an alternative, I recently initiated discussions with INP0 and Jim l Partlow to explore gaining access to the maintenance records pulled for !
the INP0 plant evaluations and to understand and sample the process that they use for plant evaluation of the NPRDS. This effort is continuir,;.
We are also exploring other alternative methods including: comparing NPRDS data to maintenance records from the PAT team for Palisades for three systems; comparing NPRDS data to licensee safety-related maintenance monthly reports; comparing NPRDS data to IPRDS records; and comparing NPRDS engineering records to EQ data base records.
In sum, we believe we need to check on whether preferential reporting of failures in LERs is occurring and, in the process, develop a firmer basis for confidence that the NPRDS is being routinely and properly implemented by l licensees.
At some point in the future we should consider revising our report frequency from semi-annual to at least annually and perhaps bi-annually. The frequency should be changed when we gain confidence that the system is in routine !
operation at all plants. However, we do not believe that we are there yet. ,
I would be pleased to provide any other information or clarification that you would desire.
Ot W. .. ,
N M M.
C. J. Heltemes. Jr., Director Office for Analysis and Evaluation of Operational Data
Attachment:
Distribution:
Proposed Response to IMPO AEOD CF F. Hebdon AE00 SF M. Williams J. Heltemes IFC : A OD:D : :
- AME :CJH
( mes:cg:
ATE : 12A\/86 : : : : :
j i
0FFICIAL RECORD COPY
a
._g .
- o .- 2
(
V E L1-28-87 DISCUSSION WITH PETE LYONS AND RON SIMARD~(INPO)
(rt M Md.O . .)Mrs / q ( AM, -
j i
l.s 1. . Discussed NPRDS contract proposal: l l
t Rec oromendat i on f or a five year and/or 700K limit was proposed by Mark. j
-* -INPO discussed the need. to recalculate the hourly usage fee'for 1987 ' r at es. -
o
- L .INPO.wants to add sentence under Scope of Work,'e.g.,.
coordinator to help users to ensure accurate searcdh and minimize system usagettime. .INPD gave the example of a Wall'ey Norris (NRC user) who generated 36" of paper because he requested a reactor trip breaker-failure printout listing each. failure on'a separate sheet. INPO stated that the NRC usage'was second to Duke Power.
- Under Task'1,,INPO'wants to-add words that ensure: .
contractors do.not have access to the password (i.e., must.obtain l
~ data from NRC coordinator).
- Under Task 3, INPO will add words that address no-more H formal training once the new IBM system is installed.- INPO will i also ' add words that' the same level of. training offered all mernber s: including -NRC.
-* .INPO proposed that AEOD's evaluation be-included in IE'sLinspection efforts. Data tapes will no longer.be provided t o - NRC. : ..Thereforer Mark recommended deleting. Task 4, Item b. The datautape issue'will be handled outside this contract.
'* INPO will.put NRC on distribution for all info provided to other' members (Task 4, Item d.).
- Under Task.6, INPO will propose words to . ensure system usage time is mot ext ended beyond ' norrnal for all searches. INPO monitors usage only once per month unless a flagged item indicaJtes a problem.
- 2. NPRDS reliability evaluation discussions
- Face tol face meetings (workshops with utilities) are used ' to resolve reporting issues. Four more meetings scheduled for.1987. INPO. was reluctant to allow NRC attendance. INPO stated that they could send'us copies of any output documentation.
I
-from the buffer and spot checks' prior to permanent NPRDS update.
a FoZM-81-V65' G //o
-INPO now knows the poor ' per f ormer s, therefore, INPO's attention will ' be pl aced there (i.e., less on good and medium per f ormers and concentrate on poor performers).
- - INPO feels conversion to IBM will reduce / del ete the NRC threat of LER Rule change (i.e., much more reliable data will be produced - because the new system is user friendly ).
evaluations have increased the NPRDS prgrammo ment em. NRC's LER check is a srna 11 e f for t cornpared ' to INPO's OA prog" am (e.g.,
curves on timeliness of .reportings, number of reports per plant, ;
plant evaluations, and all concerns directed to the CEO level ). l i
l
- INPO discussed pl snts reporting- commitment i i mp r overnen t s.
- Mar k di scussed NRC's end user needs and probl ems. NRC l evaluation is shi f ting from stressing completeness to stressing l quality of data.
- 1. Monthly process (early identi fication) indicators - computer program provides number of f ai l ur es/mont h/pl ant - fl ags e.g. , O reports j for 2 months, O report in 2 of past 6 months, j t i mel i ness, etc. (crude indicators - pr ovides paper trail from data base) !
i phone calls - auditor to reporter, Ron Si rnar d !
to plant coordinator, Lyons to VP, Pate to .
CEO. !
l LER, Monthly Operati ons Report , etc . review - !
confirms whether or not f ailure reports are i being reported as indicated by the plant ;
reporter during followup phone calls. !
Maintenance Work Request (MWR) reviews.
- 2. Plant evaluation process (every 1.5 year s on l the average, more frequent for poor per f ormers) -
100 MWRs sampled - INPO now keeping for possible review by NRC. Sample is pulled prior to INPO site visit to provide problem areas for evaluator to focus on.
Per formance objective criteria (DE.4) -
INPO evaluator reviews plants program against this criteria.
i
- --_-_ -- -- \
_3 Findings are documentec in evaluation reports. Some verbal conc erns may not be a documentated in the report but will be in the evaluators field notes and voiced i r.
the exit.
- 3. Plant "cwsist visits" - 1 week at plant going over plants compl et e progr atu. INPO requests these vi si t s.
A detailed audit of NPPDS covering compl et eness, timeliness, out of service reporting. Findings are documented in report / notes. Evaluator field notes may '
be used in this process as followup in futher plant eval uat i ons. INPD is planning 10 to 12 visits per j year as bugeted for 1987. There may be some sensitive paper that INPO may not want NRC to look at, however, i
field notes and reports are now given to IE for review.
INPO stated that this process is not documented.
- INPD expects comments from NRC as users.
- INPO can provide records on a PAT inspected plant only since PAT reports are similar to INPO reports per prior agreements with NRC.
- CFAS (component f ailure analysis system) and EFAS (equipment failure analysis system) software will be implemented by INPO . Is comparabl e to the NRC's FWCV study.
- INPO proposed that NRC resolve data problems thru INPO instesd of bugging the licensees. INPO believes that sometime i r.
1988 NPRDS data will be cleaned up. Contractor doll ar s for QA of engineeing records has been dropped each year due to other priorities. INPO has previously stressed reporting. " Release 3" will be . issued in early 1988 to OA the old data entered (pre INPO) bac k to 1974. INPO has improved the quanity of failure reports up but has not tackled quality of engineering reports yet. ,
- INPO proposed AEOD going to Partlow and arranging an inspection of NPRDS. INPO does not want to provide the data tapes anymore. INPO wants NRC to look on line for speci fi c info instead of sending tapes to EG&G. Use of a contractor is the hard spot.
- INPO proposes that NRC abould go away from discussions with INPO recognizing the weak areas and then looking indepth at only these problem areas. A " seasoned evaluation of what INPO told NRC".
- "NRC's current evaluation is redundant to INPO' process.
Not telling them anything they don't already know." INPO now needs time to install the new system. NRC is pushing faster than 7NPO is capable of ~ moving. INPO feels its their job to do the studi es AEOD i s now doi ng. INPD will do them as soon as they
-4
- o .
few; the data is reliable. INPD disapproves of the ei, t r a work required by both INPD and the uti1ities in resolving NRC's evaluation concerns.
s j
l l
1
_ _ _ _ _ _ _ _ _ _ _