ML20244E129
| ML20244E129 | |
| Person / Time | |
|---|---|
| Site: | McGuire, Mcguire  |
| Issue date: | 07/22/1986 |
| From: | Crooks J NRC OFFICE FOR ANALYSIS & EVALUATION OF OPERATIONAL DATA (AEOD) |
| To: | Mike Williams NRC OFFICE FOR ANALYSIS & EVALUATION OF OPERATIONAL DATA (AEOD) |
| Shared Package | |
| ML20238E398 | List:
|
| References | |
| FOIA-87-465 NUDOCS 8607300568 | |
| Download: ML20244E129 (11) | |
Text
. .
- . 4%
{ F_ .
g #*%'%- UNITED STATES ju NUCLEAR REGULATORY COMMISSION gI , 1 ws.m.ovou. o. c. mu a .
eeo** JUL 221996 ; J 1
/
MEMORANDUM FOR: '
Mark H. Williams, Chief
> Program Technology Branch
]1 Offfee for Analysis and Evaluation 4 e of Operational Data ).
FROM: . John L. Crooks, Chief
- Data Management Section Program Technology Branch j J
SUBJECT:
OBSERVATION OF INP0's EVALUATION OF THE-OPERATING EXPERIENCE PROGRAM AT McGUIRE From February 10 through February 21,1g86 I' accompanied an INP0 team during -
3 a plant evaluation of Duke Power Company's McGuire Station. I went at INP0's i invitation to obtato firsthand knowledge of the INP0 process for evaluating utility activities in the operating experience (0E)' review area. This activity
- .. (was'to provide information pertinent to AE00's overview of nuclear power plant
.11censees' OE review actiyities. My observations and conclusons are sumarized below, and reflect my understanding of the INPO and NRC activities.
I Background - The following provides general information on why INPO conducts
, plant evaluations and how INPC and the nuclear utility industry view the relationship between their activities and those of the NRC. INPO conducts periodic evaluation.of activities at its members' plants as a part of INP0's efforts to promote the highest levels of excellence in the operation, main-tenance, and support of operating nuclear power plants, including safety. The INP0 evaluations are peer reviews of its members' activities using performance objectives and evaluation criteria, that were developed by INP0 and reviewed v by industry representatives, as the basis.
If^is' stated in the INP0 Institutional Plan dated May 1985, INP0 reviews, I although appearing similar to NRC inspections of utilities' activities, have significant differences. INP0's criteria generally establish goals that
. provide broad statements of conditions that are subjective. INP0's criteria
.are generally based on best practices and industry's conraitment to excellence, I not on regulatory requirements. In contrast, NRC regulations are conditions ]
or' specifications that must be met to obtain or retain a license and normally should not be subjective. The purposes of the INPO reviews and NRC inspections 2 are different also. INP0 reviews stress assistance in achieving a given goal, ;
whereas NRC inspections usually stress compliance with a requirement. Industry 1 intends that NRC will take cognizance of INP0 evaluations and make determina- l tions of credit due the industry for INP0 activities. Industry anticipates j p s 1
WA ^%
L^ U46 Y&MW Iyf) _ - - - - - - _
Ma k'H. Williamsi -
that the NRC will do this through reviews of the INPO process and programs, not through fonnal audits or inspections of INPO, and not through the monitoring of INP0's interactions with its individual members. Industry expects NRC and INP0 activities to be independent, yet complementary.
f This was the first detailed observation by NRC of INP0's evaluation activities in the OE review area. Other NRC staff have accompanied INPO on corporate and plant evaluations but none concentrated specifically on the OE activities.
Observations -
My preparations began in early January and included a two day visit to Atlanta the week before the McGuire evaluation (See Enclosures 1 and 2).
The INP0 plant evaluation team consisted of the team manager, seventeen INPO
'avaluators (including several trainees), two senior reactor operator (SRO) peer evaluators, an INP0 exit representative, and three industry observers.
'The OE review area normally has one evaluator assigned. The OE evaluator for this visit was an individual who had been on loan to INP0 for almost two years from an A/E finn and had participated in eight DE evaluations. He was very knowledgeable, had a diversified background, and was thorough Aring the plant '
cvaluation in the areas that he felt needed a detailed seview.
The on-site OE evaluation began the morning of February 10 after the entrance meeting between the plant staff and the INN team. A brief discussion with the two principal OE contacts at the plant site covered the general coals of the OE program and an overview of the Duke /McGuire OE review process. A series of interviews, group discussions, and reviews of documentation were begun to address the items identified for evaluation in INP0's guidance document (INP0-85-001).
This document is a working reference for INPO evaluators and
- for utilities use in self-evaluation and is revised periodically as experience is gained through the evaluation process.
The four areas specified in INPO-85-001 to be covered in the OE review are (1) the OE review organization and administration, (2) in-house OE review, (3) industry CE review, and (4) the Nuclear Plant Reliability Data System (NPRDS). Each area has an associated performance objectee that is broad in scope and supporting criteria that are more narrow in scepe and typically describe a specific activity that contributes to the achievement of the associated performance objective. The INP0 evaluator is required to assess whether the perfomance objective for each area is being met. For each performance objective, he is to assess individual criteria, but need not assess all criteria on each visit. Because the supporting criteria stated in TNP0-85-001 may not address every activity associated with the perfonnance objective (i.e., additional activities may be addressed during the evaluation),
meeting all the criteria does not ensure that the perfomance objective is fully met. INP0 also considers that the perfomance objective can be met without meeting every specific criterion. Thus, INPO emphasizes achieving i
l I
e ,
y ,
1
~e s
- }l
)h
!. , ~
h, ' Mark H. Williams the performance objective Over focusing solely on & supporting criteria.. In addition, limited resourc% and a wide range of tasks ifmit what the evaluator can do in the two week on-site period (approximatelyweven":
' working days for '
svaluationefforts).
During this evaluation, the INP0 evaluator. spent considdable tim and effort e ['
on the review and assessment of the status of the utility's actions in tesponse to INPO recommendations and consnents contained in Significant j Operating Experience Reports (SOER) and Significant Event Reports (SFR). ,
l INP0 expects utilities to be responsive to each 50ER recommendation and each ' '
comment in an Q,, He did a thorough review of the utility's asse.asment and response to each+50ER recommendation and reviewed the responses to a sampling '
I The evaluctor had detailed guidance available of SER cos:nents (about 50).
that Nas.develcM by INPO staff which prodided a basis for determining the t
<j e4 acceptability of most responses. The lhP0 nuluator indicatedithat in the OE 7
- area completirg the SOEYand SER' review war,i frpt& tant on"all diant evaluations.
INPO compares the percent of SOER recommendations / satisfactory c:mpleted by the utility to the industry average. INP0 aisv aakes statistical comparisons on NPRDS ustve and failure reporting. Other statistics on performance are ,
generated during ,tMisit to support identWication of strengths and weak-nesses. INPO also reviews a sampling of nility responses to NRC Bulletins and Information Notices for timeliness of the utility's review and occasionally ,
a f for adequacy of' response. ' '
jL ,
s 4' Other primary object',ves for the on-site _ evaluations are to identify strengths , '
t- and weaknesses (concsrns) in the OE review area and to make suggestions for t :
4 r improvements. Upon return tp 3NPO, these evaluation results are further "
assessed. F5r example, s' concern identified at the. site may become a formal i
finding in the final INPC report for which INPO requires a Terponse and action bptheutility. Strengths may becore good practi eg. 'f)ther concerns, in rare i instances, may ndt.become findings in which case they era tddressed in further discussfor)s with the utility uMre 11@0 htresses areasithat' may Aeed some corrective >ction. Thus, the W evaluator must efficiently;alMcate.his reenurceVtb (1) assess whether the four' program objectives ari beinoiret, (2 careQJ11y effectively identify and document concerns, strengths and suggestieng, review the utility's responses to SOERs and SERsd erv' (3 assess whether indivic'ual criteria are being met in each of th'eMour OE (4 '
s areas.- ,
As mentioned above, se 1NPO technical support and management staff at Atlanta reviews the information developed by the. evaluation teaweand also makes an They meet l
assessment of whether the performanec' objectives am baing met.
f with the evaluator in Atlanta normally on the first day of the second week of the evaluation for a mid-evaluation review. At the ered of the on-site activities, the Attac., staff completes its assessments, discusses options for corrective actions with the utility,to resolve findings, identifies areas that did not receive attention in a given plent evalvation so they will.be covered in the next evaluation, and generally categorizes'the status ef the utility's CE review efforts for further geference. The overall assessment of whether g/
e
,$ y }
H
- , r t
, c ,
l Mark H. Williams .
o the perfomance objective for each OE area is met is often documented by the statements of findings and good practices and s.tatements regarding the supporting criteria assessed, rather than by a specific statement addressing whether it was met. Further discussions are held between the INPO team management and the utility management where INPO communicates the results of i i
its final assessment. A first draft report is prepared and sent to the utility for preparation of its proposed response actions. INPO then assesses l
the utility draft response actions for appropriateness, cowleteness, and timeliness of any proposed corrective actions and discusses coments and suggested changes with the utility. The report is then issued as a second draft report including the utility's corrective actions. Following utility concurrence with the report wording, the final report is issued to the utility.
The final report, the supporting details, and the backup seterial are stored at INPO for future reference. I The final report for the McGuire evaluation was issued in late June 1986. The results of the OE review were documented by (1) a statement of the perfonnance cbjective for industry 0E review; (2) SOER status information, including the statistics for the number of recommendations satisfactorily completed since the last evaluation, plus those pending-awaiting decision, pending-awaiting -
implementation, and needing further review; the cumulative percent of recom-mendations with satisfactory action taken for the planf and the industry average; and a listing of the specific SOER recommendations still open; (3) a statement of a finding in this area plus the associated INP0 recomended action and the utility response; (a) a statement of the performance objective for NPRDS; (5) NPRDS status infomation, including data on usage and failure '
reporting; and (6) a statement of a finding plus INPO recommended actions and the utility response. Both INPO findings were stated succinctly and their recommendations for action were sound. The utility's written response to the
- two concerns seemed reasonable. The utility is to report on implementation 4 in mid-December 1986, and INPO will evaluate the results in its next )
evaluation.
The OE review section of the final report conformed to the existing INPO reporting practice and procedures discussed above. It, however, seemed very focused and limited in scope considering the extensive review that was con-ducted. For example, the utility during the exit meeting queried INPO regarding what was the broad implication of the two findings identified with respect to the performance objectives. Although these matters are addressed in meetings between INPO and the utility, it would seem worthwhile that this also be addressed succinctly in the final report. The background material to be maintained for future reference also seemed too cryptic to be of much value to the next plant evaluator. For example, some specific areas that warranted further review were not highlighted for the next evaluation.
Enclosure 3 provides additional information and observations regarding the plant evaluation and INPO's OE review process. .
1
____________-__-__-__D
~
h l
l Mark N. Williams L Comparisons with NRC Activities ,
A comparison between INP0's evaluation of OE review and inspections and reviews perfomed by the NRC indicates some similarities and differences. For example, in its first round of plant evaluations addressing OE reviews, INPO focused on thn utilities' efforts to establish OE review programs. Subsequently, the plent evaluations have focused on generic and plant specific concerns in the OE review area. As noted previously, INPD as a performance assessments (see Enclosures prior to 1the and site2)part visit toof each evalu identify specific areas for possible detailed review. The INPO on-site evaluations are basically snapshots (a two-week visit once every 15 sonths or so) that focus on performance objectives and criteria, but each evaluation does not necessarily include all of the criteria (i.e., it is not necessarily ,
a thorough program review). INP0 feels they can best provide a useful service i to the utility by identifying weaknesses (i.e., concerns) when noticed and {
l describing the concerns such that the utility fully understands them and takes corrective actions. INP0 also evaluates the adequacy of a utility's responses to 50ERs and SERs as noted above. In addition, if a utility is not responsive in improving on areas of major concern identified by INPO, INP0 has procedures for escalating its concerns to obtain the appropriate response. ,
Similar to INPO's initial reviews, in 1981 the NRC had its resident inspectors perform a compliance review shortly after its requirements for an OE review l function were to be implemented. Subsequently, the NRC reviews consist of j sanplings of a utility's responses to NRC Bulletins and actions taken in response to Infomation Notices and Generic Letters as part of its routine resident inspection program. The NRC does not routinely reassess CE activities for effectiveness and compliance with NRC requirements from a program stand-point, because current policy is to do perforinance based inspections. In this regard, the NRC presently includes some OE review activities in its Systematic Assessment of Licensee Perforinance (SALP) program (i.e., some data on the quality of LER reporting and event trends are included). The NRC occasionally takes escalated enforcement actions, if an event occurs at a plant that should have been prevented by appropriate corrective actions in response to a precursor event. There is a small overlap between INP0's OE review and NRC ) -
inspection activities in that IMPO reviews a sample of utility responses to NRC Bulletins and Information Notices, the NRC has selectively reviewed utility responses to several INP0 SOERs, and the NRC occasionally reassesses selected OE program elements.
Both the INP0 evaluation and the NRC inspection activities have shortcomings that are inherent in review programs using sampling techniques. Neither organization consistently or thoroughly assesses the utility's entire OE program on an established schedule. There also has been Ifmited discussions
>etween the two organizations of the results of each organization's evaluation of the OE review function from the program standpoint. i The NRC documentation of its inspection activities, including Performance Appraisal Team (PAT) reports, often include a judgment regarding whether or not a licensees activities are in compliance with requirements.
I
Mark H. Williams .
In comparison with the AEOD study of licensees' operating experience feedback activities, the INPD plant evaluations explore many of the elements covered in AEOD's study. The AE0D study was to detemine (I) the characteristics of representative OE review activities (2) how various feedback documents are used in such programs, and (3) if there is a need for change to NRC activities.
AE00's assessment reviewed programs against the regulatory intent (stated and implied) of feeding back operating experience to increase plant operator awareness of events and to have recipients initiate actions as appropriate to prevent recurrences of undesirable events and the occurrence of more i serious events. AEOD included both NRC and INPO activities and estimates of the percent of the industry feedback resulting in some action at the plant.
The AEOD report concludes that OE review efforts are not as effective as originally intended.
In comparison, INPO develops considerably more plant-specific information prior to the plant visit and during its two week on-site INPOevaluation evaluations,than AE00 did during brief one-to-two-day visits. The For similar to the AEOD study, assess issues of performance. example, INPO implicitly /
{
addresses yield (i.e., the amount of feedback resulting in change) from the l INPO feedback process via its assessment of whether.the utility s response to "
i
' 50ER recommendations and SER connents are satisfactory < INPO also gleans how effective the information screening and assessment activities are from this activity. INP0 could also detemine how much SOER and SER information gets directly connunicated to the lowest level staff (plant operators, trades personnel, chemists, radiation technicians, etc.) based on its review of specific OE review criteria; however, this is not routinely done.
Summary The INPO periodic evaluations of the OE review activities are focused, detailed performance-based reviews covering plant activities over a period of 15 months or so and include a'two-week site visit. Each visit, however, does not neces-sarily result in a thorough program effectiveness review, because that is not INPO's intent. The INPD evaluations are also not necessarily conducted on a consistent basis plant by plant, because the criterion for judging whether the performance objectives are met need not all be assessed each evaluation (i.e.,
different criterion may be reviewed during individual evaluations). There is, however, one important element that is common to each evaluation - the review of responses to SOERs and SERs. INPO reviews of OE activities are also focused on INPO guidance and INPO feedback.
INPO reviews are more complete than the NRC inspections of these activities because the industry and the utilities have the primary responsibility to effectively implement and monitor the OE review activities. Documentation of the evaluations (i.e., the final reports)The often do not explicitly state whether performance objectives have been met.
final reports, instead, identify selected findings of fact, note concerns and highlight good practices. INPO makes improvements in the conduct and reporting of the plant evaluation efforts based on their assessments of the effectiveness of their programs and in response to their members needs. The effectiveness of INP0's plant evaluations are dependent to a great extent on the utilities' acceptance of INPO's findings and their willingness to take permanent correc-tive actions to resolve the identified concerns.
Mark H. 6lilliam's '
Conciusicns As a result of my participation in the INP0 plant evaluation, severil clarifications were made to the AEOD report on licensee OE programs to better describe INPO's OE evaluation activities and those of some utilities.
Both the discussion section and at least one finding were modified to clarify information. Infomation was also gained that altered some conclusions and recossnendations in the draft report.
d4.f t @' .-
John L. Crooks, Chief
< Data Management Section V, Program Technology Branch, AEOD cc: w/ attachments:
Robert Capra, DROGR James G. Partlow, IE . - i Robert Bauman, INPO 2
l 4
\
l L______-.
Enclosure 1 INPO forwarded plans, schedules, travel arrange [nents and Early January -
the utility's six months status report on responses to INP0's previous visit for review. .
Mid January I reviewed NRC's LER files, NPRDS data, and sections of ;
the McGuire FSAR.
I visited INPO to complete preparations. Introductory February 5 and 6 discussions were held with the Team Manager (Manager, Internal Training Department) and the OE evaluator (a member, Events Analysis Department). I reviewed material prepared by INPO as background for the evaluation team, principally the Plant Experience Report (PER) outlined in enclosure 2. A four hour training session on how to be an effective observer of plant activities was provided. The session consisted of a brief lecture and four to six exercises, where three participants reviewed video tapes of staged plant activities, noted observations, and then '
critiqued them with the instructor , I attended a six-hour training session on the McGuire plant specific radiation protection (RP) program, plant security and Duke's i i
industrial safety program. Participants had to take an exam and pass with an above 80 score in order to qualify .
for unescorted access to most areas on-site. This session i was referred to as the McGuire site-specific General EmployeeTraining(GET). To be authorized unescorted access at the site, personnel also had to document that they had received certified utility generic GET-RP training that consisted of one to two days of RP training, !
l
'had passed the INPO generic RP exam with a score of at I least 80% within the last year, and had security and psychological screening. Subsequently, I attended the second general team meeting where the team manager gave I general instructions on how the team was to function to accomplish its objectives; INPO support staff provided information on problems noted with McGuire's procedures; lead evaluators briefed the team on administrative i controls of interest to other evaluators when observing activities; and then each evaluator discussed the l background infonnation (known deficiencies and weaknesses {
in the program) for his area, his plans, and areas where support was needed from other evaluetors. The exchange of infonnation was frank and factual. Final travel l arrangements were also discussed. The team manager also reviewed how paperwork was to flow to him during the
~
l evaluation and subsequent to it.
l i
l
- 1 9
~
l Enclosure 2 INPO's Plant Experience Report
/
This document is assembled by INPO's Analysis and Engineering Group to assist evaluation teams in preparing for a plant evaluation. Plant per-formance and program activities in key areas are discussed. For example. l 1
the plant events section includes discussion on LER, SOER, and SER statistics.
Comparisons are made to previous. evaluations and to industry averages, trends are analyzed, and the causal factors are assessed. Reactor trip data receives 1 similar analysis. The document usually contains:
4 (1) an executive sumary that highlights key findings and makes comparisons to industry average perfomance in many areas and, ,
(2) sect ions with significant information (statistics, prcgram strengths, weaknesses, comparisons to other similar plants or industry averages, and INP0 assessments, particularly of '
problem areas) on the following topics: -
a.- plant description organization and management
- b. '
- c. manpower and training i
- d. operations and outages
g results)
- f. NPRDS
'h. radiation protection i INPO evaluations and assistance visits
- j. regulatory activities (SALP, enforcement history, etc.)
4 I
- k. INPO - utility interar tions
- 1. miscellaneous
)
l
y ....
p .,
i Enclosure 3 p "..
- l. Addu.ional Information and Observations
- 1. INPO plant evaluations are guided by INPO-85-001, " Performance Objectives and Criteria for Operating and Near-Term Operating 1.icense Plants."
It
- 2. All of the team members were technically knowledgeable and experienced personnel; many had twenty or acre years of nuclear experience and had participated in over 10 plant evaluations. Eight team members were on lean to INPO from utilities and other nuclear industry organizations; the remainder were INPO staff, except for the two SRO peer evaluators and the three industry observers. The role of the industry observers is to observe the INPO evaluation to learn the INPO process, to broaden their personal knowledge, and to feedback observations to INPO and their sponsoring organizations.
- 3. . Discussions at Duke's corporate offices were held with personnel who oversee the OE program, who screen and assess OE, who coordinate and "
~
c oversee NPRDS activities, and who have responsibi.11ty for controlling licensing activities. Discussions with line supervision and middle managers also took place. At the plant site, discussions were held with the McGuire Safety Review Group that reports directly to corporate staff, line supervision (including department managers), senior reactor operators (SRO), support engineers, and operations engineers. The l training, manager was also interviewed at the training facility.
- 4. The documents reviewed included corporate and plant procedures for OE, corporate program plans, reports of effectiveness reviews by the McGuire safety Review Group and Corporate Nuclear Safety Review Board, samples of OE routing action sheets, actions letters, action tracking sumary logs and computer printouts, post-trip reports, etc.
- 5. The If.Pb evaluator maintained daily communications with the team manager on ine status and direction of his activities. He also coordinated with other teaa, members who were assigned to support him through their routine ectivities. He periodically contacted his management at Atlanta to provide status and to seek advice, direction or support.
- 6. The evaluator very methodically developed his concerns and involved the corporate and plant personnel responsible ard accountable for the activities being reviewed during the process. Frequent' meetings were held to discuss facts, clarify issues and reach agreements where possible.
- 7. The OE evaluator indicated that this evaluation was more complex than ;
nomal due to the significant involvement of the Duke corporate staff in the.0E review efforts. As a result, a number of items routi6ely assessed more thoroughly did not receive much attention. For example, !
the assessment of the quality of the screening efforts and of the assessments for applicability of the information to McGuire was gleaned ,
from the 50ER and SER review activities and were not confimed by interviews and the review of supporting documentation. Nor was the i
6 4
. Enclosure 3 quality of assessments on other than INPO feedback (i.e., NRC Bulletins and Information Notices or vendor letters) reviewed. Although an assess.
ment was made of the process for communicating OE infomation to plant operators and technicians, only one position (SRO) was interviewed to confirm their receipt of OE infonnation. In addition, time was not available to review audits of training for the OE area or to review materials used in training for OE feedback.
- 8. The specifics on the status of each open item on red (imediate attention)
SOERs is currently being documented in a memorandum to the President of INPO after each evaluation for further review and action, if necessary.
- 9. The information included in the exit package that was reviewed in an exit meeting with the top corporate and plant management two weeks after the on-site evaluation appeared detailed enough to adequately support the findings and concerns identified by INPO. It contained copies of the INF0 hand-written field notes. The first draft report contained basically the same infomation as the exit package, but in a more fomal format. Both OE findings were reasonably well presented in these documents.
~
- 10. Although some notes were maintained by the INPO evaluator on some major aspects of the evaluation, copious notes on all aspects of the evaluation were not maintained. The background material left for the next evaluator did not seem comprehensive enough, particularly in providing guidance in areas of potential concern to be evaluated in the next plant evaluation.
Usually, since INPO changes evaluators after two successive visits by an evaluator, the new evaluator has discussions with the last evaluator. In this case, however, the McGuire evaluator was leaving INP0's employ.
1 ;
- - - _ _ _ _ - _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ . _ _ _ _ _ _ _ __ __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __