ML20128L177
| ML20128L177 | |
| Person / Time | |
|---|---|
| Issue date: | 10/09/1996 |
| From: | Jordan E NRC OFFICE FOR ANALYSIS & EVALUATION OF OPERATIONAL DATA (AEOD) |
| To: | Hintz D ENTERGY OPERATIONS, INC. |
| References | |
| NUDOCS 9610110331 | |
| Download: ML20128L177 (10) | |
Text
Nir. Don:Id C. Hintz, Presid:nt and Chisf Ex:cutivs Officer - Nucisar Entergy Op:rstions Inc.
P.O. Box 31995 Jackson, MS 39286-1995
Dear Mr. Hintz:
Thank y u for participating in the effort requested by the Nuclear Regulatory Commission Chairman for our office to evaluate the development of indicators that can provide bases for judging whether a plant should be placed on or deleted from the watch list. As discussed with Mr. Richard J. Barrett of my staff, we have contracted with the firm of Arthur Andersen & Co. to provide us an independent analysis of the characteristics, measures, and indicators that have been, could be, and are recommended for consideration regarding the placement of plants on and removal of plants from the NRC Watch List. In 1
conducting this analysis, Arthur Andersen has been requested to conduct several interviews with personnel from licensees who have in the past been identified as having plants that were either on the watch list or were identified as good performers. These interviews are to gather information on how the licensees measure declining and improving performance and to understand what characteristics, measures, and indicators they use to determine these trends. An excerpt from the Statement of Work for this portion of the project is included for your information.
l The interview is scheduled to be held in your Jackson offices on October 17,1996, beginning at 7:30 a.m. Two members from an Arthur Andersen team of four members will be conducting the interview. Copies of the resumes of the team members are enclosed for your information. They will be accompanied by Mr. Barrett. Mr. Barrett willintroduce the Arthur Andersen personnel and will participate in the initial portion of the interview. He will then excuse himself for the remainder of the interview.
Please contact Mr. Barrett at (301) 415-7488 if you have any questions or comments.
Sincerely, Original Signed b E.LJordan y;
Edward L. Jordan, Director Office for Analysis and Evaluation of Operational Data
Enclosures:
- 1. Statement of Work for Arthur Andersen Distribution w/ enclosure 1:
- 2. Resumes of Arthur Andersen team members File Center Public Distribution w/o encl.:
AEOD R/F KSalehi, PM-ANO WBeckner, NRR RAB R/F JDonohew, PM-Grand Gulf ECollins, R-IV
[)
SPD R/F DWigginton, PM-River Bend PHarrell, R-IV
} } () },j,,j CPatel, PM-Waterford JDyer, R-IV Vfst-7TF ru I r
y 'g> g,. p p q Lcu..
(
DOCUMENT NAME: C:\\WP51\\WPDOCS\\11NTERVW.TRW i
"See previous concurrence:
To receive a copy of this document, indicate in the box: "C" = Copy without attachment / enclosure "E" = Copy with attachment / enclosure "N"
- No copy n
OFFICE RAB lC__ AC:RABl C D:SPD l C l}D:Ap D:AJ@
pR ps0(
Ej2rpen NAME LMPadovan:mmk RBarrett CRossi 9
DATE 10/08/96*
10/09/96*
10/09/96*
1QM95
'10/ /96 9610110331 961009 cFICIAL RECORD COPY f/,-/ M h
h ff:Yk 00 PDR
Don::Id C. Hintz, Prcsident end Chief Executiva Officer - Nuclear Entergy Op2 rations Inc.
P.O. Box 31995 Jackson, MS 39286-1995
Dear Mr. Hintz:
Thank you for participating in the effort requested by the U.S. Nuclear gulatory Commission Chairman for our office to evaluate the development of in scators that can provide a bases for judging whether a plant should be placed on or leted from the watch
)
list. As discussed with Mr. Richard J. Barrett of my staff, we hav contracted with the firm of Arthur Andersen & Co. to provide us an independent ana sis of the characteristics, measures, and indicators that have been, could be, and are rec mmended for consideration regarding the placement and removal of plants from the NRC atch List. In conducting this analysis, Arthur Andersen has been requested to cond t severalinterviews with personnel from licensees who have in the past been ident' sed as having plants that were either on the watch list or were identified as good perfor/ners. These interviews are to gather information on how the licensees measure dect' ing and improving performance and to understand what characteristics, measures, and i icators they use to determine these trends. An excerpt from the Statement of Work f this portion of the project is includeo for your information.
The interview is scheduled to be held in your ackson offices on October 17,1996, beginning at 7:30 a.m. Two members fromAin Arthur Andersen team of four members will be conducting the interview. Copies of the' resumes of the team members are enclosed for your information. They will be accompanied by Mr. Barrett. Mr. Barrett willintroduce the Arthur Andersen personnel and will pa scipate in the initial portion of the interview. He will then excuse himself for the rema' der of the interview.
Please contact Mr. Barrett at (301 415-7488 if you have any questions or comments.
Sincerely, i
Edward L. Jordan, Director Office for Analysis and Evaluation of Operational Data
Enclosures:
- 1. Statement of Wor for Arthur Andersen
- 2. Resumes of Art r Andersen team members Distribution w/ enclosure 1:
Distribution w o encl.:
File Center KSalehi, PM-ANO WBeckner, NRR RAB R/F JDonohew, PM-Grand Gulf ECollins, R-IV SPD R/F DWigginton, PM-River Bend PHarrell, R-IV AEOD R CPatel, PM-Waterford JDyer, R-IV To rocpe a copy of this document,ind6cate in the box:
DOC ENT NAME: C:\\WP51\\WPDOCS INTERVW.TRW
")N opy without attachment / enclosure "E" = Copy with attachment / enclosure "N" = fo copy
/
9f FICE R%Qud l C AC:RAjBj D:SPD 4 DD:AEOD D:AEOD MAME LKAPadovan:mmk RBarptk CRospy A Dross EJordan
/ DATE 10/v/96 10/af/96 10/ ff96~
10/ /96 10/ /96 OFFICIAL RECORD COPY
O STATEMENT OF WORK PERFORMANCE INDICATORS FOR WATCli LIST PLANTS HACKGROUND Senior Manacement Meetine Process The decision to add or remove a plant from the NRC Watch List is made semiannually at the Senior Management Meeting (SMM). The Watch List plants are those whose performance warrant NRC monitoring beyond that normally required *. These meetings have occurred since 1986 and the Gnal outcomes are listed in the Summary of Senior Management Meeting Results'. Throughout the process, the information about the performance of the plants is provided primarily by the regions and the Office for Nuclear Reactor Regulation (NRR). The regions are responsible for the inspection activity of the plants, and have Grst hand information based on their observations and assessments, and NRR provides oversight, licensing and event review for the reactor program. The OfGce for Analysis and Evaluation 2
of Operational Data (AEOD) provides insights based on performance indicators, independent analysis of experience and Accident Sequence Precursors (ASP) that are based on information reported to the NRC.
The SMM process begins with the independent screening of licensee performance by different organizations of the NRC staff. Each organization does some evaluation of the plants' performance. The organizations come together in a prebrienng to form a consensus on the plants for discussion based on input from their respective organizations. Following the prebrieGng, the organizations prepare a Senior Management Meeting Executive Summary with written discussions about the performance of selected plants; these contain plant speciGc discussions and performance indicators. At SMM, the agency develops actions to address performance issues, including additions and deletions of plants from the Watch List.
Following each SMM, the licensees are informed of any NRC decisions or actions that have been taken with respect to their plants or facilities. The Commission is advised of the Watch List status, and reasons for addition or removal from the Watch List at the Pe; iodic BrieGng on Operating Reactors and Fuel Facilities. This meeting is transcribed. Following placement on the Watch List, inspection and other regulatory activity is generally refocused on the problems and the licensees generally document their improved performance. Removal of plants from the Watch List is pa t of the SMM process.
There are three categories of Watch List plants: Category 3 are shutdown plants requiring NRC authorization to startup and that the NRC will monitor closely; Category 2 plants are those authorized to operate that the NRC will monitor closely; and Category I plants are those removed from the Watch List.
4
l
\\
The SMM process is described in detail in Attachments 1 and 2 of SECY-96-093, Guidance 3
for Senior Management Meeting and Plant Evaluation Processes.
Senior Mananement Meetina Performance Indicator Study The Commission has requested that the staff evaluate the development of improved indicators that can provide a more objective basis for judging whether a plant should be placed on or deleted from the Watch List. The Commission stated that the staff should look at the dominant and recurring characteristics of those plants that have been placed on the Watch List in the past, including 1) a high level of operational events,2) inadequate engineering and technical support, and 3) management ineffectiveness. These characteristics are to be assessed through objective measures that are directly related to plant performance.
A study shall be completed with the assistance of contractors in response to the Commission request. The study needs to result in better identification of what makes a problem plant; their characterization needs to be more objective, consistent, measurable, and timely. In addition indicators must be developed from performance characteristics and measures in a logical sequence.
For the purposes of this study, the following definitions will apply:
Characteristics are aspects of a plant's behavior that are important to safety performance.
Measures are aspects of plant operation that are directly observable through data collection or inspection.
Indicators are quantitative combinations or arrangements of measures that suggest or predict a characteristic which affects performance.
The study will involve four components: 1) examination of characteristics and attributes of past problem plants and those associated with good performers which were considered important in past senior management meeting deliberations,2) identification of objective and timely indicators which relate to those characteristics,3) correlation of indicators to historic performance trends, and 4) definition of the relationship between the resulting indicators and risk.
The scope of this contract will be to complete components 1 and 2, and to prepare an overall report integrating the results of all four components.
The examination of dominant characteristics (component 1) and the identification of candidate indicators (component 2) shall involve a thorough review of existing records, including the SMM briefing books, transcripts of Commission briefings and past detailed plant reviews including major team inspections and the study of Diagnostic Evaluation Inspection Reports 2
(DET study). Interviews with senior NRC staff, management, and selected licensees shall also be conducted. The effort shall be performed by an established management consulting Grm.
AEOD has a study in progress to look at common characteristics and attributes of plants for which there was a DET. This will become an input to the contractor effort.
The correlation of indicators to past performance trends (component 3) will proceed in parallel using a technical contractor. The output of that review will be a list of data and information to be gathered in order to support the necessary analysis.
The de6nition of the relationship between the resulting indicators and risk (component 4) will be performed by NRC staff. AEOD has a long-term effort in progress to develop risk-based indicators. This work will be used to the extent practical in the current study. It is essential to maintain logical models in which other candidate indicators can be evaluated.
A simple model is shown in Figure 1. The NRC maintains the risk at a particular plant is dominated by the potential for accidents resulting in severe core damage. Probabilistic analyses have shown that such accidents result from a sequence of failures starting with an initiating event which perturbs the plant from its normal operation. Human errors are known to be major contributors to such accidents. Other contributors include design de6ciencies and safety system failures, some of which can occur from common causes. Figure I shows that the characteristics, measures and indicators of plant performance can be directly related to all of the factors that contribute to risk from core damage accidents. For example, the likelihood of design problems, human errors and equipment failures is strongly affected by the overall performance of the operating organization. The goal of component 4 of this study will be to relate the characteristics, measures and indicators to the risk-significant factors shown in Figure 1.
OBJECTIVE Identify the characteristics, measures, and indicators that have been, could be, and should be used regarding the placement and removal of plants from the Watch List. Identify the characteristics, measures, and indicators that relate to nuclear safety in a systematic manner and result in the improvement to the objectivity, consistency, quanti 6 cation, and timeliness of Watch List plant identi6 cation. In addition to examining the bases for past NRC decisions, identify new perspectives that can be applied.
QUALIFICATIONS REQUIRED The contractors shall have key personnel whose training, experience, and overall qualifications permit the conduct of an integrated management analysis study. The contractors shall also have personnel with methodological skills to design studies, interface appropriately with and interview executives, develop and deliver data collection instruments, tabulate and statistically present qualitative and quantitative Gndings, and analyze and interpret such findings into acceptable written report formats. The contractors shall have the ability to rapidly assimilate 3
~.
l l
Plant Risk l
Containment Failure Core Damage Health Effects, Probability Frequency etc.
Initiating Safety Common cause Human Design Event System Failure Error Frequency Reliability Probability Probability i
7 7
7 7
i Characteristics - Measures - Indicators Figure 1: Relationship of Characteristics, Measures, and Indicators to Plant Risk information associated with the nuclear industry regulatory environment and express findings in the terminology of that environment.
The individual who will direct the task shall have demonstrated the highest degree of professional competence. The individual shall have a minimum of 20 years of professional experience to include 10 years of upper management experience in a large organization. A working knowledge of operating and management processes of an electric utility is desired.
The individual shall have had experience with the identification and resolution of significant performance problems in industrial organizations. The individual will also have experience with strategic planning. The individual shall have had the experience with developing performance characteristics, measures, and indicators, and assessing results. A bachelors degree is mandatory and an advanced degree (s) is preferred (e.g., M.B.A., M.S., Ph.D.s in a related technical or business field).
The investigator (s) shall have at least 10 years of professional experience, preferably including experience with the electric utility industry. At least one individual or consultant shall possess a working knowledge of operating and management process of a nuclear electric utility. The 4
l
\\
P individual shall have had experience with developing performance characteristics, measurement, and indicators in an operating environment and assessing results. A bachelors degree is mandatory and an advanced degree (s) is preferred (e.g., M.B.A., M.S., Ph.D.s in a related technical or business Held).
WORK REQUIREMENTS The contractor shall provide the necessary qualified personnel, facilities, materials and services to complete the task. While in Washington, NRC will provide office space and access to NRC documents and information systems. Contractor personnel shall be available to travel and respond to NRC staff questions and comments on all phases of this project throughout the period of performance.
STATEMENT OF WORK The contractor shall submit a report that identifies the characteristics, measures, and indicators that have been, could be, and are recommended for consideration regarding the placement and removal of plants from the Watch List. The contractor shall identify the characteristics, measures, and indicators that relate to nuclear safety in a systematic manner and result in the improvement to the objectivity, consistency, quantiGcation, and timeliness of Watch List plant identincation. The report shall identify characteristics, measures, and indicators that focus on the dominant and recurring characteristics of past Watch List plants. The report should also address leading indicators, measures, and characteristics such as economic stress measures that NRC should observe to increase watchfulness for evidence of safety performance change.
As described in more detail below, the contractor shall integrate the review of NRC information, the interviews, data, analysis, findings, results and recommendations into a single report. The report shall be based on the collective analysis and evaluation of material provided by the NRC, the contractor's participation in interviews of NRC and licensee management, and the contractor's past experience.
Prior to the final report, a draft report shall be issued for NRC comment. The contractor shall l
address and resolve the comments to the NRC's satisfaction. After issuance of the final report the contractor shall be requested to present the results, possibly in a public forum.
The following work shall be performed and documented in the report:
1.
The contractor shall review all the background information provided by the NRC and any material identified as relevant by the contractor.
2.
Identify what performance characteristics, measures, and indicators have been used to put a plant on, and remove a plant from, past Watch Lists and analyze the results.
This shall be done for the plants identified (Reference 1) from January,1991, to through January 1996 (exclude Browns Ferry Unit 1). Develop a matrix to show links 5
=
between the corresponding characteristics, measures, and indicators. This matrix shall identify gaps in the information. The information shall be analyzed and documented in the report.
2.1 Using the Senior Management Meeting Executive Summaries (10 summaries about 200 pages each), the EDO list of dominant and recurring characteristics, Senior Management Meeting Summaries (10 summaries about 30 pages each), the transcripts of the Periodic Briefing oTOperating Reactors and Fuel Facilities (10 summaries about 100 pages each), summary information developed for the Senior Management Meeting premeeting (10 packages about 100 pages each), and relevant licensee correspondence (documents to be provided), identify the characteristics, measures, and indicators that have been used to put a plant on, and remove a plant from, the Watch List. Develop a matrix of the corresponding characteristics, measures, and indicators. Analyze the information and document the analysis in the report. The analysis shall discuss the objectivity, consistency, and timeliness of the information.
The example matrix in Figure 2 is provided for illustration. It shows three potentially important characteristics of plant operations; root cause assessment, personnel qualifications and preventive maintenance. For each of these characteristics, at least one measure is listed. Two are given for preventive maintenance. Numerical indicators are given for two of the four measures.
,,'n Characteristics c
rs Root Cause Assessments / Corrective Action Recurring Problems Personnel dualifications Training Program Effectiveness
- Requalification Failure Rate Preventive Maintenance Equipment Rellability
- Safety System Failure Rate
- Forced Outage Rate Material Condition Figure 2: Example Matrix of Characteristics, Measures, and Indicators 2.2 Using the Senior Management Meetiag execuuve Summaries, identify the characteristics, measures, and indicators that have been used to identify the plants selected for discussion, and highlighted as good performers. Compile the data in a matrix similar to Figure 2. The information shall be analyzed and documented in the 6
report. The analysis shall discuss how the good performers characteristics, measures, and indicators differ from those of the Watch List plants.
i 2.3 Administer interviews of selected licensees and NRC senior managers. Licensee interviews shall be scheduled and accompanied by an NRC representative. In order to meet the tight schedule, parallel interviews and analysis may be necessary. The contractor shall provide and discuss interview questions with the NRC.
2.3.1 Conduct interviews of NRC senior managers to understand the judgements made and information used to evaluate licensee performance; identify what characteristics, measures, and indicators the managers judge to be the most vital in evaluating declining and improving performance.
2.3.2 Conduct interviews at the offices of two licensees of past Watch List plants to understand what characteristics, measures, and indicators they used to measure declining and improved performance. Also, interview one good performer to identify what they consider to be important characteristics, measures, and indicators to maintain good performance. This will require review of background information to be supplied by the NRC.
2.4 Based on the reviews and interviews, provide a summary, discussion, and evaluation of the characteristics, measures, and indicators. Identify those most common, and the most relevant, characteristics, measures, and indicators used as well as gaps in the 4
information. Evaluate the extent to which the Watch List plants distinguish themselves from good plants.
t Where there is no link between characteristics, measures, and indicators, identity where additional measures could be applied.
Complete the matrix of the corresponding characteristics, measures, and indicators.
3.
Identify characteristics, measures, and indicators that could/should be obtained, and i
added to the matrix or used to fill in the gaps ofinformation. Identify new perspectives on performance assessment, beyond those which the NRC has used in past decisions. The contractor shall use their (1) observations, analysis, and evaluation of the NRC information; (2) knowledge base from experience with electric utilities and other industries; and (3) experience with the development and analysis of performance factors. The contractor shall add this information to the matrix. Document the findings and observations in the report.- Convey description of the characteristics, measures and indicators to the NRC contractor responsible for the third component,3) correlation of indicators to historic performance trends, and to the NRC for evaluation of the fourth component,4) definition of the relationship between the resulting indicators and risk.
7
REFERENCES 1.
Summary of Senior Management Meeting Results 4
2.
U.S. Nuclear Regulatory Commission, Office for Analysis and Evaluation of Operational Data, " Performance Indicators for Operating Commercial Nuclear Power 3
2 Reactors, Data Through September 1995, Parts I and II."
j j
3.
U.S. Nuclear Regulatory Commission, Guidance for Senior Management Meeting and Plant Evaluation Processes," Commission Paper SECY-964)93, May 1,1996, i
4 i
.\\
4 j
i 1
8 j
- - _,,.