ML20236W624
| ML20236W624 | |
| Person / Time | |
|---|---|
| Issue date: | 01/27/1998 |
| From: | NRC OFFICE FOR ANALYSIS & EVALUATION OF OPERATIONAL DATA (AEOD) |
| To: | |
| Shared Package | |
| ML20236W605 | List: |
| References | |
| SECY-98-045-C, SECY-98-45-C, NUDOCS 9808060114 | |
| Download: ML20236W624 (67) | |
Text
_ _ _ - - _ - - -- - - - - - - - - - - - - - - -
EXCERTS FROM:
INTERIM REPORT ON THE DEVELOPMENT OF THE PLANT PERFORMANCE TEMPLATE 1 r; Office For Analysis And Evaluation Of Operational Data U.S. Nuclear Regulatory Commission i
l l
January 27,1998
- l L .- -
- geogg49so729 99-045 C ,
1 PDR
I
- j. Interim Report on the Development ofthe Plant Performance Template i
l t-f- ' TABLE OF CONTENTS I
l l
t
- 1.0 EXECUTIVE
SUMMARY
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . I 1.1 Obj ective . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . I 1.2 Template Purpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . I 1.3 Development Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . I 1.4 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.5 Oversight . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 i-t i
t 2.0 B AC KGROUND . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 l 2.1 Purpo se . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 2.2 - Project Purpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 2.3 Project Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 3.0 DEVELOPMENT PROCES S . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 l 3.1 Introd uction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 L 3.2 Template Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 t
l; 4.0 RE S ULTS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 7 l 4.1 Plant Performance Template - Revision 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .~ 17 i 4.2 Summary of Results: Plant Discussion Database and Region II PIM Analysis !
.............................................................. 17 i j 4.3 Plant Performance Template - Revision 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 4.4 Representative List of Example Performance Issues, by Subcategory . . . . . . . 18 I i j 5.0 OVERSIGHT AND PEER REVIEW . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 i
- . 5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 )
t l 5.2 ' NRC Oversight . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 L 5.3 ACRS Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 i 5.4 Public and Industry Comment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 i l 6.0 ~ NEXT STEPS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 6.1 Discussion ofNext Steps .........................................22 6.2 ' . Assessment of the Trial implementation of the Template ................ 22
[ 6.3 Integration with the NRR Decision Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 l
6.4 Analysis of the Risk implications of the PIM Data . . . . . . . . . . . . . . . . . . . . . . 22 !
6.5 Agreement on the " Management Effectiveness Category" within the Template 22 r-January 27,1998 Page i
l Interim Report on the Development ofthe Plant Performance Template 6.6 Development of Quantitative Measures for use with the Template . . . . . . . . . . 23 6.7 Reassessment of hispection Program . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 I .
APPEN DI X . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 4 '
l B bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 A.1 Plant Performance Template - Revision 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 A.2 Summary of Results: Plant Discussion Database and Region II PIM Analysis
..............................................................45 A.3 Plant Performance Template - Revision 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 i A.4 Representative List of Example Performance Issues, by Subcategory . . . . . . . 50 1
l
! J l
l i l
l l
L l i i i I i
l L
l I
l i
January 27,1998 Pageii
i Interim Report on the Development ofthe Plant Performance Template l
1.0 EXECUTIVE
SUMMARY
1.1 Objective l
The objective of this interim report is to document the efforts to date to develop a Plant Perfonnance Template (Template). This report provides the reader with an understanding of the status of work to-date on the Template and allows the reader to see the detailed outputs of the l work. It also identifies the remaining work to complete the Template. This report is not
! intended to draw overall conclusions or speculate on the potential effectiveness of the Template.
l SECY-97-072, " Staff Action Plan to Improve the Senior Management Meeting process,"
outlined the NRC efforts to improve the timeliness, objectivity, and scrutability of Senior Management Meeting (SMM) process. Template development is one of four initiatives that are l
parallel efforts towards that end. The other three are the Performance Trending Methodology, Economic Indicators, and the Integrated Review of NRC Assessment Processes (IRAP) for operating commercial nuclear plants. The Office for Analysis and Evaluation of Operational l Data (AEOD) has the lead for these efforts except the IRAP, where the Office of Nuclear Reactor l
Regulation (NRR) has lead responsibility.
1.2 Template Purpose The purpose of the Template is to provide a structure to help display objective plant performance l information used in the discussion and assessment ofnuclear power plant performance. One of l the guiding values in the design of the Template was that the Template should provide a more prospective or leading view of plant performance.
1.3 Development Process i l
L It is important to understand that although AEOD led the effort on the development of the l Template, representatives from across NRC headquarters and regional offices provided significant input. These include staff from NRR, the Office of Nuclear Regulatory Research, and all four regions who together made up the Template working group. The group has a cross section of technical expertise including probabilistic risk assessment, human factors, training, 4 engineering, management, and power plant design and operation as well as experience in diagnostic evaluation, Management Oversight and Risk Tree, incident investigation, and the residem inspection programs.
The current Template design includes the categories Operating Performance, Material Condition, j Human Performance, Engineering and Design, Problem Identification and Resolution, and 1
January 27,1998 Page1 l^
i l
I i I t __
1
interim Report on the Development ofthe Plant Performance Template Management Effectiveness. Each of these categories is broken down into two or three subcatt:gories with the details of the design evolution described in Section 3.2 of this report.
l-l 1.4 Results l
The purpose of the Results Section is to document and present the key outputs of the Template development process described in Section 3.0. The outputs reflect various aspects of the complete Template. There is a range of outputs including quantitative analyses, qualitative descriptions, weighting factors, and definitions. Below is a list of the outputs which can be found in the Appendix of this report.
I. Plant Performance Template - Revision 1
- 2. Summary of Results: Plant Discussion Database and Region II PIM Analysis
- 3. Plant Performance Template - Revision 2 l 4. Representative List of Example Performance Issues, by Subcategory ,
! l 1.5 Oversight j 1
l ' Oversight and peer review of Template development has occurred at critical junctures in the l project. The stiff has sought feedback throughout the project and has incorporated the l constructive ideas as input to upgrade the Template. Section 5.0 of this report discusses the
! oversight and review by the Commission, Advisory Committee on Reactor Safeguards, a cross-functional project oversight committee, and public/ industry involvement, including the details of the various briefings.
2.0 BACKGROUND
2.1 Purpose l
This report is intended to document the efforts by the U.S. Nuclear Regulatory Commission (NRC), supported by Arthur Andersen, to develop a Plant Performance Template (Template).
i The Template is a tool used to facilitate NRC senior management in discussion and assessment i of nuclear power plants performance. This report documents the process used in developing the .
l Template, the results, and the lessons leamed from the process. This report also indicates l progress to date and critical next steps but does not draw conclusions or make recommendations except for suggested "next steps."
2.2 Project Purpose January 27.1998 Page 2
Interim Report on the Development ofthe Plant Performance Template l There were four key milestones that influenced the initiative to develop the Template. They were:
l
+ The Staff Requirements Memorandum (SRM) issued June 28,1996.
+ The report issued by Anhur Andersen, December 30,1996, entitled Recommendations to Improve the Senior Management Meeting Process.
I + The SRM issued March 14,1997 that included "The staff should consider the use of a
' plant perfonnance' template to help display the objective plant performance information."
l
+ SECY-97-072 - Staff Action Plan to Improve the Senior Management Meeting Process prepared for the Commission in response to the recommendations in the Arthur Andersen report.
In the June 28,1996 SRM, the Commission asked the staff to evaluate the development of improved indicators that could provide a more objective basis for assessing the performance of l nuclear power plants. Biannually, the NRC Senior Management Meeting (SMM) process l assesses the performance of every nuclear power plant in the United States. One outcome of the l
SMM is the Watch List, a list of plants which warrant increased regulatory attention. In the June 1996 SRM, the Commission asked the staff to assess the dominant and recurring characteristics of those plants placed on the Watch List in the past. Such characteristics include: (a) the number of operational events; (b) inadequate engineering and technical support; and c) management l effectiveness. The overall goal of the Commission was to develop measures or indicators of performance that result in improved objectivity, consistency, quantification, and timeliness of the Watch List plant identification process. The Commission further stated that these measures should be directly related to plant performance and risk.
Arthur Andersen was contracted in the fall of 1996, by NRC's Office for Analysis and Evaluation of Operational Data (AEOD), to assist in a review of the SMM process and to evaluate and develop improved performance measures. The Arthur Andersen report recommended that NRC implement several changes, including the ones summarized below:
+ Shift from subjective to more objective perfonnance measures.
+ Present assessment information in a more structured way.
+ Systematically assess management, operational, and organizational performance.
+ Shift from event reaction to factors which predict performance.
January 27,1998 Page 3
l
\. Interim Report on the Development ofthe Plant Performance Template
+ Improve consistency by establishing and enforcing decision criteria.
In SECY-97-072, the NRC staff stated that it would proceed with development and implementation of a Plant Performance Template. The Template will provide a systematic l methodology for review of organizational and operational factors during the SMM decision l making process. AEOD is the lead office responsible for developing the Template.
l 2.3 Project Structure l SECY-97-072 outlined an NRC-wide effort to improve the timeliness, objectivity, and l scrutability ofinformation provided to the SMM participants for use in the Watch List decision making process. The effort also focused attention on the processes used to collect and assimilate information on the performance of nuclear power plants. Four parallel implementation efforts were initiated based on the plans outlined in SECY-97-072. Three of the efforts are being led by j AEOD and focus on the development of specific performance assessment tools. AEOD's l projects are referred to as the a) Plant Performance Templafe, b) Performance Trending Methodology, and c) Economic Indicators. The fourth implementation effort encompassed the NRC inspection process and other methods of regulatory oversight. This effort is being led by the NRC's Office of Nuclear Reactor Regulation (NRR) and is known as the Integrated Review ofNRC Assessment Processes (IRAP).
l l Day to day management and organization of Template development is being led by a team of
[ AEOD staff using a process designed to involve representatives from across NRC headquarters and regional offices. These representatives make up the Template working group and include staff from NRR, the Office of Nuclear Regulatory Research (RES), and all four regional offices.
The working group consists of technical experts in probabilistic risk assessment (PRA), human i
factors, training, engineering, management, and power plant design and operation, as well as functional experts with experience in Accident Sequence Precursor (ASP), Diagnostic Evaluation, Management Oversight and Risk Tree (MORT), incident investigation, and the resident inspection programs. Oversight of Template development is provided by a Committee
- of five NRC senior managers
- AEOD Director, Deputy NRR Director, Acting RES Director, l Regional Administrator Region IV, and the Acting Deputy Executive Director for Regulatory l Effectiveness. (Please see Section 5.0 for a more detailed account of project oversight.) The l oversight group and several of the AEOD team members participate in aspects of the other three l implementation projects. This structure facilitates integration among the four projects.
l.
January 27,1998 Page 4
interim Report on the Development ofthe Plant Performance Template 3.0 DEVELOPMENT PROCESS 3.1 Introduction The development process used by AEOD to develop the Template was intended to meet the objectives outlined in SECY-097-072. The intent was a Template that was specific enough to fit <
within the current SMM decision making process, yet sufficiently versatile to be used in any future NRC-developed assessment process. AEOD, while focused on the development of the Template, remained cognizant of the need for the Template to deliver useable inputs to whatever assessment mechanism was developed by the NRR-sponsored IRAP group.
3.2 Template Design Purpose - Thc purpose of the first phase of the development process was to create a Template that would aid in the syr.ematic review of the relevant organizational and operational performance factors necessary to assess licensee performance. More specifically, AEOD sought to incorporate into the design the universe of knowledge on nuclear power plant performance, including information available from the following resources: '
+ Former Watchlist decisions used by NRC senior management. I
+ Current NRC programs on risk-informed, performance-based regulation.
+ Perspectives of representatives from across NRC headquarters and all four regional l offices.
+ Elements of past and present NRC assessment or inspection programs. ,
+ In-house and external research sources. !
+ Industry and public feedback. i l
Additionally, the Template Design phase set out to answer two questions:
+ What are the appropriate breadth and depth of operational and organizational issues to sssess?
+ What is the most effective structure for the Template given its intended audience and function?
January 27,1998 Page 5
Interim Report on the Development ofthe Plant Performance Template Work Performed - The key steps used to develop and refine the Template during the Template Design phase are described below.
Step One - Review of Management Directive (MD) 8.14'. The first work performed by AEOD l under this phase was a review of MDB.14, the guidance document for the SMM process, and the Plant Performance Review Template provided as Exhibit 1 of MD8.14. The review was l coordinated with and supported by representatives from NRR and RES and assessed the overall l _ effectiveness of the MD8.14 template in light of staff conunitments made to the Commission in l SECY-97 'J72 (Please refer to Section 2.2 of this report). It was determined that the MD8.14 template was a good starting point because its major categories were appropriately linked to risk and the queetions accompanying it raised important concems. Revisions would be focused to ensure the necessary breadth and depth of performance issues. Several meetings were held with NRR and a cooperative decision was made to keep the primary categories of the MD8.14 template, ar d proceed with the work necessary to meet the commitments in SECY-97-072.
Table 3-1 sh )ws the MD8.14 template categories:
Table 31 Management Directive 8.14 '
Template Categodes Operating Performance (Frequency of Transients)
Material Condition (Safety Reliability / Availability)
Human Perfonnance Engineering and Design Effectiveness of Licensee Self-Assessment 1
Step Two -- Database of Performance Characteristics. Following the review of MD8.14, AEOD designed and populated a database to capture the broad universe of performance characteristics to be considered in the Template. This work began with the Plant Discussion Database (PDD) j created by Arthur Andersen for its December 1996 report. The source for the PDD was several i years of NRC-written assessments of nuclear power plants that warranted discussion at a past j 8 Management Directive 8.14," Senior Management Meeting (SMM)," March 19,1997. l l
Il January 27,1998 Pege 6 j f
l
Interim Report on the Development ofthe Plant Performance Template SMM. The PDD included descriptions of plant performance. From this, AEOD created a second generation database (PDDII). The PDDII was populated by performing a keyword search of the first database and extracting each word or phrase that was. considered a performance characteristic. This resulted in a database of approximately three thousand characteristics, all of which have been used by NRC senior management to describe or assess the performance of a licensee. From the three thousand characteristics in the PDDII, multiple levels of grouping and sorting were performed. To begin , any like terms or similar means of expressing identical performance charactecsdes were grouped and their frequencies counted. Characteristics were then aggregated into groups by MD8.14 template categories and various subcategories were created based on similarity of assessment area, i.e., all characteristics discussing equipment reliability. Table 3-2 shows the summary of the results:
Table 3-2 Plant Discussion Database (PDDII) Characteristics Fesquesey7 by MDS.14 Categories - -%-
Operating Performance (Frequency of Transients) 23.1 Material Condition (Safety Reliability / Availability) 20.3 Human Performance 14.4 Engineering and Design 08.3 Effectiveness of Licensee Self Assessment 33.8 TOTAL 100.0 %
i The results were examined and showed that many of the characteristics under the category self assessment were appropriately grouped but that many more were targeted to the general notion of management effectiveness and identification and resolution of problems. Based on this analysis, AEOD decided to rerun the distribution using a new sixth category called Management Effectiveness and renaming Self Assessment to Problem Identification and Resolution. Table 3-3 shows the summary of the results of this rerun:
l l
I january 27,1998 Page 7 i
lnterim Report on the Development ofthe Plant Performance Template Table 3-3 Plant Disenssion Database (PDDH) Characteristics Frequency by Teamplete Categories - '-
Operating Performance (Frequency of Transients) 23.1 Material Condition (Safety Reliability / Availability) 20.3 Human Performance 14.4 Engineering and Design 08.3 Problem Identification & Resolution i1.7 Management Effectiveness 22.1
. TOTAL 100.0 %
i NRR and RES concurred in the decision to add the sixth category, following discussion of the results of the rerun with the oversight committee.
The analysis performed during step two indicated the relative priority placed on each performance characteristic by NRC senior managers. The result of this was a revised Temp; ate and the identification of subcategories for each major category.
Step Three - Review of Additional Sources. The result of step two was a Template developed from characteristics ofpast discussion plants. To ensure the completeness of the Template, AEOD reviewed a sizable collection of additional information sources. This included a review
- of the performance questions from the MD8.14 template. Some of the other resources are listed I l below2
+ External Research Articles - . International Atomic Energy Agency (IAEA), Journal of Public Administration Research and Theory
{
l + Industry /Public Research - World Association of Nucleat Operators (WANO), Union of !
Concerned Scientists (UCS) 1 2
See the Appendix to this report for a complete bibliography !
l January 27,1998 Page 8 l
I
l Interim Report on the Development ofthe Plant Perfonnance Template
! + NRC Assessment / Inspection Programs - Management Oversight and Risk Tree (MORT), Probabilistic Risk Assessment, Human Performance Investigation Process (HPIP)
+ NRC-sponsored Research - Behavior Anchored Rating Scale (BARS), RES/ldaho National Energy Engineering Laboratory (INEEL) Management and Operations Workshop.
l Based on this review, additional performance characteristics were identified. The Template was modified to reflect the lessons learned. Steps two and three both expanded the breadth and depth of the Template. An example of an expanded category with its accompanying subcategories and characteristics is shown in Table 3-4:
l Table 3-4 Operating Performance '
Operations Performance Transients Configuration control Event /significant transients Performance trend Response to transients l
l Program implementation Documentation of transients Adequate Stafling SupportPrograms General Performance General performance l
Operations Processes Chemistry program l Emergency procedures Emergency preparedness program Operations procedures Environmental monitoring program j Emergency exercises Fire protection program Radiation protection program Security program l l
Step Four -- Test / Validate. AEOD performed a test of the Template categories, subcategories and proposed data source. The first stage of testing was to select an appropriate data source.
Selection would be based on whether the data were available within NRC, encompassed all January 27,1998 Page 9
Interim Report on the Development ofthe Plant Performance Template plants, and included a broad array of performance insight. AEOD considered the following potential sources:
+ Licensee Event Reports database
+ Allegation database
+ Enforcement database
+ Automatic Inspection Reporting System
+ Plant Issues Matrix (PIM) database Consideration was given to develop a new assessment process thereby creating a new data source. The PIM database best met the selection criteria. 'Iherefore, the test used one year of performance data from the PIM of eight Region II plants. The next stage required the coding of each PIM entry to one or more Template subcategories. A PIM item that could not be categorized, using the existing Template subcategories, was caue to propose modifications to the Template. Table 3-5 summarizes the results of this test and shows the percentage of PIM entries by Template category.
Table 3 5 Region II PIM Data Frequency by Tenspiste Categories - %
Operating Performance (Frequency of Transients) 20.4 Material Condition (Safety Reliability / Availability) 18.3 Human Performance 15.9 Engineering and Design 12.9 Problem Identification & Resolution 12.2 Management Effectiveness 20.4 TOTAL 100.0 %
Additionally, a comparative analysis was made between the frequency and type of performance characteristics found in PIM data versus the frequency and type found in the Plant Discussion Database. Table 3-6 summarizes this comparison.
January 27,1998 Page 10 l
interim Report on the Development ofthe Plant Performance Template Table 3-6
. Plant D% Database (PDDU) Characteristics "% =%'
l Operating Performance (Frequency of Transients) 20.4 23.1 Material Condition (Safety Reliability / Availability) 18.3 20.3
! Human Perfonnance 15.9 14.4 Engineenng and Design 12.9 08.3 I
Effectiveness of Licensee Self-Assessment 12.2 11.7 Management Effectiveness 20.4 22.1 TOTAL 100.0 % 100.0 %
This test allowed NRC to clearly see the capability of the draft Template to effectively structure i
the performance assessment ofnuclear power plants and confirmed the decision to add a sixth category to capture issues of management effectiveness. The test also confirmed that the PIM database includes an adequate level of performance information for most subcategories and all of the major categories, except management effectiveness. For the most part, PIM data were easily categorized into the Template format. However, while management effectiveness data from the PDDII analysis were distributed across characteristics such as culture, oversight and discipline, and improvement initiatives, PIM data contain few references to these characteristics.
Although the PIM and PDDII data appear to have comparable frequencies for the management i effectiveness category, breakdown of the detail shows a contrast. Approximately 60% of PIM !
information in this category was derived from enforcement violations. Therefore, the assessment of management effectiveness would have to be made from a limited pool of data or inferred from the results in other categories..
Step Five - Link to Risk. To ensure a risk informed, performance-based regulatory approach, l AEOD addressed the risk perspective early in the Template development process. The NRC-accepted probabilistic risk assessment (PRA) model identifies three primary elements which are .
direct contributors to risk in a nuclear power plant. These elements are operational transients, equipment failure, and human error. Figure 3-1 demonstrates the current thinking relative to risk ;
modeling the Template.
January 27,1998 Page11 l l
Interim Report on the Development ofthe Plant Performance Template Figure 3-1 RISK -
A 1 I i
{
Maserial condition operouseal terseemanas Homes Performance i Equipment Reliability Treasiset Fregeeney Hamas Error
-> Problem Identification
& Resolution Engineering /Desigs l I
l l l l w g... en..ti.. 4 Three of the Template categories match directly to the primary elements of the risk model; )
operational transients and their root cause are captured under operating performance, equipment l
failures under material condition, and human errors under human performance. A PRA model is l
based on a specific plant design; issues associated with engineering / design may impact risk. j Problem identification and resolution is the mechanism for correcting performance concerns in l all of the a eas while management effectiveness establishes the direction and performance standard (s) that influence overall organizational performance. The output of this step was the l
Figure 3-1 above, which illustrates the current thinking for the risk model for the Template.
Step Six--Develop Guidance and Draft Operational Definitions. Accompanying guidance was l developed to support the distribution of Revision I to an NRC audience beyond AEOD. The guidance included two documents. The first, a cover memorandum introduced the structure and purpose of the Template. Second, operational definitions for each category and subcategory ,
provided the reader with a clear understanding of the performance characteristics captured by l each piece of the Template. In an effort to make the Template more " user-friendly," this guidance was intended to support the objective of the Commission, to create an assessment l
January 27,1998 Page 12
Interim Report on the Development ofthe Plant Performance Template l
mechanism that is transparent to the public and industry. Distribution of Revision 1 included the project oversight committee as well as regional offices. (Please see the Appendix of this report for a copy of the Revision 1 Template.)
Step Seven - Review. Distribution of Revision 1 of the Template was followed by a formal-review by the project oversight committee and several regional representatives. (Section 5.0 of the repon documents the oversight and review process in greater detail.) It is important to note that at this time, primary feedback from the oversight committee and regions centered around a l need for increased input to the process from NRR, RES, and the four regions. The oversight
- group also recommended changing the category name Management Effectiveness to Organizational Effectiveness. AEOD agreed with the category name change and revised the pmcess to allow representatives from each of these offices to play a formal role in the design ar d content of the Template. This occurred during several workshops discussed below.
Step Eight - Category Development Workshop. The objective of the Category Development Workshop was to create a forum for representatives from a cross-section of the NRC to provide
- direct input into the Template design. Specifically, the Category Development Workshop l allowed the participants to have impact on Template guidance, operational definitions, and subcategories. The participants in this initial workshop became the members of the Template j working group (a quasi-formal group of participants for subsequent workshops) and included l staff from NRR, RES, and all four regional offices. Members of the working group brought expertise in PRA, human factors, training, nuclear engineering, design, performance indicators,
- and management factors. External facilitation services were provided for the workshop. To l facilitate the group's contribution to the operational definitions and subcategories, a multi step approach was followed.
- 1. Created an operational definition for the overall category, i.e., Material Condition or Human Performance:
- a. Reviewed suggested definitions as included with the Revision 1 Template
- b. Broke-out into several small teams, drafted new operational definition l c. Presented break out team results to other teams I d. Developed consensus definition l 2. Identified lists of critical factors for assessing the assigned Template category, using Nominal Group Technique:
l January 27.1998 Page 13
j interim Report on the Development ofthe Plant Performance Template
- a. Generated silent list of critical assessment factors (by each member of the group)
- b. Presentedideas
- c. Clarified or explained confusing ideas or acronyms
- d. Removed duplicateideas
- 3. Created subcategories which encompassed the critical assessment factors from step 2:
- a. Designed strategy to create aflinity clusters (by break out teams)
- b. Presented break out team results to other teams
- c. Developed consensus affinity clusters
- d. Created a subcategory name for each affinity cluster
- 4. Created operational definitions for each subcategory, using the process described in step 1.
This facilitated process was used to create subcategories and operational definitions for all of the Template categories except Category 6, organizational effectiveness. Direction from the i
oversight committee and time limitations led to a decision to work on Category 6 at a later date.
The impact of this workshop was to simplify the Revision 1 Template making it succinct, l effective, and unencumbered by excess detail. Copies of the outputs developed during the Category Development Workshop are included in the Appendix of this report. Table 3-7 shows the categories and subcategories, reflecting revisions made during the workshop. j Table 3-7 ,
1 2 3 4 5 !
! Operating Material Condition Human Performance Engineering / Design Problem Identification Performance & Resolution Normal Operations Equipment Condition Work Performance Design identification Operations Dunng Programs and Processes Knowledge. Skills & Engineering Support Analysis I ;
Transients Abilities i Programs and Processes Work Environment Programs and Processes Resolution Step Nine - Organizational Effectiveness Workshop. Category 6 of the Template - i Organizational Effectiveness, was designed at a separate workshop from the first five categories.
January 27,1998 Page 14
l l
l Interim Report on the Development ofthe Plant Performance Template l
AEOD conducted the Organizational Effectiveness Workshop as a two-day meeting attended by l several representatives of the Template working group. The first proposal made at the workshop i was to change the category name from Organizational Effectiveness to Management Effectiveness, to better reflect the intended focas of the section. Group consensus was that organiutional effectiveness was a term that smamarized the entire Template and that Category Six was best referred to as Management Effectiveness. The clarification was also made that this category should focus on the effectiveness of management processes as opposed to individual managers. The workshop was held a few weeks after the previous workshop and adopted a condensed approach, due to the smaller group size and focused objectives. The following steps were used in the approach:
- 1. Created operational definition for the overall category:
l
- a. Reviewed definitions, proposed by AEOD
- b. Developed consensus definition
- 2. Created operational definitions for each subcategory, using the process described in step 1
- 3. Identified lists of ::ritical factors for assessing the assigned Template category:
- a. Presented ideas
- b. Removed duplicate ideas The outputs nom tids workshop included new subcategories and operational definitions of the management effectiveness category. The table below shows the subcategories developed during this workshop:
l l
January 27,1998 Page 15
Interim Report on the Development ofthe Plant Performance Template Table 3-8 l
6 l Management Direction and Oversight l l
Programs and Processes Organizational l
Copies of the outputs developed during the Organizational Effectiveness Workshop are included in the Appendix of this report. l Lessons Learned- These various steps resulted in the Revision 2 Template, which was distributed for comment in September 19973 (See the Appendix of this report for a copy of Revision 2 Template). This ended the Template Design phase of the project. .
I The lessons learned during the Template Design phase include:
- 1. There is a need for a balanced level of detail in the Template. Too many subcategories or too fine a line between eacl: created difficulty in properly assigning data to the correct one. Too few subcategories left an unclear or incomplete picture of plant performance. It should be easy for NRC, public, and industry to understand the intent, structure, and function of the Template. 1 1
- 2. The structure of the Template should not drive inspection behavior. The structure was j developed to capture performace information from the current inspection program. l l
- 3. The Template is a significant paradigm shift in NRC's assessment of plant performance. l It changes the focus from Systematic Assessment of Licensee Performance (SALP) !
functional areas (Maintenance, Engineering, Operations, Plant Support) to a more cross- 1 functional assessment, e.g., human performance is assessed regardless of which department the person works.
3 Memorandum From Thomas T. Martin, Director AEOD To Samuel J. Collins, Director NRR, ;
Guidancefor Trial Use ofa Plant Performance Templatnor the June 1998 SMM Cycle, September 30, 1997.
l January 27,1998 Page 16 I
t Interim Report on the Development ofthe Plant Performance Template
- 4. Future PIM entries become more focused as familiarity grows with the Template structure and objectives.
Revision 2 of the Template incorporates each of these lessons learned.
4.0 RESULTS l
The purpose of this Section is to document and present the key outputs of the work as it evolved following the development process described in Section 3.0. This Section includes a short l description of several different types ofoutputs that reflect various aspects of the complete l Template. There is a range of outputs including quantitative analyses, qualitative descriptions, i weighting factors, and definitions. Below is a list of the outputs described in this Section.
(Please refer to the Appendix of this report for copies of these outputs.)
- 1. Plant Performance Template - Revision 1
- 2. Summary of Results: Plant Discussion Database and Region II PIM Analysis l 3. Plant Performance Template - Revision 2
- 4. Representative List of Example Performance Issues, by Subcategory l
4.1 Plant Performance Template - Revision 1 l
This is the first edition of the Template that was distributed both within NRC headquarters and to the regions to get feedback on issues and recommendations for upgrade. The six major categories are shown on the first page with subcategories included on pages 2-4 with the draft of l operational definitions for each of the subcategories. A list ofissues to be considered in each I subcategory is detailed on pages 5-18.
l 4.2 Summary of Results: Plant Discussion Database and Region II PIM Analysis l The following charts show the statistical summary of performance characteristics as captured by the analysis of both the Region II PIM database and the Plant Discussion Database. Each item was coded to one of the Template categories to generate the detailed summary. The second page shows the detailed count data for each of the subcategories. The purpose of the analysis was to l understand if the various items included in both databases were covered within the design of the l
January 27,1998 Page 17
f l Interim Report on the Development ofthe Plant Performance Template l Template or if there were gaps in the Template. It also provided a preliminary view of which areas within the Template might not covered by the PIM.
4.3 Plant Performance Template - Revision 2
! This version of the Template reflects the efforts and lessons leamed from the two workshops to l upgrade and simplify the Template. A key learning from the regional input to the workshops was that the initial Template was too cumbersome with numerous subcategories and all of the attached detail.
4.4 Representative List of Example Performance Issues, by Subcategory The intent of these three products is to provide supportive guidance and information to the regional personnel who are involved in the trial implementation of the Template. The products i are intended to simplify and clarify the Template, summarize lessons leamed during the workshops, and provide a list of examples oflikely performance issues capttued under each
! subcategory.
1 i
l l
I l
l l
l January 27,1998 Page 18
l l
[ Interim Report on the Development ofthe Plant Performance Template i
5.0 OVERSIGHT AND PEER REVIEW l
5.1 Introduction j Oversight and peer review of Template development has occurred at criticaljunctures in the l
project. The staff has sought feedback throughout the project and has incorporated the constructive ideas as input to upgrade the Template. In several cases, written response was made l to those providing the feedback. Formal oversight and review had taken place on several levels within the NRC organization and external review and feedback had also taken place. This l Section documents the oversight and review process. Section 5.2 discusses NRC oversight and the review by the Advisory Committee on Reactor Safegaurds (ACRS) is covered in Section 5.3.
Finally, public and industry review is covered in Section 5.4.
l 5.2 NRC Oversight l
l The Commission has played a key oversight role on this project. The staff has briefed the Commission on several occasions. On April 24,1997, the briefing included a summary of the Arthur Andersen Report and the specific plans for implementing various recommendations
- included in the report. On June 24,1997, in SRM 97-042-4B, the commission approved the l Staff plans. On September 19,1997, a progress briefing was made on the development process used for developing the Template, the progress to date, and proposed path forward. The Commission asked questions for clarification and offered appropriate feedback.
The project oversight committee provided ongoing oversight through periodic review of Template development. This committee consisted of NRC senior managers from across the J organization, specifically: I
+ Ashok Thadani, Acting Deputy Executive Director for Regulatory Effectiveness l
+ Thomas Martin, AEOD Director
+ Denwood Ross, RES 3
+ Frank MIRAPglia, Deputy NRR Director !
+ Ellis Merschoff, Regional Administrator, Region IV I l This oversight committee helped ensure that the Template reflected the breadth and depth of ;
agency performance assessment knowledge. The oversight committee also promoted an !
l !
! I I
i i
January 27,1998 Page 19 I i
r !
L_____________-_____-_______-_-_______- ._ . - _ - - - - - - - -- - - - - - - - - - - - - - - - - - _ - --
i interim Report on the Development ofthe Plant Performance Template integrated approach between the Template work and the three other NRC projects described in SECY-97-072. The staff has met with the oversight committee periodically. Decisions made by the committee led to the following key actions:
+ The decision was made to distribute the Revision 2 Template to the Regions in October 1997, to allow for coding of PIM entries.
+ The decision was made to withhold the management effectiveness category from trial implementation because, at the time, progress on the management effectiveness category was lagging the other five categories.
On a more informal basis, AEOD worked with other NRC offices to improve Template effectiveness and create broader ownership through review and feedback of the work in-progress.
Prior to completion of Revision 1 of the Template, both NRR and RES participated in the i development and upgrades to the Template. Various offices were asked to provide feedback and to identify additional sources ofinformation that might enhance the quality and effectiveness of the Template. Representatives of NRC's regional offices reviewed and offered comments on each version of the Template and the products developed during the three workshops. The informal review and feedback contributed to the evolution of the Template, particularly in creating a thorough and user-friendly product.
5.3 ACRS Review When designing the Template, it was important for AEOD to have an impartial organization conduct a peer review. ACRS often functions in this capacity and was a natural choice. ACRS assigned the Plant Operations Subcommittee to review work on improvements to the SMM process, including the Template. AEOD has briefed ACRS on two occasions. The first briefing was to the full committee on March 6,1997 and presented the plans for upgrading the SMM. A second discussion, on August 28,1997, was held with the Subcommittee on the process used to develop the Template, progress to date, and proposed path forward with a briefing to the full Committee on September 3,1997. After this briefimg, the ACRS offered comments in a report dated September 10,1997, to which the EDO provided a formal response. The next briefmg to the Subcommittee is scheduled for February 3 and to the full Committee on February 5,1998.
5.4 Public and Industry Comment l
January 27,1998 Page 20 l
l l
! Interim Report on the Devdopment ofthe Plant Performance Template l
l integrated approach between the Template work and the three other NRC projects described in SECY-97-072. The staff has met with the oversight conunittee periodically. Decisions made by the committee led to the following key actions:
l + The decision was made to distribute the Revision 2 Template to the Regions in October 1997, to allow for coding of PIM entries.
+ The decision was made to withhold the management effectiveness category from trial l implementation because, at the time, progress on the management effectiveness category was lagging the other five categories.
On a more informal basis, AEOD worked with other NRC offices to improve Template l effectiveness and create broader ownership through review and feedback of the work in-progress.
Prior to completion of Revision 1 of the Template, both NRR and RES participated in the development and upgrades to the Template. Various offices were asked to provide feedback and to identify additional sources ofinformation that might enhance the quality and effectiveness of the Template. Representatives ofNRC's regional offices reviewed and offered comments on each version of the Template and the products developed during the three workshops. The informal review and feedback contributed to the evolution of the Template, particularly in creating a thorough and user-friendly product.
I i 5.3 ACRS Review When designing the Template, it was important for AEOD to have an impartial organization conduct a peer review. ACRS often functions in this capacity and was a natural choice. ACRS assigned the Plant Operations Subcommittee to review work on improvements to the SMM process, including the Template. AEOD has briefed ACRS on two occasions. The first briefing was to the full committee on March 6,1997 and presented the plans for upgrading the SMM. A second discussion, on August 28,1997, was held with the Subcommittee on the process used to l develop the Template, progress to date, and proposed path forward with a briefing to the full i Committee on September 3,1997. After this briefing, the ACRS offered comments in a report l dated September 10,1997, to which the EDO provided a formal response. The next briefing to the Subcommittee is scheduled for February 3 and to the full Committee on February 5,1998.
t l 5.4 - Public and Industry Comment i
January 27,1998 PagI20 L_______________..___-___ _ __ - - - _ - -
i-Interim Report on the Development ofthe Plant Performance Template l
As witli any modification to the regulatory process, soliciting the opinion of the public and the regulated industry is a required step and a valuable source ofideas. The concept of the Template has been introduced to industry and public informally because of their attendance at Commission and ACRS briefings on the topic. - AEOD also participated in a public meeting, along with the IRAP group. The staff made a presentation on November 6,1997 that introduced the main concepts of the IRAP and Template development efforts to industry and public representatives.
Severa! groups, including the Nuclear Energy Institute, Union of Concemed Scientists, and the
! news media presented industry and public opinion at this session.
l l The Template will be published in the Federal Register in the Spring of 1998 for public l comment. A formal process to solicit and incorporate industry and public input, which may include public workshops, is planned.
i l
l l
I-l l
l l
January 27,1998 Page 2i L ___ _ __ __ _ ______ __ _ _ ____ _ _________-____________________-._ ______ ______ _________- _____ ____ _______ _____ _ -..____________ _ _____ _ ___
f
> Interim Report on the Development ofthe Plant Performance Template l 6.0 NEXT STEPS l
6.1 Discussion of Next Steps l The purpose of this section is to outline the critical issues that need to be addressed by the project
! team, AEOD, NRR, or the NRC. The format of this section is an overview of the issue and an l understanding of the next steps.
I- 6.2 Assessment of the Trial Implementation of the Template Starting in October 1997, each of the regions is responsible for coding each of the PIM entries
- consistent with the Template categories and subcategories.
6.3 Integration with the NRR Decision Model l One of the key points ofintegration is to determine if the Template configuration is adequate to effectively feed into the decision model cunently under development by the NRR-sponsored IRAP team. All of the work to date on the Template assumes reasonable consequences within the decision model. However, the consequences of the decision model were not finalized prior to trial implementation of the Template.
l 6.4 Analysis of the Risk implications of the PIM Data l One of the priorities is to develop guidelines to assess the risk significance of PIM entries. RES l- has the lead for completing this task. To assist RES in this task, AEOD scheduled a meeting for
! January 27-29,1998 to Escuss this issue and gather information for use in developing guidelines. Senior Reactor Analysts from the regions and headquarters as well as representatives of NRR, RES, and AEOD participated at this meeting. Guidelines were developed and will be incorporated into the overall assessmem process under development by the IRAPP.
6.5 Agreement on the " Management Effectiveness Category" within the Template Currently, there are some unresolved issues regarding the Management Effectiveness category within the Template. These issues include:
~
January 27,1998 Page 22
interim Report on the Development ofthe Plant Performance Template
+ Defining the role " Organization" or " Management" will play in the IRAP assessment process.
+ Determining any NRC regulatory policy changes required to assess management as part of the overall inspection program i
l + Determining the methodology to collect the information required for this section of the l Template l
+ Determining the appropriate NRC personnel to conduct a management assessment
+ Reconciling the concerns ofindustry regarding the inclusion of this area in the Template. I l :
6.6 Development of Quantitative Measures for use with the Template l
l \
AEOD is currently developing a trending methodology using quantitative measures to identify plants of concern. Also, quantitative measures for the various categories within the Template were identified by the workshops for further consideration. The use of potential quantitative i' Template measures is being considered. Also, the IRAP group is investigating the feasibility of assignmg a numencal score to each PIM entry.
l AEOD met with INPO to explore the use ofINPO/WANO information in the overall assessment process. The NRC is in the process of determining how the information might be used ifit is made available.
6.7 Reassessment ofInspection Program The Template work within AEOD and the IRAP project within NRR will result in the need to
- reevaluate the current inspection program.
i-I l
l January 27,1998 Page 23 L
l
\
interim Report on the Development ofthe Plant Performance Template APPENDIX TABLE OF CONTENTS: APPENDIX !
l B bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 ll . A.1 Plant Performance Template - Revision 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 A.2 Summary of Results: Plant Discussion Database and Region II PIM Analysis . . 45 A.3 Plant Performance Template - Revision 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 A.4 Representative List of Example Performance Issues, by Subcategory . . . . . . . . . 50 1
l i r
l l i l
l l
i I
I l'
i i
l i j l
t l
L !
l l
i January 27,1998 Page 24 L -__ _ __ - - _ _ _ _ _ _ _ _ _ _ _ _ - _ - _ _ _ _ _ _ _ _ _ _ - - _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ - _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ - _ _ - _ _ _ _ _ _ _ _ _ _ _ _ _ - _ _ _ _ _ -
r.
Interim Report on the Development ofthe Plant Performance Template j Bibliography' A memora:xium from M. Wayne Hodges, Safety Programs Division Director, to Charles E.
Rossi, Division of Systems Technology Director.
Subject:
RES/ DST /ClHFB Input to L SMM Template, July 25,1997.
Chakraborty, S. " Safety of Nuclear Power Reactors in the Former Eastem European Countries."
Nuclear Safety, January-June 1995, Vol. 36, No.1, pp.46-52.
Hopper, Calvin M. " Elements of a Nuclear Criticality Safety Program." Nuclear Safety, July-December 1995, Vol. 36, No. 2, pp. 234-242.
Intemational Nuclear Safety Advisory Group, Safety Culture. Safety Series No. 75-INSAG-4.
l- IAEA, Vienna,1991.
I l Klein, Rochelle Lee, Gregory A. Bigley, and Karlene H. Roberts. " Organizational Culture in 1
. High Reliability Organizations: An Extension." Human Relations,1995, Vol. 48, No. 7, pp. 771-793.
La Porte, Todd and Craig W. Thomas. " Regulatory Compliance and the Ethos of Quality Enhancement: Surprises in Nuclear Power Plant Operations." Journal offublic Administration Research and Theory, January 1995, pp.109-137.
La Porte, Todd and Paula M. Consolini. " Working in Practice But Not in Theory: Theoretical Challenges of 'High-Reliability Organizations'." Journal ofPublic Administration Research and Theory, January 1997, pp.19-47.
Nuclear Division President's Indicators (1996). Turkey Point Nuclear Station; St. Lucie Nuclear i Station: FPL Nuclear Division.
L Paradies, M., L. Unger, P. Haas, and M. Terranova, Development ofthe NRC's Human
[ Performance investigation Process (HPIP) (U.S. Nuclear Regulatory Commission, NUREG/CR-5455, SI-92-1010, Vols. I and 2,1993). '
Roberts, Karlene H. "Some Characteristics of One Type of High Reliability Organization."
Organization Science,1990, Vol.1, No. 2, pp.160-175.
January 27,1998 Page 25 -
interim Report on the Development ofthe Plant Performance Template i
l l Rochlin, Gene I, Todd R. La Porte, and Karlene H. Roberts. "The Self-Designing High-l Reliability Organization: Aircraft Carrier Flight Operations at Sea." Naval War College Review, Fall 1987, pp. 76-91.
Rothwell, Geoffrey. " Organizational Structure and Expected Output at Nuclear Power Plants."
Review ofEconomics and Statistics,1996, pp.482-488.
Shurberg, Deborah A. and Son)a B. Haber. Techniques to Assess OrganizationalFactors:
i Progress to Date, BNL Technical Report, A-3956-1-7/94.
. U. S. General Accounting Office, Preventing Problem Plants Requires More Efective NRC Action, GAO/RCED-97-145 NRC's Oversight of Nuclear Power Plants, May 1997.
l ' Wilson, R. E. (1995). "Rickover, Excellence, and Criticality Safety Programs." Nuclear Safety, July-December 1995, Vol. 36, No. 2, pp. 243-248.
l Working Paper, The influence ofOrganization and Management on the Safety ofNPPS and i Other Complex IndustrialSystems. WP-91-28, July 1991. Laxenburg, Austria: IIASA.
i l
l l
l i
l l
l l
l January 27,1998 Page 26 l-
l l
l Interim Report on the Development ofthe Plant Performance Template L
A.I Plant Performance Template - Revision 1 l
This is the first edition of the Template that was distributed both within NRC headquarters and to l the regions to get feedback on issues and recommendations for upgrade. The six major i categories are shown on the first page with subcategories included on pages 2-4 with the draft of operational definitions for each of the subcategories. A list ofissues to be considered in each subcategory is detailed on pages 5-18.
l l
l I-l l
January 27,1998 Page 27 w- - -- -_- ---- - - - - - - - - - - - - - - _ - - - - - - - - - -
Interim Report on the Development ofthe Plant Performance Template l
PLANT PERFORMANCE TEMPLATE Revision 1 -July 31,1997 Maior Categories
- 10. Organizational Effectiveness
- 11. Problem Identification & Resolution
- 12. Operational Performance
- 13. Human Performance
- 14. Material Condition
- 15. Engineering / Design SOURCE KEY
- 1. Categories with letters assigned and type in straight print and were derived from the Plant Discussion Database
- 2. Categories with leners assigned but italic type are added from a review of other sources conducted by Arthur Andersen and AEOD. These categories are identified by the number "2"in superscript
- 3. Categories with no letters assigned and in italic type were developed by RES. These categories are identified by the number "3" in superscript l
l January 27,1998 Page 28
l i
i interim Report on the Development ofthe Plant Performance Template l
l Maior Catenories and Subcategories 1 ORGANIZATIONAL EFFECTIVENESS 1 A Objectives and Oversight Management (corporate and site) involvement in driving safe performance at the site. Management capabilities to oversee plant operations and enforce high standards. l l
- IB Communications All forms of communication internal to the licensee l and external between licensee and NRC and/or l vendors. Communication between functional teams l l
l and up and down the hierarchy of the organization.
l 1C Regulatory Compliance Response to regulatory events at the site or regulatory action taken by NRC. Indication of 4 compliance problems.
1D Culture Management's ability to establish an environment, atmosphere, attitude of safety consciousness, create l an alert and cooperative workforce that learns from l l shared experience and maintain suitable morale.
l l 1E Improvement Initiatives Capability of management to structure, implement and monitor an effective improvement program to address areas of regulatory concern.
l IF Other includes an assortment of site specific or periodic l
characteristics that do notjustify their own category l nor fit nicely in an existing category.
l l 2 PROBLEM IDENTIFICATION & RESOLUTION 2A Self Assessment Programs and processes in place to monitor and review its own perfonnance, selfidentification of emerging j l
problems l
l 2B Problem Identification Break down existing problems or events to the root l cause, identify problems as they are emerging, j properly document deficient conditions i
2C Problem Resolution Problems addressed and solutions implemented in a timely and effective manner, sufficient management l auention and resources provided to appropriately resolve problems January 27,1998 Page 29 i
interim Report on the Development ofthe Plant Performance Template 3 OPERATIONAL PERFORMANCE (Frequency of Transients) 3A OperationsPerformance Operations is mn safely and efficiently, configuration control of equipment and systems is properly maintained, adequate staffing levels are j maintained to ensure shift coverage 3B Transients Challenges to the operation of the plant including safety systems, response oflicensee programs and personnel to ongoing events, classification /
documentation of events in compliance with regulations 3C Operations Processes Development, compliance, and quality of procedures for everyday and/or emergency operating conditions 3D Support Programs Perfonnance of plant support groups in all critical areas of operations, completeness and quality of programs 4 HUMAN PERFORMANCE l
4A PersonnelPerformance/ Skill Plant personnel receive required training, qualification exams and simulation exercises are
, rigorous, personnel possess relevant skill set and l maintain at:ention to detail, operating errors l attributed to human performance are at a minimum 4B Procedures Plant procedures meet regulatory requirements, are well documented, and sufficiently easy to follow.
New or modified procedures are developed as l needed or when information is available.
4C Training and Operator Licensing' 4D Human-System interface & Human Performance
- l l
l l
l January 27,1998 Page 30
l Interim Report on the Development ofthe Plant Performance Template i
5 MATERIAL CONDITION (Safety System Reliability / Availability)
SA Systems / Equipment Reliability Systems, equipment, and instrumentation function in l a reliable manner, adequate redundancy exists and is maintained 1
58 MaterialCondition Equipment and hardware is well maintained and in l satisfactory operating condition, plant appearance is i
clean and orderly SC Maintenance Maintenance program effectively keeps-up plant equipment and systems, performs surveillance testing to ensure equipment performance, properly I
tests repairs prior to restart, and works cooperatively l l with other functionalteams !
l l' SD WorkControl Maintenance processes are well controlled and coordinated, work orders are complete and involve l all relevant departments, tagging procedures are l adhered to, backlogs and rework are mbimized l 6 ENGINEERING / DESIGN 6A Design Engineering ensures plant remains within design basis, design is up-to-date based on latest available
! information, modification are effectively
- j. implemented and properly documented, design errors j are limited t
6B Engineering Program Engineering department performs at sufficient standard to ensure plant stays within design specifications, oversight and testing of maintenance l activities, and prepares license submittals to NRC 6C Engineering Performance Engineering program effectively addresses relevant problems, minimize backlog of open issues, maintain knowledge of systems, program responds in a timely manner to needs of other departments 6D Design of the Human 4ystem Interface '
l January 27,1998 Page 31 E_______________---__ _ _ _ _ _ _ _ - - - -
Interim Report on the Development ofthe Plant Performance Template Major _Catenories. Subcatqories and Sub-Subcategories
~ ~ ~ ~ ~ ~ ~ ~ ~~ ~ ~~~
l dMERifIfiONAEIEYNCNiYfNY$5~ ~ -
TA 65Jiiiiiiinniib~v7fsi5~~~~-~~~~~ ~~-~Winnsgeiaent (corporatiand site) invoiveWint in driving safe performance at the site.
Mane: ment capabilities to oversee plant l . ._- .. - ,. .
g c.--.ations and enfo,rce.h.igh stan,d.a.rds. M
,.n..
a management commitment oloversig .--,.litT3,1scipfm.. c
~~ ~
~li~eT;ieFritio~Es coniini~nifiteTa~ridsidleis~tood Miiiii,ement communicating to staffexpectatIo~n's 67 philosophy on standards of performance c management accountab'ility (partic_ularly ng regaManagement ownerslu,polproETem-s.Semg ,
tala,rd.
n sig s.a.fe.
~ ~~ ~ ~~
aware problems ~
exist, or being ~
tolerant of problems ~
cf corporate sup;iort Parent uiiIIty's i6?porate management sEpportmg p
.._ la_nt.a.c.tivit_ie.s.o.r.P.rogr.a.m.s e management involvement in operation Plant management tal,ing an active rote in operations of the facility, including leadership, support, control,
~ ~
ownership
~~
f" manidiinent competence 7;iirlsrmance Plant management or supervisors a6flity tiffffform their responsibilities, possession of the corre:t skills, etc.
g 7o7e-~c7arityT---- Management cl early definm- g and commum"catmg
~ ~
the responsibilities
~
and job~ function to all personnel
~~
~~~'~)I support hfYe~s5EEiftraining/ q'u ilT}IEaTIbEi? Managementcommitment 5viisifC2~ i involvement in employee training, skills
~ ~ ~ ~ ~~
development work scEe~duTe~did overtimej3TcTepr---- lEUrrentfy consideridisileiTA3~a~cielu~aii siiffirigT~ ~
i attention toluumaE-rystem interfaie issuesT~~~~1cGifeT1Tly~consideli&TEiciciTAITEaE-m~acEine interface]
- ~
attention to p7anning~
antic 7Iiiu7Iig r ---][3 tYd'e~qIEvafent to i3r'k controTo7di?fileiiI'Tf~~
attention top oceduresprdgiaTnT~~~~~~~~lcErrentfy consideredisilei I Aa-management
~~~
commitment to oversight / discipline]
attention to wor'A776TlY'~~~~ i.cui~n.if.y consiT,e..ri,disicfeiTA3.~i.c,iequate.iIiffirigT.~
attention to7:uman resources .,
e.g., mcentive, discipi, mary, promotion, performance evrluation, recognition,
~
feedback programs
~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~
incompatible ~g55G"ig"Ta7Ery vs.~icimomic- r* ~ ~ ~ ~~~
~~~ ~ ~ ~ ~ ~ ~~
pir]5fmbnce trendmgprogia~mT~~~~~~~~~~~
~
commitment to aiGjithiepreventive maintenanceT*~]cGIIeTETy~ionsidere&TiicleiTCc-preventive
. - - - - - . . . .... ------ maintena.n.ce] - ----..----. ..
anonymous reportmg system 7..
).g_ . - . . . . --. ......--
licensee and external between licensee and NRC and/or vendors. Corumunication between functional teams and up and down the hierarchy of ---the o g a verticai_ro mmu.,_.,
mettion (mtra- anftnterdepartmenroTj_commu.r..aniza. tion.nication from ma
. _ _ _ . . . --. -- . personnel onpjant issues., policies t standarg___
i l
january 27,19g Page 32
Interim Report on the Development ofthe Plant Performance Template
~
~ ~~~~ii~Iioliliital~ommunliitIon (dtra- anl~~~~ CoEEEEEiiatfiiGIiiiiIifiicifolialI8r~gahIiitidual interdepartmental) teams and across te,ams c criernafcommunication Plant / corporate review aniu,.__fe_m_.___,___f np entation o _____
(OperationalEvent Review) # external information such as information provided by vendors, peers, or NRC
~~~~T copeiness to thT87f6, other regulators / other gori Plant / corporate commuitcations [nd cooperation entities 28 with NRC personnel, responsiveness to NRC
..__ requirem.e.n..ts
__.c e genera 7 performance 7
~
Overalfattributes ol_a_ plant s commumcations communicating goafs72 Expectations7 ~lcurrentfy consideredIIIiier I AS-~ expectations
~ ~ ' ~ ~ ~ ~ ~ communicated and understood) rnaT:3geIle~nT/EidEaiff~ ~
~~~~~~~liitTiiis3undant with - 1 A attentIosi31TT0T IC Regulatory Compliance Response to regulatory es ents at the site or
~~~
regulatory action taken by NRC. Indication of
. . . _ _ _ _ _ _ , . _ _ , _ _ _ .. . complian_c,e proble_m,s. .,_m . ,,
a enforcement h,istory/ violation Any v,iofations or enforcement activity (except civif_
penalties) includes conferences, pending, escalated, discretion
__.__1__c,ivil_p_e...nalty Any reTerence to civil penalues c wilif"uIIEIsconduct instances of" intentional or wl[lluffiniiddduct by licensee, violations worthy of notable exception (also probably captured under violation and civil penalty)
~~~~T~Iicense c lid [ormaice (tech sp~ccifr------ licensee compfiance with reguIitTryleilIEfial specifications for equipment, systems, or procedures 3_g _ g _ _ _ . . _ _ . . _ ..._
environment, atmosphere, atti9de of safety consciousness. Create an alert and cooperative workforce that learns from shared experience
~~ ~~~
and maintains suitable morale.
a cordmitment to safety Plant (pfant management) empTasis on safety, saIEiy~
culture, conservative decision making, safety focus,
. safe,.ty s_e_nsit __,
1 ove..ft ra s,ite teamworic.__. ..__,____
/ cooperation Abih,ty._
offu_ivity, nctional teamsmonitoring io coordinate and of safety ..
coop c empfoYie~aditudeTiidiale
~~~
~~ dine,erate, plan and faI aItliiii6Iplant perform personnei, worker work satisfaction, morale, atmosphere, sensitivity to problems
~~~~~i'Isl~h g siisdifisTeipectatissi[ goal selfirigi ~ilinagem~ent establishing hIgli standards, goals, expectations for plant and employee performance,
~~~
also promotion of those standards / goals e risli[ risk management / PRA) Plant management'ilitiliiosi5o5iifsk, use of risk and PRA in decision making, public and worker
~~~~~ ~ ~~~
exposure," risk management
~
f commitdient to imp ovement Management"idesirii3lmprove,IiidiisEip ol
_ _ _ _ _ ____ change __
l January 27,1998 Page 33 i .
I
1 1
Interim Report on the Development ofthe Plant Performance Template i
~~~~~~~
g accountability' ~~ ~~~~Ye~ri5ddil5Ec~iilabTiliy~generaipia~nTc~u]iiG5I~~
t ownership of problems (Distinct from management
~ ~ ~
accountability (1 Ac))
! h c3is~eMairilife~cisl3Tm~aling T ~
Management, supervisors aidirer ossiim'iGig-~
decisions in a well thought out, safety conscious manner, decisions which leave adequate margin for l error (currently redundant w/ commitment to safety !
~ ~~ (IDa)) ~
1 eig~n'Ea]t 5n~allearning ~~~~]Inplemiitag 1eGoils3eai5fiitivitiis*65ie~dIT~
(redundant w/OER?) # review of generic industry issues, monitorir of similar problems at peer plants, sharing of I in l 7._ --..fo.r.m_at_io_n. -_ - wit.hin.th.e.P
- ant ..__
ro
___Ee_m repor.
tmg_. ow.up--
encouragep!Teicii pr ~Iciive vs
~
epioNis~s ing r - -
[,oore ,,p,ci, cor,,,try cy,;rg,reaiirider bodifX~~ ;
l self assessment & 2Ba-root cause analysis / problem i
' ~ ID)~~~~~~~ ~~~~~ ~~ ~~~
Triciisir~a[egueniinTng~*&inu~32tattentTon io aer5ir~
~~~ ~~~ ~~~~
W~liipiTvedisiliitIstives depa5Isliy*3f dinngirim~ent to tructuTe,
~ "
l implement and anomitor an effective improvenient ;
l re a improvement unpfementation
_-- program Progress 6yt_o_ad_d_l_s_s_a,re.as..o_f_r.egu_la_t_o_7_c r_
tiie p ant unplementmg programmatic changes, typically progress against a previously improv 7_g,._g.g7,_g .___
__________ su,b.mittedg_g_e_mentplan__g g _ -_g._____
i includes technical assistance for maintenance, !
engineering, etc, also includes management
! consultants or independent reviewers to assess plant
~~~~ ~_ performance ;
~~'~T neTIdidigiinenIT6fv'nershIp diaigioIIiwiiisiiIpiir~liersonneicisiigiiidade~ii~
the corporate, manager, or supervisor level, also ;
inclu
_e.t.c )_
, __d~r_eorgan,ization__,__.
__ T_e_ih,d_e_s_s_ta_b_ilit_y - - f o_f le.a.de,rs_h.i gnment oltiie organization or personne l
l corporate or plant management levels, might include !
~~~~
reorganization of functional teams i
~~T~pcTs6ddefi6anges ~~~~Cididents about personneTiu~iover, ~~~~~~
replaced / terminated individuals, rotational assignments 7, ~~
3 g_ g; -_p y , g -_. y . 3 3 business plans also covers the quality and viability of ~the plan g resources A illibility7ade~q'u~5)~6fresources(financiaTaridToi~ 1 or sus l y- _g____,,f
____....yhysical) . ,g to improve 7 ,, _g.g_t.a.in_performan_c_e g
_ .-_____ .-- ___im_.Provement initiatives _,,, , , ,_ ,,,__
j i
l January 27,1998 ' Page 34 l'
!nterim Report on the Development ofthe Plant Performance Template l
W ~6tTer inclu3es i~n'~ssortm7it of site'sgie~cific ofErfo7Ic~
- ~ ~
characteristics that do not justify their own
~
category nor fit nicely in an existing category.
a restartreadIness' e Plants fu1HITment oliYia'sks requireTfoi~
~~
~~~~~ completion prior to restart, planning, progress, etc. ~
3 -
l 2 misceTTa~ncous ~~~~Eifeielfis~to 1 Management whicii dditI>I~ fit iii"6tIIeI~
! categories, currently references include legal,
! political, and other control issues (consider creating separate subcategories?)
i
(
l 1
i l
l l
January 27,1998 Page 35 l E_________-.__._______--_.-__--_-__ _ -- _ - - - - - - - - - - - - - - - - - - - - - --
interim Report on the Development ofthe Plant Performance Template 2 PROBLEM IDENTIFICATION & RESOLUTION g.. . _ _ _ _ _ ._ _ .___. .. . . _
review its own performance, selfidentification of
~~ emerging problems a TcIIassessment ~IIffiitividess at evaluatm"gTts owd }ErfdrIEince, self~~
~
critical ~
~ 7 ~ Q A piocess ~ ~ ~ ~~~~TapaVility oTQu~afity 'iUisiiraEEe IEEitioETEefjiidentITy~
and communicate problems at earliest possible stage, appropriate management upgrt exists __
2Y~hrobTeinTlentihestion areak down existing problems or events to the core l issues, identify problems as they are emerging, properly document deficient conditions a root-cat.se analysis / problen. identification Analyze problems, understand the breadth and depth of emerging problems or past events, identify
___. .___ _ ____. . contributingfactors to an event b reporting / documentation Accuracy and completeness when reporting events or deficient conditions to NRC 2C Problem Resolution Problems addressed and solutions implemented in a timely and effective manner, su'ficient management attention and resources provided to appropriately resolve problems,_,_________,,_____
a internal problem resolution Program and/or procedures for handling employee concerns such as allegations or harassment and intimidation l b corrective action (prioritization ofissues) manage, prio-itize and correct problems which are identified by licensee or NRC, effective and efficient resolution ofissues c managementresponsimress/ involvement 2 Management participation in addressing issues, correcting problems, adequate attention paid to ensure i work is comy,leted in a timely manner _ _____
d resourcesfor corrective action # Corporate / management allocates suflicient resources (financial and/or physical) to address correction of problems, does not encourage cost cutting without regard to safety,___, _ _ _ _ _ _ _ _
8
___,,_implementingand monitoringcogec iv,e, actions Jcurrently considered under 2Cb-corrective action]
January 27,1998 Page 36
interim Report on the Development ofthe Plant Performance Template 3 OPERATIONAL PERFORMANCE (Frequency of Transients)
~~~ ~ ~~ ~ ~
TA OpersOIGIs'Fe~rfo~r~ dance Operiti~ sis ~i'i~u~n~ safety a d'e~ffGie~atly7~~
equipment and systems are properly configured, adequate stafflag levels are malatsised to ensure shift coverage,___ __
a configuration control Operations maintains appropnate configuration of
______. _ ___ plant eggeng, systems and indicators ___
l b performance trend Recent performance of operations as compared to
-_ __ .=past history, peersto,r expytations ]
_ l l c programimplementation Current level / standard of performance of the I
operations program, status of any programmatic l
____ . . change,s tg,the departmeni,,,, ,
d odequarestaging* Sufficient operators for shift coverage, overtime not l excessive t shift tumoverperformed well ____
l e generalperformance > References to operations performance which do not l fit in other categories, currently includes references toplant condition and mode _, _
f conservative operational decision making Operations department makes well thought-out (redundant w/ culture (ID) ?) 2 decisions, maintains adequate safety margin
[ (particularly in response to transients or when
. __.__ _____ .. . _ _ approving,egipment ou,tages)_____
3B Transients Challenges to the operation of the plant including l safety systems, response oflicensee programs and personnel to ongoing events, elassifiention/
documentatica of events la complisace with
. . _ . _ _ _ _ _ _ . . . ....__ ._ - - - - regulations a event /significant transients Any challenge to the operation of the plant including those affecting safety systems and/or balance of plant systems, ranges from mild to highly risk significant, includes equipment failure,' plant trip / scram, and force,d, outage b response to transients Operations program or personnel response to an event or transient, synems actuated, decisions made, procedures followed____ _____
c documentation of transients Licensee classification and write-up of
- ___.- . __ U.'".tgya,n,sients, met ngu_latogggirements 3C Operations Processes Development, complissee, and quality of procedures for everyday and/or emergemey
( ojerstiag, conditions _,_______
! a emergencyprocedures Operations compliance with emergency procedures, i
quality and adequacy of emergency procedures, as observed duringnsponse to transients or training _
b operations procedures Quality, adequacy, and compliance with general l operating procedures includes operator walkdown, shift activity, administrative control, plant mode ch,angez etc.
January 77,1998 Page 37 l
Interim Report on the Development ofthe Plant Performance Template
_. '*R'"9' "'.'!!'_'!.'__ __ _. __ .__._ ___.___._ .___.. ._
maintenanceprocedures 8 [this should be added to 5 if not captured by the
. _______ _ _ _ _ . ._ categories presentl_,,___ ,,____, ,
3D Support Programs Performance of plant support groups in all critical areas of operations, completeness and
. ..________ _____. _ ____.1"*.li'J,orpr_og ra m s ,____, _____
a generalperformance# Quality and/or implementation of overall support program (references that don't fit under specific
______________________ __ __ ___Emem sL_________ _ ______
b chemistry program Effectiveness / adequacy of the program including procedures, QA, reacht surveillance, and decontamination c emergencypreparednessprogram Performance of the program such as procedures,
. ____ __ _______ __ 902 traininj,,a_d,e_qu_a_cy of facilipest etc. ,
d environmental monitoring program Performance of the program, monitoring of plant
___ ,_ _____, _ _. emissions, leaching _(excludes radiological) e fire protection program Performance of the program including procedures, hazard analysis, fire system monitoring and
_____ ____ ____ ____. .. adeguacy,_ _ _ _ _ ,,_ _ _ _ _ __ , _ _
f radiation protection program Performance of the program and prccedures for reducing / limiting worker exposure, collective dose for theplan_t2contaminated are,as, and source term ,
g security program Performance of the program and procedures for controlling personnel access to the plant, firearms, and vehicle access, also maintenance of security equipment (currently includes safeguard and materials control},_ ____ _____
l January 27,1998 Page 38 i
i
t Interim Report on the Dewlopment ofthe Plant Performance Template 4 HUMAN PERFORMANCE
~~ ~
TA7e'ri6an~TNHofissnce751Ml"~ a ~~~~h~aIU~perso~n'a~eT5 c~eEe~reiulid l training, qualification esams and simulation exercises are rigorous, personnel possess relevant skill set and malatala attention to detail, operating errors attributed to bemas performance are at a minimum _____ - __
a training / examination /requalification Training of personnel meets required standard, quality of the program, exam results, simulation
_ . . .__... _ ___. _______**.ercises, specific _ course offerings, etc.,
b personnelskill/ knowledge Personnel posses appropnate skill set for theirjob, personnel are knowledgeable of the plant, assigned
._ . _. _ __,, sy_ste,m,s or operating _ag,, ,,_ ,,,,,,,
c personnelerror Licensee personnel or contractor makes a mistake /
l ermr which irnpacts plant performance negatively, ib
.,_rggic3use of,ayoblem ant uted,to,,h,uman cause _
d personnelperformance General comments regarding performance of
_ _ ____ ._______ . _ . ____ ind,pidual employees angor teams of employees _
e questioning attitude / artention to detail 3 Employees maintain a vigilant attitude, do not blindly follow procedures or instructions, sufficiently attentive to notice subtle condition
_. .. ____... p changes within the,i_r area of respons iliy _
\
_.. "* P'!kC*_u"S*.'__ . _.. -_ . _..
cognitiveyegormance 8 o eg.2 cognitive,eny,rs,, stress effects;_ decision making, medicalqualtpcations '
_ .h"fo'd"Od 48 Procedures Plant procedures meet regulatory requirements, are well documented, and sufficiently easy to follow. New or modified procedures are developed as needed or when information is available.
a procedure quality Plant procedures in all functional areas are correct, documented, accessible, meet regulatory requirements, easy to comprehend and follow, drive the intended behaviors b procedureadherence/ violation Plant / contractor personnel fail to follow, are
, unfamiliar or work-around established procedures, j observed by NRC, selfidentified, or attributed as root cause ofpast e_ven.t _______..._____
e - procedure adequacy (how is this diferentfom JBa) Procedures as documented or as implemented are insufficient to meet regulatory,reguire,mgt,____m d developing, implementing, and maintaining procedures New procedures are created as necessary and validated appropriately, procedures rolled-out in a timey,m, ann,er to relevant departments and personnel j
January 27,1998 Page 39
l l
t t
imerim Report on the Development ofthe Plant Performance Template I
l l
e proceduremodification Procedures are appropriately modified or upgraded to meet new regulatory requirements or to take advantage of new vendor information, NRC
. __ - _ _ _ . publ.ications2p.e.ercompari_ son. etc.
. z miscellaneous' Miscellaneous references to procedures which do not
.. -- _ fit in othe.r.cate1o.r.i.es configuration control' [ currently considered under 3Aa-configuration control} !
computeri:ation of[wocedures *
.l"ocedure availability ' l l
I !
- 4C 1>eh.d.n.(.a.n.d D- rL. ice.n.s.u.r_e 7.'
- SAT-ba.s.e.d.t.r.ain_ing a.dequacy I
._ exam.i.n_ati.o.n.ma.tched tol'.o.b.r.9uir.e.m._ents.'..
-- e_xa.m.i.n.a.ti.o.n.. .-match.ed..to 9.u.'P..*.*.n.t/.'E.'te.m.s..' ---- .... .-- . -- .
demonstrated validity offraining/aram.8 [ currently considered under 4Aa-training /
i
.. ._. --. _-.--. examinati.o.n] .--. . .
[ . lessons learned incorporat.e.d.into trai.n.in1.' . - - -..
simulation ' e.g. simulator fidelity, appropriate use of simulator, design of simulator scenarios for training and exams, e
.--.tc --. .
l -
i ._ _ ..-- .--.. ._ --.--. ...--
l 4D H.umen.Syste.m..i.n_terfac_e & .H.u.ma_n P.e.rf.o.r.m.ance
- l SPD
.___-.---._S.'.. .. -- _. ..
i
_ D.CR.DR.'._
-_..T controlroom "s93rades.
. ..__h ybrid control rooms '____ __________ ____ _____
navigation issues *
.8 ent o2'rator aids.' .....
in.t.e.lli _- _
c.o."Eu.t.e.r.iz.a.t. ion' automation.' --- .... --- e1. . situation awaren.e.ss.,.c.o.m.munication.s . u vai ilance -
h I
l l
l i
January 27,1998 Page 40
interim Report on the Development ofthe Plant Performance Template j
5 MATERIAL CONDITION (Safety System Reliability / Availability) j
~~
~ TA~Syst Isis ~/Yq~iiipment diRaEllify~~~~~~~~~ Systiin's~, diliIp' dini~a'ad instrudentaifon
~~
function la a reliable manner, adequate
. . . . _ _ _ _ _ _ . . redundances,ists and is maintained _ _ _
a safety system reliability Condition / status of safety-related equipment and systems, reliability that these systems will respond when called upon and perform mission b equipment reliability Condition / status of individual equipment and components (may include balance of plant),
reliability tht these systems will respond when called upon ____
- c instrumentreliability Condition / status of plant instrumentation and )
l controls, reliability that these instruments will l provide accurate readings of the systems they l
~ monitor
! d amicability (Pref) 2 Availability of safety systems, appropriate redundancies in place, actions which affect core
. . _ _ _ _ . . _ . dama.Se fmi=c1____ __ ..
~ ~ ~ ~ ~ ~ ~~~ ~~ ~ ~ ~ ~ ~ ~ ~ ~ ~ -~
98 MaterIs~l'En~dition equip' dent and harilware is vreil maintained ant l In satisfactory operating condition, plant l app,earance is clean andgenerally, tidy _,
a equipment condition Condition of plant equipment, aging, deterioration, l . .. ______ .____ . . ... wear and tear 2 upgrades ,__ , __
l b hardware condition Condition of plant hardware, aging, deterioration,
._._ .... . .____ . . wear and tear,_ upgrades c plant appearance / housekeeping Cleanliness of the entire plant, paint, floors, spills, l _. .. __..... _ . leaks,1eneral tidiness of the fac_ilig_____,
l d generalperformance 2 Overall material condition of the plant, general comments, not references to specific categories l
SC Maintenance Maintenance program effectively keeps-up plant
- l. equipment and systems, performs surveillance ,
l testing to ensure equipment performance, properly tests repairs prior to restart, and works cooperatively with other functional, teams ___
a program implementation Effectiveness and adequacy of the maintenance program, current level of program implementation,
- ._ . . . . _ _ . . ._ f! pair and upk,eep of equipment _,
i b maintenance performance trend Recent performance of snaintenance as compared to
_ _ . _ . _ __.__. . __ ..pa,st, history, peerst or expectations _____,
c preventive maintenance Program ability to perform preventive maintenance, troubleshoot problems, fix equipment prior to I ;
t
_ ._ __ . . _ _ _ . . breakdo.wn-1 d testing / surveillance Assessment / testing of applied procedures and i equipment repairs prior to restarting equipment, l verification l j
January 27,1998 Page 41 i
interim Report on the Development ofthe Plant Performance Template l
e predactivemaintenance2 Predictive approach to maintenance exists, j techniques used to reduce variability in maintenance l cycles, maintenance performed on a schedule not
{ when equipment breaks down I
f supportandcommrmication 2 Maintenance program provides assistance and works in a cooperative fashion with other functional teams, j communicates effectively with other teams (particularly operations) ___ _
_ calibrationandtesting* __ ___ __ ____ __
SD Work Control Maintenance processes are well controlled and coordinated, work orders are complete and involve all relevant departments, tagging !
procedures are adhered to, backlogs and rework are minimized l a backlog Maintenance backlog of unaddressed / incomplete workorders, cause for backlog, size and age, system forprioritiziniand scheduling, efforts to reduce l b tagging Maintenance properly indicates equipment which is__ )
out-of-service, being repaired, or in-service but in
, need of f epair , ,_ _ ___
l c work controlprocess(coordination) Managing and coordinating all work orders (service, repair, installation) performed on plant equipment, i coordinating maintenance, operations, engineering and any relevant suppo,rt_ programs,_,
- d work control performance Current level of performance of the work control l
process, breakdown in coordination, procedure adherence e rework 2 Current and trended level of required rework, quantitative and qualitative discussion of maintenance work packages which are unsuccessfullygplemented f rueandcontrolofcontractor' Oversight of contracted work, quality of the work, standards and procedures enforced, amount of
__. . ...___ . . _ _ contracted work performed ______
j z miscellaneous 3 Miscellaneous references to work control which do not fit .m other_categones work arou ids 8 [cunently considered under 4Bb-procedure
_ _ _ _ . _ _ _ _ . . .. .__. adherence,[ ,,,,,,,,,,,,,,,,,,, ,,,
l l
l r
i
~
January 27,1998 Page 42 j l !
l I
I
Interim Rpar. on the Dewlopment ofthe Plant Performance Template 6 ENGINEERING / DESIGN TI lie:Va ~~TnFaee~ ring ensures piant rem Es withEilesign
~~
basis, design is up-to-date based on latest available information, modifications are effectively implemented and properly documented, design
__ _ ____ . . . ... ._ errors are limited _ _,,_____ ,
a design basis / systems and equipment design Design of systems and equipment must meet regulatory requirement, standards include reliability over time, reliability under accident conditions,
.- . . redundancy, etc. ,
b designchange/ modification Engineering update of system and equipment design
- j. based on latest information from vendors, NRC, etc.,
l quality of the design change, effectiveness of l ____ _ _ _ _ . _ _ _ _ . . . _ ,_ im,plementation___ ________
l c designerror Systems or equipment design deficiency discovered, l causes might include vendor error, installation
- .. . _____ _ _______ __ problems, etc.____, ____
- l. d upkeep of documentation / diagrams _ - . , Engineering maintains and updates design l.
documentation and schematics aAer modification or upgmdes ,_,,, ,____________ ,, ,
e plantaging 2 Design issues arising due to age of the plant, systems,
___. _ _ ___ _. . ________ _luipment ' etc.
t 2
f man-machineinterface Design of equipment and instrumentation cases worker interface, limit potential,for human erro_r_
____ designs awkwar_d t_o__ maintain, repah_operaje ' ______, ___, , _
.. poor w rk environment * , ____ __e3._h_igh noise levels
-_. J_S,_ H design errors ' ,___, _. ______ _ _______
~ ~ ~~~ ~ ~~~~~~~~~~ ~~~
68 EdiEringPrkEm
~ ~~
l Engineering deg artmenti~e~rio~rms at sufficient l standard to ensure plant stays within design i
specifications, oversight and testing of malatenance activities, and prepares license
. . . . _ _ . . ...._. submittals to NRC ___ ____,_
a engineeringoversight Capability to monitor the engineering program quality, standards, and effectiveness of the overall program, completion of 50.59 reviews, operability
- decisions and supervisor oversight of department i personnel b procedureimplementation/ adherence / adequacy Engineering develops and follows appropriate
. _ _ . - - -. _ ___... Procedures u meets regulatory _ retirements _ _
c licensesubmittal Quality and responsiveness oflicense submittals to i .. ..___ _ _ __
the NRC
- d testing Engineering tests and monitors systems following l_ design changes and updates, verifies that
_________. _ .. ___ modifications are properly, imp _lemented
( e system engineering Plant follows a " system engineering" approach, i individuals or teams are assigned responsibility for whole systems ,_ ,____ ,_
January 27,1998 Page 43
}
i
l
[
Interim Report on the Development ofthe Plant Performance Template
, f generalperformance 8 Overall performance of the engineering program, l l general comments, not references to specific l
categories 6C Engineering Performance Engineering program effectively addresses relevant problems, mirfmize backlog of l unaddressed issues, maintain knowledge of systems, program responds in a timely manner to needs of other departments a backlog Engineering backlog of unaddressed / incomplete evaluation requests or drawings, came for backlog, size and age, system for prioritizing and scheduling, efforts to reduce , ____ _,
b knowledge of plant / systems Engineers knowledge of assigned systems or areas of l _. . ____. . .__.... f*52 nsibility, familiarity _with op,erations l c performance trend Recent performance of engineering as compared to past histog, peerstor expectations __,
d responsiveness / support of other departments Availability, timeliness, and quality of engineering l response to requests for assistance from other departments, commitment and support to the activities
. . .. . .__. ..__.- of other departments _, _ _ _ _ _ , _
e assessment / diagnosis of problems Analyze potential areas for engineering concern, self-identikproblems, timely,diagnosi,s a of root causes z miscellaneous 2 Miscellaneous references to engineering performance which do not fit in other categories __, __
6D Design of the Human-System Interface ' [need to distinguish from same categoies now listed
--- under,41 _ , ____,_, _
DCRDR'
_, controlroom," upgrades"' _ _ _ ___ _ _ __ ___ _
&bridcontrolrooms '
navigation issues ' i intelligent operator aids '
i January 27,1993 Page 44 l
l I
i l
interim Report on the Development ofthe Plant Performance Template A.2 Summary of Results: Plant Discussion Database and Region II PIM Analysis The following charts show the statistical summary ofperformance characteristics as captured by the analysis of both the Region II PIM database and the Plant Discussion Database. Each item was coded to one of the Template categories to generate the detailed summary. The second page shows the detailed count data for each of the subcategories. The purpose of the analysis was to understand if the various items included in both databases were covered within the design of the Template or if there were gaps in the Template. It also provided a preliminary view of which areas within the Template might not covered by the PIM.
i i
January 27,1998 Page 45 l
l l
interim Rsport on the Development ofthe Plant Pcrforman:e Temp:::e CUMULATIVE HITS by CATEGORY Region 2 PIM Plant Discussion Dbase Row Total % Cat. Row Total % Cat.
1 Management 333 20 % 652 22.1 %
2 Operational Performance / Transients 333 20 % 681 23.1 %
3 Human Performance 259 16 % 424 14.4 %
4 Material Condition / Reliability 299 18 % 597 20.3 %
5 Engineering / Design 210 13 % 245 8.3% <
6 Problem Identification & Resolution 200 12 % 346 11.7 %
Total 1634 100 % 2945 100.0 %
January 27,1998 Page 46
interim Report on the Development ofthe Plant Performance Template DETAIL of HITS by CATEGORY (and Subcategory)
Region 2 PIM Plant Discussion Dbase Row Total % Subcat. Row Total % Subcat.
lA Oversight / Discipline 46 13.8 155 23.8 IB Communications 27 8.1 26 4.0 1C Regulatory Compliance 197 59.2 115 17.6 1D Culture 39 11.7 95 14.6 l
IE 1mprovementInitiatives 20 6.0 155 23.8 IF Other 4 1.2 106 16.3 2A Operations Performance 101 30.3 184 27.0 l
2B Transients 70 21.0 263 38.6 2C Operations Processes 24 7.2 25 3.7 2D Support Programs 138 41.4 209 30.7 3A PersonnelPerformance/ Skill 128 49.4 263 62.0 3B Procedures 131 50.6 161 38.0 4A Systems / Equipment Reliability 89 29.8 137 22.9 4B MaterialCondition 63 21.1 150 25.1 4C Maintenance 106 35.5 197 33.0 I
4D Work Control 41 13.7 113 18.9 5A Design 108 51.4 91 37.1 5B Engineering Program 77 36.7 41 16.7 5C Engineering Performance 25 11.9 113 46.1 6A Self Assessment 52 26.0 63 18.2 l 6B Problem identification 47 23.5 79 22.8 6C Problem Resolution 101 50.5 204 59.0 l
1634 600.0 % 2945 600.0 %
l I
January 27,1998 Page 47 l
interim Report on the Development ofthe Plant Performance Template A.3 Plant Performance Template - Revision 2 This version of the Template reflects the efforts and lessons learned from the two workshops to upgrade and simplify the Template. A key learning from the regional input to the workshops was that the initial Template was too cumbersome with numerous subcategories and all of the attached detail.
f
)
i l
t l
l i
January 27,199g Page 48
e tnis r t d n o n e ntnta tedn er nt a ne itptn lad cu@
m ed e, a nfe saa a e s a c a, s u a pb ts a i r sn ts cet a it e c r o v,an (.s i)ys e mgt gmgs s n m e, i n s n msd e, icio o g, ainc s ,
a mei z n pa n s s
_ o nr i n mu t
r e e e g n a e inh gdi a n e e
it gf yuit e e a mes yupai t inai odi z ss toca n r ar uf r ent ioc t
n a
ai s ri r n v r ga u t c e nainmodt mio r t g gs it m s,t n mla pd c e om e g f r
g v a ai i el e t gd aca r co ugl u r
a roe oer e s ir a e o o loe r m M Mp opc v r n DMd i ori c p a n Ppr r todi c oox geb p cu e o r
OESnhot pr pr r i r enf
_ d dd e t r e
- n nnh ot f g t a
a n
a, s a infe f o io n
t o l.
e ri n i r
p t,
n t
n e
o is lah ee t wd t
a c
es n id d e ud o ehlt nrc a p e m s
it lyi r iws e n im,e s i
n e a te na s e ct ienif v ai enn t
h os f
t r
e ifn dn in s it r p
a t
gs e mns v a, m ul e in ah o mk alo tpc an n . i fi r oed op s ge det t
5 n t
n s did n n e t s ocd s e i of i inms sni i, a, tiot a i
fodrede faf ooi t c a r
ner d it o iead e a, o ed s go nc e s a r i i
n c ue nd gn c c t, n c nmc p a t
gn nl s n un n o f nr gnt ss oo a c s a s o iouoi n ios ct mi- n a icirl a le ni o r o a iuita o t
n . s le i t it i toa mb ompup u se u te iom is it y u t s h bl f l a
b e rl u rot ;l f n
ir n efor r t r u es lord l
e s e e ipt o a e ot ar ef pm lala n v u edat i n rmr l
e h s of r a en s u cb P
r e el n dMmpocoh e ss e r ooo eh h c -
of a r
o T r ppi I t AEib cf c RT s c op 2 g g f
N isd, s ted i n
r e n g i n gn to o O e e h di n yi s s e
t r
I g
is a n n b e a et n t afi e s t t ei s h a s s mod u
pn S e nmdi ogt sdd r
od r f o og b e c p u a, D igue nd I
V /
g s cf i
oiee d dne mna p rn r n s ng E 4i n r
de d t e od ovwi o
u, dn ng pt u
S ppng o io d odHis P a kr ni s e R e e
n n r a a m,dhp r g
in ge a, s i gu o ns n n ogi a wintiev d,g
- lpnd go nai s tni nd irla a s e tnin E
ig n ei o , e nt e c n mme a
c ir t
g r ra ma h se tsd tr ty e n T Y E t d r ie el pl o i emin g u tnps.
a a l an o ci ig hc tn g f le di ei e nn s 2 w oddn ppi is ci i A O l npu a u a s fa c e o a e sa DDmh b n e a ET lp o e pi ge rh mun PTi g elic L G E
I t P T t a d M A C
e c r ht e ht n
af E n i e a ht e s s
e laku r hd ns d e o l d, c na T m r ont h s e ed c i w v o it l
a,d u a t n n a,uein uoc t isl at, d oh g n a E f o d e lsi in o .
v gc r k nif ad no dl w
si g v n e s l mmf chp or yki r s ty C 3 r e a gs upr o mi t n el n ai ne e i akii d ni ogt aes ,at ow uvla ss x :
u a t d w c o t r P r n u N n i dv inne na e fo ad i
t a ic s, inira r oh e n y qe A a ih msh t t rf n e o o s. n e s c si oef t s
s gog ivn r p st t hhc e e ne ie ion un m di Et,e s ts u n r M e nwnmt P sn cgt i s n mQ dlut e wk oa i rgpor i
k ei x s r s, o v n re m I id k R rion i t I r it u r r e p A = o te o ceopo n opkr vi m com r
loo v op ot iric t te a m S -
l l O
F I wenr p u s WAduct a a awo c KI - b ai i c b n WCp r us wen o s prc o t, r t, R n e
d na on s e m. e t o s s e
E n m d n
gc s e s e
e e P o ipde n n pg s dh ut d utf a r niai s ui n i
u n del an nl t
T d i
d qia n
s od mqp it i n r e
e a, nc o n a et n m if f ot e e w c psi n N o enie t o
f a r d n d r nk e
P r
mnmii s o A 2 C c a nl p a
aht mh o n g o o anple sa e d d pi ehta at pu n a a ni n l
L a r p Cs e n gt s y T. tn P ir mfr oaegi r t
ua e .s nei ot t nh f o a s s e e o nyr not e d oh ki s r tn m s,t n m.
t a
f r oeh h e ma tgng n on mo e e r e es nlpm M pi ei h t t
m is s s siii pr d i d t a ew emd r n u o a d
e n wt sp a t d e el l
b e ie c u sg nl r
ad u g oa pt u ml at c pmr mig h o qb c s r e o T oo t
r r ch te im se rp EOre o cin PFi met t soh c t r
p t t d, r e o o r
y e ht t r n p g ,s p o ,
c o l a a p s p t s te n
e d gd p m u a t,nt yn r i n e s ,u s d si n c e a f
te di n n p m mns ynh c e u s e sh ea nr es e is ir S m
r a
e na susa us r
r o
n g l, oio of ni c a e u e nids e v ef r
u on s
e t , u ie ht h
y
. F, el o pl s mg s n gn : to c,t a n g g ng s,r ga ns ngs c e ed od r ei nn t
v s t g: P g n yn f
o a nai n r s o n n o mhint rso n cucr ot i ai r
e migu ita irul onis duah ncocer icunitm i r
irh s
n a n et r ioipio omt Pr oivt c lpual d t oitord t
a a x a t t I P is r u u ciE r upa r ani s g
tnn npni d r c ol rd e cn c u n ou me D d di n r i e r t f la eid e t
dn ptr gcf e nlc orgel lPa mre eP p *s i s, os mp n a pn e p a a s, nf t
i iiais ,ps I, h it h od lpt c e ; Ono snna i e o o s nt s *s ted a n op o i, t o shk s npsa s y C, n.
t s . on noipco n e, l r n ecf on s t
a o r s s gd r nc r
e htebgeitns h e s m iowr/ c, d mitas lait t ognugl e
ieitai c ityt iosn a wie o rmoiu a cd m mtmwms a
s oe pe troa ion a etyirtco t iwgrugi mrdaidii ou a s a p er apod r r e l
n isb c g r r r n r t e e e nl v tnlc g g os O win o ru ata s roo r
o epnu t fnf T t lc g e a e ai is n a n no pr p tu a n o r t
mlpi d p r oe e ut pel af mec or l
l d t r e pp r NOc oh ot s c s oir c* p OTOufa n cct ebh ni aR a s u*r ya r p psf i g s of SESP mu upl o T
A C A B C B
U S
l!f
r-Interim Report on the Development ofthe Plant Performance Template A.4 Representative List of Example Performance Issues, by Subcategory l
l EXAMPLES l
! The following is a list of examples developed by the Category Development and Criteria Development workshops. The examples are intended to be a representative sample (not an exhaustive list) ofissues reflected in PIM entries. The examples are divided by Template subcategory and will assist a Resident Inspector and/or Branch Chief to appropriately categorize l PIM entries.
- 1 A. Normal Operations Configuration control Staffing C3 - Conunand, Control, Communication l Execution of support programs Number of violations of LCOs l
Number of misconfiguration TS violations Failures in tag-outs and lock-outs l IB. Operations During Transients l
l Emergency response -Identification, Classification, Follow-up l Configuration control Stafrmg C3 - Command, Control, Communication Adequacy of event documentation Execution of support programs (including activation of emergency response)
I C. Programs and Processes EP Programs Fire Protection Security Program Operations Support Program Procedural Development / Quality Information Systems 1
l January 27,1998 Page 50 l
I t-----_ - _ - -
I interim Report on the Development ofthe Plant Performance Template Number of LERs involving problems with procedure inadequacies i
Other Operations Examples i i
1 Examples below are categorizedas either 3A or 3B depending on operating condition ;
(normaUtransient) j i
number ofinadvertent safety system actuations l number ofproblems during outages difficulty during startup number of operations-induced transients number of operations-induced scrams number of operator-generated significant events number ofsafety system actuations severity of transients (and consequences) l number of significant events or precursors escalation of minor conditions due to inadequate or wrong operator response timeliness ofresponse to events number of operator errors due to poor communication number of operator errors due to poor drawings number of operator enors during response to transients due to lack of knowledge number and nature of operator errors in tesponse to transients occurrence of errors of commission during response to transients use of risk information to guide decisions adequacy of work' package tracking (configuration awareness) adequacy of outage planning (risk-informed)~
l l Configuration Control Examples (3A/3B) instances in which adequate safety margins were not maintained ,
instances in which appropriate defense-in-depth is not maintained i
instances in which operators demonstrated lack of knowledge of current plant configuration given plant activities instarices in which operators did not effectively manage the plant's risk profile 1
January 27,1998 Page51 L--_-__--___________--.--_---___
Interim Report on the Development ofthe Plant Performance Template Command, Control, Communication Examples (3A/3B) up-to-date operator knowledge of current plant conditions communications in control room in accordance with standards herence to existing programs and processes instances ofprocedure violations operator awareness as demonstrated by alertness and vigilance adequacy ofshift turnover effectiveness of communications with other departments instances of observed crowding in the control room demonstrated clarity of decision authority and accountability j involvement of appropriate personnel in event response (function allocation) i i 2A. Equipment Condition Equipment Reliability / Failures l Repetitive Equipment Failures l Safety System Reliability / Failures Availability Equipment Forced Outage Ra:e Safety System Actuations Annunciator status Unplanned LCOs Equipment response after scram Maintenance Rule Results Performance History MPFFs Equipment Importance Equipment Problems Common cause failure rate l Cleanliness Number of(operator) work-arounds Steam, water or oilleaks painting / preservation Testing Results (touch, smell)
Material Condition Tags l
January 27,1998 Page 52 .
l
Interim Report on the Development ofthe Plant Performance Template Number of components in the alert range Environmentally induced degradation Equipment Aging '
Number of temporary modules / modifications Equipment vibration Control Room Deficiency 2B. Programs and Processes Maintenance Program & Procedures Surveillance Program & Procedures Maintenance / Surveillance Procedure Implementation On-line Maintenance Maintenance Rule Program Contractor Management / Oversight Resources / Maintenance Staffing Available Parts Maintenance State -breakdown, preventive, predictive Tools & Diagnostic Methods IST/ISI Program l
Maintenance Program Testing properly accomplished Trending Program Results Test Pre-conditioning Work Package Quality Post-Maintenance Testing Work Control / Execution Foreign Material Exclusion Maintenance Backlog Overdue PMs Rework Rates Anticipated Deterioration Corrective Maintenance Backlog Instrumentation and Control Support January 27,1998 Page 53 I
j
I l
Interim Report on the Development ofthe Plant Performance Template 1
3A. Work Performance ;
Good communication, communication breakdown Procedure adherence, use of procedures, procedure use errors Job performance aids Work hand-off
- of transients that involve latent errors Team performance, teamwork Questioning attitude, pro-active vs. reactive I Formality in control room !
Stop,Think, Act, Review (STAR)
Attention to detail Cooperation Excessive overtime ,
Stop work in questionable situation HRA in PRA Goodjob brief Sleeping on duty Overconfidence Conscious vs. automatic i Fitness for duty Unmotivated personnel l Wrongly motivated personnel Appropriate people making decisions, decision making, timely decisions
, Malicious compliance /non-compliance HP contributes to event initiation Inattention to duties, attentiveness, vigilance Cognitive errors,0% errors, personnel error rate Safety focus Number of events involving the lack of use of procedures Number and nature of operator errors in response to transients Occurrence of errors of commission during response to transients Number ofwork arounds
- 38. KSA/ Training KSA's, knowledgeable ofjob tasks, systems knowledge l
January 27,1998 Page 54
interim Report on the Development ofthe Plant Performance Template Adequacy of work package l Exam faibre rate .
{
Worker task qualification process OJT process Training l Requalification failure rate l Use of mock-ups and dry runs l Requal program
)
Simulation '
Use SAT, Training program deficiency, incorporated lessons leamed in training ,
Conductjob briefings Contractor qualification Number of operator errors during response to transients due to lack of knowledge i Operator training up-to-date Adequacy of pre-briefings and training for infrequent tasks 3C. Work Environment 4 i
Shift schedule Environmental related human errors, stress induced problems, envirc nmental conditions l l Supervision, management oversight Task allocation, function allocation, workload j
Procedure quality (generic)
L _ Understanding, clear accountability .
L . Operator aids Habitual use of overtime l Rate of staff turnover Number of stafrmg level exceptions or violations 4A. Design l Environmental Design - heat issues, lighting issues, noise issues I Environmental Qualification (EQ)- seismic, environmental Maintainability FSAR updates l Up-to-date drawings January 27,1998 Page 55
interim Report on the Development ofthe Plant Performance Template Quality of calculations, Design calculations Response to engineering requests Design basis reconstitution Manual operator response time measures Digital / Analog integration CR redesigns Maintenance CDB (Current Design Basis)
Design Issues - Human Errors (HSI), Design of hardware Plant computerissues SPDS redesign .
Design basis understanding Accessibility of design information Maintenance CLB (Current License Dasis)
Instances of a mismatch between current as-built plant and drawings and other documentation 4B. Engineering Support Reactor Engineering Fuel cycle management Technical support Material certifications JCOs (Justification for Continued Operations)
Technical adequacy ofprocedures Quality ofoperability evaluations Test review and analysis 50.59 Reviews USQ (Unresolved Safety Question)
! Control of engineering backlog l Appropriate operating limits in procedures Trend system performance PRA Development OER quality Quality of safety evaluations System Engineering Ownership System engineer knowledge of system status Corporate engineering January 27,1998 Page 56
r.
Interim Report on the Development ofthe Plant Performance Template i
Engineering design backlog System reviews Plant efficiency - steam losses, etc.
Responsiveness to engineering support requests - l
' Motor Operated Valve Follow-up evaluation of plant modifications - Testing following design changes i 4C. - Programs and Processes Temporary modification System to prioritize engineering requests Technical specifications Quality ofexemption requests
' Documentation issues Quality of amendment requests l Modification controlissues l Maintenance of vendor documents Quality ofmodule packages
- Engineering organization intrusiveness Licensing issues Perfonn Part 21 reviews l Set point control Vendor / contractor oversight OER program Maintenance / Surveillance Technical Adequacy l
5A. Identification l Selfassessment
- QA findings Peer eveMtions i OER revi; .vs, industry feedback
[ Questioning attitude
! Problem identification criteria / threshold 1 Potential Problem Analysis Process (PPA) l
' Performance trending l
l January 27,1998 Page 57
\
l t_________________.____________.__ __ _ - _ - - - _ - . - - - -
Interim Report on the Development ofthe Plant Performance Template Problem report levels adhered to Quality Control / Assurance (QC/QA)
Workers initiate problem reports, problem ID available at all levels of personnel
- of problem ID problems - many or single event ProblemID tag Human performance issues are addressed Performance observation methods Systemic thinking / approach Improvement programs Role of QA in the site (reporting level)
Independent review / oversight, ISEG, PORC, off-site committee effectiveness
' Corporate culture Openness with NRC Management knowledgeable ofproblems Employee grievances, employee-management relations 5B. Analysis l
Event investigations Prioritization ofproblems Self assessment criterion defined Root-Cause Analysis, events and causal factors analysis, document reasons for Terminating RCA Trended including MIS, methods for capturing trend information Operability evaluations are performed Communications ofpriorities Analysis / evaluation criteria Identify conditions that the corrective actions are intended to achieve Qualifications of RCA evaluations Specify links between root causes and corrective actions Extent ofcondition review Use oflessons leamed from near misses PRA: to evaluate resolutions, prioritize problems, identify precuisors j PRA to communicate issues Auditable Trending data includes human performance categories Quality ofLER preparation January 27,1998 Page 58
interim Report on the Development ofthe Plant Performance Template Document conclusions ofRCA j SC. Resolution Timeliness, average time to implement fixes l Extema) commitments, response to NRC concems l Management support, adequate dollars to fix problems l Repetitive issues, repeat failure rates Corrective action program l Inter-departmental communication ofissues
- Validation and verification, post-modification testing Corrective action backlog Planning, manpower requirements considered, CA plan development, internal commitments Determined measurable objectives, measures of effectiveness of CA's l Monitoring corrective actions l Tracking i # ofoperator work arounds IPE follow-up Feedback to originator
, Allegations
! Employee concems program, safety concems program l
l t
January 27,1998 Page 59 l
l.
L--___---______----.----------__-----_---------_--------------------------------------------------------------------- - - - - - ----------- - --- -- -
DRAIT Appendix D Draft Decision Process Model I
l D-1 Appendix D l
]
8 m
'! l =
dlIl l
\ .
G s - h.i E
"' n l
- s. s s, l,
~
I , e
- 5I
, ff 1
! $] .
" e lI e f' Il I sl Ih '
111 ;y -
- ,jlIl
~
II ' _. ]
Y s
I!ldlil-ll 'lII -
I .
I a @
a d ' h j s a a s
s 1 E
'eb, K
1 :h lp. 2 I i hii t II l
b N b k ) N b E 1
i, l, a1 fil m
'hE '$ lh a .
. .I !!!!
8 l
ih'hlIl
/
. kI 1 !!!! I I!
ll'!!VkIh!=
k)'/pi
l l
1 .
sh il '
'!l' lIl -
lli hl i
A= 3 m e
k(({' {}Ij
+ $>
}>c ..h l!!
' t hg' !h!!! ijhh' hil:
l
=
1 1m miil . 1
=
itil 4
4, 6nu e
k
' "y til i} g '! I a l
l
DRAFT Appendix E FinancialIndicators l
,1 1
i l
l E1 Appendix E j
---___________u