ML102500670
| ML102500670 | |
| Person / Time | |
|---|---|
| Site: | Prairie Island  |
| Issue date: | 05/29/2008 |
| From: | NRC/NRR/DIRS/IPAB |
| To: | Atomic Safety and Licensing Board Panel |
| SECY RAS | |
| Shared Package | |
| ML102500654 | List:
|
| References | |
| 50-282-LR, 50-306-LR, ASLBP 08-871-01-LR-BD01, RAS 18574 | |
| Download: ML102500670 (29) | |
Text
NRC000050 NRC INSPECTION MANUAL IPAB MANUAL CHAPTER 0313 INDUSTRY TRENDS PROGRAM
NRC000050 TABLE OF CONTENTS 0313-01 PURPOSE ................................................................................................. 1 0313-02 OBJECTIVES............................................................................................. 1 0313-03 APPLICABILITY ......................................................................................... 1 0313-04 RESPONSIBILITIES AND AUTHORITIES................................................. 1 04.01 Director, Office of Nuclear Reactor Regulation (NRR) ............................... 1 04.02 Office of Nuclear Regulatory Research (RES) ........................................... 1 04.03 Director, Division of Inspection and Regional Support (DIRS) ................... 2 04.04 Chief, Performance Assessment Branch ................................................... 2 0313-05 BACKGROUND ......................................................................................... 2 0313-06 DISCUSSION............................................................................................. 3 06.01 Collect Indicator Data................................................................................. 3 06.02 Identify Short-Term Issues ......................................................................... 3 06.03 Identify Adverse Trends ............................................................................. 4 06.04 Analyses of Issues ..................................................................................... 4 06.05 Agency Response ...................................................................................... 5 06.06 Senior Management Review ...................................................................... 6 06.07 Communications With Stakeholders .......................................................... 6 APPENDIX A INDICATORS QUALIFIED FOR USE IN THE ITP ...................................................... A-1 APPENDIX B PREDICTION LIMITS.................................................................................................. B-1 APPENDIX C LONG-TERM TRENDING ........................................................................................... C-1 APPENDIX D BASELINE RISK INDEX FOR INITIATING EVENTS (BRIIE)...................................... D-1 EXHIBIT 1 SAMPLE PREDICTION LIMIT CHART ..................................................................... E1-1 EXHIBIT 2 SAMPLE LONG-TERM TRENDING CHART............................................................. E2-1 ATTACHMENT 1 Revision History For IMC 0313.................................................................................. A1-1 Issue Date: 05/29/08 - ii - 0313
NRC000050 0313-01 PURPOSE The purpose of the Industry Trends Program (ITP) is to provide a means to assess whether the nuclear industry is maintaining the safety performance of operating reactors, and to identify significant trends in safety performance.
0313-02 OBJECTIVES The specific objectives of the ITP are as follows:
02.01 Collect and monitor industry-wide data that can be used to assess whether the nuclear industry is maintaining the safety performance of operating plants and to provide NRC feedback to its nuclear reactor safety inspection and licensing programs; 02.02 Assess the safety significance and causes of any statistically significant adverse industry trends, determine if the trends represent an actual degradation in overall industry safety performance, and respond appropriately to any safety issues that may be identified; 02.03 Communicate industry-level information to Congress and other stakeholders in an effective and timely manner; and 02.04 Support the NRCs performance goal of ensuring safety while enhancing openness in the agencys regulatory processes.
0313-03 APPLICABILITY This manual chapter applies to all operating commercial nuclear power reactors.
0313-04 RESPONSIBILITIES AND AUTHORITIES 04.01 Director, Office of Nuclear Reactor Regulation (NRR).
- a. Provides overall policy direction for the ITP.
- b. Directs the development and implementation of policies, programs, and procedures for the ITP.
- c. Provides oversight of program effectiveness and implementation.
04.02 Office of Nuclear Regulatory Research (RES).
Issue Date: 05/29/08 0313
NRC000050
- b. Updates and maintains the ASP database, the common-cause failure database, the Licensee Event Report database, and the Integrated Data Collection and Coding System for use by NRR, RES, and the Regions.
- c. Provides annual trends and supporting information for initiating event frequencies, component performance, and system performance in support of the ITP. The results are posted on the NRC external web page.
- d. Provides support and data as requested by the Director, NRR.
04.03 Director, Division of Inspection and Regional Support (DIRS).
- b. Oversees program implementation and effectiveness.
04.04 Chief, Performance Assessment Branch.
- a. Develops policy, programs, and procedures for implementation of the ITP.
- b. Assesses ITP effectiveness and implementation.
0313-05 BACKGROUND The NRC provides oversight of plant safety performance on a plant-specific basis using both inspection findings and plant-level performance indicators (PIs) as part of its Reactor Oversight Process (ROP). Individual issues that are identified as having generic safety significance are addressed using a number of NRC processes, including the generic communications process and the generic safety issue process. The NRC staff developed the ITP to complement these processes by monitoring and assessing industry-level trends in safety performance.
A key output of the ITP is that it provides the basis for agency reporting in the Nuclear Reactor Safety arena against the performance goal measure of the number of statistically significant adverse industry trends in safety performance, as defined by the NRC's Strategic Plan. The agency reports these results annually to Congress in the Performance and Accountability Report, Fiscal Year 200X (NUREG-1542 series). In early FY 2001, NRR assumed responsibility from RES for reporting on this measure as part of NRRs overall responsibilities in the Reactor Safety arena. The current industry indicators used for assessing performance against this measure are listed in Appendix A.
In developing the ITP, the staff used the following general concepts for its approach:
- The indicators were developed using information available from current NRC programs.
Issue Date: 05/29/08 0313
NRC000050
- Industry trend information is derived from quantitative, industry-wide data.
- Trends are identified on the basis of long-term (i.e., four or more years) data, rather than short-term data. This minimizes the impact of short-term variations in data, which may be attributable to such factors as operating cycle phase, seasonal variations, and random fluctuations.
- Trends and contributing factors are assessed for safety significance. The results of inspections, analyses of significant events and abnormal occurrences, and other analyses may be used to facilitate an evaluation of the trends. The agencys response is commensurate with the safety significance.
- The current ITP indicators (identified in Appendix A) are used for the report on adverse trends to Congress in the NRCs Performance and Accountability Report.
New ITP indicators will be qualified for use prior to being used for reporting on the performance measure of the number of statistically significant adverse industry trends in safety performance. Additional indicators for the ITP will be incorporated for use in accordance with a controlled process for making such changes to the NRCs Performance Plan. In addition, the staff intends to consider refinements to the performance measure as the indicators and more risk-informed methods of assessing their safety significance are developed.
In developing the ITP, the staff used information currently available from existing NRC programs to develop an initial set of indicators for identifying adverse industry trends.
The indicators consist of the seven indicators in the NRC indicator program, the results of the ASP program, and some indicators from the ROP PIs. As a further enhancement for the ITP, the staff developed a risk-informed indicator for initiating events. The indicator, the Baseline Risk Index for Initiating Events (BRIIE), assigns a risk importance value to each initiating event according to its relative contribution to industry core damage frequency in order to calculate an annual change in core damage frequency.
BRIIE is described further in Section 06.08 and, in detail, in Appendix D and has been incorporated into the ITP.
An NRC contractor collects and trends the ITP data and provides some analysis on an as needed basis for the NRC.
0313-06 DISCUSSION The process for collecting, analyzing, reporting, and responding to the industry trend indicators is discussed below.
06.01 Collect Indicator Data.
Data for the ITP are collected from a variety of sources, including Licensee Event Reports, Event Notifications, Monthly Operating Reports, and from the Institute of Nuclear Power Operations (INPO). Each indicator currently qualified for use in the ITP is described in Appendix A.
Issue Date: 05/29/08 0313
NRC000050 In order to ensure that revised and updated data (such as data from inspection findings and ASP results) are included in the count of Significant Events, the following process will be followed:
Prior to preparation of the annual Significant Events graphs and trend line, the Performance Assessment Branch (IPAB) will request the Operating Experience Branch in the Division of Inspection and Regional Support (DIRS) to determine and identify if there were any additional or revised significant events from previous reporting periods.
If additional or revised significant events are identified, the revised data will be included in the count of significant events for the previous reporting periods and the Significant Events graph and trend line will include and reflect the revised significant events count for the previous periods as appropriate.
Any impact of the revised significant events count on prior years conclusions related to the prediction limit will also be evaluated and addressed.
06.02 Identify Short-Term Issues.
NRR adopted a statistical approach using prediction limits to provide a consistent method to identify potential short-term emergent issues before they manifest themselves as long-term trends. The prediction limits are values established at the beginning of a FY that set an upper bound on expected performance for that year for each indicator. Actual indicator values during the year can then be monitored and compared to the prediction limits. Indicators that cross the prediction limits are investigated to determine the factors contributing to the data. These factors are assessed for their safety significance and used to determine an appropriate agency response. Should obvious adverse trends emerge in the short-term data during the middle of the year, the staff should initiate a review to determine if agency action is necessary.
A more detailed description of prediction limits is provided in Appendix B.
06.03 Identify Adverse Trends.
For purposes of assessing whether there are any statistically significant adverse industry trends, only long-term data are used. The trending of long-term data minimizes reacting to potential false positive indications that may emerge in short-term data.
Long-term was defined to be greater than or equal to four years to ensure that sufficient data (i.e., data for at least two typical nuclear plant operating cycles) are available and valid trends can be distinguished from operating cycle effects such as refueling outages and from random fluctuations in the data.
The staff applies statistical techniques to the long-term indicator data to identify trends.
These techniques have been previously adapted and used extensively by the NRC in reactor operating experience analyses over the past several years. In general terms, a trendline is fit to each indicator using appropriate regression techniques. Once a statistically significant fit of a trendline is made to each indicator, the slope of the Issue Date: 05/29/08 0313
NRC000050 trendline is examined. Improving or flat trendlines are not considered adverse and need not be investigated further. Degrading trendlines are considered adverse.
A more detailed discussion of long-term trending and the determination of a statistically significant adverse trend is discussed in Appendix C.
06.04 Analyses of Issues.
Once an adverse trend is identified, the staff conducts an initial analysis of information readily available in the databases used to compile the indicator data to determine whether the trend is unduly influenced by a small number of outliers and to identify any contributing factors. If the trend is the result of outliers, then it is not considered a trend requiring generic actions, and the agency will consider any appropriate plant-specific actions using the ROP. For example, the affected plants unduly influencing the adverse trend may have already exceeded plant-level thresholds under the ROP, and the NRC regional offices would conduct supplemental inspections at these plants to ensure the appropriate corrective actions have been taken. If the plants did not exceed any thresholds, the NRC would not take regulatory actions beyond the ROP, however, the NRC may gather additional information regarding the issue within the scope of the ROP using risk-informed baseline inspections. The results of these inspections would be examined to determine if a generic issue existed requiring additional NRC review or generic inspections.
If no outliers are identified, the staff conducts a broader review to assess whether larger groups of facilities are contributing to the decline and to assess any contributing factors and causes. For example, the data review is expanded to include a review of various plant comparison groups, contributing factors such as the operational cycle stage of the facilities (shutdown, at-power, startup from refueling, etc.), and the apparent causes for the data (equipment failures, procedure problems, etc.). The staff will also conduct a more detailed review of applicable Licensee Event Reports. Should a group of plants be identified, the staff will examine the results of previously conducted inspections at these plants, including any root causes and the extent of the conditions.
Once this information is reviewed, the staff assesses the safety significance of the underlying issues. The staff is mindful that trends in individual indicators must be considered in the larger context of their overall risk significance. For example, a hypothetical increase in automatic scrams from 0.4 to 0.7 per plant per year over several years may be a statistically significant trend in an adverse direction. However, it may not represent a significant increase in overall risk since the contribution of a small number of scrams is relatively low, and it is possible that overall risk may actually have declined if there were reductions in the frequency of more risk-significant initiating events or the reliability and availability of safety systems had improved. Depending on the issues, the staff may perform an additional evaluation using the most current risk analysis tools or an evaluation by the ASP Program.
06.05 Agency Response.
Should a statistically significant adverse trend in safety performance be identified or an indicator exceed a prediction limit, the staff will determine the appropriate response Issue Date: 05/29/08 0313
NRC000050 using the processes described above and the NRC's established processes for addressing and communicating generic issues. The generic issue process is described in SECY-99-143, Revisions to Generic Communications Program.
In general, the issues will be assigned to the appropriate branch of NRR for initial review. The branch will engage NRC senior management and initiate early interaction with the nuclear power industry. Depending on the issue, the process could include requesting industry groups such as the Nuclear Energy Institute (NEI) or various owners groups to provide utility information. As discussed in SECY-00-0116, Industry Initiatives in the Regulatory Process, industry initiatives, such as the formation of specialized working groups to address technical issues, may be used instead of, or to complement, regulatory actions. This can benefit both the NRC and the industry by identifying mutually satisfactory resolution approaches and reducing resource burdens.
Depending on the issues, the NRC may consider generic safety inspections at plants.
In addition, the issues underlying the adverse trend may also be addressed as part of the generic safety issue process by RES. The NRC may consider additional regulatory actions as appropriate, such as issuing generic correspondence to disseminate or gather information, or conducting special inspections for generic issues. The process also includes consideration of whether any actions proposed by the NRC to address the issues constitute a backfit.
06.06 Senior Management Review.
The ITP results and agency response are reviewed annually during the Agency Action Review Meeting (AARM). In general, the AARM is intended to review the appropriateness and effectiveness of staff actions already taken, rather than to make decisions on agency actions. NRC senior managers review the industry trends information and, if appropriate, recommend any additional actions beyond those implemented by the staff.
06.07 Communications With Stakeholders.
The NRC communicates overall industry performance to stakeholders by publishing the ITP indicators on the Nuclear Reactors portion of the agencys public Web site at http://www.nrc.gov/reactors/operating/oversight/industry-trends.html. The staff believes that communication of the industry-level indicators, when added to the information on individual plants from the ROP, should enhance stakeholder confidence in the efficacy of the NRCs oversight of the nuclear industry.
The staff informs the Commission of the results of the ITP in an annual report prior to the AARM. Some of the indicators are also published annually in the NRCs Information Digest 200X (NUREG-1350 series). In addition, NRC managers have historically presented industry indicators and trends at major conferences with industry and other external stakeholders.
The NRC reports industry trend information and selected indicators to Congress annually in the NRCs Performance and Accountability Report, Fiscal Year 200X (NUREG-1542 series), and in the NRCs Budget Estimates and Performance Plan Issue Date: 05/29/08 0313
NRC000050 Fiscal Year 200X (NUREG-1100 series). The indicators demonstrate how the agency has met the measure of number of statistically significant adverse industry trends in safety performance for the agency strategic goal of ensuring safety. Statistically significant adverse trends would be reported, but indicators that exceeded short-term prediction limits need not be included in these reports since these are tools to monitor industry performance rather than desired thresholds of performance.
In addition, the Commission has historically used the ITP indicators when presenting the status of industry performance to the NRCs oversight committees.
06.08 Baseline Risk Index for Initiating Events (BRIIE).
As a first step in enhancing the ITP, the staff has developed the Baseline Risk Index for Initiating Events (BRIIE), a performance indicator that provides a mechanism for determining the risk significance of changes in performance, at both the individual initiating event level and at the integrated cornerstone of safety level.
The BRIIE concept consists of a two-level process. The first level provides short-term trending information and an action point for NRC engagement. The second level provides a risk perspective of industry performance as a change from a baseline core damage frequency value.
First level, short-term monitoring (referred to as Tier 1 performance monitoring) is accomplished by monitoring yearly industry performance for 9 risk-significant initiating events for boiling-water reactors (BWRs) and 10 events for pressurized-water reactors (PWRs) (the additional event category is steam generator tube rupture) against performance-based predictions limits. These activities, similar to the process described in Appendix B for the ITP indicators, help NRC identify degrading industry performance before the emergence of any long-term adverse trends.
Second level monitoring (referred to as Tier 2 performance monitoring) is accomplished by assigning a risk importance factor to each initiating event according to its relative contribution to industry core damage frequency in order to calculate a change in Core Damage Frequency (CDF) from a baseline CDF. This indicator is reported as the Baseline Risk Index for Initiating Events (BRIIE). Annual BRIIE values are reported separately for BWRs and PWRs in addition to a combined industry value. BRIIE results may be included for information in the NRCs Performance and Accountability Report, Fiscal Year 200X (NUREG-1542 series), and in the NRCs Budget Estimates and Performance Plan Fiscal Year 200X (NUREG-1100 series). However, BRIIE results would be included in the report to Congress if the BRIIE combined industry value reaches or exceeds a threshold value of 1x10-5 per reactor critical year along with actions that have already been taken or are planned in response.
The BRIIE is intended to enhance and complement the ITP and not as a replacement.
Appendix D to this IMC contains a detailed discussion of the BRIIE and its function within the ITP. Historical results and the technical basis for BRIIE are provided in NUREG/CR-6932 (INLEXT-06-11950), Baseline Risk Index for Initiating Events (BRIIE).
Issue Date: 05/29/08 0313
NRC000050 END Appendices:
A. Indicators Qualified for Use in the ITP B. Prediction Limits C. Long-Term Trending D. Baseline Risk Index for Initiating Events (BRIIE)
Exhibits:
- 1. Sample Prediction Limit Chart
- 2. Sample Long-Term Trending Chart
Attachment:
- 1. Revision History for IMC 0313 Issue Date: 05/29/08 0313
NRC000050 APPENDIX A INDICATORS QUALIFIED FOR USE IN THE ITP Automatic Scrams While Critical Definition: The number of unplanned automatic scrams that occurred while the affected reactor was critical.
Data Source: Scram data are primarily derived from 10 CFR 50.73, Licensee Event Report (LER), information and is supplemented as necessary from 10 CFR 50.72, Immediate Notification, reports.
==
Description:==
This indicator monitors the number of unplanned automatic scrams that occurred while the affected reactor was critical. Examples of the types of scrams included in this indicator are those that resulted from unplanned transients, equipment failures, spurious signals, or human error. Also included are those that occurred during the execution of procedures in which there was a high chance of a scram occurring, but the occurrence of a scram was not planned. The reactor was "critical" if the report so states. Otherwise, criticality is determined from a detailed review of the other operational information.
Safety System Actuations (SSA)
Definition: Safety system actuations are manual or automatic actuations of the logic or equipment of either certain Emergency Core Cooling Systems (ECCS) or, in response to an actual low voltage on a vital bus, the Emergency AC Power System.
Data Source: Input for this indicator is derived from LERs and supplemented by 10 CFR 50.72 reports.
==
Description:==
In determining which events should be counted by this indicator, the following conventions are used:
- 1. Only actuations of the High Pressure Injection System, Low Pressure Injection System, or Safety Injection Tanks are counted for pressurized water reactors (PWRs). For boiling water reactors (BWRs), only actuations of the High Pressure Coolant Injection System, Low Pressure Coolant Injection System, High Pressure Core Spray System, or Low Pressure Core Spray System are counted. Actuations of the Reactor Core Isolation Cooling System are not counted.
- 2. Actuations of Emergency AC Power Systems are counted only if they were in response to an actual low voltage condition on a vital bus. Specifically, actuations are counted only if the Emergency AC Power System's output breaker closed, or should have closed, to power a dead bus. Actuations resulting from momentary Issue Date: 05/29/08 A-1 0313
NRC000050 low voltage conditions that do not result in emergency output breaker closure are not counted.
- 3. Logic actuations of any of the equipment associated with the specific ECCS or Emergency AC Power System are considered necessary and sufficient to constitute a data count. For example, if only a valve in a system is commanded to move to its emergency operational position, this is counted as an actuation. A pump does not have to be commanded to go to its emergency mode of operation and fluid does not need to be injected for an occurrence to be counted.
- 4. Only one ECCS actuation is counted in any one occurrence, even if multiple ECCS systems actuate during the occurrence. For example, actuation of both the High Pressure Injection and the Low Pressure Injection Systems at a PWR during the same occurrence counts as only a single ECCS actuation.
- 5. Only one Emergency AC Power System actuation is counted in any occurrence, even if multiple emergency generators actuate during the occurrence. For example, actuation of all four emergency diesel generators (EDGs) at a unit counts as only a single actuation for that occurrence.
- 6. Occurrences involving actuations of both an Emergency AC Power System to power a dead bus and an ECCS are given a count of two, one for the Emergency AC Power System actuation and one for the ECCS actuation.
- 7. At multi-unit sites that share equipment (e.g., a swing EDG or shared buses),
actuations are counted and assigned to the unit at which the actuation signal or loss of power originated. If the signal source cannot be associated with one unit, the actuation is assigned to both units.
Significant Events Definition: Significant Events are defined as
- 1. A Yellow or Red Reactor Oversight Process (ROP) finding or performance indicator
- 2. An event with a Conditional Core Damage Probability (CCDP) or increase in core damage probability (CDP) of 1x10-5 or higher
- 3. An Abnormal Occurrence as defined by Management Directive 8.1, Abnormal Occurrence Reporting Procedure
- 4. An event rated two or higher on the International Nuclear Event Scale Data Source: The screening process includes the daily review and discussion of all reported operating reactor events, inspection report findings, as well as other operational data.
Issue Date: 05/29/08 A-2 0313
NRC000050
==
Description:==
Significant events are those events identified by NRC staff through detailed screening and evaluation of operating experience.
Safety System Failures (SSF)
Definition: Safety system failures are any events or conditions that could prevent the fulfillment of the safety function of structures or systems.
Data Source: Input for this indicator is derived from LERs and supplemented by 10 CFR 50.72 reports.
==
Description:==
If a system consists of multiple redundant subsystems or trains, failure of all trains constitutes a safety system failure. Failure of one of two or more trains is not counted as a safety system failure. The definition for the indicator parallels but is not identical to NRC reporting requirements in 10 CFR 50.72 and 10 CFR 50.73.
Forced Outage Rate (FOR)
Definition: The forced outage rate is the number of forced outage hours divided by the sum of unit service hours and forced outage hours.
Data Source: Monthly Operating Reports or data provided by the Institute of Nuclear Power Operations (INPO) as obtained from licensees through their consolidated data entry (CDE) process.
==
Description:==
Forced outages are those required to be initiated no later than the end of the weekend following the discovery of an off-normal condition. Unit service hours are the hours that a generator is on-line.
Equipment Forced Outages per 1000 Commercial Critical Hours (EFO)
Definition: This indicator is the number of forced outages caused by equipment failures per 1000 critical hours of commercial reactor operation.
Data Source: Monthly Operating Reports or data provided by INPO (as obtained from licensees through CDE).
==
Description:==
It is the inverse of the mean time between forced outages caused by equipment failures. The inverse number was adopted to facilitate calculation and display.
Collective Radiation Exposure (CRE)
Definition: This indicator is the total radiation dose accumulated by unit personnel.
Issue Date: 05/29/08 A-3 0313
NRC000050 Data Source: The radiation exposure data are obtained from INPO.
==
Description:==
Prior to the third quarter of 1992, values at multi-unit sites were reported as site averages, with the exception of the Indian Point and Millstone sites which reported individual unit values. Beginning with the third quarter of 1992, some multi-unit sites reported site average values, while other multi-unit sites reported individual unit values. For industry level indicators, reporting in either manner is acceptable.
Accident Sequence Precursors (ASP)
Definition: Accident sequence precursors are events with a conditional core damage probability (CCDP) or increase in core damage probability (CDP) that is greater than or equal to 1x10-6. A significant accident sequence precursor is an event with a CCDP or CDP greater than or equal to 1x10-3.
Data Source: To identify potential precursors, NRC staff reviews plant events from LERs, inspection reports, and special requests from NRC staff.
==
Description:==
RES is responsible for the ASP program and provides the data and analysis for the Industry Trends Program. Only long-term trending is provided for ASP.
Unplanned Power Changes Definition: Total unplanned power changes at all plants each year multiplied by 7000 hrs, and then divided by the total critical hours for all plants each year.
Data Source: ROP PI data submittals from licensees.
==
Description:==
This indicator is based on data submitted by licensees to support the ROP.
The definition of an unplanned power change is contained in the latest version of NEI 99-02, Regulatory Assessment Performance Indicator Guideline.
Reactor Coolant System (RCS) Specific Activity Definition: Sum of maximum percentage of Technical Specification RCS specific activity within each year at all plants, divided by the total number of plants with data.
Data Source: ROP PI data submittals from licensees.
==
Description:==
This indicator is based on data submitted by licensees to support the ROP.
The requirements for calculating the maximum percentage of RCS specific activity are contained in the latest version of NEI 99-02, Regulatory Assessment Performance Indicator Guideline.
Issue Date: 05/29/08 A-4 0313
NRC000050 Reactor Coolant System Leakage Definition: Sum of maximum percentage of Technical Specification RCS leakage within each year at all plants, divided by the total number of plants with data.
Data Source: ROP PI data submittals from licensees.
==
Description:==
This indicator is based on data submitted by licensees to support the ROP.
The requirements for calculating the maximum percentage of RCS leakage are contained in the latest version of NEI 99-02, Regulatory Assessment Performance Indicator Guideline.
Drill/Exercise Performance Definition: Total number of classifications at all plants each year multiplied by 100, and then divided by the total number of classification opportunities at all plants each year.
Data Source: ROP PI data submittals from licensees.
==
Description:==
This indicator is based on data submitted by licensees to support the ROP.
The requirements for determining the number of classifications and opportunities are contained in the latest version of NEI 99-02, Regulatory Assessment Performance Indicator Guideline.
Emergency Response Organization (ERO) Drill Participation Definition: Total number of key ERO members participating in drills at all plants each year multiplied by 100, and then divided by the total number of key ERO members at all plants each year.
Data Source: ROP PI data submittals from licensees.
==
Description:==
This indicator is based on data submitted by licensees to support the ROP.
The requirements for determining the number of key ERO members are contained in the latest version of NEI 99-02, Regulatory Assessment Performance Indicator Guideline.
Alert and Notification System Reliability Definition: Total number of successful alert and notification system tests at all plants each year multiplied by 100, and then divided by the total number of tests at all plants each year.
Data Source: ROP PI data submittals from licensees.
Issue Date: 05/29/08 A-5 0313
NRC000050
==
Description:==
This indicator is based on data submitted by licensees to support the ROP.
The requirements for determining the number of tests are contained in the latest version of NEI 99-02, Regulatory Assessment Performance Indicator Guideline.
Baseline Risk Index for Initiating Events (BRIIE)
Definition: A performance indicator that provides a mechanism for determining the risk significance of changes in performance, at both the individual initiating event level and at the integrated cornerstone of safety level by combining frequent and infrequent initiating event category frequencies with different risk measures (i.e., Birnbaum importance measures).
Data Source: Input for this indicator is derived from LERs and supplemented by 10 CFR 50.72 reports.
==
Description:==
The indicator is calculated as follows:
m BRIIE = B i ( ic* ib ),
i =1 where B i = industry - average Birnbaum for initiatin g event i ic* = common industry current frequency for initiating event i ib = baseline frequency for initia ting event i m = number of initiatin g events covered in the BRIIE The BRIIE formulation above uses PWR- or BWR-average Birnbaum importance measures and combines the industry-wide data to generate the common industry current frequency for each initiating event category. (See Appendix D for additional detail)
Issue Date: 05/29/08 A-6 0313
NRC000050 APPENDIX B PREDICTION LIMITS To develop prediction limits, two steps are required. The first is to establish a baseline period for each indicator, during which the data can be regarded as fairly constant.
The second step is to select an appropriate constant model for the data in the baseline period. The prediction limit is based on this constant model and corresponds to a selected probability that future data points will not exceed the prediction limit.
Baseline Period and Prediction Limits For each indicator considered, a baseline period must be established. The baseline period data are then used as input to the predictive limits analysis.
To guide the determination of the baseline period, the following characteristics were identified:
- The baseline period is representative of current industry performance.
- The baseline period is long enough to give a good estimate of the frequency, and is not strongly influenced by random variation.
- The baseline period is limited in length to ensure that the true frequency is approximately constant.
Because a long enough period is needed to give a good estimate, it was decided that every baseline period should contain at least four years. For each indicator, the history is examined back to the earliest year of data, FY 1988. Candidate baseline periods are analyzed for successively shorter time periods down to the minimum of four years. For each candidate baseline period, a trend model is fit to the data. That is, the slope of the trend model is estimated from the data for the corresponding period. A statistical test is then performed to see whether the slope could be equal to zero (a slope equal to zero would reflect a frequency that is constant). The statistical significance of this test is a probability (called a p-value). If the significance of the test is less than 0.05, then the slope is treated as being different from zero.
In this way, a significance value is calculated for each candidate baseline period year.
The closer the significance value is to 1.0, the more assurance one has that the slope is zero. The baseline period with the largest significance value and with reasonable mean value is selected as the baseline period. The baseline period is selected to balance the other competing criteria described above.
Using information from the baseline period (e.g., number of occurrences and reactor operating or calendar years) and an estimate of the reactor operating or calendar time for the next year, a predictive distribution is determined. This distribution is derived on Issue Date: 05/29/08 B-1 0313
NRC000050 the assumption that it will predict future performance subject only to random fluctuations in the data. The 95% percentile of this distribution is the prediction limit. Random fluctuations should only exceed this value about 5% of the time based on current conditions. However, if those conditions should degrade, then the prediction limit could be exceeded more often. This situation would indicate that further investigation is warranted. See Exhibit 1 for an example prediction limit chart.
When such a period and its associated statistical model have been established, the prediction limit threshold does not change from one year to the next unless a clear change has been noted in the performance of an indicator. Such a change would need to be both noticeable and persistent. Thus, data from a single year would not be sufficient to change a model and its prediction limit.
Issue Date: 05/29/08 B-2 0313
NRC000050 APPENDIX C LONG-TERM TRENDING As mentioned in the main body of this Manual Chapter, one objective of the Industry Trends Program (ITP) is to determine the number of statistically significant adverse trends in industry performance. The ITP uses long-term trending of the data to report against this performance measure.
The NRC has been collecting and trending industry level data since at least 1987. The ITP previously established long-term trends based on data back through FY 1988.
When most indicators were continuing to show improving performance, trending from 1988 worked well. However, continuing to display trends driven largely by the 1987 to mid-1990s data is no longer valid for representing recent industry performance.
In order to ensure current performance trends are not masked by the use of historical data, the NRC has removed some historical data from consideration during the trending analysis. The NRC now uses a 10-year rolling period to establish long term trends. A 10-year period was chosen to ensure enough data was available for meaningful statistical analysis. See Exhibit 2 for an example long-term trending chart.
Using a 10-year period through the current years data, a contractor develops trend lines using statistical regression techniques. Statistical goodness of fit tests are then conducted to determine which trend line model best fits the data. Statistical analysis is performed to determine the significance of the trend. Confidence levels are used in a manner very similar to that discussed in Appendix B. If the confidence level in a trend is 95% or greater, there is strong evidence that the trend is not due to random data fluctuation. At this point, the trend is considered statistically significant.
In order for a trend to be considered a statistically significant adverse trend, the slope of the trend line model has to be in the adverse direction (performance degrading) and statistical analysis has to determine that the trend is statistically significant.
Issue Date: 05/29/08 C-1 0313
NRC000050 APPENDIX D BASELINE RISK INDEX FOR INITIATING EVENTS (BRIIE)
As a first step in enhancing the ITP, the staff has developed the Baseline Risk Index for Initiating Events (BRIIE), a performance indicator that provides a mechanism for determining the risk significance of changes in performance, at both the individual initiating event level and at the integrated cornerstone of safety level.
A. Enhancement Process for Initiating Events Cornerstone of Safety A three-step process is used to enhance the ITP coverage of the Initiating Events Cornerstone of Safety:
Step 1 - identify appropriate risk-significant categories of initiating events Step 2 - trend and establish performance-based prediction limits for these individual event categories (referred to as Tier 1 performance monitoring)
Step 3 - calculate an integrated, risk-informed indicator by assigning a risk importance factor to each initiating event category according to its relative contribution to industry core damage frequency (CDF) in order to calculate a change in CDF (CDF) from a baseline CDF (referred to as Tier 2 performance monitoring)
Step 1 - Identification of Risk-Significant Initiating Events The list of risk-significant initiating event types consists of 10 initiating event categories applicable to pressurized-water reactors (PWRs) and 9 applicable to boiling-water reactors (BWRs) as listed below in Table 1:
Table 1. Risk-significant initiating event categories covered by the BRIIE Pressurized Water Reactors (PWRs) Boiling Water Reactors (BWRs)
- 5. Very small loss of coolant accident (VSLOCA) 5. Very small loss of coolant accident (VSLOCA)
- 10. Steam generator tube rupture (SGTR)
In general, these risk-significant initiating event types, cover approximately 60% of the internal event core damage risk (excluding internal flooding) from the operating commercial nuclear power plants in the United States. Also, these initiating events do not overlap.
Issue Date: 05/29/08 D1-1 0313
NRC000050 Step 2 - Performance Monitoring of Risk-Significant Initiating Event Categories (Tier 1 Performance Monitoring)
Tier 1 performance monitoring activity consists of trending risk-significant initiating event categories and monitoring yearly industry performance against prediction limits. To accomplish this, the staff established up-to-date baseline frequencies for each of the risk-significant initiating event categories, and then determined performance-based prediction limits using these baseline frequencies and estimated yearly industry-total critical-reactor years of operation. The process is similar to the prediction limit process described in Appendix B for the ITP indicators. An example is presented in Figure 1.
These Tier 1 activities are intended to help NRC identify degrading industry performance as an adjunct to the plant-specific performance assessment performed as part of the ROP. Potential NRC responses if one or more of the prediction limits are reached or exceeded are outlined in Section E of this appendix. Also, example scenarios are presented in Section E for illustrative purposes. Tier 1 activities and results are not reported to Congress but are used by NRC as a diagnostic tool to identify degrading industry performance before the emergence of any long-term adverse trends.
The Tier 1 results will be placed on the NRC Web site for access by interested stakeholders.
Figure 1 Baseline period: CY1998-2002 Rate for PWR general transients Baseline period maximum likelihood estimate (n/T) 95% prediction limit 3.2 Events per reactor critical year 2.4 1.6 0.8 0.0 19 19 19 1 1 1 1 1 1 1 1 1 2 2 2 2 2 8 8 89 90 991 992 993 994 995 996 997 998 999 000 001 002 003 004 Baseline log model p-value = 0.26 Fiscal Year PQ-11-Jan-2006 Step 3 - Risk-Informed Monitoring of Initiating Events Cornerstone of Safety (Tier 2 Performance Monitoring Activity)
Tier 2 performance monitoring provides an evaluation of the risk significance of changes in industry initiating event category performance in an integrated, risk-informed performance indicator -- the Baseline Risk Index for Initiating Events (BRIIE).
Issue Date: 05/29/08 D1-2 0313
NRC000050 Risk significance is evaluated in terms of CDF. This indicator combines operating experience for risk-significant initiating event categories with information associated with internal event, CDF-based importance. Although Tier 2 does not provide strict trending information, it does provide an annual risk perspective of industry performance as a deviation from a baseline value and the proximity of the deviation from a set threshold.
Using BRIIE, the staff is able to appropriately combine frequent and infrequent initiating event category frequencies with different risk measures (i.e., Birnbaum importance measures). The main use of the BRIIE is to combine individual initiating event category performance changes into an integrated, risk-informed indicator at the Initiating Events Cornerstone of Safety level. The BRIIE solves several deficiencies in the present ITP:
(1) no systematic and defined method for determining whether individual initiating event category performance changes or adverse trends are risk significant, (2) no systematic and defined method for integrating individual initiating event category performance changes into an overall risk result at the Cornerstone of Safety level, and (3) untimely risk-informed industry trend results. The staff uses the results of the BRIIE in the ITP, along with the other qualified indicators, when reporting the number of adverse trends to Congress.
B. BRIIE Calculation The quantification method used for formulating the related changes in CDF (CDF) is given by the following:
m BRIIE = B (
i =1 i
ic ib ),
where B i = industry - average Birnbaum for initiatin g event i
- ic = common industry current frequ ency for initiating event i ib = baseline frequency for initia ting event i m = number of initiatin g events covered in the BRIIE (1)
BWRs and PWRs have different core damage frequencies, which depend to some extent on different initiating event types. The risk weights for various initiating events also are different for the two types of reactors. Therefore, BRIIE results are provided for each reactor type and the two BRIIE results are also combined into a single index that provides an indication of overall industry performance.
The BRIIE formulation in Equation 1 uses PWR- or BWR-average Birnbaum importance measures and combines the industry-wide data to generate the common industry current frequency for each initiating event category.
In order to ensure that the BRIIE indicator reflects current industry performance, industry performance for components and initiating events will be reviewed and baseline CDF will be recalculated periodically. The industry-average Birnbaum importance measure for each initiating event category also will be recalculated periodically.
Issue Date: 05/29/08 D1-3 0313
NRC000050 C. Reporting Thresholds BRIIE results, although representing industry-wide results, are presented as average results per plant. The PWR-wide impact is (PWR BRIIE result per PWR) x (total number of PWRs). Similarly, the BWR-wide impact is (BWR BRIIE result per BWR)x (total number of BWRs).
These industry-wide impacts were considered in establishing reporting thresholds for the BRIIE, together with the following information:
I. Uncertainty in the BRIIEs and the 95% and 99% prediction limits from simulations II. Distribution of the Birnbaum importance measures for each initiating event category and understanding of the groups of plants that have large values for each category III. Major contributors (i.e., dominant initiating event categories) to the BRIIEs IV. Sensitivity of BRIIEs to initiating event categories, especially those with lower frequencies V. Other factors, such as the NRC safety goal policy and Regulatory Guide 1.174.
An expert panel was convened in July 2006 with the objective of reviewing the BRIIE and establishing a threshold value for reporting to Congress. The panel reached the following conclusions:
- 1. The two-level process for BRIIE provides an accurate and full picture of industry performance. The two-level process provides trending information and action level for NRC engagement if the 95% prediction limit of Tier 1 is reached, as detailed in Appendix B to this NRC Inspection Manual Chapter. Tier 2 provides an annual risk perspective of industry performance as a deviation from a baseline value and the proximity of the deviation from a set threshold.
- 2. The presentation for BRIIE should be in a bar graph that provides three separate annual values: one bar providing industry-wide results, one bar for BWR results, and the third bar for PWRs. All three bars for each will be presented on one graph. This presentation provides more information than simply aggregating industry-wide results into one number or presenting BWRs and PWRs individually.
- 3. The BRIIE should be in the form of CDF. The absolute CDF form will be calculated and be available, if requested, as a communication tool to provide additional insights to interested stakeholders. The CDF form of BRIIE is preferred because the absolute CDF form of BRIIE would result in different values for BWRs and PWRs. The CDF form shows the change Issue Date: 05/29/08 D1-4 0313
NRC000050 from the baseline for both types of reactors and hence is more understandable. Infrequently occurring initiating events such as Loss of Service Water, Loss of Component Cooling Water, Small Loss of Coolant Accident, and others would not be included in the calculation of BRIIE.
These events, if they occur, would be captured in the calculation of absolute CDF.
- 4. The threshold for reporting BRIIE results to Congress should be set at 1x10-5 per reactor critical year (rcry). It should be associated only with the CDF BRIIE calculations for industry-wide results. This threshold value was arrived at from considerations of the NRC safety goals, RG 1.174, and consistency with the ROP and ASP programs. The threshold was derived from coherency with current agency metrics and the surrogates for the safety goals discussed in RG 1.174. Two scenarios were discussed.
The first was that a single event at 1x10-3 /rcry CDF (e.g., the current ASP indicator threshold for reporting significant events to Congress) would make the aggregate industry performance about equal to 1x10-5 /rcry CDF (1x10-3 divided by 100 plants). Also, if 10% of plants had a problem at about 1x10-4 /rcry CDF, then this would also make industry performance about equal to 1x10-5 /rcry CDF. The 10% number was chosen to provide a distinction between an industry problem in contrast to issues with individual plants.
D. BRIIE Historical Performance Figure 2 provides a representative comparison of industry BRIIE results for 1988 through 2005 with the PWR and BWR plant type BRIIE values calculated with a LOOP frequency that excludes the August 2003 LOOP events. As already stated, the baseline CDF value will be recalculated periodically to reflect ongoing industry performance. It should be noted that values below the 0.0E0 line indicate that the industry/PWRs/BWRs performed better than the baseline.
E. Potential Regulatory Responses to BRIIE Results In this section we present two examples to show how the enhanced ITP might treat initiating event performance changes.
As a first example, suppose we observe four events in one year that are classified as very small break LOCAs (VSLOCA), and each event occurred at a different plant. A VSLOCA as an initiating event is rare. The 95% prediction limit (used in the Tier 1 analysis) is two events. Therefore, we have exceeded the 95% prediction limit.
Because the number of actual events exceeds the prediction limit, this initiating event is a candidate for further investigation.
Because VSLOCAs do not occur very often, NRC would examine and review each event in more detail after it occurred. The ITP would look at these events to see if there were similarities among the events and to provide any lessons learned from this Issue Date: 05/29/08 D1-5 0313
NRC000050 evaluation. These lessons would be communicated to the industry via appropriate generic communication. Further regulatory action would be taken as necessary based on the results of a detailed NRC inspection of each event.
If all of these events had occurred at PWRs, the PWR CDF-BRIIE would be significantly below the 1x10-5 CDF threshold. The plant-type average Birnbaum importance measure for the PWR VSLOCA is the smallest (least important) of the Birnbaum importance measures. This means that a much higher count of events would have to occur before the average PWR would exceed its BRIIE threshold or to have any significant impact in the composite industry CDF BRIIE value. Therefore, since neither the PWR nor industry BRIIE reporting threshold is exceeded, NRC would not make a report to Congress. However, the staff would analyze and take appropriate action in response to these events.
Since the threshold for reporting to Congress is associated with the CDF BRIIE calculations for industry-wide results, the NRC would also not make a report if the PWR CDF BRIIE value would exceed the 1x10-5 threshold but the industry average CDF BRIIE value does not.
In a second example, there is a marked increase in the number of BWR loss of DC (LODC) events, resulting in seven LODC events for the year. This exceeds the 95%
prediction limit for Tier 1, which is three events for all plants. As such, this initiating event is a candidate for further investigation and NRC engagement. NRC would examine and review each event in more detail as it occurred to see if there were similarities among the events and to provide any lessons learned from the evaluation.
These lessons would be communicated to the industry via appropriate generic communication. Further regulatory action would be taken as necessary based on the results of a detailed NRC inspection of each event.
However, unlike in the previous case, both the BWR CDF BRIIE and the industry CDF BRIIE would exceed 1x10-5 for this hypothetical case and NRC would report this result to Congress along with actions that have already been taken or are planned in response to these events.
Issue Date: 05/29/08 D1-6 0313
NRC000050 Figure 2. Baseline Risk Index for Initiating Events 1988 - 2005 1.0E-05 L ~ E- 05
, L OE-05 u
0
'i
- t ~.0E- 06 B
~ ., 1j11h III In O.OE-+OO r IJ ROUttJ '" 'ul IJl r
_ L OE-05 1988 1989 1990 199 1 1991 1993 1994 1995 1996 1997 1998 1999 1000 1001 1 001 1003 1004 l00~
I_ I ndustry D BWR D P\VR 1 Issue Date: 05/29/08 D1-7 0313
NRC000050 EXHIBIT 1 SAMPLE PREDICTION LIMIT CHART The chart below is for information only.
Automatic Scrams While Critical 3
Industry Average 2.41 Prediction Limit (0.680)
Baseline Period Industry Average (0.534) 2 Scrams per Plant 1.87 1.61 1.57 1.52 1.18 1.05 1.03 1
0.80 0.75 0.64 0.54 0.57 0.56 0.48 0.52 0.44 0
1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 Fiscal Year Issue Date: 05/29/08 E1-1 0313
NRC000050 EXHIBIT 2 SAMPLE LONG-TERM TRENDING CHART The chart below is for information only.
Collective Radiation Exposure 600 Average Exposure per Plant Industry Average Industry Average Trend 450 300 y = 199.98e -0.0692x (Person-REM) 202 R2 = 0.8431 178 176 140 128 150 115 123 111 125 102 0
1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 Fiscal Year Issue Date: 05/29/08 E2-1 0313
NRC000050 Revision History for IMC 0313 Commitment Issue Date Description of the Change Training Training Comment Tracking Required Completed Resolution Number Accession Number NA 05/29/08 Added Appendix D and description of NA NA ML081430600 CN 08-016 Baseline Risk Index for Initiating Events (BRIIE) throughout text of Manual Chapter Revised Section 06.01 to describe a process to ensure that revised and updated data (such as data from inspection findings and ASP results) are included in the count of Significant Events Issue Date: 05/29/08 Att 1-1 0313