ML103620379

Gap Analysis of the Reactor Oversight Process (ROP)

ML103620379
Person / Time
Issue date:	12/28/2010
From:	Office of Nuclear Reactor Regulation
To:
Vaughn, Stephen, NRR/DIRS/IPAB 415-3640
Shared Package
ML103620319	List: ML103620349 ML103620379
References
Download: ML103620379 (11)

Text

Gap Analysis of the Reactor Oversight Process (ROP)

Purpose: Review the capability of the Performance Indicator (PI) Program in concert with the Reactor Inspection Program to ensure adequate performance insights in each safety cornerstones key safety attributes as defined in the ROP basis documentation.

Background:

On April 21st, 2010 the NRC held a category 2 public meeting to discuss a framework in considering new PIs. The initial approach focused on potential new PIs that would supplement the existing suite of indicators.

While this approach has merit, the NRC and industry agreed to hold off on dedicating resources to consider any new PIs without a well defined problem statement. As a result, the NRC agreed to perform a gap analysis with the goal of revealing potential areas of the ROP that warrant additional oversight through inspection or PIs. The results of this gap analysis could be considered for the biennial ROP Realignment currently in progress.

Process: The following steps were used for the gap analysis:

1) Review recommended future PIs stated in SECY 99-007 that were never designed and/or implemented.

2) Review recent sources of information on ROP implementation (e.g.,

annual ROP self assessments, feedback forms, action matrix deviations, task force reports, industry initiatives) to see if any potential gaps are revealed.

3) Evaluate the reviews from steps 1) and 2).

4) Make recommendations on the evaluations made in step 3) and categorize each recommendation as either:

a) Pursue Now b) Defer c) Not Currently Viable

5) Summarize the Gap Analysis.

1

INITIATING EVENTS

1. Shutdown Safety Margin

Description:

Shutdown safety margin was recommended as a potential PI in SECY 99-007. This indicator is designed to track the number of unplanned reductions in the safety margin for (1) reactor coolant inventory, (2) reactor coolant temperature, and (3) reactivity while the plant is shut down. Overall, this indicator would count the events that jeopardize the capability to remove decay heat from the reactor while shut down or could lead to unplanned criticality.

Experience has shown that plant activities while shut down with safety equipment out of service can, under certain circumstances, have serious consequences. It is important that reactor coolant level and temperature be controlled to maintain the heat removal capability and to prevent inadvertent criticality.

Evaluation: Creating a shutdown safety margin PI was explored in NUREG-1753 Risk-Based Performance Indicators (RBPIs). One of the major challenges identified, which still applies, was that initiating events do not accumulate statistical data quickly enough to support timely detection of declining performance. As a result, it is too difficult to create an initiating event PI for shutdown operations.

Recommendation: Not Currently Viable as a risk-based PI. Instead of developing a shutdown safety margin PI, the inspection program should continue to cover this area by sampling licensee performance at shutdown. The resultant research and data from NUREG-1753 could be used to support the ongoing development of significance determination process (SDP) tools for low-power and shutdown operations.

MITIGATING SYSTEMS

2. Shutdown Operations Performance

Description:

Shutdown operations performance was recommended as a potential PI in SECY 99-007. This indicator is designed to measure the percent of outage time that defense-in-depth was compromised. Most licensees manage shutdown risk in accordance with NUMARC 91-06, "Guidelines for Industry Actions to Assess Shutdown Management." They manage defense in depth, through configuration control, for key safety functions (decay heat removal, inventory control, electrical power availability, reactivity control and containment).

This PI would measure the percent of outage time that each key safety function lacked defense in depth, either from installed equipment or contingency actions.

2

Evaluation: Creating a shutdown operations performance PI was explored in NUREG-1753 Risk-Based Performance Indicators (RBPIs) and the following challenges were identified and still currently apply:

• Because risk changes from configuration management are detectable in short time intervals, monitoring shutdown risk has an episodic, SDP character, rather than a longer-term, trending-indicator character.

• Development of a baseline would require characterizing a nominal outage.

• Operational exigencies drive variations in shutdown risk that are substantially greater than the risk changes that are associated with PI thresholds in the ROP. PIs that measure changes in shutdown risk would therefore capture influences whose relationship to licensee performance in configuration management is indirect. This could potentially lead to unintended consequences.

Recommendation: Not Currently Viable as a risk-based PI. However, deterministic, qualitative considerations could be applied to establish thresholds for performance bands. Technical basis for those thresholds would need to be developed. Alternatively, the inspection program can continue to cover this area by sampling licensee performance at shutdown. The resultant research and data from NUREG-1753 could be used to support the ongoing development of significance determination process (SDP) tools for low-power and shutdown operations.

3. Maintenance Rule Implementation

Description:

Maintenance Rule implementation was recommended as a potential PI (or PIs) in SECY 99-007. One option could be a PI that indicates changes in cumulative core damage frequency (CCDF) based on the changes resulting from on-line and shutdown safety assessments (threshold values would be plant-specific). Another option might be a PI that indicates structures, systems, and components (SSCs) that either remain in an (a)(1) status for long periods or that have entered into an (a)(1) status on multiple occasions over relatively short duration.

Evaluation: The performance based nature of the Maintenance Rule (10 CFR 50.65) allows a diverse set of maintenance programs to exist throughout the industry. Although most licensees use internal performance metrics to manage their maintenance effectiveness programs, it may be difficult to establish a standardized indicator since each site or utility can create a program that is slightly different but still meets the intent of the Maintenance Rule.

3

In addition, the number of SSCs in (a)(1) is not necessarily an indication of declining performance in the area of maintenance effectiveness. An evaluation of the SSCs in (a)(1) would need to be performed on a case-by-case basis to detect a declining performance trend.

Recommendation: Not Currently Viable. The industry uses their own specific internal maintenance metrics to verify safety and non-safety significant SSCs are able to perform their intended function. Standardizing these metrics would be an extremely difficult and large resource effort and there is no clear indication that a Maintenance Rule PI would complement current inspection efforts. Since the inspection program adequately covers the key safety attributes and is flexible enough to adapt to minor differences among licensees in their implementation of the maintenance rule, a PI is not needed.

4. Passive Systems (New Reactor Design)

Description:

Defer. Passive systems in new reactor designs are likely to have PIs dissimilar to the current PIs used to assess the operating power reactor fleet.

As a result, new possibilities for passive system PIs should be explored.

Evaluation: Currently the evaluation of potential passive system PIs is on hold until further work can be completed on new reactor risk metrics, assessment, and oversight activities.

Recommendation: Any recommendations are deferred until further work can be completed on new reactor oversight.

BARRIER INTEGRITY

5. Reactor Coolant System (RCS) pressure boundary leak(s) frequency

Description:

The frequency of RCS pressure boundary leak(s) was recommended as a potential PI in SECY 99-007. This indicator would monitor the frequency of pressure boundary leaks as defined by technical specifications (excluding steam generator tubes).

Evaluation: The RCS leakage rate described as a percentage of technical specifications is currently used as a PI. The frequency of RCS leakage events PI would attempt to capture a slightly different data set (i.e., the number of RCS leakage events) involving the same key safety attribute. Since the overall RCS leakage rate PI is a percentage of technical specification limits, which is an 4

inspectable area, a PI that would monitor the frequency of RCS leaks would not add much value.

Recommendation: Not Currently Viable. Another PI that monitors the frequency of RCS leaks would not add substantial value to the Barrier Integrity cornerstone since the current PI (RCS leakage rate) already addresses the same key safety attribute.

6. Reactor Coolant System (RCS) individual in-service inspections (ISIs)

Description:

RCS individual in-service inspection(s) was recommended as a potential PI in SECY 99-007. This indicator would monitor the percentage of individual in-service inspections that require disposition against ASME standards.

Such an indicator can be objectively derived and a threshold set that is related to historically good industry performance. By using a percentage indicator, instead of an absolute number indicator, it is less likely to influence the assessment of non-destructive examination (NDE) examiners as the number count of flaw indications increases. Verification and validation of this indicator would likely include ensuring that industry operating experience is being applied to the selection of areas for NDE.

Evaluation: Monitoring the percentage of ISIs that would require disposition against ASME standards would augment performance insights in a key safety attribute that currently does not have an indicator. However, counting the number of times that an ISI requires an ASME standard disposition does not necessarily reflect declining performance. The inspection program currently samples in-service inspection activities every refueling outage and is capable of evaluating the details of risk-significant inspections on a case-by-case basis.

Recommendation: Not Currently Viable. Although a PI that monitors RCS ISIs that required an ASME standard disposition would add coverage to a key safety attribute that does not currently have a PI, the inspection program provides a sufficient level of effort to ensure the safety cornerstone objective are met.

7. Containment Leakage

Description:

In SECY 99-007 a containment leakage PI was defined and used during the pilot phase of the ROP. The indicator was designed to estimate the "as-found" integrated leak rate for the containment, provide a reasonable indication of effluents during operation, and provide an indication of the leak-tight integrity of the containment barrier. Measurement data would be based on the last integrated leak rate test result, informed by the results of subsequent local leak rate tests. The data would be reported as a fraction of the design basis leak 5

rate (La). Licensees currently collect these data as required by 10 CFR Part 50 Appendix J.

Evaluation: Two major limitations with this indicator were identified and still apply:

• "As-found" leak rate data are not collected in a consistent manner at all plants. Specifically, some plants perform the Type C tests at the end rather than at the beginning of the refueling outage. The leak rate data for those plants may not reflect the actual leak rate that existed during power operation, particularly if the isolation valves are cycled during the outage. Some changes to licensee practices would be needed to achieve consistency.

• The data obtained from integrated and local leak rate tests is gathered relatively infrequently. In accordance with Appendix J, licensees are required to perform integrated leak tests (Type A tests) on a frequency of 3 tests every 10 years, and to leak-test Type B and Type C components during each reactor shutdown for refueling, but in no case at intervals greater than 2 years. Licensees adopting Option B of Appendix J can extend the integrated leak test frequency to one test every 10 years, and extend the test interval up to 60 months for Type B penetrations (except personnel airlocks) and Type C components (except main steam and feedwater isolation valves in BWRs, and containment purge and vent valves in PWRs and BWRs). The extended test interval for those excepted components would be limited to 30 months. Thus, depending on the licensees test program, updates to the performance indicator would occur on an infrequent basis.

Recommendation: Not Currently Viable. Since each licensee has a variety of options for meeting the requirements of 10 CFR Part 50 Appendix J, creating a standardized metric is not practicable. The containment portion of the Barrier Integrity cornerstone should continue to be monitored under the inspection program to account for the various design and operational differences among the fleet.

PUBLIC RADIATION SAFETY

8. Groundwater Contamination

Description:

The purpose of the public radiation safety cornerstone is to assess the performance of the radiological effluent control program. The only PI in this cornerstone counts the number of radiological effluent release occurrences 6

(liquid and gaseous) per site that exceed pre-determined values. The thresholds are 1 and 3 for white and yellow, respectively. There is no red threshold for this PI.

There is a growing concern about groundwater contamination from leakage.

Unless the leakage reaches a measuring point (i.e., monitoring well) and indicates a sufficient dosage, it will not count against this PI. Even though the leaks/spills that the NRC is aware of (see ML101270439) have not posed a hazard to human health, they have impacted public confidence for some stakeholders and led them to question NRCs interest in environmental protection.

Currently there are two action matrix deviations at Vermont Yankee (ML100960321) and Indian Point (ML083590057) to increase NRC oversight because of ground water contamination. The events at both plants did not adversely affect public health and safety. For Vermont Yankee the deviation was proposed because of the extraordinary level of interest and concern by public stakeholders. For Indian Point, the deviation was proposed because some processes and procedures important to the overall effectiveness and quality of the licensees long-term program for monitoring groundwater contamination were not sufficiently developed and implemented in time for the NRC to perform a proper assessment in CY 2008. In addition, the staff has received feedback from the regional inspectors that questions the adequacy of the PI in identifying declining performance.

The Groundwater Task Force issued a final report in June 2010 (ML101680435) that recommended enhancing the NRCs response to groundwater contamination events. This recommended action included potential development of a predictive PI. The report states that An effective PI for groundwater protection would change as a function of the number, quantity, and type(s) of radionuclide, and/or locations of leaks/spills for groundwater protection.

Evaluation: The industry undertook a ground water protection initiative to address potential environmental concerns. NEI 07-07, Industry Groundwater Protection Initiative, was developed to provide guidance on improving utilities management of, and response to, instances where the inadvertent release of radioactive substances may result in low but detectible levels of radioactive materials in subsurface soils and water. The limits set by the NEI 07-07 guidance are well below the NRC limits.

The public radiation safety cornerstone is of significant interest to the public.

When the ROP was being developed the NRC and external stakeholders recognized that a licensees control of its radioactive material is a significant issue for members of the public even when very low levels of radioactive material are involved. As such, public confidence in the PI and SDP is an important consideration in assessing ROP effectiveness.

7

Recommendation: Pursue Now. The two action matrix deviations at Vermont Yankee and Indian Point, internal feedback, industry initiatives in groundwater monitoring, and the groundwater task force collectively indicate that a gap exists in the ROPs ability to address licensee performance in monitoring and controlling releases to groundwater. Development of, or changes to, inspection program tools or a PI should be pursued to address this potential gap in public confidence even though the key safety attribute may be sufficiently satisfied.

9. Unauthorized Radioactive Material Release Occurrence

Description:

Monitoring the unauthorized radioactive material release occurrences was recommended as a potential PI in SECY 99-007. Specifically, the PI would monitor the release of radioactive material(s) from licensee control that could reasonably result in public exposure in excess of 1 millirem per year (mrem/yr) total effective dose equivalent (TEDE). This PI would be used to assess licensee performance in effectively monitoring and preventing measurable dose to members of the public from the unconditional release of solid materials from the plant protected area.

Evaluation: An indicator that monitors the occurrences of uncontrolled solid releases of radioactive material would supplement the current PI, which only monitors liquid and gaseous releases. Currently the inspection program would address these radioactive releases to the public on a case-by-case basis; however, there could be value in monitoring the occurrences assuming that there are enough data to support a statistically valid indicator. The low number of unauthorized radioactive material releases may be insufficient to establish a statistically valid indicator. This would pose a significant challenge for a potential PI.

Another potential challenge would be in defining the phrase could reasonably result in public exposure. Determining what is reasonable and what is unreasonable involves subjectivity, which can make creating an objective PI difficult.

Recommendation: Defer. Monitoring the uncontrolled solid radioactive releases to the public would provide additional insights for the public radiation cornerstone. However, there could be some challenges in establishing a robust data set and setting a threshold for the indicator.

8

MULTIPLE SAFETY CORNERSTONES

10. Operator Licensing Examination (Initial and Requalification)

Description:

Operator licensing examination performance is a potential area for future PI development. Currently there is not an indicator that covers operator licensing examination and training; however, there is an inspection procedure that directs inspectors to review the licensed operator requalification program. A quantitative approach has been discussed internally that would monitor the pass-fail rate of initial and requalification examinations administered by the licensees.

Evaluation: Accumulating and processing data for this potential indicator would not require too much effort. However, analyzing and evaluating the data with the hope of identifying a performance trend may prove to be difficult. For example, at two different sites a pass to fail ratio is 5 to 1 when the fleet average is 15 to 1.

Although it might seem apparent that these two sites with a higher failure rate have a negative performance trend, the numbers alone can be misleading. For instance, one licensee might have a poor, low-quality training program that results in a high failure rate while the other might have an extremely rigorous training program that sets a very high standard of performance. In both cases the data indicate a low pass to fail ratio, but the reasons are completely different.

Recommendation: Not Currently Viable. Although data for an operator licensing examination PI might be easy to collect, interpreting the data to determine performance trends is difficult to do consistently. Also, since each sites program is slightly different, oversight of the operator licensing program is more conducive to inspection than a performance indicator. Inspection Procedure (IP) 71111.11, Licensed Operator Requalification Program, is currently undergoing a comprehensive revision to make it a more performance-based (rather than a prescriptive) inspection. One area being pursued is assessment of the responsiveness of operator training programs to industry events and station performance through a review of available data and trends (problem identification and resolution). Also, increased emphasis on observation of operator performance during actual plant operations is being considered. Enhanced focus in these areas is aimed at assessing operator performance and detecting weaknesses for correction in light of recent significant industry events involving licensed operators. Overall, inspection is the most effective method to monitor and evaluate licensed operator training and requalification programs.

11. Pre-initiator Human Performance

Description:

Pre-initiator human performance was recommended as a potential PI in SECY 99-007. Although the safety cornerstone approach to licensee assessment relies on objective indicators of plant performance to make inferences about human reliability, a more direct measure of pre-initiator human 9

performance could provide a leading indication of changing plant performance.

For example, errors during maintenance, testing, and operations affecting plant configurations will eventually, if not corrected, be evidenced through degraded equipment availability and reliability and increased frequency of transients and scrams.

Evaluation: Currently human performance is evaluated through the assignment of a cross cutting aspect (CCA) to an inspection finding. Aggregated human performance CCAs could result in a substantive cross-cutting issue (SCCI);

however, the SCCI alone would not drive movement in the action matrix. A human performance PI could cause movement in the action matrix.

The overriding challenge with creating a PI that measures soft casual relationships is subjectivity. No matter how quantitative the indicator is, it is difficult to engineer all (or almost all) subjectivity from the PI framework.

Recommendation: Defer. Although it may be difficult to generate an objective and truly leading PI based on human performance, this effort should be kept open to allow new ideas and proposals to be considered. New risk models such as the Bayesian Belief Networks (BBNs) are being used by other industries to quantify soft causal relationships. In the future, BBNs (or other new models and frameworks) might be useful in creating a human performance PI.

10

SUMMARY

The eleven potential gaps that were identified and the proposed indicators are listed below in the Summary Table. Next to each indicator title the recommendation and a short comment describes the proposed path forward.

Summary Table Indicator Recommendation Comments

1. Shutdown Inspection is the preferred method Safety Margin Not Currently Viable of covering this area of performance.

2. Shutdown Although not currently viable as a Monitoring risk-based PI, deterministic and Performance Not Currently Viable qualitative considerations could be applied to establish thresholds for performance bands.

3. Maintenance Inspection is the preferred method Rule Not Currently Viable of covering this area of Implementation performance.

4. Passive Pending design of passive Systems (NRO) Defer systems for new construction.

5. RCS leakage - Current PIs and inspection for frequency Not Currently Viable Barrier Integrity provide enough coverage .

6. RCS leakage - Current PIs and inspection for ISI vs ASME Not Currently Viable Barrier Integrity provide enough Code coverage.

7. Containment Inspection is the preferred method Leakage Not Currently Viable of covering this area of performance.

8. Groundwater The Public Radiation Cornerstone Contamination PI and inspection tools are not effectively supporting public Pursue Now confidence. A new or reformed PI or inspection program tool could be a viable solution.

9. Unauthorized Monitoring solid radioactive Radioactive release occurrences is a potential Defer option to supplement the Material Release Occurrence cornerstone.

10. Operator Inspection is the preferred method Licensing Exam Not Currently Viable of covering this area of performance.

11. Pre-initiator NRC staff are exploring ways to Human Defer possibly meter human Performance performance in the form of a PI.

11