ML052090158

2004 External Stakeholder Response; 2004 Reactor Oversight Process External Survey - Attachment

ML052090158
Person / Time
Site:	Oconee, Cooper, Prairie Island, Waterford
Issue date:	08/02/2005
From:	NRC/NRR/DIPM/IIPB
To:
Sanders S, NRR/DIPM, 415-2956
Shared Package
ML02090174	List: ML052090158
References
Download: ML052090158 (81)

Text

CONSOLIDATED RESPONSE TO THE 2004 REACTOR OVERSIGHT PROCESS EXTERNAL SURVEY This document contains the consolidated results of the Federal Register notice (FRN) that solicited external stakeholder comment and feedback on the Reactor Oversight Process (ROP).

The FRN, entitled Solicitation of Public Comments on the Implementation of the Reactor Oversight Process, was published on October 25, 2004 (69 FR 63411). This notice was part of an ongoing effort by the staff to obtain external stakeholder input regarding the efficacy of the ROP. The comment period expired on December 16, 2004.

In an effort to actively solicit feedback on the implementation of the ROP, the Nuclear Regulatory Commission (NRC) staff: (1) mailed approximately 700 surveys directly to stakeholders, (2) placed a direct link to the survey information on both the ROP Web page and the Documents for Comment page of the NRCs external Web site, and (3) issued a press release and posted it on the NRCs external Web site.

The external surveys were sent out a month earlier than in 2003 to ensure that stakeholders had ample time to participate. Unfortunately, the Agencywide Documents Access and Management System (ADAMS) Publicly Available Records System (PARS) was unavailable to the public due to a security review of documents that could potentially contain sensitive information. For approximately 6 weeks of the comment period, external stakeholders were unable to access PARS. On December 7, 2004, PARS was partially restored and available for public viewing; thus, the NRC granted an additional 30 days for those stakeholders that needed additional time to secure public documents for the purpose of participating in the survey.

In the past, the staff has sent letters to all survey respondents acknowledging receipt of their comments and noting that all comments were considered and many of the comments were addressed throughout the annual ROP self-assessment SECY paper. However, to address the continued concerns that the NRC has been unresponsive to stakeholder feedback, the staff has consolidated the comments by survey question and is providing a response categorized by survey question. The respondent comments for each question are listed in chronological order as received. For those respondents who made general comments that were not directed to a specific question, the comments were listed as a response to question #20 (additional information or comments on other program areas related to the ROP). To manage the size of this consolidated response, comments that are repetitive (same comment provided for more than one question) are addressed for the first question, but deleted from later questions or referenced back to the more detailed response.

Comments were received from the following respondents (listed in chronological order as received). The staff attempted to represent stakeholder comments in this document exactly as they were received, but has also provided the Accession numbers from the Agencywide Documents Access and Management System (ADAMS) after each respondent for access to the official record copy of the specific FRN response.

< Union of Concerned Scientists (ML043150198)

< T. Gurdziel, Private Citizen (ML043210419)

< Region 5/6 Emergency Management, NE (ML043230584)

Attachment

< Alabama Emergency Management Agency (ML043230586)

< First Selectman Connecticut, Town of Waterford (ML043230590)

< M. Mulligan, Private Citizen (ML043350273)

< Minnesota, Department of Public Safety (ML043350267)

< Union of Concerned Scientists (2nd submittal) (ML043480285)

< Prairie Island Nuclear Generating Plant (ML043550216)

< State of New Jersey, Department of Environmental Protection (ML043620075)

< Pennsylvania Department of Environmental Protection (ML043620080)

< Nuclear Management Company (ML043620068)

< Blue Ridge Environmental Defense League (ML043620071)

< Entergy Operations (ML043620073)

< Region IV Utility Group (ML043650145)

< Duke Energy (ML043650168)

< Southern California Edison (ML043650149)

< Tennessee Valley Authority (ML043650450)

< Nebraska Public Power District (ML043650153)

< Exelon Generation Company and AmerGen Energy Company (ML043650154)

< Nuclear Energy Institute (ML050050419)

< Strategic Teaming and Resource Sharing (ML050120343)

Eleven of the 21 responses came from the Nuclear Energy Institute (NEI) or utilities endorsing the NEI response, while 6 responses came from State or local agencies and 4 responses came from public interest groups or members of the public. The Union of the Concerned Scientists (UCS) commented twice, hence one response was not counted. UCSs first response acknowledged the groups inability to do a meaningful review due to the unavailability of ADAMS, but ADAMS was restored soon thereafter and UCS resubmitted a second comprehensive response.

Each question number includes all comments received followed by the NRCs response to those comments. The Table of Contents on the following page can be used to find the comments and responses to specific survey questions. To the extent practicable, we reference the relevant portions of the annual staff paper to the Commission (SECY-05-0070) and the annual ROP metric report to demonstrate how the staff addressed the comments. These documents are available on the ROP Program Documents Web page (http://www.nrc.gov/reactors/operating/oversight/program-documents.html) and can also be obtained in the NRC document management system (ADAMS). The document accession numbers are ML050630297 and ML050670162 respectively.

In some cases the staff plans to consider the specific comments and suggested improvements in future revisions to program guidance. Accordingly, some issues will be entered into the ROP issue tracking system as feedback forms in accordance with IMC 0801, Reactor Oversight Process Feedback Program, to ensure that these issues are considered and tracked to resolution.

This consolidated response, along with the Commission paper and the annual ROP performance metric report, will be posted to the ROP Web page and sent to each respondent to the survey.

1. Does the Performance Indicator Program promote plant safety? 2. Does appropriate overlap exist between the Performance Indicator Program and the Inspection Program? 3. Is the reporting of PI data efficient? 4. Does NEI 99-02, Regulatory Assessment Performance Indicator Guideline provide clear guidance regarding Performance Indicators? 5. Is the information in the inspection reports useful to you? 6. Does the Significance Determination Process yield equivalent results for issues of similar significance in all ROP cornerstones? 7. Does the NRC take appropriate actions to address performance issues for those licensees outside of the License Response Column of the Action Matrix? 8. Is the information contained in assessment reports relevant, useful and written in plain English? -

30-

9. Are the ROP oversight activities predictable (i.e., controlled by the process) and reasonably objective (i.e., based on supported facts, rather than relying on subjective judgement)? 10. Is the ROP risk-informed, in that the NRCs actions are graduated on the basis of increased significance? 11. Is the ROP understandable and are the processes, procedures and products clear and written in plain English? 12. Does the ROP provide adequate regulatory assurance when combined with other NRC regulatory processes that plants are being operated and maintained safely?

13. Does the ROP improve the efficiency, effectiveness, and realism of the regulatory process? 14. Does the ROP ensure openness in the regulatory process? 15. Has the public been afforded adequate opportunity to participate in the ROP and to provide inputs and comments? 16. Has the NRC been responsive to public inputs and comments on the ROP? 17. Has the NRC implemented the ROP as defined by program documents? 18. Does the ROP reduce unnecessary regulatory burden on licensees? 19. Does the ROP minimize unintended consequences?

20. Additional Information or Comments 1. Does the Performance Indicator Program promote plant safety?

Respondent Comments:

Region 5/6 Emergency Management, NE I like this approach.

Union of Concerned Scientists (2nd submittal)

Not really, but it sure is colorful.

The performance indicators (PIs) originally promoted safety, but the industry has figured out how to "game" them and render the whole process a regulatory joke. The PI Summary on the NRC website shows four (4) PIs colored gray with a "T" in the box and a total of six (6) PIs that are colored white, yellow, and red. The 'T' boxes mean that the "thresholds are under development." Under development? The reactor oversight process is mature. Under development? Thresholds were developed years ago. Under development? None of the plants with the 'T' boxes was built or significantly modified in recent years. Under development?

There's no legitimate reason for these facilities not to have PI thresholds.

But plant owners can, and do, avoid white, yellow, or red PIs by asking the NRC questions about the thresholds. Not surprisingly, these questions are not asked when a facility is operating firmly in the green but "suddenly" arise when conditions degrade and PIs might go white, yellow, or red. The NRC plays the game and assigns 'T' to the boxes while the questions are being answered. There are nearly as many PIs in NRC limbo-land because the owners asked some silly questions as there are PIs colored white, yellow, or red. That's unacceptable.

The plant owners must not be able to hijack the PIs by posing silly questions to the NRC.

There must be zero (0) 'T' boxes in the performance indicator summaries. If any thresholds have to be developed, let them be developed in parallel with the existing thresholds to replace them when the development is completed.

Plant owners found other ways of undermining the performance indicators.

FirstEnergy "gamed" the Alert and Notification System PI to avoid a White color. When the emergency sirens failed to sound during a monthly test, FirstEnergy improperly increased the testing frequency so as to throw in sufficient "passes" to mask the failures and keep the PI green. That's unacceptable performance.

PSEG "gamed" the Unplanned Power Changes PI to avoid a White, or perhaps even a Yellow, color. They repeatedly reduced the power level of their Salem Unit 1 reactor down to just a tad above 80 percent power to avoid the >20 percent power change trigger for this PI. The PI is intended to count power changes caused by recurring equipment problems with inadequate resolutions. Salem Unit 1 definitely fit that bill, but PSEG kept the PI green via "gaming." That's unacceptable performance.

Theres a longstanding disconnect between PIs such as Unplanned Power Changes and the NRCs Notice of Enforcement Discretion (NOED) policy. If two plants with equivalent technical specification limiting conditions for operation (LCOs) encounter an equipment problem and Plant X shuts down within the LCO time while Plant Y obtains an NOED from the NRC to continue operating with the equipment problem beyond the LCO period, the Pls would record an event for Plant X but not for Plant Y. This is inequitable and just plain wrong. When the NRC issues an NOED, a condition of that approval should be that counts are applied toward applicable PIs.

The new Mitigating System Performance Indicator (MSPI) is awesome. It requires more number crunching to provide less information than any other mathematical equation in history.

Why grown-ups would expend so much time and energy on the MSPI is baffling to UCS. And we refuse to expend any more time and energy trying to figure out why so much time and energy is being wasted on MSPI-mouse.

UCS rating for the ROP in this area: Yellow (1)

State of New Jersey, Department of Environmental Protection

• Having all ROP performance indicators green doesn't indicate whether the plants are promoting plant safety or not.

• Emergency Preparedness performance indicators are weak.

• Licensees overly concerned maintaining green due to stock holder implications regardless of plant safety.

• Performance indicator thresholds need re-examination.

Pennsylvania Department of Environmental Protection The PIs are actual plant data and provide a mechanism for objective criteria for evaluating plant performance. Also, some of the PIs promote plant safety. However, the basis for setting the existing PI thresholds is inconsistent; some are based on PRAs and others are based on regulatory requirements or technical specification limits. Therefore, some PIs and their associated thresholds do not directly correlate with risk. We encourage the NRC to expedite the development of the risk-based PIs.

Region IV Utility Group In most cases, but not all, the data elements that make up the Performance Indicators (PI) reflect that appropriate actions are being taken in support of safe plant operation. The Emergency Preparedness indicators are a good example of this. Industry performance has improved over the period that the performance indicators have been used. The Mitigating Systems Cornerstone System Unavailability PIs have potential to cause Licensees to take actions that can adversely impact plant safety. The System Unavailability PIs are not risk informed and are not consistent with Maintenance Rule Program goals in most cases. Planned maintenance schedules are managed in order to maintain plant performance in the GREEN band". A significant effort has been made by the industry and the NRC to aggressively address the problems associated with the System Unavailability PIs by better risk-informing them. The Mitigating Systems Performance Index (MSPI) has been piloted and accepted as a replacement for the System Unavailability PIs. We continue to support the implementation of the MSPI and are supportive of the scheduled implementation in January of 2006.

Duke Energy Decisions made by NRC dont always align (or are based upon) PIs.

Nuclear Energy Institute Yes, the Performance Indicator Program, in conjunction with the inspection findings of the ROP Inspection Program, promotes plant safety. Both are used as indicators of safety performance and were designed to provide objective, risk-informed outcomes. Licensees have programs and take actions that minimize the potential for outcomes that adversely impact safety. If performance begins to degrade, the licensee is required to determine the cause(s) for declining performance and provide effective corrective action. The NRC also increases its inspection activity in a graduated manner as performance starts to decline, as indicated by the safety significance of inspection findings. Thus the combined program elements provide incentives to minimize the potential for licensees to take actions that adversely impact plant safety, and provide early warning should performance begin to decline. Since the onset of the program, performance has either improved or remained stable and well within the acceptable band of performance. For example:

• Heat removal (RCIC and AFW) availability has improved 19%

• Safety System Functional Failures dropped 53%

• Unplanned Power Changes have decreased 48%

• ERO Drill Participation has increased 13%

• Protected Area Security Equipment Index has improved by 66%

• Occupational Radiation Safety has improved by 43%

There are two performance indicators which have the potential to influence licensees to take actions that could adversely impact plant safety; however, action is underway to revise these two indicators.

The first is the Safety System Unavailability. This indicator only measures system unavailability (it does not address unreliability), and has generic, rather than plant specific thresholds. The potential exists to minimize down time for preventive or predictive maintenance because that will minimize unavailability and there is no penalty for train failure in the PI. This potential problem can be eliminated by replacing the safety system unavailability indicator with the proposed and successfully piloted Mitigating System Performance Index (MSPI) which balances availability and reliability and sets plant specific thresholds of performance aligned with risk.

Industry and NRC have established a target date of January 2006 to have this replacement in place.

The second problem indicator is the Scrams with Loss of Normal Heat Removal. The initial wording and intention of this indicator has been changed over the past several years, sending an implied message that operators should be focusing their attention on the status of non-safety related equipment (main feedwater pumps) during transients, whereas the emergency operating procedures emphasize focusing on safety-related equipment (auxiliary feedwater pumps). In addition, the staff appears to believe that intentionally closing the main steam isolation valves (MSIVs) under conditions of very low decay heat warrants being considered as a count against performance. Closing the MSIVs in this situation is entirely appropriate, is not a loss of heat removal capability, and should not be considered an action which increases risk. These interpretations do not promote safety. Industry and NRC are working aggressively to develop a replacement for this indicator which will identify those scrams which are the most challenging to operators, and hence are more risk-significant.

Strategic Teaming and Resource Sharing The performance indicators have had a positive effect on promoting plant safety since initial implementation of the Reactor Oversight Process. Over the course of the past five years, improvement trends have been realized in most of the performance indicators. The most pronounced improvements have been in the performance indicators that licensees have the most control over such as unplanned power changes, occupational radiation safety, and protected area equipment. This indicates that licensees do focus on improving the areas considered by the ROP to be important to safety. Enhancements such as the adoption of the Mitigating Systems Performance Index (MSPI) will further promote plant safety.

NRC Response:

The NRC and stakeholders have been working to resolve a number of these issues. An ROP Working Group Task Force has been formed and is working to resolve issues with the Scrams with Loss of Normal Heat Removal PI. In addition, MSPI is planned for implementation in 2006 and will address the issues with the Safety System Unavailability PI.

As noted in the PI evaluation in Attachment 1 to SECY-05-0070, the staff has addressed, or has work in progress to address, a number of the specific issues identified by stakeholders, including:

• An option available to the staff when the staff concludes that a PI is not being properly reported is to implement Inspection Procedure (IP) 71150, Discrepant or Unreported Performance Indicator Data. Under this process the staff declares a licensees PI data report invalid and colors the associated PI gray until the staff determines the correct color through inspection. During 2004, the staff implemented the discrepant PI process for the first time. The associated inspection found that the licensee had taken steps inappropriate for the program, which influenced the PI result. The PI was subsequently colored white. Two other cases of potentially discrepant PIs are currently under review by the staff that could be treated similarly.

• The staff plans to evaluate the Unplanned Power Changes per 7,000 Critical Hours PI to determine whether power changes that were avoided because the NRC issued a Notice of Enforcement Discretion or a temporary technical specification change should be included. In addition, the staff plans to clarify the guidance regarding the 72 hour8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br /> rule for planning a power reduction and the 20 percent power change minimum requirement for counting the change as a PI.

• To address a number of issues with the Scram with Loss of Normal Heat Removal PI, a task force was formed with members from both the NRC and the industry. The task force is evaluating whether to count events or conditions that require operators to depart from their normal scram recovery procedure, as a possible modification to the PI definition.

• The Mitigating Systems Performance Index (MSPI) is being developed by the staff and the industry to replace the existing Safety System Unavailability (SSU) PIs. While there are a few remaining issues to be worked out, the MSPI is scheduled for implementation in 2006 following a series of three industry-sponsored workshops in 2005.

• The staff has concluded, based on the historical results, that the PI program has succeeded in focusing industry attention in selected areas. However, the number of PIs that are consistently green at all plants bring into question the present value of those PIs to the program. In addition, the staff has concluded that the PI program has not contributed to the early identification of poor performing sites to the degree envisioned by the staff. As a result, the staff intends to interact with NEI and other stakeholders at a senior management level during CY 2005 to define steps to improve the effectiveness and efficiency of the PI program, and to identify enhancements to the program intended to increase its contribution to the identification of plants with poorer performance.

• The staff recognizes that the process for resolving PI questions from specific licensees has not been consistently timely nor efficient. Because the PI program is not required by regulation, and industry participation is voluntary, the program was set up such that questions about the PI guidance are to be resolved by consensus of a joint NRC/industry working group which typically meets once each month. In some cases, the working group has not been able to reach consensus, resulting in open questions lingering. The working group has agreed to a process to resolve these issues. If after a question has been discussed at several meetings the question remains unresolved, it is referred to the NRC Director of the Division of Inspection Program Management (DIPM).

The decision of the DIPM Director is binding on the working group. The backlog of open site-specific questions has been largely eliminated via this process.

• The staff intends to interact with NEI and other stakeholders at a senior management level during CY 2005 to define steps to improve the effectiveness and efficiency of the PI program, and to identify enhancements to the program intended to increase its contribution to the identification of plants with poorer performance.

UCS commented that several PIs are coded gray with a "T" (thresholds under development) in the box on the ROP Website. Response: There are currently four BWR units whose heat removal system PI (MS03) is coded gray with a "T" (thresholds are under development) in the box on the website summary page. These units (Oyster Creek, Nine Mile Point 1, Dresden 2, and Dresden 3) are older plants that were designed with isolation condensers (IC) to perform the heat removal function; the Reactor Core Isolation Cooling (RCIC) system was used in the Monticello BWR 3 design and later plants. The green-white threshold for the RCIC plants was selected to identify approximately 5 percent of the plants whose RCIC unavailability was significantly greater than the population mean. For the IC plants, however, it is difficult to identify outliers because: (1) there only four plants with the IC system, (2) the mean can be significantly affected by a single plant, and (3) the differences in the number and capacity of the isolation condensers at each unit vary considerably, as evidenced by the data the staff collected. Nevertheless, the heat removal function for these four units will be included in the Mitigating Systems Performance Index, currently planned for implementation in 2006. Contrary to UCSs comments, the staff has not assigned any additional T designations since the beginning of the ROP in FY 2000 and will not do so in the future. The staff has never assigned a T designation while an issue is under discussion or to prevent a plant from crossing a threshold.

UCS also commented that licensees "game" the Unplanned Power Change PI to avoid a greater than green color. Response: For a PI to be efficient and effective, the criteria associated with it should be objective, if possible. Objective criteria minimize interpretation issues and the associated delays in assessing the PI result. In the case of the Unplanned Power Change PI, the objective criteria selected was an 80% power reduction. It is not practical to apply a subjective criteria on what constitutes an Unplanned Power Change without introducing significant difficulties with the implementation of this PI. If a licensee reduces power to some level just above 80% power and complies with the Technical Specifications and other associated regulatory requirements in the operation of the unit, then the staff does not view this action as gaming the system. Nevertheless, the staff recognizes the problems that can occur when numbers are used in the definition of a PI and plans to address both the 20 percent power reduction and the 72 hour8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br /> planning window in a future revision to this PI.

UCS also questioned the usefulness of the Mitigating System Performance Index (MSPI).

Response: The MSPI is being implemented to provide a risk-informed PI for the systems that it covers. This is consistent with the Commissions intention to increase the staffs use of risk information in the regulatory process. MSPI will also address several shortcomings of the PI that it will replace.

Several commenters noted the difference in the way PI thresholds have been established.

Some PI thresholds are coupled with risk insights, while others, such as Emergency Preparedness (EP) and Radiation Protection (RP) PIs, have thresholds that are more deterministic. Some commenters are critical of the more deterministic EP and RP PIs.

Response: The staff considers licensee performance in the areas of EP, RP, and security to be significant elements of our regulatory oversight responsibility, and therefore that PIs for these areas are needed. In setting thresholds, however, it is difficult to connect performance in these areas to measures such as core damage frequency or large early release frequency. In establishing the PI program, the staff worked with industry to define thresholds that would indicate that additional regulatory attention was warranted. The staff believes that these PIs have been effective in focusing licensee attention on the areas of performance addressed by the PIs. As previously discussed above, the staff is working to risk inform safety system PIs by the implementation of MSPI. The staff meets monthly with industry representatives and is open to discussion on how to further improve the PI program. As noted above, the staff intends to discuss with industry possible changes to increase the effectiveness of the program.

Duke Energy commented that decisions made by NRC dont always align with (or are based upon) PIs. Response: Staff actions regarding the results of the PI program, combined with inspection findings, are dictated by the ROP Action Matrix. The staff believes that the Action Matrix ensures a relatively predictable range of NRC actions for a given level of licensee performance.

2. Does appropriate overlap exist between the Performance Indicator Program and the Inspection Program?

Respondent Comments:

Region 5/6 Emergency Management, NE I have seen this demonstrated at open meetings.

Union of Concerned Scientists (2nd submittal)

Technically, yes. The inspection program provides useful insights on safety issues and the performance indicator program provides useless insights. The overlap is use.

The NRC should seriously consider dropping the regulatory charade that is the performance indicator program and instead devote ALL of those full-time equivalents (FTEs) to inspections at operating power reactors.

UCS rating for the ROP in this area: White (1)

Prairie Island Nuclear Generation Plant Too much baseline inspection w/o real tie back to what the PI/Cornerstone reflects. Baseline inspections should be better defined and look for the critical success factors rather than compliance to old regulations.

State of New Jersey, Department of Environmental Protection

• Something must be missing if cross-cutting issues, such as the Davis-Besse event, and safety conscious work environment concerns exist.

Pennsylvania Department of Environmental Protection Yes, but there are areas of improvement. The ROP Inspection Program, including the SDP, is more focused on risk significant issues than the PI Program. This inconsistency has reduced the overall effectiveness of the ROP.

The PI verification inspection is a positive aspect of the ROP and it should continue.

Considering that currently there are no PIs for cross-cutting areas (human performance, safety-conscious work environment, and corrective action program), we recommend NRC's continuous attention to these areas.

Region IV Utility Group In most cases appropriate overlap exists between the Performance Indicator Program and the inspection Program. There are, however, a few exceptions to this. For example, NRC performs an SDP for each occurrence of single equipment failure, even though the safety system unavailability PI covers the equipment. Another example of excessive overlap is the Scrams with Loss of Normal Heat Removal. NRC performs a risk assessment of every scram to determine the need for additional inspection; therefore, there is no need to have a redundant PI.

Nuclear Energy Institute In general, yes. If anything, there is excessive overlap. For example, NRC performs an SDP for each occurrence of single equipment failure, even though a safety system unavailability PI already exists. Implementation of the MSPI will assess unavailability and unreliability; however, NRC has stated that it will continue to conduct SDPs for simple, single failures. Another example of excessive overlap is the Scrams with Loss of Normal Heat Removal. NRC performs a risk assessment of every scram to determine the need for additional inspection; therefore, there is no need to have a redundant PI, the Scrams with Loss of Normal Heat Removal, which purports to identify risk significant scrams. The replacement indicator described above will be an improvement, but NRC has stated that it will continue to perform assessments of every scram.

We believe that there is a significant opportunity for the NRC to streamline and make more efficient the inspection procedures for the occupational and public radiation safety cornerstones. The scope, frequency, and duration of inspection in these areas appear unnecessarily burdensome when compared to the current level of industry performance and types of findings that have been issued. It would be better use of resources if the NRC would reduce baseline inspection in areas in which performance indicators indicate good performance, and determine what, if any, areas require more inspection, as evidenced by operating experience or newly emerging generic safety areas of concern.

Finally, industry believes that NRC is spending too much time verifying performance indicators, and questioning items that could only have trivial impact. For example, raising questions regarding a few hours of safety system unavailability over a three year period (e.g., a half dozen hours out of 26,280, or 0.02% unavailability) is counterproductive. The inspection program is supposed to be risk-informed. Spending time addressing trivial questions is not an appropriate use of NRC or licensee resources.

Strategic Teaming and Resource Sharing The performance indicators work in concert with the inspection program under normal conditions to form a strong oversight program. For events that occasionally occur at plants, the regions have made liberal use of management directive 8.3 to determine the need for additional inspection. There is no need to have a performance indicator for Scrams with Loss of Normal Heat Removal if the decision whether to perform additional inspection is not driven by that indicator. In that case there is unnecessary overlap.

NRC Response:

Early in initial implementation, the staff believes the PI program was effective in focusing licensee attention and improving licensee performance in selected areas, particularly the Protected Area Security Equipment Performance Index and the Emergency Response Organization Drill Participation PI. However, the number of PIs that are consistently green at all plants brings into question the present value of those PIs to the program. As noted in the PI evaluation in Attachment 1 to SECY-05-0070, and discussed in more detail in the staffs response to question #1, the staff intends to interact with NEI and other stakeholders at a senior management level during CY 2005 to define steps to improve the effectiveness and efficiency of the PI program, and to identify enhancements to the program intended to increase its contribution to the identification of plants with poorer performance.

As noted in the PI evaluation in Attachment 1 to SECY-05-0070, to address a number of issues with the Scram with Loss of Normal Heat Removal PI, a task force was formed with members from both the NRC and the industry. The task force is evaluating an industry proposal to count events or conditions that require operators to depart from their normal scram recovery procedure, as a possible modification to the PI definition.

As discussed in Attachment 1 to SECY-05-0070, the Mitigating Systems Performance Index (MSPI) is being developed by the staff and the industry to replace the existing Safety System Unavailability (SSU) PIs. While there are a few remaining issues to be worked out, the MSPI is scheduled for implementation in 2006 following a series of three industry-sponsored workshops in 2005.

As noted in the inspection program evaluation in Attachment 2 to SECY-05-0070, the staff performed its annual review of each baseline inspection procedure in CY 2004. Based on this review, the staff did not identify any significant changes to the inspection program, although the staff is making minor adjustments to some inspection procedures. In addition, the staff plans to perform a more detailed analysis of the scope and level of effort of the inspection procedures in CY 2005. The purpose of the analysis will be to better align inspection resources to inspected areas where there is an indication of risk-significant performance deficiencies. As a result of this systematic analysis, the staff plans to adjust existing resources within the baseline inspection program for CY 2006.

As described in the PI evaluation in Attachment 1 to SECY-05-0070, as part of its continuing reassessment of the PI program, the staff will look into the feasibility of new indicators, including those for cross-cutting issues, as recommended by the Advisory Committee on Reactor Safeguards (ACRS) and other stakeholders.

The State of New Jersey commented on the Davis-Besse event. Response: The staff agrees that there are lessons to be learned from the Davis-Besse event and from safety conscious work environment (SCWE) concerns that occurred at Salem/Hope Creek. There have been a number of program changes made to the ROP as a result of Davis-Besse. A summary of agency actions and their status is located on the agency website at http://www.nrc.gov/reactors/operating/ops-experience/vessel-head-degradation/lessons-learned

/lessons-learned-files/lltf-web-page-status022205.pdf. This document was last modified in February 2005 and is due to be updated in August 2005. Additionally, at the Commissions direction, the staff proposed options to enhance staff actions regarding SCWE and safety culture in Commission paper SECY-04-0111, Recommended Staff Actions Regarding Agency Guidance in the Areas of Safety Conscious Work Environment and Safety Culture. The Commission directed the staff to take action in this area as outlined in the associated Staff Requirements Memorandum (website address http://www.nrc.gov/reading-rm/doc-collections/commission/srm/2004/2004-0111srm.pdf). The staff is working to carry out the Commissions direction in this area.

The industry commented that excessive overlap exists in that the NRC performs an SDP for each occurrence of a single equipment failure or event, even though the equipment or event may be addressed by a performance indicator. Response: The staff disagrees with the concept that an event or equipment failure should be addressed by either a PI or the SDP, but not both. The PIs typically monitor long-term performance trends, which can indicate relatively weak licensee performance that may not manifest itself in any individual event. For example, plant scrams are normally not individually risk significant, and do not typically cross a threshold under the SDP, however a trend of a number of scrams over a period of time would appropriately result in increased NRC attention. Conversely, the SDP identifies risk significant occurrences that appropriately warrant increased NRC attention, which might otherwise not occur under the PI program. For example, some PIs consider performance over a 36 month period. Considering PI data alone, significantly poor performance regarding a single event or equipment failure might be offset by prior good performance years in the past. The SDP appropriately counter-balances this potential problem. The NRC staff conclusion is that the PI program and the SDP must work in conjunction to direct increased inspection to weaker performing licensees.

In general the staff agrees with the industry comment that inspections should be risk-informed and not focused on trivial issues.

Strategic Teaming and Resource Sharing commented that the regions have made liberal use of management directive 8.3 to determine the need for additional inspection. Response: The staff believes that the guidance in MD 8.3 has been appropriately applied to follow up on events at sites. The comment lacks detail to support the conclusion of the comment.

3. Is the reporting of PI data efficient?

Respondent Comments:

Union of Concerned Scientists (2nd submittal)

Nope. Since the PI data is currently useless, its reporting cannot possibly be efficient.

UCS rating for the ROP in this area: White (1)

State of New Jersey, Department of Environmental Protection

• The reporting of performance indicator data, as well as, inspection findings take too long. For example, performance results from January, February, and March are reported in late May.

• Since the performance indicators are lagging indicators and not leading indicators, the time frame for posting information is excessive and does not lead to proactive resolution of performance issues.

Pennsylvania Department of Environmental Protection It would be more appropriate for the licensees to provide comments on reporting of PI data.

Region IV Utility Group Yes, reporting conflicts continue to exist between the ROP, INPO, WANO, and the Maintenance Rule. Maintenance Rule monitors performance with one set of goals while ROP monitors system performance using another set of goals. In most cases, the guidance for the ROP, INPO, WANO, and the Maintenance Rule differ slightly. However, of equal concern is the continuing amount of duplicated effort by the plant support staff to develop and maintain, "customized" indicators. We recognize that INPO is working with the industry and the NRC to reduce the conflicts and duplication of effort that currently exists. Implementation of MSPI will solve some of these conflicts. We encourage continued industry and NRC support of INPO's work with the Consolidated Data Entry program.

Duke Energy CDE Process is much improved over original PI web.

Nuclear Energy Institute Yes, the reporting of PI data is efficient. A significant improvement was made when industry shifted to the Consolidated Data Entry (CDE) technology, which allows better management of data for different uses. However, there continue to be differences in reporting and definitions amongst the ROP, WANO/INPO and maintenance rule. Implementation of the MSPI will address many of the differences. In addition, several minor changes will be necessary in maintenance rule implementing guidance to improve consistency and efficiency.

Strategic Teaming and Resource Sharing Reporting performance indicator data is efficient. Making use of internet technology and electronic media reduces burden on licensees. While differences still exist in the plant data reported to other organizations such as INPO and WANO, improvements have been made to lessen the burden on licensees by using technologies like Consolidated Data Entry (CDE). The concept of breaking data into elements and using the algorithms in CDE to manipulate the data element to derive the performance indicators to suit the needs of multiple organizations was an improvement. Additional work is needed to further align the performance indicator definitions between INPO, WANO, and the NRC.

NRC Response:

As noted in the PI evaluation in Attachment 1 to SECY-05-0070, the staff completed its review of INPOs Consolidated Data Entry (CDE) Program and found that it is consistent with the staffs needs for licensee data used in various NRC programs. Use of CDE for the MSPI indicators is currently under development by INPO with licensee and staff involvement.

The State of New Jersey commented that the time frame for posting information is excessive and does not lead to proactive resolution of performance issues. Response: It should be noted that the identification and corrective actions for performance issues are taken promptly; it is the roll up and assessment of these issues that is delayed. The time frame for posting plant performance information is provided in IMC 0306, Information Technology Support for the Reactor Oversight Process. In summary, PIs are submitted quarterly by licensees and inspection reports are typically issued quarterly by the NRC. This information is posted as soon as practicable after it has been gathered and verified in accordance with the established schedule. PIs are available externally within 35 days after the end of each quarter as is the latest available inspection information and the current Action Matrix designation. Within 60 days after the end of each quarter, the external ROP web page is updated to include any newly issued inspection reports from late in the previous quarter so that the ROP Web page reflects a complete picture of PIs and inspection findings from the previous quarter in a reasonably timely manner. The ROP Web page will also be updated between quarters to correct any discrepancies or noted omissions in previously posted information or to reflect changes in NRC actions (i.e., Action Matrix designation) based on a final significance determination that resulted in a risk significance that is more than very low significance (i.e., white, yellow, or red) or if a licensees PI change submittal caused a threshold to be crossed. Gathering performance indicator data and/or issuing inspection reports more frequently would make NRC evaluation of plant events and issues available to the public sooner, but this would result in an additional resource burden on the inspection staff, NRC management, and licensees. As noted earlier, performance issues are addressed and corrected as soon as they are identified and do not rely on the timeliness of documenting the issues in NRC inspection reports or on the ROP Web page.

4. Does NEI 99-02, Regulatory Assessment Performance Indicator Guideline provide clear guidance regarding Performance Indicators?

Respondent Comments:

Union of Concerned Scientists (2nd submittal)

Apparently not since FirstEnergy was unable to follow the guidance and properly report the Alert and Notification System performance indicator to the NRC. Or, maybe it is clear enough and FirstEnergy deliberately faked its submittal. In that case, the guidance in NEI 99-02 about not faking PIs is apparently too vague or understated.

UCS rating for the ROP in this area: U State of New Jersey, Department of Environmental Protection

• This document supports more prompt reporting of performance indicator information.

Pennsylvania Department of Environmental Protection Overall, the NEI Guidance Document is very helpful in defining the PIs. It would be more appropriate for the licensees to comment on the effectiveness of this document.

Region IV Utility Group NEI 99-02 provides adequate guidance regarding Performance Indicators. The current revision of NEI 99-02 does contain some confusing verbiage. For example, the clarifying notes in the Scrams with Loss of Normal Heat Removal PI include several special considerations, which have complicated the PI. Considerable efforts by the NRC and Industry to resolve issues with this indicator have failed; we support the recommendation by NEI to eliminate or modify this Performance Indicator.

Duke Energy The FAQ process is a good tool to resolve special issues.

Nuclear Energy Institute Yes, NEI 99-02 provides clear guidance regarding performance indicators. As described above, replacements are needed for the Scrams with Loss of Normal Heat Removal and the Safety System Unavailability PI. These replacement indicators will provide better indication of safety and will be more clearly defined.

For the most part, the Frequently Asked Questions (FAQ) process has proven useful and efficient in resolving questions on the PI guidance. The backlog of FAQs has remained small (with the exception of the to-be-replaced Scrams with Loss of Normal Heat Removal PI). A revision of NEI 99-02 will be published early in 2005 which incorporates FAQs since the previous revision. This revision provides more specific guidance in preparing an FAQ which should further improve the process.

Strategic Teaming and Resource Sharing The guidance in NEI 99-02 for the most part is clear. There are still areas that need to be improved, most notably the guidance for IE 02 (Scrams with loss of normal heat removal). The Frequently Asked Questions process is effective in providing real time clarification to NEI 99-02 and should continue. Every effort should be made to use the guidance in NEI 99-02 as written, not as interpreted. When examples arise where the guidance in NEI 99-02 produces an unintended consequence, the guidance should be revised, not worked around.

NRC Response:

The NRC continues to work with stakeholders to resolve the open issues. Specifically, an ROP Working Group Task Force has been formed to address the issues with the Scrams with Loss of Normal Heat Removal PI and modifications were made to NEI 99-02 to improve the FAQ process.

As noted in the PI evaluation in Attachment 1 to SECY-05-0070, the staff has addressed a number of these specific issues, including:

• The staff implemented the discrepant PI process in 2004 and two other cases of potentially discrepant PIs are currently under review by the staff that could be treated similarly (as discussed in more detail in the staffs response to question #1).

• To address a number of issues with the Scram with Loss of Normal Heat Removal PI, a task force was formed with members from both the NRC and the industry. The task force is evaluating whether to count events or conditions that require operators to depart from their normal scram recovery procedure, as a possible modification to the PI definition.

• The staff recognizes that the process for resolving PI issues has not been consistently timely nor efficient. The lack of clear concise guidance for selected PIs has contributed to the timeliness and efficiency issues. The staff has had trouble dealing with a number of potentially white PIs with which interpretation issues have arisen. This is partly because the PI guidance document is subject to interpretation and the joint NRC/industry working group has had difficulty agreeing on which events or conditions are to be included in the PI calculation. As a result, the staff intends to interact with NEI and other stakeholders at a senior management level during CY 2005 to define steps to improve the effectiveness and efficiency of the PI program, and to identify enhancements to the program intended to increase its contribution to the identification of plants with poorer performance.

• As discussed under question #1, a process has been implemented to reach a resolution on PI questions when the NRC/industry working group is deadlocked.

In addition, NEI 99-02 was revised and reissued in April 2005 to incorporate the approved FAQs and additional guidance aimed at making the FAQ process more efficient and effective (reference Appendix E). This revision is effective for data collection as of April 1, 2005, and reported on July 21, 2005, and is available on the ROP Web page at:

http://www.nrc.gov/NRR/OVERSIGHT/nei_9902rev3.pdf.

5. Is the information in the inspection reports useful to you?

Respondent Comments:

Region 5/6 Emergency Management, NE They provide a better understanding of plant operations.

Union of Concerned Scientists (2nd submittal)

Yes. The usefulness of the inspection reports has been high since the onset of the reactor oversight process and has only improved over time. The inspection reports do a fine job of describing what was examined and what was found. It is rare that my review of an inspection report raises more questions than it answers. Prior to the reactor oversight process, my review of inspection reports often prompted the need to contact the NRC for clarifying or additional information. That need rarely happens any more. When it does occur, its my opinion that my questions reflect a special interest in the subject rather than indicate a deficiency in the information provided in the reports.

The inspection reports frequently contain the final answer to questions raised in other forums.

For example, a Daily Event Report in late summer 2004 discussed cracking of the coating protecting the berm around the Condensate Storage Tank at the Limerick Generating Station.

I contacted the Senior Resident Inspector at the facility for additional information. The Senior Resident Inspector answered my question and referred me to a recently issued inspection report that covered the matter in considerable detail. This example is by no means an isolated case and that fact is my evidence that inspection reports are very useful. That inspection reports are far more frequently the end points instead of the starting points for question threads is, to me, also highly suggestive of their usefulness.

I can suggest only two things to improve the usefulness of the inspection reports. The report numbering scheme is cumbersome. For example, NRC Inspection Report 05000440/2003010 dated January 20, 2004, documents an inspection at the Perry Nuclear Power Plant. The Summary of Findings highlights the key issues discussed in the report. The first finding is summarized and Section 40A2 is cited as containing additional information. Finding Section 40A2 in the report becomes a challenge. Look at page 18 of the report, a typically formatted page within the report. It starts out at the top with b. Findings followed by

.3 Radiation Protection Technician Instrument Use. You have to flip back two pages to page 16 to see that the .3 in question is a subsection of Section 20S3, Radiation Monitoring Instrumentation and Protective Equipment. Finding this reference point, I know that I now need to flip towards the back of the report to hopefully locate Section 40A2. Flipping down to page 23, I see .2 Annual Sample Review at the top with no clue on the page as to what Section this subsection belongs to. Moving backwards to page 22, I find its part of Section 40A2. The NRC inspection reports should be reformatted so that section numbering is more useful to the reader. Each page of the report should provide some clue to the reader as to the section.

The other suggested improvement for inspection reports would be to expand the inclusion of diagrams, when appropriate. A good example of such an inclusion is NRC Inspection Report 05000528/2004012 dated July 16, 2004, for the Palo Verde grid event. Attachment 7 to this inspection report is an offsite power electrical diagram. This graphic makes the discussion in the report much easier to follow and comprehend. When inspection reports describe in detail a condition involving numerous components, a simplified graphic should be used.

UCS rating for the ROP in this area: Green Green Prairie Island Nuclear Generation Plant The report is too heavy on compliance rather than real insight on how the success factors are being accomplished so that the public is protected.

State of New Jersey, Department of Environmental Protection

• By the time we receive an NRC inspection report the information is old so the report becomes archival.

• The most useful information is the analysis of any findings because this has consequences to the licensee.

Pennsylvania Department of Environmental Protection The information contained in the inspection reports is very useful and overall, the quality of these reports has improved.

Region IV Utility Group The organization of the inspection reports with the ties to cornerstones help in providing better definition and focus in problem areas. The listing in the reports of inspection scope is duplicative of the Inspection Procedures and should be eliminated.

Nuclear Energy Institute Information in the inspection reports is useful and acceptably formatted. The reports should continue to focus on risk and safety significant issues, leaving any suggestions for improvements to be discussed at the inspection exit meeting.

Strategic Teaming and Resource Sharing For the most part information in the inspection reports is useful and acceptably formatted. The inspection reports do not always provide an adequate understanding of the basis for a crosscutting aspect being applied to an inspection finding. The reports should continue to focus on risk and safety significant issues, leaving any suggestions for improvements to the exit meeting venue.

NRC Response:

The majority responded that the inspection reports were clearly written and provided a better understanding of plant operations. Stakeholder satisfaction was generally favorable and consistent.

The availability of NRC assessment and discussion of an issue documented in an inspection report may be several months after the event date for those issues which may have occurred early in the inspection period because most inspection reports describe inspection activities over a three-month period (one quarter) and the reports are issued about 45 days after the end of the calendar quarter. Issuing inspection reports more frequently would make NRC evaluation of plant events and issues, as documented in inspection reports, available to the public sooner, however, this would result in an additional resource burden on the inspection staff and NRC management. As noted earlier in the staffs response to question #3, performance issues are addressed and corrected as soon as they are identified and do not rely on the timeliness of documenting the issues in NRC inspection reports or on the ROP Web page.

As noted in IMC 0612, Power Reactor Inspection Reports, it is important to describe in detail how the inspection was conducted and what was inspected in the inspection scope because this information can be used to verify that the inspection was completed satisfactorily.

The staff also plans to consider several additional suggested improvements to the inspection reports in future revisions to IMC 0612, including the specific comments provided by UCS.

6. Does the Significance Determination Process yield equivalent results for issues of similar significance in all ROP cornerstones?

Respondent Comments:

Union of Concerned Scientists (2nd submittal)

Nope. The SDP yields less and less equivalent results with each revision to the reactor oversight process. The problem seems to come from how one views issues of similar significance. The reactor oversight process was developed to measure licensee performance.

In its original incarnation, the reactor oversight process did a fairly good job of evaluating licensee performance fairly in areas that were "apples and oranges" from a core damage frequency perspective. For example, it originally did a fairly good job of assigning equivalent colors to similarly good, or bad, licensee performance in mitigating systems and in occupational exposure areas. Poor licensee performance that allowed degraded plant conditions to elevate the chances of core damage properly received a color comparable to that color assigned to poor licensee performance that allowed workers to be exposed to elevated levels of radiation.

But heavy, and misguided, industry pressure over the years since the reactor oversight process was launched have bent and twisted it to near uselessness. For example, the emergency exercise SDP is now comical. A licensee can, and sadly has, fared very badly on a graded emergency exercise only to have that dismal performance dissipated by the unwarranted - and just plain stupid - reason that the breakdown occurred during a test vice an actual emergency.

[UCS notes that the biennial exercises are supposed to emulate real emergencies except, apparently, when it comes to evaluating poor licensee performance during the real emergencies. Hogwash!] That's lame and allowed to proceed to the extreme will result in only two colors - Green and Red. The colors should be changed to skunk-like Black and White because this practice stinks.

The reactor oversight process and its SDP were supposed to evaluate licensee performance. If the grading system is so perverted that it renders virtually all miscues and foibles Green when they don't involve a core meltdown and Red if they do, then the reactor oversight process is no more insightful than a radiation detector mounted on the perimeter fence. The reactor oversight process needs to be more discerning so that all reasonable regulatory actions are taken to prevent those radiation detectors from ever being used to monitor actual accidents.

The reactor oversight process needs to return to its roots of providing insightful evaluations of licensee performance. Since September 1984, more than two dozen U.S. nuclear power reactors have been shut down for a year or longer while licensee performance problems were corrected. The reactor oversight process must provide early detection and warning of declining performance so it can be corrected before year - plus outages are necessary - or worse, before it triggers a nuclear catastrophe. The reactor oversight process cannot achieve this goal by downplaying bad licensee performance simply because it didn't result in disaster. "No blood, no foul" is the wrong regulatory approach to take and the NRC must abandon it ASAP.

UCS rating for the ROP in this area: Yellow (1)

Prairie Island Nuclear Generation Plant SDP when applied to EP & Security is following the rigid Rx process. EP/Security are too much what if and based on exercises rather than actual system or organizational performance.

State of New Jersey, Department of Environmental Protection

• Judgement about the Significance Determination Process depends on the inputs it receives and most of the inputs are in the mitigating systems cornerstone so it is impossible to conclude anything about this question.

• The significance determination process should be designed in favor of promoting plant safety, and therefore, generate more colors that move licensees in promoting plant safety. The current system forces licensees to avoid any color, in spite of plant safety.

• We believe that the overall significance determination process is more relaxed now then when the ROP was implemented. This is evidenced by all NRC findings moving from higher color to lower color in every case and the increased use of "potentially greater then green" when the NRC indicated that this would be rarely used.

Pennsylvania Department of Environmental Protection The SDP is a resource-intensive process and the lack of standardized risk analysis tools has further complicated the process. Therefore, SDP may not always yield equivalent results for issues of similar significance in all ROP cornerstones.

Region IV Utility Group No, the Significance Determination Process (SDP) does not apply the same risk significance to issues across the seven cornerstones. Some SDPs are still deterministic in nature and do not characterize the appropriate risk insights - especially in the areas of security and emergency preparedness and to a lesser degree, public radiation protection. Deterministic thresholds have the effect of aggregating lesser items of minor risk significance to create findings with a final significance out of proportion to the risk presented by any credible situation. We recognize that both the industry and the NRC have worked over the past year to better risk-inform the Emergency Preparedness and the Public Radiation Safety SDPs.

Nuclear Energy Institute No. While the reactor SDP provides a risk-informed, structured, and generally consistent result for similar issues, the more deterministic SDPs do not result in equivalent results for issues of similar risk significance across different cornerstones. In general, they represent a deterministic escalation for various types of regulatory noncompliance, and more reflect NRC concern over programmatic issues. We have observed improvements in consistency in assessing outcomes of similar issues in revisions to the Emergency Preparedness, Occupational Radiation Safety, and Public Radiation Safety SDPs. There have been several new SDPs produced over the past year. We believe additional time will be necessary to determine their quality in appropriately assessing performance, such as the Fire Protection, Steam Generator, Containment, and Shutdown SDPs. We are, however, especially concerned about the draft Maintenance Rule SDP and the Physical Security SDP, and have made those concerns known to the staff.

Industry shares NRC's concern with improving the SDPs, so that the inspection findings can play their appropriate role in informing NRC decision making regarding assignment of its inspection resources. To be effective, the SDP results must be accurate, timely, and resource efficient (more resources should not be invested in determining the color than in inspecting the corrective action). This "triple constraint" has not been achieved; in fact, the resources expended on determining the SDP outcome have increased, while the timeliness has decreased. Both NRC and industry are working on solutions which will result in improvements to timeliness and resource expenditure while not sacrificing accuracy of the results.

We recommend that NRC conduct semiannual lessons learned workshops on SDPs.

The purpose of these workshops would be to share information from completed SDPs to improve efficiency in assessing similar situations or analyses; to inform the industry and public of changes to SDPs; to explore ways to improve the timeliness of SDPs; to enhance consistency of NRC and industry across regions; and to address PRA issues of generic interest.

Strategic Teaming and Resource Sharing The Significant Determination Process (SDP) does not yield equivalent results for issues of similar significance across all ROP cornerstones. Specifically, issues and events such as Emergency Preparedness, Security, Radiation Protection, and Fire Protection are evaluated using processes that are more deterministic in nature. These deterministic SDPs tend to exaggerate the actual risk. The reactor safety SDP is the most risk informed and should be used whenever possible. "Specialized" SDPs should be minimized.

NRC Response:

The staff believes that, in general, an appropriate balance has been achieved among the cornerstones based on the potential impact on public health and safety and the NRC response that the staff considers appropriate to specific findings.

Several commenters noted the difference in the way SDPs determine significance across the cornerstones. Some SDPs are highly risk-informed, while others, such as Emergency Preparedness (EP) and Radiation Protection (RP) SDPs are very deterministic. Generally commenters are critical of the more deterministic EP and RP SDPs. Response: The staff considers licensee performance in the areas of EP, RP, and security to be significant elements of our regulatory oversight responsibility, and that increased regulatory oversight is needed when weaker performance is indicated. In establishing SDPs in these areas, however, it is difficult to connect performance in these areas to measures such as core damage frequency or large early release frequency. The staff has worked to define SDPs in the EP and RP areas that result in the agency response that is considered appropriate for a range of performance problems. Because of the inherent differences between the various cornerstones, the staff does not presently envision being able to define a single risk-informed SDP approach that would address all cornerstones.

Regarding specific comments made by UCS on the EP SDP, performance deficiencies that occur during an actual emergency could place the public at risk if adequate protective actions are not implemented. No direct risk to the public is created by an exercise weakness if the licensee identifies the weakness, enters it into the corrective action program, and initiates appropriate corrective actions. The NRC views exercises as opportunities for training and identification of needed program improvements. Nonetheless, the EP SDP will lead to regulatory action if the licensee does not identify the weakness during the exercise critique, if the licensee fails to correct the weakness, or if the weakness reveals inadequacies in the licensee's plan or procedures.

UCS also commented that the ROP must detect declining licensee performance, and that the ROP cannot achieve this goal by down-playing bad licensee performance. Response: The staff agrees that a goal of the ROP is to detect declining performance, however, we disagree that the ROP downplays poor licensee performance, thereby allowing plants to operate in an unsafe condition. A recent example of achieving the goal suggested by UCS is the Perry plant, which entered the multiple/repetitive degraded cornerstone column of the Action Matrix due to an accumulation of white findings. This has resulted in a substantial increase in NRC oversight of the Perry plant, although Perry has not experienced a recent relatively highly safety-significant event.

As noted in the SDP evaluation in Attachment 3 to SECY-05-0070, the staff has addressed a number of specific issues in this area, including:

• The timeliness of final significance determinations remains a concern and focus for the staff. Several significant initiatives relied upon by the staff to improve the timeliness, such as the revised fire protection SDP and improvement in the containment SDP, have been completed, but their impact on improving timeliness is not yet known. The staff also recognizes that the new and revised SDPs will not fully resolve the timeliness issues. Therefore additional initiatives are being considered, including expanding the definition and scope of Phase 2, improving guidance on risk-informed decision making based on the best available information within agreed-upon time constraints, and grading the timeliness requirements based on the complexity and risk significance of the finding. As a result of these changes, the staff anticipates notable improvement in SDP timeliness but recognizes continued challenges ahead.

• Initial efforts to benchmark and standardize the Phase 2 risk-informed inspection notebooks have been completed. However, a further standardization effort based on lessons learned is currently underway and will be completed in 2005, at which time Revision 2 of the risk-informed notebooks will be issued. The Revision 2 notebooks will be a tool to both aid inspectors in their understanding of their assigned facility, and to make risk assessments of findings easier. The Revision 2 notebooks will address any major changes in the licensees probabilistic risk analyses (PRAs) to date. It is expected that the notebooks will continue to be evaluated and updated in response to future licensee-implemented changes to plants and the PRA models. Each Revision 2 notebook will include or reference basic pre-solved tables which will identify the value of each sequence when a particular component or human action is degraded. A spreadsheet for each notebook will be completed and available for staff review and use after release of the Revision 2 notebooks. Additionally, the staff has completed development of all Level 1 Revision 3i Standardized Plant Analysis Risk (SPAR) models and will schedule onsite quality assurance (QA) reviews during notebook benchmarking visits to develop a more reliable Phase 3 SDP analysis tool for at-power internal events.

• The Commission requested that the staff provide an evaluation of the effectiveness of recent changes made to improve the timeliness of the fire protection SDP. Inspectors and senior reactor analysts have indicated that the new fire protection SDP issued in May 2004 is providing the expected guidance for evaluating the significance of fire protection findings. A formal evaluation will be conducted to determine whether the SDP meets its intended purpose, including facilitating timely significance evaluations. The results of this evaluation will be included in the CY 2005 ROP self-assessment Commission paper.

The State of New Jersey commented that the staff has used the potentially greater than green designation more often than they had indicated. Response: Only three potentially significant findings have been preliminarily designated as potentially greater than green. Designating findings as potentially greater than green recognizes that at times there is a relatively large amount of uncertainty in the significance assessment. In those cases, the staff considers it appropriate to seek additional information from the licensee, instead of designating a preliminary color to the finding that might change significantly, thereby potentially misleading stakeholders about the importance of the issue.

7. Does the NRC take appropriate actions to address performance issues for those licensees outside of the License Response Column of the Action Matrix?

Respondent Comments:

Union of Concerned Scientists (2nd submittal)

Yes, but with a qualifier.

When a facility wandered out of the Licensee Response Column of the Action Matrix, the NRC took the prescribed steps in response.

But the NRC needs to take more appropriate actions to address performance issues detected at a facility but not quite rising to the level where a move out of the Licensee Response Column occurs. Specifically, these situations arise when the NRC detects a substantive cross-cutting performance issue at a facility. In UCSs view, an alternative to the cross-cutting issue would be to issue greater-than-green inspection findings that easily could move a facil0ity out of the Licensee Response Column. UCS is not contesting or disputing the NRCs use of cross-cutting issues, but we feel that more appropriate actions are necessary by the NRC at these times to ensure that the desired, intended performance improvement results.

We cite the NRCs actions to address the cross-cutting performance issue in the safety conscious work environment (SCWE) at Hope Creek/ Salem as a suitable model. In the mid-cycle assessment letter, the NRC staff identified SCWE as a cross-cutting issue. The NRC staff went on to list five specific regulatory actions it was going to take about SCWE at Hope Creek/Salem. This is a welcome departure from how the NRC typically handles cross-cutting issues. In those other cases, the NRCs assessment letters would inform licensees about the cross-cutting issues (most often in the corrective action area) and the reasons for the NRC s identification. The NRC stopped short of detailing what actions it intended to take about the cross-cutting issues.

When the NRC identifies a substantive cross-cutting performance issue in an assessment letter, the NRC must also clearly delineate the actions it will take regarding that issue. Simply citing cross-cutting performance issues in assessment letter after assessment letter is totally improper regulatory response.

UCS rating for the ROP in this area: White (1)

Prairie Island Nuclear Generation Plant NRC tries to use the rules/regulations to force fit action which the NRC Leadership feels is needed. Getting at plant performance issues through EP compliance or design 50.54(f).

State of New Jersey, Department of Environmental Protection

• The NRC follow-up actions for licensees outside the Licensee Response Column focuses on a very narrow area of inspection.

• Usually by the time the follow-up inspection takes place the event is old news and the issue has long since resolved itself.

• The NRC should expand follow-up inspections to make them more valuable when they occur.

• It appears that non-ROP identified events are driving the NRC to inspect larger areas of plant performance and obtaining better results in promoting plant safety.

Pennsylvania Department of Environmental Protection Based on our experience with the ROP implementation at the PA power plants, the NRC Region 1 has taken appropriate actions to address performance issues for those licensees outside the Licensee Response Column of the Action Matrix. This may or may not be the case in other NRC regions.

Region IV Utility Group The NRC follows the Action Matrix and takes prescribed actions to address performance issues. The NRC actions in the Regulatory Response Column are appropriate, however the NRC engages at too low a level for column 3. Two low level White inputs to the Action Matrix currently create an unnecessary regulatory response based on the risk significance. A change to the Action Matrix should be considered such that three white findings are required to transition to the Degraded Cornerstone column. The additional effort required to prepare for and implement a 95002 inspection is rarely warranted for two low safety significance issues. A threshold of three white findings to change from the Regulatory Response column to the Degraded Cornerstone column in the Action Matrix would be a better use of available resources.

We continue to believe an improvement that should be considered is limiting the length of time a finding is reflected against licensee performance. A graduated approach should be considered correlating the length of time a finding remains visible (or effective in the action matrix) to the severity of the finding (e.g., a green finding stays for one quarter, a white finding stays for 2 quarters, etc.) Rather than retaining all findings for four quarters, this approach results in retaining the finding for a period of time commensurate with its significance. Of course, those findings should continue to be retained until the NRC is satisfied that the issue has been satisfactorily resolved.

Duke Energy Based on our experiences, the NRC action is delayed, inappropriate, and very burdensome for issues that have been resolved. Due to the long lead times in determining the significance color, many issues have been long resolved before a 95001 or 95002 inspection occurs. These inspections are very burdensome for utilities, especially when the performance deficiency has been corrected, as was the case in each of our findings. While the 95002 process may be appropriate, the timing and level of effort involved pulls folks away from solving current issues to supporting closure of past issues for which corrective actions have already been completed.

Nuclear Energy Institute Yes, for the most part we have found that NRC takes appropriate actions to address performance issues for those licensees outside of the Licensee Response Column of the Action Matrix.

Strategic Teaming and Resource Sharing Actions taken by the NRC to address performance issues for licensees outside the Licensee Response Column conform to the current ROP program. While the program is being followed as written, improvements should be considered. The consequences of being outside the Licensee Response Column are severe enough to encourage licensees to resist any type of input greater than green, especially findings that are categorized using deterministic SDPs.

Consideration should be given to increasing the number of white inputs into the Action Matrix from 2 to 3 in order for a licensee to be placed in the Degraded Cornerstone Column or reduce the length of time a finding remains active to a time period that is commensurate with the significance of the finding.

Licensees that are inside the Licensee Response Column are sometimes moved into a defense posture because of additional NRC scrutiny or oversight. We recommend that the NRC resist the temptation to act on a licensee's performance trend in the Green Band.

NRC Response:

UCS commented that the NRC should take more action for cross-cutting issues identified at plants that are in the licensee response column. Response: This recommendation is contrary to a fundamental principle of the ROP. That being that a given level of licensee performance results in the actions defined by the Action Matrix. Substantive cross-cutting issues are not an input to the Action Matrix. The identification of substantive cross-cutting issues is a pro-active element of the ROP to inform licensees that the NRC has identified a performance trend that may warrant the licensees attention, but in the case of licensees in the Licensee Response Column, the issue has not manifested itself in identified performance deficiencies at a significance level above green. Based on a number of comments from both external and internal stakeholders, IMC 0305 has been significantly revised to enhance the documentation and followup of substantive cross-cutting issues. (See paragraph 0606.I of IMC 0305)

With regard to the UCS comments about Salem/Hope Creek, as addressed in the staff response to question #2, the Commission has directed the staff to enhance the ROP in the area of safety culture.

As noted in the assessment evaluation in Attachment 4 to SECY-05-0070, the staff recognizes that additional guidance is needed in the area of cross-cutting issues. As a result, the staff plans to revise the guidance documents in order to support the mid-cycle review meetings scheduled for August 2005. Improvements include providing examples or screening criteria to define a minimum threshold for assigning a cross-cutting element to a finding, better definitions of the human performance bins, and revisiting the exit criteria for substantive cross-cutting issues.

As discussed in the SDP evaluation in Attachment 3 to SECY-05-0070, the timeliness of final significance determinations remains a concern and focus for the staff. Several significant initiatives relied upon by the staff to improve the timeliness, such as the revised fire protection SDP and improvement in the containment SDP, have been completed. Additional initiatives are also being considered to improve SDP timeliness as discussed in the response to question #6.

As outlined in the assessment evaluation in Attachment 1 to SECY 04-0053 (the CY 2003 ROP self-assessment), the staff performed a detailed analysis of the industrys recommendation to increase the threshold for a degraded cornerstone from two to three white PIs or inspection findings, as directed by the Commission SRM dated June 10, 2003 (ADAMS accession number ML031900342). As documented in a memorandum to the Commission, dated August 29, 2003, the staff did not support changing the existing threshold of two white inputs to three white inputs, in part, for the following reasons:

• The staff reviewed the plants that have entered the degraded cornerstone column or multiple/repetitive degraded cornerstone column of the Action Matrix during the 3-year period from April 1, 2000, through March 31, 2003. That review revealed that 4 of the 11 plants that entered the degraded cornerstone column would not have entered that column if the entry threshold had been three white inputs, rather than the current threshold of two white inputs. As a result, those plants would have received a less-intensive IP 95001 inspection instead of an IP 95002 supplemental inspection. After further review of the IP 95002 inspections that were performed, the staff concluded that in these four cases, IP 95002 was the appropriate inspection for the issues at the plants, and that the degraded cornerstone column of the Action Matrix was the appropriate action level.

• The SDP Task Group concluded that the current threshold of two white inputs in the same cornerstone as the criterion for a degraded cornerstone was reasonable and there was no information to suggest that it was inappropriate.

The industry also recommended a graded approach for removing inspection findings from consideration in the assessment program. This recommendation involved applying a graded approach based on safety significance, such that white findings would remain in the assessment program for two quarters, yellow findings for three quarters, and red findings for four quarters. As discussed in the assessment evaluation in Attachment 1 to SECY 04-0053 (the CY 2003 ROP self-assessment), the staff disagreed with this approach because the range of actions across the Action Matrix is graded, such that increased regulatory actions occur with the accumulation of greater than green assessment inputs. One concern with the industrys recommendation is that inspection findings would not remain in the assessment program long enough to allow increased NRC action with degrading performance, as envisioned during the development of the ROP. This would be inconsistent with the PI program, in which the indicators reflect performance over the past year or more based on specific algorithms.

Additionally, experience since the inception of the ROP indicates that, in many cases, the licensees corrective actions were not completed and were not deemed adequate within the four quarters for consideration in the Action Matrix in accordance with the existing program. The staff does not currently plan to change this policy or expend additional resources to further evaluate this industry recommendation. However, the staff will continue to review the Action Matrix annually as part of the self-assessment and the Agency Action Review Meeting (AARM),

to assess the appropriateness of the criteria for determining a licensees placement in the Action Matrix.

8. Is the information contained in assessment reports relevant, useful and written in plain English?

Respondent Comments:

Region 5/6 Emergency Management, NE It is understandable to me.

Union of Concerned Scientists (2nd submittal)

Nope. Unlike the inspection reports that contain relevant and useful information written in plain English, the assessment reports contain irrelevant and useless information written in plain English.

The inspection reports and assessment reports are both written using a formal process designed to promote conformity in the products generated. As detailed in the response to Question (5) above, the inspection reports provide relevant, useful information about what was looked at and what was found. The conformity as applied to the inspection reports causes them to look the same, but not to say the same things.

Excessive conformity renders the assessment reports garbage. They now all say virtually the exact same thing - nothing. If you compare the mid-cycle assessment report for the best performing plant in the country to the mid-cycle assessment report for the worst performing plant in the country, youll see that the difference is a mere handful of words. The assessment reports should clearly explain why the NRC believes the end-point plants are at their respective endpoints and why the NRC believes the remaining plants are in between.

UCS has tried very hard over the years to figure out what the assessment reports are saying.

They tell us nothing about what has happened in the past and give us little clue about what NRC intends for the future. There is little difference between NRC assessment reports and blank pages in terms of informational value.

The assessment reports need to become as relevant and useful as are the inspection reports.

[UCS observes that the August 30, 2004, mid-cycle assessment letters sent by the NRC to the Hope Creek and Salem licensees are the exception they alone have been useful from among all the prior assessment letters.]

UCS rating for the ROP in this area: Yellow (1)

Prairie Island Nuclear Generation Plant Well written, regulatory relevant, but doesnt really tell you how the regulator feels about your performance. The lack of issues seems to indicate the site is doing OK.

State of New Jersey, Department of Environmental Protection

• The NRC licensee assessment letter is one page long - what report?

• Since this letter is the final word on the assessment of the plant for a set period of time and an opportunity to outline future plant expectations more assessment and value added is necessary.

• The application of the ROP ignores assessment and focuses on color.

Pennsylvania Department of Environmental Protection The initial assessment reports were very stilted and sometimes unclear. However, the reports continue to improve in readability and content and usefulness.

Region IV Utility Group Yes, the information contained in the assessment reports is relevant, useful, and written in plain English. There are, however, continued questions about the basis and closure process for Substantive Crosscutting Issues identified in the assessment letters.

Duke Energy Relevant - yes, Useful - yes, In plain English - no. The information contained in an assessment would be very difficult for members of the public to digest, and is often confusing to the licensee. This is evidenced by the local media's non-understanding of either the ROP process and the results.

Nuclear Energy Institute The new format is brief and focused on objective performance measures. Industry has found the reports to be relevant, useful, and written in plain English. We are concerned, however, about the bases and closure process for substantive crosscutting issues. Finally, we believe NRC should consider using the public meetings associated with the annual assessment reports as an opportunity to do more outreach/education work with the public on the reactor oversight process and its value in maintaining safety, reducing unnecessary regulatory burden, improving effectiveness and efficiency, and informing the public.

Strategic Teaming and Resource Sharing Yes, the information contained in assessment reports is relevant, useful, and written in plain English.

NRC Response:

UCS commented that if you compare the mid-cycle assessment report for the best performing plant in the country to the mid-cycle assessment report for the worst performing plant in the country, that the difference is a mere handful of words. UCS also stated that the assessment reports should clearly explain why the NRC believes the end-point plants are at their respective endpoints and why the NRC believes the remaining plants are in between. Response: The staff disagrees with these comments. First the NRC does not rank plants from top to bottom.

Plants are assigned to columns in the Action Matrix consistent with their inspection findings and PI inputs. There is no curving of the assessment. Conceivably all plants could be assigned to a single column. As such, the assessment letters should explain why a given plant is assigned to its respective column. We believe that the assessment letters accomplish this. As UCS suggested, the staff compared the most recent annual assessment letter of a plant in the Licensee Response column (McGuire Nuclear Station) to the assessment letter of a plant in the Multiple/Degraded Cornerstone Column (Point Beach Nuclear Plant), and found the difference to be more than a few words. For example, the Point Beach annual assessment letter dated March 2, 2005, included significant discussions of the red and yellow inspection findings, the associated CAL, Agency monitoring activities, substantive cross-cutting issues in human performance and PI&R, and a severity level III violation with a $60,000 civil penalty. The scope and depth of this letter was significantly more than the letter issued for McGuire, which had no performance indicators and inspection findings that were greater than green.

Program guidance directs that the level of detail in the assessment letters increases as the plant performance decreases as detailed in IMC 0305, Operating Reactor Assessment Program. For example, IMC 0305 requires additional detail for those plants outside of the licensee response column of the Action Matrix and/or those plants that have substantive cross-cutting issues. In accordance with IMC 0305, the mid-cycle and annual assessment letters shall contain:

1. A summary of safety significant PIs and inspection findings for the most recent two quarters as well as discussion of previous action taken by the licensee and the agency relative to these issues.

2. A discussion of any deviations from the Action Matrix during the assessment period.

3. A qualitative discussion of substantive cross-cutting issues, if applicable.

4. A discussion of the licensees progress in addressing a substantive cross-cutting issue, if documented in the previous mid-cycle or annual assessment letter.

5. A discussion of non-SDP (severity level III or greater) enforcement actions.

6 A discussion of findings that are currently being evaluated by the significance determination process that may affect the inspection plan.

7. A statement of any actions to be taken by the agency in response to safety significant issues, as well as any actions taken by the licensee.

8. An inspection plan consisting of approximately 18 months (from the issuance of the mid-cycle letter) of activities.

The staff considers that the above response addresses similar comments received from the State of New Jersey.

As previously discussed under question #7, the staff recognizes that additional guidance is needed in the area of cross-cutting issues and is revising the guidance accordingly.

NEI commented that the NRC should consider using the public meetings associated with the annual assessment reports as an opportunity to do more outreach/education work with the public on the ROP. The staff believes that this already occurs, but is willing to discuss specific details that the industry may have to offer.

9. Are the ROP oversight activities predictable (i.e., controlled by the process) and reasonably objective (i.e., based on supported facts, rather than relying on subjective judgement)?

Respondent Comments:

Union of Concerned Scientists (2nd submittal)

I guess so.

UCS rating for the ROP in this area: U Prairie Island Nuclear Generation Plant There is a predictable process but the finding is still very dependent on the predisposition of the inspector.

State of New Jersey, Department of Environmental Protection

• Enough evidence exists through the number of fines and the amount of fines that the ROP is slanted toward licensees.

• The ROP maintains the status quo and is not designed to promote continuous improvement over the long anticipated life of nuclear power plants.

• The goal of the ROP should include ensuring that plants operate seamlessly, without incident, and safe for potentially 60 years.

Pennsylvania Department of Environmental Protection The new ROP is more objective and predictable than the previous process. This is due to the combination of Performance Indicators and a more objective and better structured Inspection and Assessment Program.

Region IV Utility Group For the majority of the normal baseline inspections, the ROP oversight activities are predictable and objective as reported in the end product (i.e. the inspection report). During the course of the actual inspection activities this is not always the case. Most inspectors follow the guidance but a few still appear to use aggregation and "reverse SDP" techniques. (Reverse SDP means predetermining significance of an issue based on subjective judgment then developing the supporting arguments.) The subjective nature of some of the SDP screening questions reduces the predictability of the ROP oversight activities.

There are not clear criteria for the documentation of crosscutting aspects. What guidance does exits is not definitive or consistently applied, and relies on individual subjectivity. There are also continued questions about the basis and closure process for Substantive Crosscutting Issues identified in the assessment letters. There do not seem to be established criteria for identification and resolution of these Substantive Crosscutting Issues and the process appears to be very subjective.

Duke Energy Predictability is clearly an area that needs improvement. We have our PRA model and the NRC has theirs. We can both have supportable inputs which differ and yield different results, and the NRC will always default to the most conservative "because they can. In an example, the NRC uses an unapproved draft model to predict PSV failure rates. A utility charters an EPRI sanctioned Expert Elicitation Panel to study the issue and arrives at slightly different failure rates. The EPRI value is a clearly more recent, and arguably more accurate, document yet the NRC will not accept it. This seems contrary to the notion of using the most accurate available information. It seems that the most predictable and objective approach would be to inspect our use of our PRA to see if our inputs are reasonable and our results are valid.

Nuclear Energy Institute For the majority of the normal baseline inspections, the ROP oversight activities are predictable and objective as reported in the end product (i.e. the inspection report). During the course of the actual inspection activities this is not always the case. Most inspectors follow the guidance but a few still appear to use aggregation and "reverse SDP" techniques. (Reverse SDP means predetermining significance of an issue based on subjective judgment and then developing the supporting arguments.) The subjective nature of some of the SDP screening questions reduces the predictability of the ROP oversight activities.

There are not clear criteria for the documentation of crosscutting aspects. What guidance does exist is not definitive or consistently applied, and relies on individual subjectivity. There are also continued questions about the bases and closure process for substantive crosscutting issues identified in the assessment letters. There do not seem to be established criteria for identification and resolution of these substantive crosscutting issues and the process appears to be very subjective.

Strategic Teaming and Resource Sharing The ROP oversight activities are predictable and objective when a plant is in the Green Band of the Action Matrix with no events. Inspectors typically follow the guidance provided in the baseline inspection procedures. However, once an event occurs, and management directive 8.3 is being used, the oversight activities are not as predictable.

Categorizing findings is not as predicable as the inspection process. When a finding doesn't neatly fit in an SDP, there appears to be an effort to classify it as a 10 CFR 50 Appendix B, Criterion XVI violation or a cross cutting issue which are more subjective in nature. An example of this is in the area of corrective action associated with conditions classified as adverse to quality. Criterion XVI specifies that conditions adverse to quality need to be promptly identified and corrected. If a problem classified as a condition adverse to quality recurs, some inspectors view the recurrence as ineffective corrective action because the root cause was not addressed.

Although significant conditions adverse to quality do require actions to determine the cause and actions to preclude repetition, there is no requirement in Criterion XVI to address the root cause of all condition adverse to quality.

Also, the current ROP guidance lacks criteria for defining the specific aspects that make up cross cutting issues. As such, the application of cross cutting issues is very subjective. The identification of substantive cross cutting issues often lacks the clarity required to ensure the appropriate focus for corrective actions. As a result, the criteria to be used to assess removal of the substantive crosscutting issue are very subjective.

There are not clear criteria for the documentation of cross-cutting aspects. What guidance does exist is not definitive or consistently applied, and relies on individual subjectivity. The bases for documentation of cross-cutting aspects are not consistently discussed with licensees at the level of detail that findings are discussed. They sometimes get added as almost an afterthought with little or no discussion. In addition, a Problem Identification and Resolution (PI&R) cross-cutting aspect may be tagged to a finding contrary to corrective action program requirements. Corrective action program requirements do not require "prevention of recurrence" for all issues and, therefore, to tag a recurrence of such an issue with a PI&R cross-cutting aspect is incorrect.

NRC Response:

Prairie Island commented that there is a predictable process but the finding is still very dependent on the predisposition of the inspector. Response: All findings that are potentially greater than green receive considerable management attention and are reviewed by an SDP and Enforcement Review Panel (SERP). As part of the process, licensees are invited to formally comment on potentially greater than green findings. The predisposition of an inspector is unlikely to influence the outcome. Findings that are green are subject to regional management review. Program documents provide guidance on when to assess a finding as green.

The industry comments that some inspectors use subjective judgement, aggregation and "reverse SDP" techniques is also addressed by the above response.

The State of New Jersey commented that the number of fines and the amount of fines indicate that the ROP is slanted toward licensees. Response: Consist with the Commissions Enforcement Policy, inspection findings under the ROP are not considered for monetary fines unless the finding meets specific criteria such as willfulness or actual consequences.

Consequently, the number of fines assessed to power reactor licensees is small.

The State of New Jersey also commented that the ROP maintains the status quo and is not designed to promote continuous improvement over the long anticipated life of nuclear power plants. Response: This is a correct statement and reflects the Commissions policy. In implementing the ROP, the Commission recognized that the overall performance of the industry has significantly improved. The staffs latest annual paper to the Commission on the Industry Trends Program (ADAMS accession number ML050820301) contains graphs of performance indicators which demonstrate this trend. Recognizing the general good performance of the industry, the ROP is focused on identifying plants with relatively weaker performance, so that additional regulatory oversight can be brought to bear, while at the same time carrying out a baseline level of inspection at all plants to monitor for declining performance.

As previously noted under question #7, the staff recognizes that additional guidance is needed in the area of cross-cutting issues and is revising the guidance accordingly.

Duke Energy commented that SDP results between the licensee and the NRC may differ, and that the NRC will always default to the most conservative result "because they can.

Response: The data does not support this comment. To date, 21 findings have been changed from their preliminary significance determination, in part because the licensee provided information that supported the change. As stated in ROP program guidance, it is the responsibility of the NRC to objectively consider the available information and to reach a final significance determination. The staff believes that this is an appropriate process.

As noted under question #6, efforts are underway to benchmark and standardize the Phase 2 risk-informed inspection notebooks.

10. Is the ROP risk-informed, in that the NRCs actions are graduated on the basis of increased significance?

Respondent Comments:

Union of Concerned Scientists (2nd submittal)

No, but the fault is the silly SDP used to determine graduation. The ROP is set up to provide commensurate NRC responses if it wasnt for the wrong answers coming out of the SDPs. If the SDPs were fixed, the NRCs responses under the ROP would be more commensurate with the significance of the performance deficiencies.

UCS rating for the ROP in this area: U (the problem lies elsewhere and UCS did not feel it fair to assign a second greater-than-green rating for the same root cause)

State of New Jersey, Department of Environmental Protection

• Overall, the ROP is moving toward a risk-informed process.

• The biggest concern is the way the findings are handled by the NRC and the licensee.

• We reviewed most findings and most are aggressively defended by the licensee to lower the color finding because the licensees know that the probabilistic risk assessments have a lot of uncertainty and can be massaged in the direction of the lower color.

Pennsylvania Department of Environmental Protection Overall, the ROP is more risk-informed than the previous process and the NRC actions are generally graduated on the basis of increased risk significance. However, the lack of standardized risk analysis tools has diminished the effectiveness of the process.

Region IV Utility Group The majority of the ROP is risk-informed due to actions taken over the past three years of implementation to further risk-inform the process. Actions that result from findings that are classified using the Reactor Safety SDP, IMC 0609 App A, are the most risk-informed and are the ones most graduated on the basis of an actual increased significance. Actions resulting from findings that are classified based on SDPs that are still deterministic in nature are not as likely to be graduated consistent with actual significance. For example, the number of occurrences does not equate readily to the "significance" of an issue. It would seem that the "significance" of each occurrence would have to be the overriding consideration, rather than the aggregation of a few "minor" items or the sheer number of insignificant occurrences. We believe that a degraded cornerstone should result from three, rather than two, white inputs (inspection findings and PIs), and the period of time findings remain in the action matrix should be graduated based on safety significance.

Duke Energy Yes, but...Actions taken for CDF results of 1.04 E-6 are significantly above 9.99 E-7, and there is no real increase in risk. There should be judgment applied, but NRC approaches in a "black or white" manner. The NRC seems to follow the process "blindly" with no room for judgment or interpretation. The default position is that in time where judgment can be applied, the NRC just says potentially > green and calls the utility down to defend itself. This is incredibly resource intensive and again, diverts resources from solving current safety issues to defend an issue which may be over 2 years old and already solved. It also puts the utility in the "guilty until proven innocent" column with no incentive for the NRC to come to any conclusion except "guilty" as the NRC has already invested great resources into the issue.

Nuclear Energy Institute The majority of the ROP is risk-informed due to actions taken over the years of implementation to further risk-inform the process. Actions that result from findings that are classified using the Reactor Safety SDP, IMC 0609 Appendix A are the most risk informed and are the ones most graduated on the basis of an actual increased significance. Actions resulting from findings that are classified based on SDPs that are still deterministic in nature are not as likely to be graduated consistent with actual significance. As we stated last year, we believe that white inspection findings should be closed out and removed from the action matrix when the NRC follow-up inspection has determined that effective corrective action has been achieved.

Strategic Teaming and Resource Sharing Many areas of the ROP are risk-informed such that NRC actions are on the basis of increased significance but there are some areas that are still deterministic in nature such as Emergency Preparedness, Security, and Operator Training. The areas covered by Manual Chapter 0609, Appendix A are the most risk informed. Efforts to better risk-inform the ROP should continue.

NRC Response:

As previously noted, the staff recognizes and is working towards more standardized risk analysis tools.

As discussed under question #6, and as noted in the SDP evaluation in Attachment 3 to SECY-05-0070, the staff believes that a relative parity has been achieved among the cornerstones. Additionally, the verification of parity for all findings other than green is an ongoing process conducted by the significance determination process and enforcement review panel (SERP). One of the roles of the SERP is to examine all the facts associated with the findings and determine the appropriateness of the recommended color and the corresponding NRC response. The preliminary and final color associated with a finding reflects its safety significance due to impaired or loss of function, root cause and extended condition results, NRC appropriate response level, and responsibility for public health and safety, among others. For the more deterministic SDPs, all of which include various measures of risk in their bases (such as limits for exposure), the non-risk inputs play a more important role when safety significance is considered.

As noted in the assessment evaluation in Attachment 1 to SECY-04-0053 (the CY 2003 ROP self-assessment), the staff performed a detailed analysis of the industrys recommendation to increase the threshold for a degraded cornerstone from two to three white PIs or inspection findings, as directed by the Commission SRM dated June 10, 2003. As documented in a memorandum to the Commission, dated August 29, 2003, the staff did not support changing the existing threshold of two white inputs to three white inputs for several reasons. Additional detail is provided in response to question #7.

As noted in the assessment evaluation in Attachment 1 to SECY-04-0053 (the CY 2003 ROP self-assessment), the industry had recommended a graded approach for removing inspection findings from consideration in the assessment program. The staff disagreed with this approach for several reasons as previously discussed in response to question #7.

Duke Energy commented that actions taken for [delta] CDF results of 1.04 E-6 are significantly above those actions for results of 9.99 E-7, although there is no real increase in risk. Duke Energy further stated that there should be judgment applied, but NRC approaches the decision in a "black or white" manner, and that the NRC seems to follow the process "blindly" with no room for judgment or interpretation. Response: The staff disagrees with this comment. All potentially greater than green findings are reviewed by a SDP and Enforcement Review Panel (SERP). For findings in the reactor safety cornerstones (except EP), the SERP consists of a regional manager, a manager from the NRR Probabilistic Assessment Branch, a manager from the NRR Inspection Program Branch, and a representative from the Office of Enforcement.

The SERP is usually supported in their discussions by a number of technical staff. The SERP recognizes that typically there is a level of uncertainty in the calculation of the delta CDF for a finding. Because the program is risk-informed and not risk-based, the SERP is not bound or required to define the finding significance based on a single calculated delta CDF number.

Rather the SERP conducts its business very much like Duke suggests it should, using judgement by considering the uncertainties associated with the delta CDF calculation, and other inputs, to arrive at the significance determination.

Duke Energy also commented that where judgment can be applied, the NRC just says potentially > green and calls the utility down to defend itself. This is incredibly resource intensive and again, diverts resources from solving current safety issues to defend an issue which may be over 2 years old and already solved. It also puts the utility in the "guilty until proven innocent" column with no incentive for the NRC to come to any conclusion except "guilty" as the NRC has already invested great resources into the issue. Response: The staff disagrees with this comment. Designating findings as potentially greater than green recognizes that at times there is a relatively large amount of uncertainty in the significance assessment. In those cases, the staff considers it appropriate to seek additional information from the licensee, instead of designating a preliminary color to the finding that might change significantly, thereby potentially misleading stakeholders about the importance of the issue.

Thus licensees are offered the opportunity to provide input to the process, either in writing or in a meeting format. Licensees are not required to do either. The licensee, not the NRC, determines the level of resources to invest in their participation in the process. With regard to SDP timeliness, we agree that the process needs to be more timely, and as previously discussed under question #6, we are working to improve in this area. We also disagree with the comment that the NRC will default to "guilty" because of the staff resources invested in an issue. One of the over-riding principles of the ROP is to be objective. Again, one of the purposes of the SERP is to ensure results are objective. There have been several cases where the significance of findings were changed based on licensee input.

11. Is the ROP understandable and are the processes, procedures and products clear and written in plain English?

Respondent Comments:

Region 5/6 Emergency Management, NE Most understandable.

Union of Concerned Scientists (2nd submittal)

Si.

UCS rating for the ROP in this area: neerG State of New Jersey, Department of Environmental Protection

• We understand it more now then we did when it was first introduced.

Pennsylvania Department of Environmental Protection Overall, the ROP is an understandable process. However, there are certain aspects of the new process that are not always as clear as they could be. For example, the SDP (particularly phase 2 and 3) is a complex and complicated process.

Region IV Utility Group In general the ROP is understandable and the processes, procedures, and products are clear and written in plain English. There is no guidance or established process for the closing of a Substantive Crosscutting Issue. Additionally, some of the newer SDPs do require a technical background to understand. The Fire Protection, Shutdown, and Steam Generator SDPs have been particularly difficult to follow.

Duke Energy No - Especially in the case of the 95002 inspection guidance. The scope of this type of inspection can be expanded or contracted based solely on inspector preference. There is absolutely no predictability in this area, nor will NRC engage in meaningful conversations ahead of a 95002 inspection to bound the issue.

Nuclear Energy Institute In general the ROP is understandable and the processes, procedures, and products are clear and written in plain English. There is little guidance or established process for the closing of a substantive crosscutting issue. There are some areas that need clarification such as the definitions and distinction amongst self identified, self revealing, and inspector identified.

Because of their technical complexity, some of the newer SDPs do require a specific technical background to understand. The Fire Protection, Shutdown, and Steam Generator SDPs have been particularly difficult to follow. The Fire Protection SDP is more technically accurate than its predecessor in determining risk significance, but it also requires more knowledge of risk techniques.

Strategic Teaming and Resource Sharing Generally the ROP is understandable and the process, procedures, and products are clear and written in plain English. There are some areas that need clarification such as the definitions "self-identified," "self-revealing," and "inspector identified" in Manual Chapter 0612. Also, the term "mining" used in the context of "mining the corrective action program" has a different meaning to licensees than it does to NRC inspectors.

NRC Response:

As previously discussed under question #7, the staff recognizes that additional guidance is needed in the area of cross-cutting issues.

As previously noted under question #6, an SDP standardization effort is currently underway.

Two industry commenters stated that definitions and distinction amongst self identified, self revealing, and inspector identified need clarification. A revision to IMC 0612, Power Reactor Inspection Reports, which addresses this concern, has been drafted and should be issued in the next few months.

Duke Energy commented that in the case of the 95002 inspection, the scope of this inspection can be expanded or contracted based solely on inspector preference. There is absolutely no predictability in this area, nor will NRC engage in meaningful conversations ahead of a 95002 inspection to bound the issue. Response: The staff considers this comment to be an inaccurate portrayal of the process. The scope of the inspection is defined within the inspection procedure. The inspection is not without bounds. Regional management and the inspection staff apply judgement in determining the scope of the inspection, within the limits defined by the inspection procedure. It is appropriate to not share the responsibility for defining the scope of the inspection with the licensee being inspected.

12. Does the ROP provide adequate regulatory assurance when combined with other NRC regulatory processes that plants are being operated and maintained safely?

Respondent Comments:

Union of Concerned Scientists (2nd submittal)

No. Davis-Besse and Hope Creek/Salem prove this point beyond even the shadow of a doubt.

The NRCs own Lessons Learned Task Force (LLTF) following the Davis-Besse debacle made 51 recommendations on things the NRC could do to prevent recurrence of this regulatory breakdown. The Commission approved 49 of the recommendations. But many of these known weaknesses and faults in the NRC regulatory processes remain uncorrected today. The answer to this question cannot be Yes when these known deficiencies exist. After all, these deficiencies allowed Davis-Besse to be operated and maintained with inadequate safety levels.

In addition, the ROP missed serious problems at Hope Creek/Salem. The Institute for Nuclear Power Operations (INPO) had dropped its rating for the site to 3 while the ROP tracked performance largely as Green, particularly for Hope Creek. In early 2004, the Utility Services Alliance (USA) company rated performance at Hope Creek/Salem as less than competent in 73 of 92 areas. The NRCs assessment of conditions at Hope Creek/Salem were disparate from the assessments by INPO and USA. This grade inflation by the NRC allowed the Hope Creek/Salem plants to operate too long with unacceptable safety levels.

There is some good news to temper these reports. In mid 2004, the NRC identified safety conscious work environment problems at Hope Creek/Salem as cross-cutting issues. The ROP was revised to allow the NRC to identify safety culture problems sooner and intervene earlier when potential warning signs are detected. This is a very positive step.

UCS rating for the ROP in this area: Yellow (1) (it would have been Red (1) but the safety culture revision to the ROP was credited as ample evidence of improvement in this area)

Prairie Island Nuclear Generation Plant The plants are much safer and predictable today than just 5 years ago. But the NRC process can not/should not be viewed as being predictive. The NRC can measure/inspect that minimum processes/systems performance is in place but not that the leadership can improve it.

State of New Jersey, Department of Environmental Protection

• Davis Besse, Hope Creek, Salem 1, Salem 2, Oyster Creek, and Indian Point are a few examples that put into question the ROP.

Pennsylvania Department of Environmental Protection There are no signs of declining plant safety at any of the nine operating reactors in Pennsylvania since the implementation of the ROP. Although, performance at one of the older plants has declined over the past few years due to materials condition of the plant, human performance issues, and problems with plant procedures. We are also concerned about the problems at Davis-Besse plant and the ability of the ROP to detect problems or weaknesses in cross-cutting areas in a timely manner. It should be mentioned that the problems at Davis-Besse have eroded public confidence in the new ROP.

Region IV Utility Group Yes, the ROP provides adequate assurance that plants are being operated and maintained safely as indicated by the continuously improving industry trends.

Duke Energy Due to the long time for development of issues and assessment in SDP space, many issues are already corrected by the time a finding is assessed. The ROP needs to provide more timely results and assessments, and timing seems to run from 1 year to over 3 years before an issue is finally resolved and a risk color assigned.

Nuclear Energy Institute Industry believes that the ROP provides adequate regulatory assurance, when combined with other NRC regulatory processes, that nuclear plants are being operated and maintained safely within the known design basis and operating experience of the industry. The process is designed to identify weaknesses of lower safety significance before safety breakdowns occur which result in the potential for risk to the public. The Davis-Besse event, however, made it clear that continual improvement is needed in the ROP and in licensees oversight programs to incorporate operating experience and new phenomena which were unknown and therefore were not included in inspection or oversight. NRC and industry must continue to assess and strengthen operating experience and corrective action programs to avoid complacency and blindness to evolving degradation.

Strategic Teaming and Resource Sharing The ROP does provide adequate regulatory assurance that plants are being operated and maintained safely.

NRC Response:

UCS commented that the Davis-Besse LLTF made 51 recommendations, and the Commission approved 49 of the recommendations, but that many of the weaknesses and faults in the NRC regulatory processes remain uncorrected. Response: As previously discussed under question

1. 2, the staff agrees that there are lessons to be learned from the Davis-Besse event. There have been a number of program changes made to the ROP as a result of Davis-Besse. A summary of agency actions and their status is located on the agency website at http://www.nrc.gov/reactors/operating/ops-experience/vessel-head-degradation/lessons-learned

/lessons-learned-files/lltf-web-page-status022205.pdf. This document was last modified in February 2005 and is due to be updated in August 2005. However, all of the recommendations associated with the ROP have been acted on.

UCS also commented that the ROP missed serious problems at Hope Creek/Salem and that the NRC allowed the Hope Creek/Salem plants to operate too long with unacceptable safety levels. Response: Also as previously noted under question #2, at the Commissions direction, the staff proposed options to enhance staff actions regarding SCWE and safety culture in Commission paper SECY-04-0111, Recommended Staff Actions Regarding Agency Guidance in the Areas of Safety Conscious Work Environment and Safety Culture. The Commission directed the staff to take action in this area as outlined in the associated Staff Requirement Memorandum (website address http://www.nrc.gov/reading-rm/doc-collections/commission/srm/2004/2004-0111srm.pdf). The staff is working to carry out the Commissions direction in this area. The staff disagrees with the comment that the plants operated with unacceptable safety levels.

In response to the comment regarding the NRCs missed opportunity to consider independent evaluations in the NRCs assessment process, the staff revised IMC 0305, Operating Reactor Assessment Program, in December 2004 to include the consideration of independent evaluations as a result of a Davis-Besse Lessons Learned Task Force recommendation.

Specifically, IMC 0305 requires that the mid-cycle and end-of-cycle reviews consider the conclusions of any independent assessments of a licensee to provide a means of self-assessing the NRC inspection and assessment process.

As previously discussed under question #6, the staff agrees that SDP timeliness needs to improve.

13. Does the ROP improve the efficiency, effectiveness, and realism of the regulatory process?

Respondent Comments:

Region 5/6 Emergency Management, NE Marked improvement.

Union of Concerned Scientists (2nd submittal)

Yes, but the NRCs Systematic Assessment of Licensee Performance that it replaced was so inefficient, ineffective, and unreal that it would be virtually impossible for the ROP not to show improvement in these areas.

UCS rating for the ROP in this area: Green State of New Jersey, Department of Environmental Protection

• The ROP is more efficient and more real but, as we have already addressed, not as effective as it could be.

Pennsylvania Department of Environmental Protection In general, the ROP has improved the effectiveness of the regulatory process. However, one of its weaknesses is in the area of timeliness. There continues to be challenges to the SDP greater-than-green findings by the licensees. These challenges, along with the lack of standardized risk analysis tools have resulted in delays in the determination of the final SDP findings. Also, additional time and data is needed to assess the ability of the ROP to detect, in a timely manner, adverse trends in cross-cutting areas.

Region IV Utility Group Yes, in comparison to the SALP process the ROP improves the efficiency, effectiveness, and realism of the regulatory process. However, in some cases the efficiency and effectiveness are decreasing in the existing ROP process. The scope and resources needed for the baseline Radiation Protection Cornerstone inspections seem excessive relative to overall industry performance. The NRC should consider reevaluating the frequency of these inspections.

Many new and specialized SDPs are being added to the program without a complete evaluation of the adequacy of the existing SDPs. The development of many SDPs are complicating the ROP process and causing significant training issues for the NRC inspection staff as well as licensees. The NRC has provided routine opportunities for the industry and the public to participate in monthly task force meetings designed to improve the ROP. The NRC has been receptive to industry and public comments made during these meetings.

Duke Energy Not entirely. There is nothing efficient about the ROP, as the process drags on and does not promptly assess possible performance deficiencies. We have several items open for over 2 years without being informed of a performance deficiency. If a PD is identified, the issue will then enter the SDP process and it will be even longer before a finding is issued for something that was corrected years ago. Regarding realism, I would offer that once a performance deficiency is found, the SDP process allows too many "what ifs" that are not realistic. For instance, you could say "what if you had a 3g earthquake, and that probability would be entered into the risk equation. A 3g earthquake "could" occur, but the plant is not licensed to that. This would appear to be a defacto backfit in that we are not required to factor in things beyond our approved licensing basis.

Nuclear Energy Institute The ROP is a vast improvement over the old SALP process. Inspections are more focused, findings are evaluated using a more structured tool, and performance assessment is more objective. Efficiency could be gained by combining related inspections activity and evaluating the need for some inspection activity that may be excessive, such as in the area of occupational radiation protection which has been a very good performing area. As discussed above, additional efficiencies could be gained by moving on in an SDP when the point of diminished returns is reached and the effort expended by licensees and the NRC to evaluate the significance of a problem is more than performing an inspection. (However, this approach will require a consideration of the impact on the action matrix as a whole. We will be prepared to meet with NRC early in 2005 to discuss this issue.) In addition, we believe that effectiveness and efficiency could be improved significantly by allowing licensees to employ their own assessments with NRC oversight to substitute for some inspection modules every other inspection cycle, such as the design engineering, radiation protection, and fire protection areas.

NRC intends that fire protection inspections during the next triennial cycle will provide enforcement discretion for findings related to fire-induced circuit failures if a self-assessment using NEI 04-06 or other assessment method is performed. This is a useful step toward increasing the amount of credit provided in inspection space for self assessments covering a broader range of fire protection issues. We encourage NRC to continue work on exploring the possibility of licensee assessment substituting for NRC inspection through a pilot program with industry.

Strategic Teaming and Resource Sharing The ROP is an improvement over the old SALP process. Inspections are more focused, findings are evaluated using a more structured tool, and performance assessment is more objective. Efficiency could be gained by combining related inspection activity and evaluating the need for some inspection activity that may be excessive such as in the area of occupational radiation protection which has been a very good performing area. Additional efficiencies could be gained by moving on in an activity when the point of diminished returns is reached such as in the SDP process when the effort expended by licensees and the NRC to evaluate the significance of a problem is more than performing an inspection.

NRC Response:

A common concern among most respondents was the efficiency of the SDP in regards to timeliness. The staff agrees and as discussed under question #6, is working towards this end.

As noted in the inspection program evaluation in Attachment 2 to SECY-05-0070, the staff began evaluating a process that would allow licensees to receive credit for certain self-assessments. The staff is considering allowing licensees to substitute a self-assessment for specific, predetermined baseline inspections, as long as the self-assessment is conducted in accordance with an NRC-approved industry self-assessment process. The staff would still monitor these self-assessments, but the staff anticipates that resource savings to the NRC and its licensees could be significant for these inspectable areas. The staff plans to conduct a pilot program to ascertain the feasibility of a licensee self-assessment process. However, these efforts were deferred until completion of the design/engineering pilot program. After completing the pilot inspections and assessing the results, the staff intends to evaluate the proposed policy of granting licensee credit for their self-assessment activities.

As noted in the inspection program evaluation in Attachment 2 to SECY-05-0070, the staff performed its annual review of each baseline inspection procedure in CY 2004. Based on this review, the staff did not identify any significant changes to the inspection program, although the staff is making minor adjustments to some inspection procedures. In addition, the staff plans to perform a more detailed analysis of the scope and level of effort of the inspection procedures in CY 2005. The purpose of the analysis will be to better align inspection resources to inspected areas where there is an indication of risk-significant performance deficiencies. As a result of this systematic analysis, the staff plans to adjust existing resources within the baseline inspection program for CY 2006.

As previously discussed under question #6, the staff agrees with the need to increase standardization of SDP tools.

Duke Energy commented that the SDP process allows too many "what ifs" that are not realistic, and that this would appear to be a defacto backfit in that licensees are not required to factor in things beyond their approved licensing basis. Response: The staff considers this comment to be contrary to the actual process. Given that a performance deficiency exists (an issue that is the result of a licensee not meeting a requirement or standard where the cause was reasonably within the licensees ability to foresee and correct, and which should have been prevented), then the staff is required to make reasonable and realistic assumptions in the basis for its significance determinations. Typically this approach benefits the licensee in that the significance determination considers non-safety equipment and operator actions that are outside the licensing basis. It is also a correct statement that significance determinations make realistic assumptions about such inputs as multiple equipment failures and the impact of external events, however it would not be appropriate to arbitrarily consider only those factors which mitigate an event, and not realistically consider those inputs that may negatively impact the event. The staff believes that the SDP process appropriately integrates both mitigating and escalating factors, and assigns probabilities that are consistent with the best available information. Additionally, the SDP process does not backfit requirements on licensees.

Strategic Teaming and Resource Sharing commented that additional efficiencies could be gained by moving on in an activity when the point of diminished returns is reached such as in the SDP process when the effort expended by licensees and the NRC to evaluate the significance of a problem is more than performing an inspection. Response: The staff agrees that the SDP process can be more timely and more efficient, and is working toward that goal.

However, the NRC has no control over the level of resources that licensees invest in the process.

14. Does the ROP ensure openness in the regulatory process?

Respondent Comments:

Union of Concerned Scientists (Initial Submittal)

The NRC must be joking to ask such a question at a time when it has denied the public access to agency records on the reactor oversight process.

Union of Concerned Scientists (2nd submittal)

Less now than in past years. The Commissions lack of discernible interest in the Davis-Besse matter, the closed-door efforts for the Davis-Besse Lessons Learned Task Force recommendation resolutions, and the unfortunate events of August 4, 2004, illustrate this position.

By the NRC staffs own admission, the Davis-Besse reactor head degradation was one of the most risk significant events in the history of nuclear power in the United States. Yet the Commission conducted a grand total of one (1) briefing in public with the 0350 Panel. During that same time period, the Commission held many more public briefings on burden reduction and industry initiatives. The Commission should at least pretend to show interest in safety. In addition, the Commissions lack of interest did a tremendous disservice to the NRC staff working on and supporting the 0350 Panel. Many NRC staff members devoted considerable time inspecting Davis-Besse and ensuring that the facility was ready for restart in March 2003.

Both the public and the NRC staff deserved to see the Commission take an interest in this matter. The Commission has zero excuse for ducking this issue.

The ROP was developed via a very open, accessible process. Since its adoption, the ROP has featured monthly public meetings between NRC staff and public stakeholders to discuss ROP implementation details and proposed revisions. The NRCs Davis-Besse Lessons Learned Task Force (LLTF) recommended 51 revisions to NRC regulatory processes to prevent recurrence. Many of these recommendations involved or affected the inspection, assessment, and enforcement components of the reactor oversight process. Therefore, it was UCSs expectation that the monthly public ROP meetings would be forums for discussing proposed changes to resolve the LLTF recommendations. That was not the case. With the notable and commendable exception of the proposed changes to address the LLTF recommendation about Operating Experience Reviews, the LLTF recommendations have been resolved behind closed doors, even those with direct impact on the ROPs inspection, assessment, and enforcement modules. The NRC should have used its established, public process for implementing LLTF recommendations impacting the ROP. The NRC opted not to do so and therefore cannot receive high grades for openness.

On August 4, 2004, the NRC announced that it was removing security-related information from the ROP. The NRC failed to explain the reason for this removal. When the ROP was created, there were extensive discussions over what security-related information could be provided within the ROP. After the tragic events of 09/11, the NRC pulled the ROP from the public arena and then returned it after a careful review with regard to that security-related information that could be made publicly available. These two NRC deliberations determined that the security-related information provided in the ROP could not be exploited by our enemies.

For some unexplained reason, the NRC reversed these decisions and yanked the security-related information from the ROP. When asked about this reversal, all that the NRC would say is that the yank followed a long, careful consideration of the balance between the publics need to know and the need to protect the facilities.

But the two prior NRC decisions were not rushes to judgment. The two prior NRC decisions also balanced need to know with the need to say no. The public deserved more from the NRC about the basis for its decision to yank security-related information from the ROP nearly three years after 09/11 than the mealy-mouthed, lame, trite excuses.

The NRC cannot claim that the regulatory process is open when it refuses to provide straightforward, frank reasons for its actions. The NRC should be ashamed of its performance on August 4, 2004.

UCS rating for the ROP in this area: Red (1)

State of New Jersey, Department of Environmental Protection

• Even in an environment where the NRC issues no financial penalties, the licensee still fights any color, other than green, as a financial penalty, instead of an opportunity to improve performance.

Pennsylvania Department of Environmental Protection We have no additional comments.

Region IV Utility Group Yes. The ROP provides an objective, repeatable process for assessing plant performance. PIs and inspection results are readily available for public review and scrutiny. The public has participated in the development and revisions to the process.

Duke Energy It certainly could if the workings in the Phase III determination were not so "secret. Basically, the assumptions made for a specific scenario by the NRC are, many times, not known by the utility until the "choice letter" is received. An example of this is the non-industry reviewed tool for evaluating RV head cracks on heads that were replaced years ago. The utility had no idea a Phase III determination was underway until informed during a phone call. From our perspective, ROP is far from transparent.

Nuclear Energy Institute For most areas of the ROP, the regulatory process is open. However, security has been very closed and disassociated with the rest of the ROP. Also, examples of some communication breakdowns have been realized especially when inspection information is processed in the Region. Licensees have reported cases when field inspectors debrief with licensee staff and leave the station with the understanding that an item or event observed is not an issue or one of very little significance. Then a day before (or in some cases the day of) the exit meeting, licensees are told that the finding or observation will be characterized as having a higher level of significance. We are concerned that these changes in characterization are happening during regional review without an open dialogue with the licensee.

Strategic Teaming and Resource Sharing For most areas of the ROP, the regulatory process is open. However, Security has been very closed and disassociated with the rest of the ROP. Also, examples of some communication breakdowns have been realized especially when inspection information is processed in the Region. Field inspectors will routinely debrief with licensee staff and leave the station with the understanding that an item or event observed by the field inspector is not an issue or one of very little significance and then a day before (or in some cases the day of) the exit meeting, licensees are told that the finding or observation will be characterized as having a higher level of significance. We believe these changes in characterization are happening during regional review without an open dialog with the licensee.

NRC Response:

As discussed in more detail under question #15, the staff utilizes a variety of communication activities to ensure that all stakeholders have access to ROP information and results, and have an opportunity to participate in the process and provide feedback. The public outreach and stakeholder involvement in the decision making process during development and implementation of the ROP have both been unprecedented, and the staff continues to focus on stakeholder involvement.

UCS commented that the Commission conducted one briefing in public with the 0350 Panel on the Davis-Besse event, and opined that the Commission should have been more involved.

Response: This comment does not accurately or fairly portray the Commission involvement with Davis-Besse. The Commission recognized the significance of the event and responded accordingly. As UCS has noted, the Commission chartered a Lessons Learned Task Force (LLTF) to carry out an extensive review of the event to identify recommendations for staff action. In accordance with agency procedures, Davis-Besse was placed under the IMC 0350 process and was subjected to a detailed and prolonged review prior to restart of the plant. In part due to Davis-Besse, the Commission directed the staff to explore options for enhancing the ROP in the area of safety culture, and directed the staff to enhance our inspection of engineering. In addition to the Commission briefing by the IMC 0350 Oversight Panel in February 2003, the status of Davis-Besse actions and oversight was discussed with the Commission in each of the last three annual briefings on the results of Agency Action Review Meeting (in May of 2003, 2004, and 2005). The Commission has also been briefed three times on the status of the Davis-Besse LLTF action items and receives semi-annual status reports. A summary of NRC actions and their status as of February 2005 is also available on the NRCs Website at http://www.nrc.gov/reactors/operating/ops-experience/vessel-head-degradation/lessons-learned

/lessons-learned-files/lltf-web-page-status022205.pdf. Finally, the NRC provided many opportunities for public input on Davis-Besse issues and developments and was very open to receiving public comments/questions. The NRC held numerous noticed public meetings, most near the site, which were transcribed and/or included a phone line that the public could listen in on; issued monthly newsletters at monthly public meetings near site; developed and maintained a Web page devoted to Davis-Besse; and responded to thousands of letters and e-mails from the public.

UCS commented that it was UCSs expectation that the monthly public ROP meetings would be forums for discussing proposed changes to resolve the LLTF recommendations. Response: It is not the intent of the staff to seek external stakeholder comment on detailed changes to the ROP. Nonetheless, contrary to UCSs assertion, Davis-Besse LLTF topics were discussed during several of the monthly ROP public meetings, including discussions of the development of a new RCS leakage PI, the use of operating experience, changes to inspection program guidance, and others. In addition, the Davis-Besse LLTF recommendations and status of action items were discussed in detail in at least eight noticed public meetings, including four Commission briefings as noted above. The Davis-Besse LLTF recommendations and action items, as well a specific subtopics resulting from the DBLLTF recommendations (i.e., safety culture, operating experience, ROP improvements) have also been the topics of several breakout sessions at the annual Regulatory Information Conference over the past three years.

UCS commented negatively on the Commissions decision to remove from public access information regarding security. UCS opined that the Commission should explain this action to the public in a more thorough manner. Response: NRC's primary mission is to protect the public health and safety, and the environment from the effects of radiation from nuclear reactors, materials, and waste facilities. We also regulate these nuclear materials and facilities to promote the common defense and security. The NRC takes its responsibility to be open to the public seriously. However, openness must be weighed against the primary mission of public health and safety. The Commission considers this balance with the best available information, which is continually evolving. The Commission removed security information from public access because the information available to the Commission indicated that it was the appropriate action to take to achieve the agencys primary mission. The Commissions decision and reasons for the decision were publicly announced in a press release, at a public meeting on security held on August 4, 2004, and in documents provided to UCS through a Freedom of Information Act request.

The State of New Jersey commented that licensees fight any color finding, other than green, as a financial penalty, instead of an opportunity to improve performance. Response: The staffs experience is that licensees typically take action to address findings, even when they disagree with the NRCs risk significance determination. In fact the process appropriately invites licensees to comment on risk significance, regardless of whether the licensee agrees with the NRC assessment or not. There have been a number of greater than green findings where licensees agreed with the NRC risk significance determination.

Duke Energy commented that the SDP process should be more timely, and that the NRC should be more open with licensees in assessing the risk significance of findings. Response:

We agree that the SDP process needs to be more timely. However, implementation of the SDP is not a shared responsibility with the licensee. Although program guidance encourages staff to communicate and share information with the licensee as the process progresses, the program requirement is that the staffs preliminary assessment be formally shared in writing with the licensee, and the licensee invited to comment and/or meet in public with the NRC to discuss the issue. It is then the NRC staffs responsibility to consider the best available information, and to reach a final significance determination. The staff considers the process appropriately open to the licensee.

NEI commented that security has been very closed and disassociated with the rest of the ROP.

Response: The security cornerstone remains a part of the ROP. In March 2004, the Commission directed the staff to develop a separate method for assessing appropriate NRC actions in response to security performance, and to keep all security-related information out of the public domain. The Office of Nuclear Security and Incident Response (NSIR) works closely with NRR to ensure program consistency. Nonetheless, the staff recognizes that fundamental differences between cornerstones and the nonpublic nature of security information result in unique differences between the security cornerstone and the other six ROP cornerstones.

The industry commented that when inspection information is processed in the Region, the characterization of a finding may change to a higher level of significance. The industry stated concern that these changes may occur without an open dialogue with the licensee. Response:

Program guidance encourages inspectors to share preliminary information with licensees as inspections progress. Licensees should recognize that regional review is necessary before inspection findings are finalized. The exit meeting is an opportunity to continue the dialogue on the significance of findings, before the inspection report is issued. The staff is open to further discussion on this issue at the monthly ROP meeting.

15. Has the public been afforded adequate opportunity to participate in the ROP and to provide inputs and comments?

Respondent Comments:

Union of Concerned Scientists (2nd submittal)

Not quite. While UCS had adequate opportunity to participate in the ROP via processes like the session at the Regulatory Information Conference and the monthly ROP public meetings conducted at NRC headquarters, we heard from many people across the country about their problems engaging the NRC about the ROP. Thus, UCS must conclude that the public has not been afforded adequate opportunity to participate due to this inconsistency.

The recurring theme heard by UCS from members of the public was that the annual assessment meetings conducted by the NRC in reactor communities were their foremost opportunity to participate in the ROP, but that problems prevented these opportunities from being productive. Seeking to understand this area better, I attended one of the annual assessment meetings and talked with NRC staff and public stakeholders about several other meetings. In addition, I reviewed the NRC presentation slides prepared for many of the annual assessment meetings. My comments from the meeting I attended, the discussions I had, and the documents I reviewed:

a. The NRC presentation slides do a commendable job of explaining the assessment for the specific site and where that assessment stands in context of overall fleet performance. For example, the slides for the presentation by NRC Region II for the Watts Bar annual assessment (ML04100242) provide insights on the NRC staff responsible for monitoring performance (slide 4), how the elements of the reactor oversight process fit together (slide 6), levels of inspection effort expended within the reactor oversight process (slide 7), the overall breakdown of performance indicator and inspection findings in 2003 (slide 9), and sources for additional information (slides 18 and 19). NRC Region IV supplemented that commendable model with information on the NRC s response to security threats and also with very good information on bulletins issued by the NRC in the past year (ML042120487).

To enhance the utility of the NRC presentation slides developed for the annual assessment meetings, UCS recommends that the NRC make these documents accessible from the reactor oversight process webpages. For example, the NRC uses http:www.nrc.gov/NRR/OVERSIGHT/ASSESS/FERM2/ferm2 chart.html to post information for the Fermi Unit 2 reactor. At the top of this webpage (and at the bottom of the webpages for the other operating reactors) is a section titled Additional Inspection

& Assessment Information. UCS recommends that the NRC staff make the annual assessment presentation slides accessible from this section of the reactor webpages.

b. The NRC staff made great progress this past year alerting people in the communities about the upcoming annual assessment meetings. For example, NRC Region I sent out at least thirty three (33) invitations to individuals for the Three Mile Island Unit 1 annual assessment meeting (ML041260093).

c. The NRC staff struggles with anticipating the topics of interest to the public attending the annual assessment meetings and therefore whom to bring to address those topics.

d. The public struggles with non-responses to their questions raised at the annual assessment meetings.

UCS concludes that both the NRC staff and the public enter the annual assessment meetings in good faith, but that communication barriers exist that undermine the utility of these meetings.

To lessen the communication barriers, UCS recommends the following revisions to the annual assessment meeting process:

1) Concurrent with the public release of the meeting notice for each annual assessment meeting, the NRC should post the presentation slides to the NRC website and reference that URL in the meeting notice. This provides members of the public the opportunity to review the presentation materials prior to the meeting and thus engage the NRC staff during the meeting at a more informed level.

2) The meeting notice for each annual assessment meeting should encourage members of the public to provide the NRC with questions and/or topics of interest in advance of the meeting. The meeting notice should explain the purpose for this measure is to allow the NRC to bring the appropriate staff members to the Meeting to address those issues.

3) During each annual assessment meeting, the NRC staff should clearly communicate its preferred point-of-contact for members of the public for additional and follow-up questions.

A related issue for the annual assessment meetings is attendance, particularly low public attendance at many of the sites. UCS recommends that the NRC consider the option of conducting regional annual assessment meetings in selected locales. For example, the Limerick, Peach Bottom, and Salem/Hope Creek sites are fairly close together as are the Catawba, Summer, and McGuire sites. In selected cases like these, it might be prudent for the NRC to conduct the annual assessment meeting for all sites within that region near one site and rotate the location such that all sites periodically host the annual assessment meeting.

In addition, UCS re-recommends that the NRC consider having the annual Commission briefing on the reactor oversight process rotate between its four Regions. The Commission convening this annual briefing in Atlanta, or Lisle, or King of Prussia, or Arlington would likely attract far more local public and media attention than is drawn to this meeting in Rockville. I understand that no news is good news, but good news is better news and these road shows would provide the NRC with ample opportunities for presenting good news. Even when the road show is to the region with the worst plant (e.g., a Commission briefing on the reactor oversight process in Region III in 2003 with Davis-Besse in the middle of its problems), it would provide an opportunity to place that situation in context and better illustrate the performance levels the NRC could expect once the specific problems were corrected.

UCS rating for the ROP in this area: White (1)

State of New Jersey, Department of Environmental Protection

• The NRC does an assessment of the ROP each year but it is for the benefit of the NRC staff to make slight adjustments to the process.

Pennsylvania Department of Environmental Protection The NRC has been actively seeking stakeholders' input to further improve the ROP, but the level of participation by the general public has been very low and the public confidence in the process does not appear to be increasing.

We recommend that the NRC develop and implement an effective mechanism to receive public input continuously and on a plant specific basis. The NRC resident inspectors should play a more active role in the agency's public involvement activities within the local communities. The posting of plant specific information (i.e., PIs, inspection and assessment reports, etc.) on the NRC Website can help improve public confidence in the process and should continue.

Unnecessary changes to the ROP may reduce public confidence in the process and should be avoided.

Region IV Utility Group Yes, the public is afforded adequate opportunity to participate and provide inputs and comments. Public representatives attend the monthly ROP Task Force meeting.

Nuclear Energy Institute Industry has found that the NRC has provided adequate opportunity to all external stakeholders to participate in the ROP and to provide input. Open public meetings at sites and the monthly ROP meeting provide opportunities to ask questions and provide comments. We hope that this same openness and sharing of information which allows joint problem solving and a more effective, efficient, and safety focused oversight process will continue.

Strategic Teaming and Resource Sharing The public is afforded adequate opportunity to participate in the ROP process. On a monthly basis, the NRC has held public ROP meetings to discuss improvements in the ROP process and answer Frequently Asked Questions. The monthly ROP meetings have been effective in maintaining open lines of communication between the NRC, industry, and other stakeholders.

NRC Response:

In general, the staff utilizes a variety of communication activities to ensure that all stakeholders have access to ROP information and results, and have an opportunity to participate in the process and provide feedback. Specifically, in addition to providing a significant amount of ROP information and results on the ROP Web page, the staff provided the opportunity for public feedback through the following avenues: (1) the staff conducted an external survey on ROP effectiveness, (2) the staff conducts monthly public working-level meetings with NEI, the industry, and other stakeholders to discuss the status of ongoing refinements to the ROP, (3) the staff conducted public meetings in the vicinity of each operating reactor to discuss the results of the NRCs annual assessment of the licensees performance and to engage interested stakeholders on the performance of the plant and the role of the agency in ensuring safe plant operations, (4) the staff sponsored three breakout sessions at the 2005 Regulatory Information Conference regarding cross-cutting issues, performance indicators, and the ROP in general. The public outreach and stakeholder involvement in the decision making process during development and implementation of the ROP have both been unprecedented, and the staff continues to focus on stakeholder involvement. The staffs public outreach efforts resulted in valuable feedback and contributed to program improvements.

The UCS recommendations to post meeting slides to the website, to solicit questions for the meeting in advance via the meeting notice, and to clearly identify the staff point of contact, appear to be good recommendations. The staff will either implement these changes or communicate back to UCS why we did not. With regard to providing the meeting slides at the time of the meeting notice, we intend to encourage this practice, but not to make it a program requirement.

Regarding the UCS recommendation to hold common meetings for several plants, we do not intend to implement this recommendation. We agree that the practice may increase the size of the audience, but it could also discourage attendance if members of the public were required to travel an additional 50 miles or more to attend. It would also make the meetings significantly longer. Some members of the public would have to wait until the end of the meeting to hear their plant discussed, which would likely cause a level of frustration. Others may be concerned that the time available to address public questions regarding one plant may need to be shortened in order to move on in the meeting agenda.

The staff considers the UCS recommendation for the Commission to hold the annual Commission briefing on the ROP in the regions to be an interesting suggestion, but not practical. We note that Commission meetings are video streamed over the internet, and meeting materials are available on the NRC website.

The Pennsylvania Department of Environmental Protection recommended that the NRC develop and implement an effective mechanism to receive public input continuously and on a plant specific basis and that the NRC resident inspectors play a more active role in the agency's public involvement activities within the local communities. Response: The NRC staff is open to public input on a plant specific basis, either verbally or in writing. We routinely receive correspondence, e-mails, and telephone calls from members of the public, either to express opinions, seek agency action, or to seek information. We agree that the NRC resident inspectors play an important role as an interface with the local public, however we believe that they should be focused on carrying out safety inspections at the plant.

16. Has the NRC been responsive to public inputs and comments on the ROP?

Respondent Comments:

Union of Concerned Scientists (Initial Submittal)

Denying public access to agency records on the reactor oversight process is technically a Response:, but not one that should have occurred or should ever be replicated.

Union of Concerned Scientists (2nd submittal)

Not only no, but Heck No. Unlike the Frequently Asked Question (FAQ) process the NRC uses to respond to industry inputs and comments, theres no discernible Response: from the NRC to public inputs and comments. If the public accidentally delivered its comments to the National Zoo instead of the NRC (not so far-fetched given that both are destinations along Metros Red Line), the NRC Response: would seemingly be the same - none.

UCS was particularly incensed by the NRC staffs galling statements in SECY-04-0053:

The two most troubling aspects of the survey results are the anonymous NRC employee submission and the perceived lack of NRC Response: to comments .... Additionally, a common theme in many Responses was the apparent lack of NRC Response: to comments. Many of those surveyed believe the NRC has ignored their previous comments or, at the very least, been slow to act, and that the respondent has no way to obtain feedback or responses from the NRC.

The staff was surprised by both of these issues .... The staff believes that there is a distinct difference between being unresponsive and not adopting all recommended improvements to the program. The staff must carefully consider the appropriate balance between all stakeholder points of view and the goals of the ROP when considering any significant changes to the process. The staff will continue to acknowledge each FRN response, indicating the staffs plans to address the comments in this SECY paper as appropriate. However, the staff does not have the resources to provide a direct reply to each FRN response detailing how it handled the respondent's specific comments.

First, how can the staff possibly be surprised by the 'lack of response' charge when it repeatedly hears the same comments (e.g., the "common theme") from many stakeholders spanning several years? The staff must be suffering from Group Attention Deficient Disorder (GADD).

Second, UCS has trouble understanding how the staff can lack the resources to provide direct replies yet have the resources to review the same comments year after year. UCS suggests the NRC staff try responding this year and see if the hours saved NOT reviewing repetitive comments next year more than exceeds the puny number of hours actually responding to comments this year. As the old ad went, "try it, you might like it."

Third, as demonstrated by my citation of the passages above from SECY-04-0053, I and other external stakeholders are in fact diligently reading the staffs publicly available documents on the reactor oversight process. We diligently seek to find out how the NRC staff considered our comments. But we cannot find evidence that the staff considered our input when achieving the "appropriate balance between all stakeholder points of view."

UCS does not insist that the NRC staff expend the time and effort to individually respond to us with its comment-by-comment reaction to our input. We would welcome that treatment (since the NRC staff sees fit to address comments from industry via its Frequently Asked Question process), but do not require it. If the NRC staff instead opts to respond to external stakeholder comments generically through vehicles such as the annual SECY on the reactor oversight process assessment, then the NRC staff has the obligation to provide sufficient information in those publicly available documents to allow the trail from public comment to NRC staff consideration to NRC staff decision. The fact that so many external stakeholders - not just UCS - continue to make the same comments year after year is prima facie evidence that the NRC staff is not properly responding to public comments, either individually or collectively.

UCS and other public stakeholders have provided comments since the reactor oversight process was developed - often the very same comments year after year - without any indication that the NRC agreed with, disagreed with, did not understand, or had even bothered to read them. To be fair, perhaps UCS will check with the National Zoo to see if they've received a bunch of public comments about the reactor oversight process.

UCS rating for the ROP in this area: Red (1) (only 'cause lower ratings are unavailable)

State of New Jersey, Department of Environmental Protection

• The licensees were heard and the ROP evolved to their benefit.

• More public involvement existed in the beginning of the ROP.

• The ROP has not changed to include leading indicators, and as such is only documentation of past performance and not a predictor of future performance. Yet it is used to determine future NRC resource assignments. This does not make common sense and has not been addressed.

Pennsylvania Department of Environmental Protection The NRC has been slow to respond to public inputs and comments on the ROP. The past five years have yielded numerous comments on the inconsistent bases for the existing PI thresholds, the delay in issuing a final SDP finding, and the lack of standardized risk analysis tools. We recognize that the NRC has taken measures to address these issues or concerns, however the agency's response has been slow and these measures have not yet been fully implemented.

Region IV Utility Group Yes, the NRC makes special efforts to recognize the public representatives at public ROP meetings and allows the public to have an opportunity to voice their opinion on the issues discussed. Public comments are received, evaluated, and dispositioned in a professional manner.

Nuclear Energy Institute For the most part, yes. The NRC makes special efforts to recognize the public representatives at public ROP meetings and allows the public to have an opportunity to voice their opinion on the issues discussed. Public comments are received, evaluated, and dispositioned in a professional manner.

Strategic Teaming and Resource Sharing The NRC has for the most part been responsive to public input and comments on the ROP.

This past year a system was developed and used during the monthly ROP meetings to track and status issues discussed or brought to the meetings. The tracking system has been effective in ensuring input and comments made on the ROP are assigned and tracked to closure.

NRC Response:

The staff is attempting to increase its responsiveness to all stakeholders. As noted in the stakeholder survey results discussion in Attachment 6 to SECY-05-0070, to address the continued concerns that the NRC has been unresponsive to stakeholder feedback, the staff plans to consolidate the comments by question and provide a comprehensive response to each question. As in previous years, the staff will acknowledge receipt of each FRN response by correspondence indicating that the staff has considered and generally addressed the comments in SECY-05-0070, as appropriate. In addition, the staff has consolidated the comments by question and provided a comprehensive response to each question. The SECY paper, the annual ROP performance metric report, and the consolidated response will be posted to the ROP Web page and sent along with the acknowledgment letters to each survey respondent.

The State of New Jersey commented that the ROP has not changed to include leading indicators, and as such is only documentation of past performance and not a predictor of future performance. Yet it is used to determine future NRC resource assignments. This does not make common sense and has not been addressed. Response: The staff disagrees with the assertion that the ROP is wholly reactive. The staff considers some of the PIs to be possible indicators of future performance. Substantial cross-cutting issues are also an indicator of possible future plant performance problems. One reason that each plant receives the baseline level of inspection is to look for indications of declining performance prior to poor performance becoming self-revealing. Given the generally good performance of most licensees, the staff considers the present process to allocate inspection resources to be appropriate. Nonetheless we remain open to specific suggestions for objective indicators of future licensee performance.

17. Has the NRC implemented the ROP as defined by program documents?

Respondent Comments:

Union of Concerned Scientists (2nd submittal)

Not entirely. In the past year, several NRC inspectors and NRC contractors contacted me with complaints about NRC inspections. The most disturbing complaints involved NRC inspection team leaders instructing the inspectors and contractors on the teams that they were not to find any thing at the facilities. I have also heard numerous complaints from several inspectors/contractors that NRC team leaders are burying significant findings. It makes little sense to send inspectors out with directions to close their eyes and to turn deaf ears upon their voiced concerns. The ROP inspection procedures may look good on paper, but they are not being faithfully followed.

UCS rating for the ROP in this area: Red (1)

State of New Jersey, Department of Environmental Protection

• We have noticed an increase in the use of potentially greater then green when the process stated that this would be used rarely.

• We have noticed that the licensee still has tremendous influence when we expected that the ROP would support the NRC inspector findings.

Pennsylvania Department of Environmental Protection We have no additional comments.

Region IV Utility Group Yes, where adequate guidance exists. The NRC as a whole has implemented the ROP as defined by the program documents. More program definition is needed in the area of cross cutting issues and how to document and close these issues.

Nuclear Energy Institute For the most part, yes. More program definition is needed in the area of cross cutting issues and how to document and close these issues. Occasional differences in interpreting the ROP documents have occurred, the most noticeable being the way some inspectors interpret NEI 99-02. Some inspectors try to make problems that occur in a cornerstone count in an indicator when the problem does not relate to the performance indicator. These problems are more appropriately evaluated and dispositioned using the inspection program and SDP.

Strategic Teaming and Resource Sharing For the most part, the NRC has implemented the ROP as defined by program documents.

Occasional differences in interpreting the ROP documents have occurred, the most noticeable is in the way some inspectors interpret NEI 99-02. Some inspectors try to make problems that occur in an area covered by a performance indicator count in the indicator when the problem does not fit the definition of the performance indicator. Some problems are better evaluated and dispositioned using the inspection program. The program does give us a way to raise these issues using FAQs which we never had before.

NRC Response:

As noted in the SDP evaluation in Attachment 3 to SECY-05-0070, only three potentially significant findings have been preliminarily designated as potentially greater than green. The staff will continue to monitor the effectiveness of this change as more cases are run through the new process.

The UCS comment regarding NRC inspector concerns was forwarded to the Office of the Inspector General.

As discussed under question #7, the staff agrees that additional guidance is needed in the area of cross-cutting issues and is revising the guidance accordingly.

The staff agrees that improvements can be made with the PI program, as discussed under questions #1 and #2.

18. Does the ROP reduce unnecessary regulatory burden on licensees?

Respondent Comments:

Union of Concerned Scientists (2nd submittal)

Dont know, dont care.

UCS rating for the ROP in this area: U Prairie Island Nuclear Generation Plant Costs are escalating much faster than the return of value of NRC activities.

State of New Jersey, Department of Environmental Protection

• It reduces necessary and unnecessary regulatory burden on the licensee.

Pennsylvania Department of Environmental Protection It is our observation that the licensees are spending less time responding to issues of low safety significance (i.e., non-cited violations, etc.). However, the ROPs significance determination process is resource-intensive and the differences between plant PRAs and the NRC's SPAR models have further complicated the process. We recommend that the NRC continue to conduct its surveys of NRC regional staff and the licensees to determine whether the ROP is making progress toward achieving this goal.

Region IV Utility Group Yes, the ROP has reduced unnecessary regulatory burden on licensees. While we have not seen a marked reduction in baseline inspection hours, we have realized reduced burden in the area of resolving minor violations. We appreciate the opportunity to use our Corrective Action Program to resolve the problems that used to be characterized as level IV violations, which required formal written responses.

Duke Energy This certainly depends on the plant. In many cases, this is true but for older plants, the opposite is true. Overall regulatory burden has actually increased significantly at our facility.

Nuclear Energy Institute The ROP has significantly reduced unnecessary regulatory burden. The ROP eliminated the requirement to respond in writing to minor violations (green findings). This practice permits licensees to focus on fixing problems, not generating time-consuming, non-value added correspondence. Industry has found that improvements in inspection planning and schedule performance allow for better utility planning and resource utilization. Improvement in the SDP resolution process (timeliness and resource burden while maintaining accuracy), as discussed above, would further reduce unnecessary regulatory burden. Less inspection time spent on minor reporting issues in the PIs would also reduce burden without affecting safety. After some experience with the MSPI, further burden could be reduced if NRC would eliminate the practice of conducting an SDP for simple, single equipment failures on MSPI monitored equipment.

This practice has been characterized as chasing random failures.

Strategic Teaming and Resource Sharing The ROP has reduced unnecessary regulatory burden but more could be realized if more confidence were placed in the performance indicators. Based on the reluctance of the NRC to accept the Mitigation System Performance Index without the provision to perform redundant SDPs on failures is an indication that some in the NRC have little confidence in the ability of performance indicators to adequately characterize significance.

NRC Response:

As previously noted under question #6, the staff has undertaken efforts to standardize SDP analyses.

As previously discussed under questions #1 and #2, the staff agrees that improvements in the PI program are warranted.

The industry commented that further burden could be reduced if NRC would eliminate the practice of conducting an SDP for simple, single equipment failures on MSPI monitored equipment. As discussed in more detail under question #2, the staff disagrees with this recommendation. The NRC Staff conclusion is that the PI program and the SDP are both required to appropriately direct increased inspection to weaker performing licensees.

19. Does the ROP minimize unintended consequences?

Respondent Comments:

Union of Concerned Scientists (2nd submittal)

Insufficient data to analyze.

UCS rating for the ROP in this area: U Prairie Island Nuclear Generation Plant It doesnt control hidden agendas by individual inspectors or leaders. See #7 for more info.

[NRC tries to use the rules/regulations to force fit action which the NRC Leadership feels is needed. Getting at plant performance issues through EP compliance or design 50.54(f).]

State of New Jersey, Department of Environmental Protection

• Not sure.

Pennsylvania Department of Environmental Protection Based on our experience in Pennsylvania and as it relates to plant safety, the ROP has not yet resulted in any unintended consequences. However, we cannot conclude at this time that the ROP is capable of minimizing the unintended consequences. Some industry representatives have expressed concerns that there is a potential for licensees to inadvertently take actions that might adversely impact plant safety, particularly as it relates to "unplanned power reduction" and "unplanned scrams."

Region IV Utility Group Some Performance Indicators could cause unintended consequences as discussed earlier in question # 1 above. In addition, the trend towards lower regulatory threshold in documentation of very low safety significance issues based on items identified by licensee employees could have a chilling effect on the willingness of employees to bring forth minor issues for trending.

Duke Energy The answer here would tend to be no, as often is the case, the original issue has long since been solved via the corrective action program before ROP issues a finding and SDP determines the risk. This ties back to timeliness, as the unintended consequence is often "punishment" that occurs years after the event in question. This delay is not effective in driving the desired utility behaviors in the right direction.

Nuclear Energy Institute The ROP does minimize unintended consequences with negative safety implications. However, the following examples of unintended consequences could occur under the current process:

• As discussed above, the scrams with loss of normal heat removal PI and the safety system unavailability PI have the potential for unintended consequences. Action is underway by NRC and industry to correct these indicators.

• Findings of low safety significant based on condition reports written by licensees are being documented in inspection reports. When inspectors ask for all condition reports for the past year or two and document the ones that rise to the level of low safety significant findings, licensees view this as mining the corrective action program. This type of inspection activity provides a disincentive for licensees to document problems.

Strategic Teaming and Resource Sharing The ROP does minimize unintended consequences. However, the following examples of unintended consequences could very easily occur under the current process:

The Scrams with Loss of Normal Heat Removal performance indicator as currently written includes verbiage that penalizes licensees for taking conservative actions especially when the decay heat load is very low and a scram occurs.

A proposed change is being made to the guidance in NEI 99-02 to prevent flooding the indicator with "opportunities" that would encourage licensees to reduce reliability maintenance on Alert and Notification sirens.

Findings of low safety significance based on condition reports written by licensees are being documented in inspection reports. When inspectors ask for "all condition reports" for the past year or two and document the ones that rise to the level of low safety significant findings, licensees view this as "mining" the corrective action program. This type of inspection activity has the potential, as an unintended consequence, to provide a disincentive for licensees to document problems.

NRC Response:

The NRC and stakeholders have been working to resolve the issues associated with the unintended consequences for the Scrams with Loss of Normal Heat Removal PI and the Safety System Unavailability PIs as discussed under question #1. An ROP Working Group Task Force has been formed and is working to resolve issues with the Scrams with Loss of Normal Heat Removal PI. In addition, MSPI is planned for implementation in 2006 and will address the issues with the Safety System Unavailability PI.

The NRC staff continues to improve PI and overall ROP effectiveness to minimize potential unintended consequences, however we must acknowledge that any open regulatory process which assesses performance in a meaningful way will likely result in some level of unintended consequences. The staffs priority is to ensure that the unintended consequences do not have a negative impact on plant safety. It is also the responsibility of licensee management to ensure that operation of their facility is driven by safety, and not by a desire to meet performance monitoring criteria.

20. Additional Information or Comments NOTE: Due to the high volume of comments that fell into this category, we have added the NRC response (indented) following each significant comment as opposed to one comprehensive response at the end.

Union of Concerned Scientists (Initial Submittal)

[From Cover Letter]

On November 1, 2004 (one week after the agency denied public access to its Agencywide Document Access and Management System, ADAMS), the Nuclear Regulatory Commission published a notice in the Federal Register (Vol. 69, No. 210, pp. 63411-63413) soliciting comments from the public on the reactor oversight process (ROP). It is quite simply unbelievable that the NRC would ask the public for comments after the agency intentionally and deliberately took action to deny public access to agency records needed to formulate comments. It is shockingly poor regulatory performance.

On behalf of the Union of Concerned Scientists, it is my duty to formally provide comments in response to the agency's Federal Register notice so as to officially exhaust our administrative remedies before we can pursue this agency malfeasance in the courts. My comments are enclosed.

Union of Concerned Scientists (Initial Submittal)

[Response to Most Questions 1-19, Except as Specifically Noted]

UCS is unable to provide a meaningful response to this [any] question. During the pilot program conducted for the reactor oversight process and during prior annual public comment periods for the reactor oversight process, UCS searched documents in the NRCs Agencywide Document Access and Management System (ADAMS) to answer the NRCs specific questions by citing examples that supported our conclusions. On October 25, 2004, the NRC closed down public access to ADAMS and even disallowed members of the public from perusing documents in the agencys Public Document Room. The NRC cannot expect the public to develop and submit meaningful comments when the NRC denies that public access to agency records on the reactor oversight process.

Union of Concerned Scientists (Initial Submittal)

[Response to Question 20]

UCS intends to provide comments on the reactor oversight process within thirty (30) days of the NRCs restoring public access to agency records in ADAMS.

NRC: UCS and all other potential respondents were provided 30 additional days to comment after the ADAMS ROP information was returned to public availability.

T. Gurdziel, Private Citizen

[From Cover Letter]

I believe that the US commercial nuclear industry is safer today than it was, say, before Three Mile Island, Unit 2. However, I am not comfortable saying that this is the result of current US NRC efforts.

It is hard to believe ANYTHING is important enough to get the NRC to actually tell a plant operator to shut down an operating reactor where there is a question of safety, Davis-Besse being an excellent example.

The reliance on risk measurements, where, no matter what, there is never enough risk to worry about has led to the present downfall of regulation efforts, in my opinion.

I may have a lot more to say on this topic when I have sufficient time, but what you (US NRC) need right now is a group to do fruitful investigations that lead to convictions for wrong doing and massive fines, perhaps equal to what investors can procure independently of the NRC.

NRC: In the past, a number of facilities have been shutdown for prolonged periods to address regulatory concerns. If the NRC considers a plant unsafe to operate, the NRC will take prompt action to ensure the plant does not operate until the safety issues are addressed.

M. Mulligan, Private Citizen

[From Cover Letter]

So how come there is not an easy NRC advertised modern internet questionnaire - such that the public could just click on the answer dots - or submit their word answers for this important question in the of, is the ROP adequate. It's too much work for you and the public to transfer the question and answers onto a separate paper or e-mail. You could even give us a running percentage of the results immediately as you finish the poll for those who answered on the internet.

NRC: We will consider the recommendation for next years survey.

Union of Concerned Scientists (2nd submittal)

[From Cover Letter]

By letter dated November 8, 2004, I provided comments on behalf of the Union of Concerned Scientists to the notice published by the Nuclear Regulatory Commission in the Federal Register (Vol. 69, No. 210, pp. 63411-63413) soliciting comments on the reactor oversight process (ROP). At that time, it was impossible to provide meaningful, substantive comments because the NRC had plugged the plug on public access to records within the Agencywide Document Access and Management System (ADAMS) on October 25, 2004. I did convey our intention to provide comments on the reactor oversight process within 30 days after the NRC ended this injustice. The NRC restored limited public access to ADAMS on December 7, 2004.

I hereby supplement the original comments as promised.

I have commented on the reactor oversight process each and every year since its inception (and often in between the NRC's annual solicitations). Many of these comments are repetitive and I apologize for the duplications within these supplemental comments. The reason I keep submitting the comments again and again and again is because I see no signs whatsoever that the NRC considers the comments. I do not get a response from the NRC and do not see my comments addressed by the NRC in any publicly available document. Thus, I have no clue if the NRC agrees with, disagrees with, does not understand, or even bothered to read my comments. Until I find some reason to believe that the NRC considered our comments, I will keep submitting them.

To help end this stalemate, I decided to annotate my comments this time with color-coded ratings. These colors were assigned using a Significance Determination Process for NRC performance in each of the reactor oversight process areas. A Green rating was assigned when NRC performance was judged acceptable. A White (1) rating was assigned when NRC performance needs improvement. A Yellow (1) rating was assigned when NRC performance needs lots of improvement. A Red (1) rating was given when NRC performance was unacceptable. And a U rating was assigned when NRC performance didn't matter to us and when insufficient information exists publicly to rate performance.

If this approach fails to yield a communication breakthrough with the NRC staff, next year we may arrange our comments in columns with the more significant comments over in the third and fourth columns and the less significant comments over in the first and second columns to emulate the NRCs Action Matrix. If that ruse fails, we also have plans to submit the comments on paper with a letterhead similar to (but distinct enough to avoid copyright infringement charges) that of the Nuclear Energy Institute. We lack confidence that any of these approaches will succeed, but at least it makes submitting the same comments over and over again a little more engaging for us. After all, we expend considerable effort researching the matter and documenting our findings and need something to assuage our complete and utter frustration at the NRC staffs aversion to considering our comments.

NRC: The staff has sent letters to all survey respondents acknowledging receipt of their comments and noting that all comments were considered and many were addressed throughout the annual ROP self-assessment SECY paper. These letters are available in ADAMS under ML041190018 for the CY 2003 survey and ML031400842 for the CY 2002 survey. As noted in the survey discussion in Attachment 6 to SECY-05-0070, the staff will acknowledge receipt of each FRN response for the CY 2004 survey by correspondence indicating that the staff has considered and generally addressed the comments in the annual ROP self-assessment, as appropriate. In addition, to further address the ongoing concern that the NRC is not being responsive to stakeholder comments, the staff has consolidated the comments by question and has provided a comprehensive response to each question. The SECY paper, the annual ROP performance metric report, and the consolidated response will be posted to the ROP Web page and sent along with the acknowledgment letters to each survey respondent.

Union of Concerned Scientists (2nd submittal)

[Response to Question 20]

(a) The reactor oversight processs self-assessments continue to be commendable. For example, the NRC e-mail dated January 16, 2004 (ML042380284), described an upcoming peer review of two NRC Region II inspection reports against the guidance contained in program document Manual Chapter 0612. Combined with many similar efforts, these self-assessments promote consistency and efficiency.

UCS rating for the ROP in this area: Green-plus (b) For some reason, the NRC did not ask any specific questions about the enforcement component of the reactor oversight process. The enforcement component is very bad. It is simply unacceptable and inexcusable that the NRC has taken no enforcement action against FirstEnergy or any individuals for the incomplete and inaccurate information provided to the NRC in fall of 2001 about the condition of the reactor vessel head at Davis-Besse. The NRC can continue to point fingers at DOJ and shrug its shoulders that its out of their control, but thats a lie. The NRC has the power to sanction FirstEnergy and individuals but sheepishly chooses not to do so. The NRC s impotent enforcement stance is inexcusable and must be corrected ASAP.

UCS rating for the ROP in this area: Red (1)

NRC: Since the time these comments were provided in late 2004, the NRC has taken significant enforcement action and imposed civil penalties against FirstEnergy for the incomplete and inaccurate information provided to the NRC in the fall of 2001 about the condition of the reactor vessel head at Davis-Besse (reference EA-05-071). On April 21, 2005, a Notice of Violation and Proposed Imposition of Civil Penalties in the amount of

$5,450,000 was issued for multiple violations (some willful) related to the significant degradation of the reactor pressure vessel head identified in February and March 2002.

The significant violations included: (1) operation with reactor coolant system pressure boundary leakage (associated with a Red SDP finding, $5,000,000), (2) failure to provide complete and accurate information (Severity Level I, $110,000), (3) failure to promptly identify and correct a significant condition adverse to quality (Severity Level II,

$110,000), (4) failure to implement procedures (Severity Level II, $110,000), (5) failure to provide complete and accurate information (Severity Level I, $120,000), (6) failure to promptly identify and correct a significant condition adverse to quality (associated with a Red SDP finding), (7) failure to implement procedures (associated with a Red SDP finding), and (8) failure to provide complete and accurate information (Severity Level III).

Regarding the UCS comment that the enforcement component is bad and needs to be fixed, UCS provided only a single example to make that case and not just any example, but one of the rare cases where the delay is the result of the federal government attempting to balance the needs of both the civil and criminal enforcement processes. A fairer assessment would be to look at all cases over a given time frame. The fact is that enforcement timeliness for all escalated cases including those associated with ROP findings is tracked, trended, and reported in the Enforcement Program Annual Report and all timeliness goals were met last year and in all recent years.

(c) I have frequently commended the NRCs Industry Trends Program (ITP). The ITP has produced many charts of various parameters doing various things over various periods of time.

I confess that the geek in me was bedazzled by the data plots and the associated linear regression.

The charts and graphs this year are as impressive as past years, but their luster has faded. I can find no evidence to suggest that the NRC changed its course because of the ITP math or stayed its course because of the ITP math. In other words, theres no evidence I can find that the NRC would have done anything different at any time had the ITP never existed. If the ITPs output neither shapes nor informs the NRCs regulatory decisions, my question is why bother?

If the NRC had recruited a troupe of mimes and jugglers and stationed them outside their One White Flint North offices, their performances would undoubtedly have captured my attention as I walked to and from the NRC building. Does the ITP provide any more regulatory value than the hypothetical band of street performers? I would like to think that it does, but I know of no such evidence. If the NRC is as unable to show the value added from the ITPs charts and graphs as I am, Id recommend discontinuing the effort and reallocating the resources to other activities.

UCS rating for the ROP in this area: White (1)

NRC: An objective of the Industry Trends Program (ITP) is to assess whether the nuclear industry is maintaining the safety performance of operating reactors. The indicators monitored by the program are assessed for adverse trends. The NRC response to safety significant trends would include adjustments to the inspection and licensing programs if necessary. A lack of an adverse trend provides feedback to the inspection and licensing programs that no changes are needed due to the ITP. Although no trends have been identified to date, future trends may develop and the NRC would respond as appropriate.

In addition to the trending of the data, the ITP also evaluates the indicators for potential short-term year-to-year emergent issues before they become long-term trends. In fiscal year 2004, several indicators exceeded these prediction limits, primarily due to electrical gird issues and the plants connection to the grid. Although agency actions were already underway to resolve electrical grid concerns, the ITP provided information to assist in this effort. The NRC believes the ITP has value and plans to continue this effort. The staffs latest annual paper to the Commission on the Industry Trends Program is available on the ROP Program Documents Web page (http://www.nrc.gov/reactors/operating/oversight/program-documents.html) and can also be obtained from ADAMS (accession number ML050820301).

(d) The NRC deserves credit for placing much of the reactor oversight program process information and results on its website. But that information is so poorly organized and presented as to make it unusable. When I field calls from citizens living around nuclear plants, reporters, or elected officials staffers, I cannot in good conscience direct them to the reactor oversight process webpage. That would be cruel and unusual punishment to inflict on anyone.

The information about the reactor oversight process is so bizarrely grouped that I almost wear out my computer mouse during frustrating cyber safaris looking for documents I know exist.

There seems to be little rhyme or reason to where documents are stashed. Ive essentially given up on the NRC website as a source of information about the reactor oversight process.

Instead of running this gauntlet, I instead opt for finding the information in the NRCs Agencywide Document Access and Management System (ADAMS). ADAMS is equipped with a search engine that allows me to quickly find documents I know to exist. The search engine for the NRCs website is as likely to guide me away from known documents as to them. The NRC should hire people who can discard the abomination that is the web-based ROP info and replace it with something that people can use.

UCS rating for the ROP in this area: Yellow (1)

NRC: Given more specific feedback, the staff would consider modifying the ROP Website arrangement. The staffs goal continues to be to make the ROP Web pages as informative and user-friendly as reasonably possible.

Prairie Island Nuclear Generation Plant

[Response to Question 20]

The NRC is trying to use regulatory process to mandate organizational culture or values.

It should concentrate instead on organizational behavior which can be shown leads to or condones inappropriate plant/equipment performance.

NRC: The staff disagrees with this comment. The ROP is significantly more objective than the previous inspection and oversight process. It is also much more performance based, in contrast to the relatively higher level of programmatic focus in the previous process.

State of New Jersey, Department of Environmental Protection

[From Cover Letter]

The experience of the NJDEP-BNE with the Reactor Oversight Process (ROP) has not been a favorable one for the four nuclear power generation reactors located in New Jersey (Hope Creek, Salem 1 & 2, Oyster Creek).

Over the past four and one half years, since the April 2000 implementation of the ROP, significant enforcement actions (Levels White, Yellow, Red), referred to as escalated, totaled one for Hope Creek, two total for both Salem 1 & 2, and one for Oyster Creek (a second for Oyster Creek related to Emergency Planning is anticipated). All of these significant enforcement findings were categorized as "White, or the lowest of the escalated levels. When compared to pre-ROP findings for the similar length period of January 1996 to April 2000, there were "Severity Level III" findings totaling five for Hope Creek, three total for both Salem 1 & 2, and three for Oyster Creek. This comparison shows a significant decline in significant enforcement findings while lessor findings (Green) remained at a relatively constant level. It is NJDEP-BNE experience that this comes from "Potentially Greater than Green" findings, identified as such on-site by NRC Inspectors, are later determined to be "Green" by NRC management. The final "Green" determination is often justified based on laborious analysis presented by the licensee to show that while the finding/condition is not in dispute, probable risk for the condition can be computed to be low. This obfuscates the obvious conclusion that licensee performance was poor. An obvious example of this was the recent case of the "Green" determination for the Oyster Creek Diesel Generator maintenance finding.

The above comparison is especially disturbing for the three reactors, Hope Creek and Salem 1

& 2, located at one common site and operated by a common licensee. As you are aware, the NRC has recently determined that there are severe challenges to the existence of a safety conscious work environment at this location. As a result of this concern by the Administrator of NRC Region 1, special NRC team inspections and enhanced oversight of these facilities are ongoing. The safety culture concern was not identified by the ROP process as is evidenced by only three total "White" findings found over the past four and one half years for this location. It can be concluded that the ROP process has significant flaws since it allowed this a poor safety culture and failed to identify prescriptive preemptive actions.

Finally, you will find attached the completed survey form enclosed with the subject letter. It is our hope that the future existence and application of the ROP will show significant improvement and will better serve to ensure the safe operation of nuclear power generation facilities in New Jersey and throughout the country.

NRC: The staff appreciates that the New Jersey Department of Environmental Protection acknowledges the substantial response by NRC Region I to SCWE concerns at Salem/Hope Creek. As discussed under question #2, the Commission has directed the staff to enhance the ROP in the area of safety culture and SCWE. Regarding the comments about the significance of findings at Salem/Hope Creek, the staff believes that the significance determination process arrived at appropriate objective conclusions.

It is difficult to compare results with enforcement from prior years because that would assume unchanged licensee performance, and the prior process was much more subjective.

State of New Jersey, Department of Environmental Protection

[Response to Question 20]

• Indirectly, this question is asking if anything is being left out of the ROP - either missing performance indicators or baseline inspections.

• Safety Conscious Work Environment has emerged as an area which is overlooked in the ROP.

NRC: See the response to the prior question above.

Pennsylvania Department of Environmental Protection

[Response to Question 20]

We encourage NRC to continue to conduct public workshops to: 1) discuss the results of the NRC's most recent self assessment of the ROP; 2) review recent changes or proposed changes to the ROP; and 3) seek input and comments from external stakeholders including utility representatives, states and members of the public.

NRC: Comment noted for consideration. The staff meets monthly in public with the industry to discuss a variety of ROP topics.

Nuclear Management Company

[From Cover Letter]

The ROP constitutes a significant improvement over the previous enforcement paradigm. NMC shares the views expressed by the Nuclear Energy Institute (NEI) in their comments dated December 16, 2004. In particular, the frequency of baseline Radiation Protection inspections does not represent an efficient allocation of NRC resources given the industry's performance in this area. NMC is also concerned by the level of effort expended by the NRC staff on characterizing issues that have low safety significance.

In addition to NEIs comments, NMC believes that the definition of licensee-identified findings per Manual Chapter 0612, Power Reactor Inspection Reports, should be broadened to encompass and credit self-reported deficiencies in addition to those defined in the Manual Chapter. For example, a design discrepancy discovered by a training instructor as a result of following up on a question raised by a student should be credited as licensee-identified.

Currently, this is precluded because the issue was not discovered through a process specifically intended to identify the problem.

NRC: Changes to the frequency of baseline Radiation Protection inspections will be considered as part of the annual review of baseline inspection procedures as described in IMC 0307, Reactor Oversight Process Self-Assessment Program. In addition, as discussed under question #13, the staff is considering the need to shift inspection program resources. As discussed under question #11, the staff is revising program guidance to more fully address licensee-identified issues.

Blue Ridge Environmental Defense League

[Response to Question 20]

On October 23, 2003, the NRC issued an inspection report for Catawba Nuclear Station, Units 1 and 2 (IR 05000413/2003-004, IR 05000414/2003-004; 612912003- 9/2712003). Under NRC-identified findings, the report stated that barrier integrity was GREEN. However, the non-cited violation involved Duke's failure to comply with 10 CFR 50 Appendix B, Criterion 3, Design Control, "due to inadequate design measures." At issue were relief valves which were too small to prevent excess water pressure in the event of reactor pump thermal barrier rupture. Why has this been deemed "green"?

On September 27, 2004, the NRC held a conference call with Duke Energy regarding steam generator tube inspections. At issue were cracks in the Westinghouse Model D5 steam generator tubes. Catawba Unit 2 has four such steam generator units and each unit contains 4570 tubes with diameters of 0.75 inches and nominal wall thicknesses of 0.043 inches.

Overexpansion in some of the tubes resulted in "circumferential indications," that is, cracks.

The "indication" in the tack roll segment was 330 degrees in circumference, i.e., nearly all way round, and 100% "through-wall," i.e., broken. If the licensee plans to plug the damaged tube, there was no indication of when this might happen. This looks like an accident waiting to happen which would be compounded by the inadequate relief valves cited in the NRC's October 23rd inspection report.

On December 6, 2004, Catawba Unit 1 suffered an automatic turbine trip which caused a reactor trip (Event No. 41246). The event report stated, "All Emergency Core Cooling Systems are fully operable if needed." This would seem to be contradicted by the noncited violation cited on October 23, 2003 for inadequate "relief valve sizing to prevent exceeding the design pressure of the component cooling water (KC) piping in the event of a reactor coolant pump (RCP) thermal barrier rupture." (IR 05000413/2003-004, IR 05000414/2003-004; 6/29/2003-9/27/2003) and cracks in steam generator units at Catawba Unit 2 which was simultaneously "operating at 100% power."

Meanwhile, Duke Energy seeks to extend the time between checking Actuation System slave relays from 92 days to 18 months, six times as long between inspections. NRC has requested additional information regarding the greatly reduced surveillance regime for this safety system.

But one must ask why would the agency even consider such an extension in an aging plant with the aforementioned weak points in critical systems?

The reactor oversight process must not allow event reports and inspection reports to simply serve as bystanders in the operation of nuclear power plants. The litany of noncited violations and exemptions from requirements is a path to certain failure. I have attached a list of such reports to these comments. Why in the world would NRC not cite a violation of the regulations?

Citations and fines would censure licensees and prompt better operations. Nuclear utilities seem to enjoy a privileged place in the sight of the agency empowered to write and enforce safety regulations. During hearings before the Atomic Safety and Licensing Board, the NRC staff counsel parrot utility opinions. We who have brought cases are outnumbered two to one, with industry and NRC counsel acting as if they are playing on the same team instead of as opponent and referee.

Someone at the agency has got to recognize the inherent danger of such a system.

NRC: The violations described above and all other violations were assessed for risk significance and determined to be of very low safety significance and therefore are being resolved by the licensee within their corrective action program. There have been no exemptions from the regulations granted to Catawba during this period. As noted in our Annual Assessment Letter to the Catawba licensee dated March 2, 2005, (ML050610760), overall, Catawba operated in a manner that preserved public health and safety and fully met all cornerstone objectives. Plant performance for the assessment cycle was within the Licensee Response Column of the NRCs Action Matrix based on all performance indicators and inspection findings being classified as having very low safety significance (Green) requiring no additional NRC oversight.

Entergy Operations

[From Cover Letter]

Entergy Nuclear (EN) is pleased to submit our comments in the above captioned matter. In general, we believe the Reactor Oversight Process (ROP) is meeting your established performance goals. Since implementation, ROP has improved the overall transparency of the regulatory process and communications have improved between EN and the NRC.

EN comments are summarized into 4 areas below. Entergy comments have also been included in a response provided by the Region IV Utility Group (RUG IV). We endorse the RUG IV comments. Grading of the questions provided in the Register notice is provided in an attachment to this letter.

1. The Program lacks some clarity and definition. Instead of evolving to a clearly defined, scrutable process, the opposite may be occurring. As such, this is increasing licensee resources devoted to the process and in general, increasing licensee concern with the process.

a. The Performance Indicators (PIs) have a Frequently Asked Question (FAQ) process that was initially helpful but the process has become tedious and even counterproductive as the number and complexity of the questions increased. Some of the solutions advanced by NRC to make this process more workable (for example, NRC would make a final determination after some time period) could impede the consensus aspects of the current process.

NRC: The number and complexity of the FAQ questions is generally dictated by licensees asking those questions. As discussed under question #1, the development of a process for the NRC to make a final determination after some time period is intended to resolve issues that otherwise would linger.

b. Item 1.a is especially true regarding the Scram with Loss of Normal Heat Removal indicator, where several FAQs have been unresolved for over two years. That indicator has evolved away from its original design; attempts to improve the indicator have been unsuccessful. The value of the indicator is uncertain since scrams are usually inspected by the Resident Inspector and the more significant ones are often inspected via the Management Directive 8.3 process.

Regardless, the industry and NRC are currently evaluating a new approach for this indicator and Entergy encourages the continued work in this area.

c. How a licensee and NRC interface in the Significance Determination Process (SDP) process is unclear and inefficient.

• The process may be entered without licensee knowledge.

• The process the NRC uses to preliminarily develop an issue's significance is usually not open to the licensee until a preliminary significance is determined.

• The SDP phase 2 process is sufficiently conservative as to almost always warrant more thorough analysis. However, the NRC's tool for this more thorough analysis is the Standardized Plant Analysis Risk (SPAR) model, which has been shown to be overly conservative. As a result, many issues may be characterized as being more significant than they would be using more realistic tools, such as the licensee's Probabilistic Risk Assessment (PRA) model.

• The overall process, from issue identification to resolution (final significance determination) is untimely, sometimes taking 6-12 months. Industry and NRC have recognized this and are working in parallel to develop solutions. Some of the solutions being considered by NRC may hinder rather than improve timeliness. For example, using the Phase II notebook results in the choice letter could result in more regulatory conferences and more process time, not less.

Strictly enforcing a "90 day" goal for resolution may result in overly conservative results and ultimately present an inappropriate picture to the public regarding Licensee performance. This is especially true if at resolution, the finding is determined to be of much less significance than originally reported which would tend to erode public confidence.

The net result of the use of the SDP is an over-application of licensee resources for an extended period of time in order to address potential issues. EN has learned to engage the NRC early in the process in order to help characterize an issue properly. But sometimes we must be intrusive in the process in order to ascertain assumptions and characterizations used in the NRC analysis and to influence the use of more realistic inputs. While interventions are possible at the site (Resident Inspector) level, they are less likely at the regional or national level, especially when NRC employs contractors for PRA results. The net result is that EN (and most likely NRC) expends unnecessary resources evaluating potential issues.

Several SDPs are being or have recently been developed that are difficult to use. The value of some of these SDPs is also questionable. For example, the Fire Protection SDP is a vast improvement over the previous version, but is still cumbersome and complex. The Shutdown and Steam Generator SDPs are complex and appear to be of limited value.

NRC: NRC actions directed at improving the significance determination process have been outlined in more detail under previous questions. Overall, the staff seeks to complete SDPs in a timely, efficient, objective, and public manner. In some respects these goals work against each other. Our actions seek a balance between these goals, recognizing that all of the goals can not likely be fully satisfied.

d. The practice of characterizing findings as self-revealing in order to document them in the PIM is not consistent with the enforcement manual. Instances used within the enforcement manual to assign identification credit are more appropriate and the ROP (specifically Inspection Manual Chapter 0612) should not be deviating from the enforcement definition and guidance just for the sake of documentation.

NRC: As discussed under question #11, the staff is revising program guidance to more fully address self-revealing and licensee-identified issues.

e. The process for establishing and resolving a "cross cutting" issue is unclear. The lack of clear definition for this type of issue appears to make them subjective rather than objective.

The Licensee is often made aware of substantive cross cutting issues in the semi-annual assessment letter. The NRC and Licensee could benefit from a discussion of these types of issues as they arise; a better characterization of the issue would occur, as would a more timely resolution of the issue.

NRC: As discussed in more detail under question #7, the staff recognizes the need for more guidance in the area of cross-cutting issues.

f. Movement in the Action Matrix should be clarified, especially when moving from more oversight to less oversight, e.g., multiple degraded cornerstone to degraded cornerstone.

While it is clear how performance results are used to increase assessment when moving "left to right," the actions necessary to reverse that movement appear unclear and inconsistently applied.

NRC: The staffs review of lessons learned from exercising the ROP at Indian Point 2 and Cooper Nuclear Station indicated that additional guidance and criteria were needed to address closing greater than green inspection findings at plants that reach the multiple/repetitive degraded cornerstone column of the Action Matrix. Program improvements were included in two separate revisions of IMC 0305, Operating Reactor Assessment Program, dated February 19, 2003 and January 29, 2004 to address this concern. The first revision added guidance to ensure that actual performance improvements (which typically take longer than several quarters to achieve) have been made prior to closing out the inspection findings and exiting the multiple/repetitive degraded cornerstone column of the Action Matrix. Specific criteria included:

(a) New plant events or findings do not reveal similar significant performance weaknesses.

(b) NRC and licensee performance indicators do not indicate similar significant performance weaknesses that have not been adequately addressed.

(c) The licensees performance improvement program has demonstrated sustained improvement.

(d) NRC supplemental inspections show licensee progress in the principal areas of weakness.

(e) There were no issues that led the NRC to take additional regulatory actions beyond those listed in the Multiple/ Repetitive Degraded Cornerstone Column of the Action Matrix. Additionally, the licensee has made significant progress on any regulatory actions which were imposed (i.e. CALs, orders, 50.54 (f) letters) because of the performance deficiencies which led to the multiple/repetitive degraded cornerstone designation.

The second revision added additional actions that the regional offices may utilize to ensure the appropriate level of agency oversight of licensee improvement initiatives at plants that have exited multiple/repetitive degraded cornerstone column of the Action Matrix. These actions, which do not constitute a deviation from the Action Matrix, include senior management participation at periodic meetings/site visits focused on reviewing the results of improvement initiatives (such as efforts to reduce corrective action backlogs and progress in completing the Performance Improvement Plan),

non-baseline IP 95003 and CAL followup inspections (not to exceed 200 hours0.00231 days <br />0.0556 hours <br />3.306878e-4 weeks <br />7.61e-5 months <br /> of direct inspection without IIPB concurrence), the annual public meetings, and authorization of the contents of the subsequent assessment letters.

2. The Program still has unintended consequences due to its definition and implementation. The Program may influence actions in order to mitigate PI or inspection consequences. This has led to the process being less objective and predictable than expected. Many issues provided in the previous Solicitations of Public Comments are still applicable. Two issues we would like to highlight again follow:

a. The thresholds for action matrix "triggers" that result in movement from column to column in the matrix may be too low. Industry has recommended that the number of white inputs that result in movement from the "licensee response band" to the "regulatory response band" be increased. With the thresholds currently set as they are, licensees are disposed to challenge any white finding. The Industry has also suggested that a graded approach to the length of time an inspection finding is considered in the action matrix be implemented. All findings regardless of color are considered for one year. It might be more advantageous to retain white findings in the action matrix for a period of time less than yellow or red findings, for example, utilize white findings for 2 quarters, yellow findings for 3 quarters and red findings for 4 quarters. NRC has evaluated this recommendation and declined to pursue it.

NRC: As discussed under question #7 and as noted in the assessment evaluation in Attachment 1 to SECY 04-0053 (the CY 2003 ROP self-assessment), the staff performed a detailed analysis of the industrys recommendations to increase the threshold for a degraded cornerstone from two to three white inputs as well as the graded approach for removing inspection findings from consideration in the assessment program and did not implement these recommendations for the noted reasons.

b. Entergy monitors findings across the industry and has noted what appears to be a threshold difference between the regions, especially considering what is "minor." Appendix E of IMC 0612 has guidance and examples of minor violations. Further, the guidance may not be clear in that findings that do not match an Appendix E example may be characterized by the Inspector as more than minor. To a lesser extent, differences in greater than minor findings have also been observed across the regions. In both cases (minor and greater than minor), determining what to learn from the finding may be an inefficient activity since one could focus on activities in one region that are not important to another.

NRC: The staff is revising IMC 0612, Power Reactor Inspection Reports, to clarify the guidance and examples for minor violations.

3. The Program was initiated with several Performance Indicators (PIs) and Significance Determination Processes (SDPs) lacking sufficient risk attributes. While positive changes have been made or are underway to improve this situation (e.g., the Radiation Protection SDPs), the progress has been slow. In the meantime, licensees are being unnecessarily burdened.

a. The Emergency Planning and Security SDPs lack risk based thresholds for actions, are more deterministic in nature and the resulting findings are not equivalent (riskwise) to those emerging from the At Power Reactor Safety SDP. The Security SDP is being further complicated by the NRC revising that SDP with little industry input.

NRC: As discussed under question #6, the staff believes that, in general, an appropriate balance has been achieved among the cornerstones based on the potential impact on public health and safety and the NRC response that the staff considers appropriate to specific findings.

The staff has met with NEI and other industry representatives numerous times during the development of the various aspects of the NRCs process for overseeing security performance. And the staff continues to meet with industry representatives on aspects of our oversight process that are not yet developed. Although the new physical protection SDP was developed in house, the staff worked with the industrys Nuclear Security Working Group in refining and piloting it, and continues to meet with the working group to establish a trial of NEIs proposed security SDP. The staffs meetings with industry representatives included SDP workshops for plant security managers near NRCs regional offices.

b. External event risk effects are included in the At Power Reactor Safety SDP but the SDP lacks clear guidance on how to do this. This has resulted in the misapplication of external risk to findings.

3. The SDP allows the estimation of external events contributions without any contextual guidance. The risk analyst is referred to the licensee's IPEEE analysis for insights.

While the IPEEE results were reviewed by NRC and a SER was issued, the regulatory bases for fire PRAs and IPEEEs is not the SDP.

4. The use of the IPEEE in these cases is tantamount to imposing a new regulatory requirement while at the same time lowering the threshold for characterization of findings.

5. Entergy understands NRC's initial efforts to quantify external event risk in order to determine its impact to the risk attributed to internal events - the inspection manual' chapter directs it. Nevertheless, Entergy contends that this use is inappropriate. If the results of this evaluation are to be used in the significance determination process, they should be taken within the context of the IPEEE process - if it screened out in the IPEEE it is insignificant. It is understood within the PRA discipline that these IPEEEs were overly conservative in many areas.

NRC: As noted in the SDP evaluation in Attachment 3 to SECY-05-0070, consideration of the contribution to overall risk due to external events is a requirement of the SDP for findings that may be greater-than-green. The method for performing this portion of the analysis is currently developed on a case-by-case basis, which has been an additional challenge to meeting SDP timeliness goals. Development of a methodology which could be used to account for the added risk contribution from external events is under consideration by a task group.

A pilot report issued in November 2004 proposed a 3 phase development: (1) develop a Green Screen, a flow chart attached to IMC 0609, to help the inspectors determine whether external events contribution needs to be considered for green findings (tentative schedule for availability is the end of 2005), (2) develop external event worksheets to be attached to the phase 2 worksheet of the risk-informed inspection notebooks (tentative schedule for completion is the end of 2006), and (3) update the existing SPAR models to incorporate external events (completion date not yet determined).

4. Many of the concerns above have been provided to NRC previously through the NRC's yearly requests for comments on ROP, through Licensee/NRC/NEI Task Force participation and, in 2004, a Commission briefing. While progress has been observed in resolving our comments (and other stakeholders as well), the resolution process appears slow. We would appreciate any feedback on these comments, especially their usefulness and any actions you may consider to address them.

NRC: Your comments are useful and appreciated. This document is intended to provide more detailed feedback to commenters than was the case in prior years.

Region IV Utility Group

[From Cover Letter]

In general, we believe the ROP is meeting the established performance goals. We appreciate the opportunity to meet on a monthly basis with the NRC and the public to provide direct input to revisions and enhancements of the ROP and look forward to ongoing discussions in the coming year.

Region IV Utility Group

[Response to Question 20]

• There seems to be a rush to develop specific SDPs that are not very risk informed. This complicates the ROP and should be minimized. A better change management process to determine the regulatory benefit, cost effectiveness and test for consistent valid output of proposed SDPs should be implemented.

NRC: The specifics of the above comments should be suggested for the agenda of the monthly ROP meeting.

• The changes in the security oversight programs are being developed and implemented in a near vacuum. The input from stakeholders is not being widely sought. When stakeholders provide comments or proposals they are not received well.

NRC: As discussed above, the staff has met with, and continues to meet with, NEI and other industry representatives numerous times during the development of the various aspects of the NRCs process for overseeing security performance.

Southern California Edison

[From Cover Letter]

Southern California Edison (SCE) believes that the U. S. Nuclear Regulatory Commission's (NRC's) revised Reactor Oversight Process (ROP) continues to be a significant improvement over the prior deterministic approaches and we continue to support this important effort.

During the fifth year of the ROP, significant progress has been made on several new initiatives.

SCE has been actively involved in the development of the revised Reactor Oversight Process, served on the Initial Implementation Evaluation Panel, is a pilot in the Mitigating Systems Performance Index pilot program, and continues to support other improvement initiatives.

SCE endorses the comments, provided separately, by the Nuclear Energy Institute (NEI). The following SCE comments are provided to augment those of NEI, and include programmatic issues/comments we have identified previously.

SCE concludes that the NRC revised Reactor Oversight Process has been successful in providing a more risk-informed framework. There are several areas, however, that we believe require continuing attention:

• As in all things, Performance Indicators (PIs) and other aspects of the Reactor Oversight Process (e.g., Significance Determination Process (SDP), etc.) can create unintended consequences. There is a continuing need for a robust and ongoing process to identify and address such situations as they arise.

• While some conservative "false positives" are acceptable from any such processes (i.e.,

Performance Indicators, SDPs), it is also necessary that the Reactor Oversight Process identifies and resolves potential opportunities for "false negatives." "False negatives" have the potential to significantly undermine the credibility of the entire Reactor Oversight Process.

• While much improvement has been realized, there is a continuing need to improve the public's understanding of all the elements of the Reactor Oversight Process. It appears that much of the public continues to perceive the new Reactor Oversight Process as solely the "Performance Indicators", and is less aware of the revised Inspection Process, SDPs, Action Matrix, and Enforcement Policy.

• SCE appreciates the Commission decision to proceed with the Mitigating Systems Performance Index (MSPI) which is a new, risk-informed unreliability and unavailability metric. This effort is important, as the GREEN/WHITE threshold for current Safety System Unavailability (SSU) Performance Indicators was previously set at the 95%

performance level based on historical industry data. Other PI thresholds (including the GREEN/WHITE thresholds for assessing Inspection findings using the SDPs) were established based on risk. Having an inconsistent logic for the bases for setting the thresholds continues to create inconsistency and confusion. SCE believes the MSPI can be an important improvement over the SSU.

• SCE remains concerned with various proposals to revise upward some of the Performance Indicator thresholds. Changing the PI thresholds would impose a de facto "rising standard." SCE supports the original NRC position that the thresholds were set with the expectation that, while licensee performance would be expected to improve, performance at the current thresholds represent "acceptable licensee performance."

• Difficulties continue to be experienced with the development, precision, and robustness of the Significance Determination Processes. Several SDPs are not as robust as they should be, and do not produce consistent and/or accurate results.

• SCE supports the recent proposal for the formation of a joint NRC/Industry task force to examine SDP timeliness issues, and formulate process improvements to reduce unnecessary delays.

• The opportunity to provide comments on the NRC's revised Reactor Oversight Process Program is appropriate and appreciated. Unfortunately, the staff has not provided formal public feedback on the disposition and/or resolution of the comments received.

We recommend that the NRC staff provide formal, timely, and public feedback on comments received from the external stakeholders.

NRC: These comments are similar to comments previously addressed under questions #1 through #19.

Tennessee Valley Authority

[From Cover Letter]

TVA appreciates the opportunity to comment on the request for public comment on the implementation of the reactor oversight process. TVA endorses the comments and survey score results provided by Anthony R. Pietrangelo, Nuclear Energy Institute, on the subject proposed generic communication which parallels this submittal.

Nebraska Public Power District

[From Cover Letter]

NPPD endorses the comments submitted both by the Region IV Utility Group on December 14, 2004 and those by the Nuclear Energy Institute. In addition, NPPD has the following comments.

1) Change management within the ROP is inconsistent. Changes to Inspection Manual Chapters (IMC) 0305, 0308 and 0609 are not consistently communicated to stakeholders prior to the change occurring. While the ROP and its supporting documents are the responsibility of the Nuclear Regulatory Commission (NRC), timely communication to stakeholders of projected changes is important. Stakeholders need to be afforded an opportunity to comment and to be given adequate time to implement any changes.

NRC: It is not the staffs intent to discuss all changes to ROP program guidance with industry representatives prior to making the revision. In some cases, such as PIs, the industry is very involved. In other areas, the staff will normally seek external stakeholder comment for significant policy issues that rise to the Commission level.

2) NPPD recognizes that timely evaluation of licensee performance deficiencies through the Significance Determination Process (SDP) needs improvement. However, recent proposals to use the Phase II notebook results in the "choice letter" to meet a 90 day SDP evaluation goal could be counter-productive. More regulatory conferences will likely result due to the conservative nature of the results and a potentially inaccurate picture of licensee performance provided to the public.

NRC: NRC actions directed at improving the significance determination process have been outlined in more detail under question #6. Overall, the staff seeks to complete SDPs in a timely, efficient, objective, and public manner. In some respects these goals work against each other. Our actions seek a balance between these goals, recognizing that all of the goals can not likely be fully satisfied.

3) The practice of characterizing findings as self-revealing is not consistent. Instances used within the enforcement manual to assign identification credit are more appropriate. The ROP should not be deviating from the enforcement definition. In addition, changing of a licensee-identified finding to one that is inspector-identified based on a perceived "inspector added value" is subjective and not consistently implemented with licensees. Finally, the guidance in Appendix E of IMC 0612 concerning what constitutes a minor finding may not be clear in that findings that do not match an Appendix E example may be characterized as more than minor.

Since minor issues are not documented in inspection reports, they are not subject to agency reviews for consistency between plants or across regions.

NRC: As discussed above and under question #11, the staff is revising program guidance to more fully address self-revealing and licensee-identified issues.

4) Changes in the security area have been handled without significant interaction and communication with stakeholders. As a result, the significance of issues in the security arena is not consistent with the rest of the ROP. For instance, the revised security SDP does not take into account the important concept of "predictable and exploitable" at the beginning of a significance determination. Also, evaluating force on force drills and equating the results to an actual intrusion could have unintended consequences.

NRC: As previously noted, the staff has met with, and continues to meet with, NEI and other industry representatives numerous times during the development of the various aspects of the NRCs process for overseeing security performance. The staff worked with the industrys Nuclear Security Working Group in refining and piloting the new physical protection SDP, and continues to meet with the working group to establish a trial of NEIs proposed security SDP.

5) NPPD understands NRC's initial efforts to quantify external event risk in order to assess its impact on the risk attributed to internal events. However, if this evaluation result is to be used in the SDP, it should be taken within the context of the Individual Plant Examination of External Events (IPEEE) and the significance of events within the IPEEE.

NRC: As discussed above, the development of a methodology which could be used to account for the added risk contribution from external events is under consideration by a task group.

6) NPPD recognizes, as does the NRC (IMC 0609, Appendix A, Attachment 1), that there are several human reliability analysis methods to quantify human error probabilities for use in the SDP. NPPD believes that the NRC and industry should work toward resolution of these differing models to improve efficiency of the SDP, rather than attempting to resolve them on individual dockets.

NRC: NRC is in the process of issuing a Standardized Plant Analysis Risk (SPAR) Human Reliability Analysis Method. This is a simple tool for the on the spot analysis to estimate human error probabilities associated with human actions. The process is based on the combination of recognized methodologies. This is not a detailed assessment and could be performed by inspectors without plant specific information. However, knowledge of plant specific deficiencies such as procedural or process weaknesses, and certain environmental conditions would help in refining the estimate and the assessment.

Exelon Generation Company and AmerGen Energy Company

[From Cover Letter]

The ROP continues to be viewed as a significant improvement over the previous Systematic Assessment of Licensee Performance (SALP) program in that it is objective, safety focused and predictable. This approach, for the most part, provides an objective measurement of performance, avoids unnecessary regulatory burden, and focuses NRC and licensee resources on safety significant issues. Further, the ROP provides a timely and understandable assessment of licensee performance, which leads to an increase in public confidence regarding the nuclear industry.

Over the past year, the ROP has continued to evolve and the program continues to provide more than adequate assurance that nuclear plants are being maintained and operated safely.

Since the beginning of the ROP, nuclear plant performance has demonstrated improving performance, which is clearly a positive result of the program.

More work, however, is required to further improve the ROP. Reference 2 provides detailed comments and responses to the questions provided by the NRC. EGC and AmerGen endorse the industry responses provided in Reference 2. There are several issues that need to be highlighted and require increased senior NRC management involvement for resolution. The subsequent paragraphs describe the more significant concerns that need to be addressed.

Over the past three years, the NRC has initiated a significant number of new physical security requirements for nuclear power plants. Many of these requirements were implemented without significant consultation with the industry. The NRC is now developing a revised assessment process for the physical protection cornerstone of the ROP, including an overall structure for the process, revised performance indicators (Pls), and revised significance determination processes (SDPs). The original ROP was developed with significant industry collaboration.

EGC and AmerGen are concerned that the NRC is developing the revised assessment process for the physical security cornerstone without substantial participation from the industry.

Opportunities for participation have been limited to solicitation of stakeholder comments, as opposed to collaborative development. Further, we have not seen indication that the comments provided by industry have been addressed in any substantial manner.

We are particularly concerned about the two proposed SDPs for baseline physical security inspections and force-on-force exercises. We believe that these SDPs have the potential to overstate the relative significance of security findings. The proposed baseline security SDP does not differentiate between programmatic deficiencies and more limited failures to properly implement programs. The force-on-force exercise evaluation process should be modeled after the process by which the NRC evaluates licensees in emergency preparedness exercises, relying on NRC oversight of licensee self-assessment and correction.

In summary, although the implementation of new security requirements without substantial industry participation over the past three years may have been the result of overriding factors, we see no reason to minimize industry participation in the development of the assessment process.

NRC: As discussed above, the staff has met with, and continues to meet with, NEI and other industry representatives numerous times during the development of the various aspects of the NRCs process for overseeing security performance.

More generally, EGC and AmerGen are concerned with elements of the SDP process beyond the physical security cornerstone. First, because the NRC is not meeting its internal performance measures for timeliness in reaching a final significance determination, the agency has proposed measures that essentially use best available information to expedite this process.

While we support the NRC's timeliness goals, we must point out that accuracy of the process is at least as important as timeliness. The ROP process is designed to focus NRC and licensee resources based on safety significance, and thus, the safety significance of findings must be properly characterized. Further, assigning significance based on best available information and then relying on an appeal process to allow for consideration of new information will have a negative affect on public confidence. The industry has asked NRC to host a public workshop to better understand this issue, but this has not occurred. EGC and AmerGen suggest that the NRC and industry work together to evaluate the factors impeding timeliness, and develop solutions that do not sacrifice the accuracy of the final significance determinations.

NRC: As EGC and AmerGen stated above, the ROP process is designed to focus NRC and licensee resources based on safety significance. If the SDP does not contribute to the redirection of inspection resources in a timely way, then it is failing in one of its primary purposes. Numerous other commenters have also expressed frustration with the lack of SDP timeliness. Additionally, it was never the intent of the SDP to expend considerable resources on assessing risk, but rather to use best-available information. The staff goal is to return the process to the original intent. The industry recently provided written input to the staff on improving SDP timeliness (See NEI letter of March 24, 2005; ADAMS accession number ML051220253). The staff responded to NEI in a letter dated June 8, 2005 (ADAMS accession number ML051330334). In summary, the industry proposed, in part, that after readily available information is used to determine the preliminary significance of a finding, and there is a reasonable difference in the results of the licensee and NRC staff evaluations, the NRC should proceed with its 95001 follow up inspection. Then, if the licensees corrective action on the initial finding is found adequate, the initial finding should be considered green. If not, the NRC should label the finding as a preliminary white and follow the current process. The staff disagrees with this proposal, because it would not effectively allow us to fulfill the ROP requirement to assess current licensee performance using the Action Matrix, as described in Inspection Manual Chapter (IMC) 0305, Operating Reactor Assessment Program.

The staff does not see the value in conducting a workshop at this point, however we remain open to dialogue during the monthly ROP meetings with industry.

Further, we are concerned that the process does not, in all cases, yield equivalent results for issues of similar significance in all ROP cornerstones. This was noted in the NRC's 2003 self assessment of the ROP (SECY 04-0053), and continues to be an issue. Generally speaking, the initiating events, mitigating systems, and barrier integrity cornerstones have been demonstrated to be consistent and implement a risk-informed basis. However, the emergency preparedness and occupational radiation safety cornerstones still contain SDPs with considerable subjectivity. Further, the proposed physical security baseline SDP contains considerable flexibility in application, which will inevitably lead to inconsistent application. We recommend that the NRC review and revise these SDPs, in collaboration with the industry.

NRC: See the response to similar comments under question #6.

As noted previously, nuclear plant performance has demonstrated an improving trend since the beginning of the ROP. Continued improving performance will result in an increasing number of green performance indicators (PI) and inspection findings and a decrease in non-green outcomes. We are concerned that some regional staff continue to view this decrease in non green outcomes as a negative aspect of the ROP. The PIs and inspection program act in a synergistic manner to assess licensee performance and permit appropriate escalation of NRC oversight if performance declines. Changes to the ROP cannot be considered or created simply to increase the number of non-green outcomes.

NRC: As stated in our annual Commission paper on the ROP, the staff recognizes that a number of licensees have improved performance in areas monitored by the PI program.

The staff concern is that the PI program has not been fully effective in identifying significantly declining performance in a number of instances, such as Davis-Besse. The staff intents to work with industry to seek changes to the PI program to enhance its effectiveness.

EGC and AmerGen support the revisions to the ROP currently in progress, including the implementation of the Mitigating Systems Performance Index (MSPI) and revision of indicators for Scrams with Loss of Normal Heat Removal and Reactor Coolant System Leakage. These new and revised indicators will further improve the safety focus and consistent application of the process.

EGC and AmerGen recognize and support the need for further refinements to the ROP and will work to achieve these improvements with NRC, industry, and other stakeholders in a collaborative manner. We believe the ROP is a very effective process for assessing performance and our comments are intended to further reinforce the importance of this program.

Nuclear Energy Institute

[From Cover Letter]

The ROP, initially implemented in April 2000, has been an enormous improvement over its predecessor, the Systematic Assessment of Licensee Performance (SALP) program. In general, we believe the ROP is meeting the established performance goals. We appreciate the opportunity to publicly meet with the NRC staff on a monthly basis to provide direct input to revisions and enhancements of the ROP, and we look forward to ongoing discussions in the coming year.

Nuclear Energy Institute

[Response to Question 20]

Industry believes an ROP public workshop would be appropriate to discuss the status of the program, address lessons learned (particularly in the SDP area, see Question 6), and brain storm new ideas to continuously improve the ROP.

Industry suggests that NRC continue its efforts to refine inspection scope, inspection frequency, and inspector-hour commitments based on experience. In particular, industry supports efforts to integrate radiological controls inspections and coordinate with outage activities. Industry would also suggest that NRC look for additional ways to conduct single inspections for utility programs that are common to multiple sites (e.g., access authorization, fitness for duty, and environmental monitoring).

NRC: A consolidated inspection for some security-related programs that are common across a utilitys sites may afford some resource savings for the NRC and could be pursued.

However, implementation of programs, even common ones, is site specific and would still need to be inspected at each affected facility. In addition, as previously discussed under question #13, the staff plans to perform a more detailed analysis of the scope and level of effort of the inspection procedures in CY 2005 to better align inspection resources to inspected areas where there is an indication of risk-significant performance deficiencies. As a result of this systematic analysis, the staff plans to adjust existing resources within the baseline inspection program for CY 2006.

Strategic Teaming and Resource Sharing

[From Cover Letter]

Since implementation in April 2000, the ROP has exhibited marked improvement over the former inspection and enforcement process. Subjecting the ROP to continuous improvement by way of the routine ROP public meetings and the periodic solicitation of public feedback has assisted the ROP in effectively meeting the intended objectives, i.e., to maintain reactor safety; to enhance public confidence; to improve the effectiveness, efficiency, and realism of the oversight process; and to reduce unnecessary regulatory burden. STARS supports and looks forward to assisting in the continuing efforts to further develop and improve the ROP.

Strategic Teaming and Resource Sharing

[Response to Question 20]

An SDP is being developed to address issues associated with the Independent Spent Fuel Storage Facilities. Independent Spent Fuel Storage Facilities are currently not part of the ROP.

We understand that the NRC has regulatory oversight responsibly in this area and others that are not currently within the bounds of the ROP. Care should be taken to either properly integrate them into the ROP or clearly define the oversight process that bounds them.

NRC: In general we agree with the comment.