View source for 3F1098-09, Comment Re Integrated Review of Assessment Process for Commercial Npps.Recommends That Assessment Process Be Reviewed with Consideration of Enforcement Process,Insp Process & Reporting Process

{{Adams
| number = ML20154G593
| issue date = 10/05/1998
| title = Comment Re Integrated Review of Assessment Process for Commercial Npps.Recommends That Assessment Process Be Reviewed with Consideration of Enforcement Process,Insp Process & Reporting Process
| author name = Grazio R
| author affiliation = FLORIDA POWER CORP.
| addressee name = 
| addressee affiliation = NRC OFFICE OF ADMINISTRATION (ADM)
| docket = 05000302
| license number = 
| contact person = 
| case reference number = FRN-63FR42439, RULE-PR-MISC
| document report number = 3F1098-09, 3F1098-9, 63FR42439-00004, 63FR42439-4, NUDOCS 9810130244
| document type = LEGAL TRANSCRIPTS &amp; ORDERS &amp; PLEADINGS, PUBLIC COMMENTS ON PROPOSED RULES &amp; PETITIONS FOR
| page count = 17
}}

=Text=
{{#Wiki_filter:___-                       - - - - -         -       -
Firrida               ##
  .              P5wer                Zf/ye                                      m' . p ,               4 p p i/ & M PT CORPORATION                                                     '
3 n=,.
                 %     u.=... m."
                        .                                                                                       u M m -6 El 3: 24                          [
FiU% 6 L ; :.            , ~;i October 5,1998                                                            US IE 3F1098-09 Chief, Rules and Directives Branch Division of Administrative Services Office of Administration, Mail Stop: T-6D-59 U.S. Nuclear Regulatory Commission Washington, D.C. 20555-0001

==Subject:==
Integrated Review of the Assessment Process for Commercial Nuclear Power Plants,63 Federal Register 42439, dated August 7,1998

==Dear Sir:==

This letter provides Florida Power Corporation's (FPC) reply to the questions contained in 63 Federal Register 42439 concerning the integrated Review of the Assessment Process (IRAP). FPC endorses performance of an IRAP to develop a new NRC method for assessing licensee performance at commercial nuclear power plants. The objectives of ensuring the health and safety of the public can be more effectively achieved at a lower cost to the NRC and the industry with an integrated process that has focus on safety standards using a tiered approach, as discussed in the Enclosure.
FPC appreciates the opportunity to provide comments on this important proposed process.
The comments in the Enclosure also reflect insight gained by NRC's valuable September 28 through October 1,1998, workshop on the process. While good progress was made during the workshop, it is clear that additional effort will be required. FPC would be pleased to participate in developing a workable assessment process by November 1998, if you have any questions regarding this information, please contact Mr. Walter J. Pike, Manager, Nuclear Regulatory Compliance, at (352) 563-4988.
Sincerely, M                                                                                                 )
Robert E. Grazio Director                                                                                               )
Nuclear Regulatory Affairs k
REG /dwh Enclosure xc:      NRC Document Control Desk I
CRYSTAL RIVER ENERGY COMPLEX: 15760 W. Power Line Street
* Crystal River, Florida 34428-6708 * (352)795-6486
'                 ~
9810i30244 981005 PDR    PR 63FR42439            PDR
                                                                                                                            ]

   ,   U.S. Nucinr R::gulatory Commission 3F1098-09 Prg) 2 of 17 ENCLOSURE FPC REPLY TO QUESTIONS INTEGRATED REVIEW OF THE ASSESSMENT PROCESS This Enclosure contains FPC's reply to the questions contained in 63 Federal Register 42439 on the integrated Review of the Assessment Process for Commercial Nuclear Power Plants.
The questions contained in 63 Federal Register 42439, dated August 7,1998, are restated below for convenience. The corresponding FPC reply to each question has been italicized.
A. Regulatory Oversight Approach
: 1. The NRC currently has a low threshold for initiating increased interaction with licensees above the core inspection program.             For example, procedure      I adherence errors or program implementation weaknesses with low actual safety consequences may _ result in increased inspection activity in these areas.
Alternatively, if these regulatory oversight thresholds were raised, the NRC would j
wait until actual safety significant events occurred (such as those measured by performance indicators) before increasing interaction with licensees.
: a.      At what threshold should the NRC take action to assure the adequate protection of public health and safety?
t      FPC Reply:     The NRC should establish thresholds for action based on objective, measurable performance indicators which relate to protecting public health and safety.         Two thresholds are recommended which create three performance bands or tiers as defined in the industry white paper, "A New l                    Regulatory Oversight Process. " The first (the regulatory threshold) would distinguish between a utility response band, in which NRC would conduct its          l l                    core inspections and monitor utility actions, and below which NRC would              1 l                    increase its inspection activity and regulatory action (the regulatory response band).      The regulatory threshold would be set at a level of safety l                    performance such that there is sufficient safety margin to enable the NRC to         '
I increase its oversight and take regulatory action before performance fell to a second threshold (the safety threshold), below which performance is such that adequate protection of the public health and safety may not be available.
Thresholds should be created for a set of objective performance indicators l                    directly related to protecting the public health and safety, that is, indicators l                    which: assess the defense in depth barriers to release of radionuclides, the l                    events which could challenge the barriers, and the key mitigation systems.
.                                                                                                         \
NRC's inspection activity should be guided by the principal that above the regulatory threshold, the utility should be allowed to perform its duties and to     l correct its errors through its own corrective action system. Below the               '
threshold, NRC should increase its review of specific processes and procedures. Below the second threshold (the safety threshold) the NRC and 1

u        .
L, U.S. Nucisar Rzgulitory Commission 3F1098-09                                                                                                       i l                ' Prg3 3 of 17 1:.
                                    . the utility should ensure an assessed basis for continued power operation
                                    . exists and effective corrective actions are in progress.                                     )
i
: b. What is the basis for this threshold?                                                 ,
FPC Reply:          The thresholds should allow room for the utility to operate with minimal NRC oversight while above the regulatory threshold, and yet provide enough
                                    . safety margin above the safety threshold such that there is time for                         ,
corrective action to avoid falling below the safety threshold. (Note that even                i below the safety threshold there is not an actualimpact on the public health and safety, only that continued power operation requires an assessed basis and effective corrective actions are in progress.)
FPC endorses the following industry proposed guidance on thresholds:
Each indicator has an objective regulatory threshold and safety threshold value.
Regulatory threshold defines the level of performance at which l                                                    the safety performance margin has declined to a point where
..                                                   regulatory attention may be warranted.
Safety threshold defines the level of performance at which the safety performance margin has declined to a point where plant V                                                    operation requires an assessed basis and corrective actions are being taken to restore margin.
l                        2.          What range and specific types of NRC actions should be taken if licensees
!                                    exceed the regulatory thresholds discussed in Question A.17                                   '
FPC Reply:         For areas covered by the safety performance indicators, regulatory actions would depend on the performance results. For example, performance
:_                                   discrepancies that did not cause the results to drop below the utility l
response band could be documented in the inspection report as an inspector follow up item without the need to take enforcement action. This would                        ;
l                                    avoid the expenditure of NRC and licensee resources on matters of low
: i.                                   safety importance. For performance within the regulatory response band,
[                                    regulatory actions would depend on the available margin to safety. Actions j                                    could include increased inspection activities. For performance below the V
safety threshold, the NRC could do one of the following: (1) determine whether the licensee's corrective action has corrected, or is expected to correct, the performance problem, or (2) if the licensee is not taking effective corrective action, take enforcement action, i                                    For areas not covered by the safety performance indicators, the degree of regulatory action should be commensurate with the safety significance of the
}
discrepancy. Consideration should also be given to the overallperformance z                           ,,       ,

  ,   U.S. Nuctur R:gul: tory Commission 3F1098-09 Page 4 of 17 i
l of the plant. That is, if a plant is generaHy operating in the utility response           i band for a broad set ofindicators, this provides confidence that the licensee 1
has an effective corrective action program and further regulatory action may              .
not be warranted. If the plant is operating in the regulatory response band for a broad set of indicators, this may provide evidence that the corrective action program has weaknesses. NRC inspection of the corrective action program and performance areas may be warranted. A subjective analysis should be used which would include information from the NRC Resident Inspector to determine appropriate enforcement action.
: 3. The current regulatory oversight process focuses discretionary inspection                 l resources on a selective sample of all aspects of licensee performance, such as           I human performance, procedure quality, and program implementation,
: a.      Could an enhanced use of high level performance indicators (e.g.,
operational transients and safety system availability) reduce the need for discretionary inspection if particular levels of licensee performance are          l achieved?                                                                         I FPC Reply:    Yes. FPC endorses the industry proposed approach which suggests a set of objective safety performance indicators that wiH provide the NRC and the                  ;
public with a good overaH indication whether the plant is being operated and              '
maintained in a manner that protects the public health and safety.              If performance declines in any of the indicators, it indicates specific areas in which more inspection attention may be appropriate. If performance in these areas remtins high, it means that those utility processes and procedures which relate directly to public health and safety are being adequately performed. The NRC core inspection should then be viewed as adequate.
: b.      Would this approach result in a regulatory oversight process which is             f timely and comprehensive enough to assure the adequate protection of               1 the public health and safety?
FPC Reply:   FPC suggests a performance assessment based on the indicators on a quarterly basis. This should be timely enough to take effective corrective action before performance declines to the safety threshold. The actual indicators :ould, and usuaHy would, have data points at a greater frequency l
than quarterly.                                                                           )
i                  FPC believes the industry proposed indicators resulting from the assessment workshop should be comprehensive enough to assure the adequate protection of the public health and safety. There is no effect on public health and safety unless (1) there is an event; (2) the mitigation systems do not                l function as necessary; and (3) the three barriers to radionuclide release (fuel, reactor coolant system, and containment) all fail. The industry proposal includes performance indicators for aH of these areas.
l l
+
I

,   U.S. Nuclxr Regulttory Commission 3F1098-09 Prg3 5 of 17
: 4. What should the role of licensee audits, inspections, and self-assessments be in the regulatory oversight process?
FPC Reply:   FPC endorses the industry proposal which recommends that NRC rely more on licensee audits, self-assessments, and licensee sponsored inspections by outside organizations. This approach would allow the NRC to focus its inspection resources on areas not covered by the licensee and on areas where performance indicators show poor performance. As stated in the industry proposal:
The NRC would develop its inspection plan based on the results of its assessment of licensee performance in the performance indicators, its review of licensee corrective actions on previous regulatory actions, and its requirements to assess deterministic regulatory areas not covered by the performance indicators.
For areas covered by the performance indicators, the scope of future inspection activities could be determined by the performance results relative to the response bands. For example, performance in the utility response band would mean that the NRC would only conduct core inspections for that performance area. In addition, NRC could make expanded use oflicensee self assessments and audits, in lieu of conducting its own inspections, which are often redundant to the licensees' efforts. Performance in the regulatory response band would warrant increased inspection activity to review licensee determination of the cause of performance problems and corrective action taken.
For areas not covered by the performance indicators, the NRC could plan to perform core inspections or opt to evaluate or participate in licensee self assessments and audits. By reviewing the licensee's self assessment and audit schedule for the next inspection cycle, the NRC could conserve resources by opting to evaluate licensee self assessments and/or audits rather than conduct redundant inspections.
A precedent for this approach already exists: Inspection Procedure 40501, "1.icensee Self Assessments Related to Team Inspections. "
: 5. Would an enhanced use by the NRC of licensee audits, inspections, and self-assessments (and a corresponding reduction in NRC discretionary inspection) result in a regulatory oversight process that was sufficiently independent?
FPC Reply:    Yes. NRC will continue to conduct its core inspections, will have resident inspectors onsite, and will review the safety performance results on a regular basis. The NRC will be able to monitor and observe licensee audits and assessments to determine that they are being conducted in an effective, thorough and accurate manner. The NRC can also verify the performance indicators during inspections.

       - . _.       ~      -. .        .        -          .    -         .      .          .    - -    ..        -  .
   ,          U.S. Nucinr Rrgulatory Commission 3F1098-09                                                                                                )
l Pcge 6 of 17                                                                                             j I
B.      Integrated Assessment Process                                                                    I l
: 1.        Objectives and Attributes
: a.      The objectives developed by the staff for an integrated assessment             !
process include the following: (1) Provide early warning of declining          )
licensee performance and promote prompt, timely corrective action; (2) provide checks and balances with other processes; (3) allow for the integration of inspection findings and other relevant information; (4) focus NRC's attention on those plants with declining or poor performance; (5) effectively communicate assessment results to the licensees and the public; and (6) allow for effective resource allocation. What changes could be made to these objectives and why?                                     l FPC Reply:        Objectives 2 and 3 imply that inspection and enforcement are independent of the assessment process. In fact, assessment should drive inspection focus and resource allocation. Enforcement should be driven by safety significance and safety performance, as measured by objective safety indicators. The                ,
NRC still also has the obligation to force compliance with existing                    1 regulations. FPC believes the purpose, objectives, and program attributes to achieve the objectives of an integrated assessment process are more completely stated in the industry proposal:                                            \
l Pumose The purpose of this framework is to define a safety focused                   \
regulatory oversight process for those activities that can be effectively    monitored     using   risk-informed,    performance-based       1 approaches.      The process acknowledges the need to preserve the             1 current regulatory requirements (e.g., rules, regulations, operating license) that define the design and licensing basis of plants. It is l                                      recognized that those activities for which objective measures of safety cannot be provided, traditional oversight will be required. Itis suggested, however, that this oversight should rely more on evaluating licensee self assessments as an alternative to NRC team inspections.                                                                    1 Objectives r
{                                       The new risk-informed, performance-based approach is designed to i                                      meet the nuclear power plant stakeholders needs for an effective regulatory oversight program:
Accurately and objectively measure the safety performance of j                                               nuclear power plants in protecting the public health and safety.
a-1 i

  ~
U.S. Nuclerr Regulitory Commission i     3F1098-09 i     Page 7 of 17 i
l Provide accurate and understandable safety performance information to the public, news media, and other stakeholders.
1 Provide utility licensees and the NRC with objective indicators    !
to assess safety performance and trends, to rationalize the NRC Enforcement Policy, and to allocate resources in an effective and efficient manner.
Provide Congress with objective information to assist in performing its oversight and authorization responsibilities.
Program Attributes Necessary To Achieve Objectives The following attributes are considered necessary to achieve the desired objectives:
The program should be directly linked to the NRC's mandate to assure protection of public health and safety.
The     program      should preserve       current   deterministic requirements of the regulations (e.g., defense in depth, single-failure, redundancy).
The program should apply the concepts of risk-informed, perforr.rance-based oversight.
Safety performance assessment should be based on public health and safety thresholds and regulatory thresholds, not on relative plant performance.
Assessment conclusions should be supported by the direct measurement of the performance indicators.
Attributes of appropriate indicators are:
A direct relationship should exist between the indicator and safety performance expectations Data necessary to measure the indicator should be available or capable of being generated Indicators should be capable of being expressed in quantitative terms that are not ambiguous Indictors should be meaningful and readily understood i
Indicators should be able to be validated

  ~
    . U.S. Nuc!e r R:gulttory Commission 3F1098-09 Prga 8 of 17 4
Program implementation should:
Provide clear roles and responsibilities of the NRC and licensees Communicate results to the public Include a decision model or criteria so that NRC actions are predictable Be simple, non-redundant, and resource efficient
: b.      The new integrated assessment process would not formally recognize superior licensee performance, nor would it include a Watch List. Should the NRC recognize superior licensee performance?
FPC Reply:    If NRC adopts objective performance indicators, superior performance will be evident, as would poor performance. The NRC would have less need to exercise subjective judgment as in the SALP or Watch List approach. NRC would also not need to apply subjective scores to diverse events, LERs and inspection findings and then arbitrarily decide a certain numerical score indicates poor performance.
: c.      The integrated assessment process would not provide a measure of how good licensee performance was. This was due in part to the significant resources involved and the lack of clear guidance against which good performance can be measured. Therefore, performance issues involving solely good or neutral licensee performance would not be included in the evaluation. To what extent and how should positive inspection findings be factored into an assessment process?
FPC Reply:    Positive and negative inspection findings could be used to verify the performance indicators. There may be areas that are not covered by performance indicators but should be included in the assessment process. These areas would be             <
covered by inspection results.
: d.      The integrated assessment process would include an assessment report for each licensee and a public meeting with the licensee to review this assessment.        How should the NRC's assessment results be             i communicated to the licensees and to the public?                          I FPC Reply:   FPC endcrses the industry's proposed approach which would make l                    information publicly available on a quarterly basis.            The performance indicators would serve as the primary communication tool to the public. The NRC could present inspection based verification of the performance                 ,
indicators and any safety impacting concems.           Where performance was       l l
l I

l
  . . U.S. Nucinr R:gul: tory Commission 3F1098-09 Pag] 9 of 17 below the regulatory threshold, NRC could provide a description of actions to monitor licensee corrective actions.
: e. The integrated assessment process would provide several opportunities for the licensee and the public to be made aware of the issues being considered and to provide feedback and input on these issues and assessment results. What are the most desirable ways to include licensee and public input and feedback during the implementation of the assessment process?
FPC Reply:     NRC can use publication of the proposed process for public comment and work shops, such as the one held from September 28 through October 1,1998. Even an industry proposed process that would be endorsed by the NRC could be issued forpublic comment.
: 2.      Assessment Criteria
: a.      In the integrated assessment process, a plant performance matrix is used to categorize performance findings into assessment areas in order to provide better structure for the information and to better communicate assessment results. What additional or alternate information should be used and how should it be integrated?
FPC Reply:      The NRC should focus on objective safety performance, and not on individual errors. Running a nuclear power plant is an industrial process, which will include error. It is the job of utility management to control processes, procedures, plant equipment and human performance to ensure that the safety outcomes are achieved. NRC should allow the utility to perform its job without involvement in the process, which can easily skew management action away from the safety important issues to respond to non-safety impacting regulatory pressures. If licensee performance declines below the regulatory threshold, it is then appropriate for NRC to increase its review and look at processes in more depth,
: b.      Under the integrated assessment process, individual performance issues were numerically graded on the basis of safety and regulatory significance. As stated in the SRM for SECY-98-045 dated June 30, 1998, the Commission did not approve this approach. Are there alternate methods by which the NRC could provide quantitative input into the assessment process so that the significance of issues can be assigned in a scrutable way?
FPC Reply:      Yes. FPC endorses the industry's approach which relies on a set of comprehensive, objective, scrutable, safety performance indicators that relate directly to protecting public health and safety.
l

i 1
  ,     U.S. Nuclur RIgulatory Commission 3F1098-09 Page 10 of 17                                                                                                 1
  .                                                                                                                   I
: c.      In developing a new assessment process, it was essential that the results of the assessment could be clearly communicated to the licensees and                   !
the public. The staff chose color category ratings for each assessment                 ,
area for the integrated assessment process. As stated in the SRM for SECY-98-045 dated June 30,1998, the Commission did not approve of this approach. What alternate presentations could be used to clearly convey the results of licensee performance assessments.
FPC Reply:    FPC believes the industry proposed approach, which includes safety performance indicators that can be objectively measured and thresholds for performance, can provide a meaningfulintegrated assessment program.
: 3.       Decision Mode!
                                                                                                                      )
The staff developed a decision model to provide for a structured and predictable application of NRC actions in response to assessment results.              Are there additional or better ways to optimize the scrutability and predictability of the NRC outcomes of the assessment process?
FPC Reply     FPC endorses the industry approach which would allow NRC to review actual safety performance results and how they could affect public health and safety. The individual errors or problems identified by NRC and licensees can then be understood within a true safety context by validation of the perfonnance indicators. As performance results declined, NRC would be able to focus its inspection efforts on areas of significance to safety.
: 4.      Assessment Periodicity The staff recommended that an annual performance assessment be performed for each plant to allow for a periodic assessment report and a public meeting to discuss the assessment results. Is there a more appropriate periodicity for accurately assessing changes in licensee performance.
4 FPC Reply:    FPC endorses the industry proposal which recommends a quarterly reporting                       ,
period for the safety performance indicators. (The quarterly report may include data from the previous year or several years, depending on what is
:                     appropriate to judge performance changes.) A quarterly report is often l                     enough to determine a trend in performance and alert licensees of the need                      !
to take corrective action, and the NRC of the need to increase its oversight.                   !
Experience has shown that performance deterioration actually takes several                      \
years.
: 5.      Success Criteria
: a.       The integrated assessment process was designed to produce NRC assessments that are more scrutable and predictable. For comparison, i
(

f          .
U.S. Nucl:ar Rcgulatory Commission 3F1098-09 Pag 311 of 17 how scrutable, predictable, and objective are the current assessment                      '
i                                    processes? .
FPC Reply:      The SALP process was based on relative performance of licensees. As the industry's performance as a whole increased, licensees have had to perform                         +
l; better and better to maintain SALP scores. Thus SALP was based more on relative performance than on objective safety performance measures. The SALP process is not scrutable, objective, or predictable. The staff proposed process is also not based on objective indicators. Its system of subjective scores on a subset of plant events is not scrutable, objective, or predictable.
Other shortcomings of the current assessment process apply also to the NRC proposed process:                                                                                   :
The system of assessment is based primarily on the number and type of violations and findings discovered by NRC inspectors or reported by licensees.             The system does not provide any balancing credit with what is working and meeting or exceeding regulatory requirements.
This system is negatively biased.
The system does not have an objective safety focus. The system collects all the violations (in each of the SALP areas) but does not have consistent logic with which to determine what violations are
{
safety significant, and what they mean as a whole.                                          :
The SALP scoring is based on subjective judgment which is influenced by perceived responsiveness of licensee management.
The system is oriented to a "zero-defect" mentality. Yet, we know that no industrial process is zero defect, emphasizing that there will always - be deviations around a mean.               The real objective is to               '
determine what level of deviation is acceptabie and to work to raise the mean to an acceptable level.
The system requires enormous amounts of NRC management attention andis often based on violations without safety rationale.
The system is not integrated with inspection activity and enforcement activity, with the result that resources are not effectively allocated.
The threshold for issuing a violation is subjective and varies from inspector to inspector, and region to region.
i                                   Results, such as SALP scores and watch list designations, are neither l                                   cleady defined nor well understood by the public, industry or news
: j.                                  media.
i
: l. -              ,.      -               .

U.S, Nucle r R;gulatory Commission l        3F1098-09 l        P ge 12 of 17 An underlying problem with the current approach is that it does not measure the actual safety results. It interprets safety in abstract and subjective numbers rather than readily available measures of safety system performance and numbers of challenges to the plant. The more abstract the assessment, and the more numbers that are used that are subjective rather than objective, the less the indicators reflect the underlying reality.
The system measures the relative performance of plants with the result that as average performance improves, the standard for what is acceptable also rises, with no change in regulatory requirements.
While it is appropriate for the industry to take the initiative to improve performance, the NRC should oversee against the current regulations, or revise the regulations if deemed necessary through the established rulemaking process, not through assessment and enforcement.
: b.      The integrated assessment process was intended to be less resource intensive for both the NRC and the licensee. How do the estimated licensee costs compare with the costs of the existing assessment processes?
FPC Reply:     FPC endorses the industry approach which can save significant resources by providing objective indicators of safety rather than relative performance. The      1 industry approach allows an accurate picture of safety performance and              '
focuses both NRC and licensee resources on safety outcomes. When performance is above the regulatory threshold NRC can save inspection resources. A revised enforcement approach based on this assessment methodology would save both NRC and licensee resources in that minor violations would not absorb resources that could be better used to address safety issues.
C. Risk-informed Assessment Guide
: 1.      Effective risk management is necessary to ensure the safe operation of nuclear power plants. How should indications of risk-management performance be considered in the assessment of plant safety?
FPC Reply: Risk insights gleaned from Independent Plant Evaluations and the Maintenance Rule can be used to identif, risk important plant indicators and to set thresholds for performance. FPC believes that providing adequate protection to the public health and safety requires assessment of three areas: (1) the events which can challenge plant safety systems; (2) the mitigation systems; and (3) the three barriers to radionuclide release. Risk insights should be used to determine what indicators and thresholds to apply. The thresholds could differ from plant to plant depending on risk identified by the plant specific PRA.
i l

U.S. Nucizr R gulitory Commission 3F1098-09 Pigs 13 of 17
: 2.       One aspect of a risk-informed regulatory process is that plant performance measures are considered commensurate with their impact on plant safety and risk. Are there questions presented in " Guidance for Assessing the Risk inherent in Plant Performance" sufficient to ensure that inspection findings are interpreted in a risk-informed manner?
FPC Reply:    No.      Inspection findings do not measure plant safety performance, they           \
measure compliance with the regulations. They should not be used, whether            1 interpreted in light of risk or not, to form the basis for an integrated assessment process. The integrated plant assessment process should focus on those aspects important for protecting public health and safety: (1) the events which can challenge plant safety systems; (2) the mitigation systems; and (3) the three barriers to radionuclide release.
Most of the questions in the guidance document can be better answered by looking directly at performance indicators of safety performance. Engineering and design is an exception, which may lend itself to objective safety outcome indicators. These areas can best be measured through audits, self assessments, and inspections.            The areas of human performance and problem identification and resolution are only germane if performance is below regulatory thresholds of objective safety performance indicators. If performance is above the thresholds, the performance in these areas is, by definition, acceptable.
: 3.      Regulatory Guide 1.174, "An Approach for Using Probabilistic Risk Assessment in Risk-Informed Decisions on Plant-Specific Changes to the Current Licensing Basis," presents a framework, principles, and staff expectations relative to regulatory decision-making.
l l                 a.        What role, if any, should such guidance play in risk-informed assessments
;                           of plant performance?
FPC Reply:     This guidance can be used in establishing safety thresholds for risk significant structures, systems or components. For example, the safety threshold for mitigating systems performance could be set at a level that corresponds to a level that could cause a change in conditional core damage probability that would require prior NRC approval.
: b.        What role should PRA techniques and risk metrics play in theassessment i                           of plant performance?
FPC Reply:    It is important to use PRA and risk insights as much as practicable in order to orient the integrated assessment toward what is known about risk.
Risk insights gleaned from Independent Plant Evaluations and the Maintenance Rule can be used to identify risk important plant indicators and to set thresholds for performance. To the extent possible, risk insights

U.S. Nucl=r Rrgulttory Commission 3F1098-09 Pcg314 of 17                                                                                     !
should be used. FPC believes that providing adequate protection to the public health and safety requires assessment of three areas: (1) the events which can chauenge plant safety systems; (2) the mitigation systems; and
                                                                                                          \
(3) the three barriers to radionuclide niease. Risk insights should be used to determine what indicators and thresholds to apply.                                 !
: 4.      How should patterns of degrading human performance, equipment performance,         !
and risk management at a nuclear power plant be factored into the plant            )
performance assessment process?
l FPC Reply:     The NRC's primary responsibility is to ensure that its licensees provide          ;
adequate protection to the public health and safety. The licensees are             '
responsible for managing their people, processes, procedures, and plant             l equipment to achieve safe and reliabln nuclear power. If human or                  1 equipment performance is degrading such that it affects safety performance, then it will show up in the safety performance indicators.             If safety performance is above the regulatory thresholds, degradation is not significant and correction is the purview of plant management. If performance fans below the regulatory threshold, the cause may be human performance, or equipment degradation, or some other cause, which should be evaluated by the plant corrective action program. NRC should at this point increase its focus and look more closely.
: 5.      Are the questions raised in " Guidance for Assessing the Risk Inherent in Plant Performance" sufficient to provide a risk-informed assessment of plant safety that addresses the influence of human performance and equipment performance on plant safety?
l       FPC Reply:    No. The questions are intended to be used by NRC staff to score and bin             ,
l                     that subset ofissues the NRC decides to include in its matrix. The questions       l l                     win certainly help the staff to do that. However, a risk informed assessment        l of plant safety requires a set of objective safety performance indicators          ,
which relate to protection of the public health and safety, to wit, indicators l                     that measure: 1) the events which can chaHenge plant safety systems; (2)           l the mitigation systems; and (3) the three barriers to radionuclide release.         l Equipment performance wiH be apparent in the safety indicators. Human              ;
performance, as it is important to plant safety, wiu also be reflected in the      ;
safety indicators.
i D. Indicators                                                                                  i
: 1.      General 1
The trending methodologies can be used as part of an integrated assessment l
process that uses both quantitative and qualitative information. The trending       ;
  ;                   methodologies are not intended to be used in isolation as the only definitive       l identifying element in plant performance assessment.                               !
4

l       .
U.S. Nucle:r Rrgulatory ' Commission
            - 3F1098-09 Paga 15 of 17
: a.              How should the NRC use quantitative measures of performance?
FPC Reply:          NRC assessment should be based largely on objective quantitative indicators that directly- measure safety performance. That is, the indicators should measure safety outcomes which directly relate to protection of the public health and safety, to wit, indicators that measure: 1) the events which can
                              - challenge plant safety systems; (2) the mitigation systems; and (3) the three                      l barriers to radionuclide release. Trended over time, these indicators provide                     ;
a clearpicture of the plant safety and whether it is improving or declining.                       '
: b.              What methodologies and/or performance measures would be useful to quantitatively monitor plant performance trends?
FPC Reply:          The best methodology would be to measure and trend objective safety performance indicators against risk-informed thresholds of performance that provide adequate assurance of protecting the public health and safety. This would provide an objective standard for performance.
1
: 2.         Trending Methodology                                                                               !
: a.              The staff considered more than 20 variables during the development of both the trend and the regression models.
: 1.              Are there other variables that should be considered?
FPC Reply:          The variables considered by the staff for its trend and regression models                        >
were based on statistical correlation and the ability to distinguish levels of
[                               relative performance, benchmarked against previous Senior Management i                               Meeting discussion plants. Variables which are inherently important to protecting the public health and safety are incompletely covered, to wit: 1) the events which can challenge plant safety systems; (2) the mitigation systems; and (3) the three be:rriers to radionuclide release. Trended over time, these indicators provide a clear picture of the plant safety and whether it is improving or declining against risk-informed objective safety standards.
: 2.               Are the data for the suggested variables publicly available?
FPC Reply:          Yes.           They are made available by the NRC.                    Although the variables are generally publicly available, they are not direct!y related to plant safety performance.
!                                               3.               Are the data for the suggested variables reported to the NRC7
:            FPC Reply:          Yes, some directly and others in LERs and other reporting requirements.
Some of the variables, such as the cause codes, require NRC staff analysis.
}
  ^
i 5

U.S. Nucl:ar Rcgulitory Commission                                                                     !
3F1098-09 Pcg316 of 17 l
However, they are not variables that measure plant performance safety                    l trends.                                                                                 i
: 4.      How frequently are the data for the suggested variables available?
FPC Reply:     This depends on the NRC's resource allocation and the time it takes to process information provided by licensees and for analysis, by NRC, of cause codes. There may be as much as a one year lag in the availability of this                l data to the public and the industry.                                                     \
l
: b.       The staff considered a variety of time periods for monitoring plant performance during the development of the trend model. The proposed trend model uses a four-quarter moving average. Should a different time period be used?                                                                  l FPC Reply:     This depends on the indicator.          FPC recommends the use of quarterly indicators.
: c.       The proposed trend model uses a " hit" threshold that is based on a fixed       ,
2-year average of one standard deviation beyond the quarterly industry mean for the period from July 1995 through June 1997. Should a different threshold be used?
FPC Reply:    First, NRC should use indicators in its trending which are based on objective measurement of plant safety: 1) the events which can challenge plant safety systems; (2) the mitigation systems; and (3) the three barriers to radionuclide release. Trended over time, these indicators provide a clear picture of the plant safety and whether it is improving or declining against risk-informed objective safety standards.
To the extent practicable, thresholds should be based on safety standards and risk insights. They should not be based on relative performance of the industry, which has continued to improve. The NRC should be concerned whether or not each plant meets safety performance indicators, not who ranks lowest. When risk-informed thresholds cannot be established, it may be appropriate to use industry performance data, keeping in mind risk insights.
: d.       The proposed trend model uses a discussion candidate threshold value of two hits. Should a different threshold be used?
FPC Reply:    Do not use a " hit" methodology based on deviations in performance beyond an industry mean. Performance trends and safety standards should be used.
: 3.      Financial Indicators
: a.       Financial indicators can be used to gain insight into licensee performance in conjunction with other assessment measures. They would not be relied

e U.S. Nucber R::gulatory Commission
    . 3F1098-09 P gs.17 of 17 upon solely to draw conclusions on licensee performance in an integrated assessment process. How should financial indicators be used in the assessment of licensee performance?
FPC Reply:     They should not be used. The purpose of indicators should be to determine how welllicensees are performing in protecting the public health and safety.
Measures of safety such as minimizing events, maintaining mitigation systems, and maintaining the barriers to release of radioactivity are directly           '
related to plant safety. Measures of Operating and Maintenance (O&M) costs, revenue factor, coverage and total production costs are merely statistical correlations which do not prove causality.
: b.      Are there other financial methodology processes that will provide a more useful set of financial variables?
l FPC Reply:     This is an issue for utility management and the financial community to measure as a business and market predictor, it is not a predictor of plant                i safety outcomes.          Depending on the situation, low O&M costs could                 '
represent a very effective management team, or a situation in which safety is being shortchanged. It is not a viable predictor of plant safety. Several of the indicators are related to the total production of the site. If the plant              i management decides to take a shutdown to correct a problem, even though it was not required to do so by regulation, this would penalize them with                 l respect to those indicators.                                                              '
: c.      The financial variables are based on publicly available data. Are there other financial data that could be made available that would be more useful?
FPC Reply:    FPC does not believe that any financialindicator can provide accurate insight into licensee safetyperformance.
E. Additional Comments FPC Reply:    FPC recommends that the Assessment Process be reviewed with consideration of the Enforcement Process, the Inspection Process, and the Reporting Process. These processes should be consistent and be structured to meet the objectives of the NRC without duplication of activities and with               ,
attention to safety impacting performance.
,}}