ML20210R994
| ML20210R994 | |
| Person / Time | |
|---|---|
| Issue date: | 08/21/1997 |
| From: | Callan L NRC OFFICE OF THE EXECUTIVE DIRECTOR FOR OPERATIONS (EDO) |
| To: | |
| References | |
| M970129A, SECY-97-192, SECY-97-192-01, SECY-97-192-1, SECY-97-192-R, NUDOCS 9709040216 | |
| Download: ML20210R994 (5) | |
Text
.~
r.J. _,.,_. m i,._ m ww u
%,,.....f
. n... o.........ms:
POLICY ISSUE (Information)
Auoust 21. 192Z SECY-97-192 E@:
The Commissioners VI SEP -2 N1:22 EB@:
L. Joseph Callan Executive Director for Operatior.si,UBUC DOCUMENT nr se SUMECT: PEER REVIEW 0F THE ARTHUR ANDERSEN METHODOLOGY AND USE OF TRENDING LElTERS PURPOSE:
The purpose of this Comission Pa)er is to respond to staff requirements memorandum (SRM) M970129A. dated r bruary 14, 1997. in which the Comissicn e
requested the staff to conduct a detailed peer review of the Arthur Andersen methodology, compare the results of the application of that methodology to the results of the last senior management meeting (SMM) evaluate and describe the reasons for any differences, and to address whether the Arthur Andersen methodology is more appropriate at the scracning meetings or the SMMs themselves.
In addition, this pa)er responds to a separate request in SRM M970129A that the staff consider t1e advisability of reissuing adverse trend letters at each SMM for which they are left in effect, as well as, issuing trending letters between SMMs (or some other defined evaluation interval) in warranted situations. The Commission asked that the staff provide the results of its evaluation and provide any recommended changes for approval.
BACKGROUND ON THE ARTHUR ANDERSEN METHODOLOGY:
In an SRM dated June 28, 1996, the Comission directed the staff to assess the SHM process and evaluate the development of indicators that can provide bases
Contact:
NOTE:
To BE MADE PUBLICLY AVAILABLE IN Mick?1 R. Johnson NRR S WORKING DAYS FRoM THE DATE OF THIS 301-415-1241 PAPER 9709040216 970821
-19 PDR I
i The Comissioners for judging whether a plant should be placed on or deleted from the Watch 1.ist.
In response to this request, the staff contracted with Arthur Andersen Consulting to perform an independent assessment of the SMM process. The contractor's report was issued in December 1996.
Following the January 1997 periodic briefing for the Comission on operating reactors and fuel facilities, the Commission issued SRM M970129A.
In the SRM, the Comission requested that the staff initiate the actions described in the Purpose section, above.
The staff's actions in response to SRM M970129A i
related to the Arthur Andersen methodology are described below.
DISCUSSION OF THE PEER REVIEW OF THE ARTHUR ANDERSEN METHODOLOGY:
The trend plots presented in the Arthur Andersen report were intended to illustrate that quantitative data could be used in a systematic way to indicate trends in plant performance.
Arthur Andersen Consulting recognized the ap)roximate nature of the method, and did not intend that their trend plots >e used to evaluate the performance of operating reactors. The Arthur Andersen report recommended that NRC formalize a methodology based on a thorough review of available performance indicators. Since issuance of SRM M970129A. the staff has pursued a program to develop a validated NRC Trending Methodology.
On April 2,1997, the staff forwarded its alans to address the recommendations made by the Arthur Andersen Company in SECY-97-072, " Staff Action Plan to Improve the Senior Management Meeting Process." and the staff subsequently briefed the Commission. As indicated in SECY-97-072, several of the improvement actions had already been initiated.
In response to the briefing on staff plans, the Comission issued SRM M970424B, dated June 24, 1997.
In it, the Comission approved the staff's plans to improve the SMM process but directed the staff to critically evaluate the proposed approach, assumptions, etc.
In addition, SRM M9704248 directed that the Arthur Andersen algorithm, or any other >erformance algorithm developed by the staff, should be peer reviewed, >enchmarked, and trial-tested before being relied upon for making assessments or comparisons.
The staff has factored the direction provided by the Comission in the February and subsequent SRMs into current plans and activities. For example, the NRC Trending Methodology will be peer reviewed in detail by the Advisory
. Committee for Reactor Safeguards (ACRS) and will be published for industry and public review and coment.
The staff is moving ahead with development of the improved NRC Trending Methodology and plans to validate the proposed methodology as part of the trial testing and peer review activities as directed by the June 24, 1997. SRM (M970424B).
Consistent'with this additional guidance, the staff has not gone back to compare the trend plots that were presented for illustrative purposes by Arthur Andersen with the results of the January 1997 SMM since the staff expects there to be substantial differences between the initial and final trending methodologies.
Additionally, the staff proposes to consider whether the NRC Trending Methodology is best utilized in the screening meetings or the SHM as it develops plans to implement the final. validated methodology in the SMM process.
1 l
4 i
The Commissioners For record purposes, the staff considers the activities requested by SRM M970129A to be subsumed by actions associated with implementing Commission direction in response to SRM M970424B. Future periodic update reports to the 3
Commission on progress of the peer review and analyses of the NRC Trending Methodology will be made in response to SRM M9704248.
BACKGROUND ON USE OF TRENDING LETTERS:
In 1992, the Executive Director for Operations directed the Office of Policy Planning (OPP) and NRR to conduct a broad review of the reactor inspection 3
grogram. The results of this review were sumarized in OPP 92 010, Assessment of the Reactor Inspection Program." One of the issues identified by this review was that senior licensee managers indicated they would have initiated robust actions to remedy poor performance if they had known the NRC was considering their plant as a potential candidate for the Watch List.
After being briefed on the results of the assessment, the Commission issued SRM M9304232 in which it suggested that the staff begin notifying the utility i
corporate president or board of directors when a )lant has been identified at i
a SMM as trending towards placement on the Watch.ist.
Following the June i
1993 SMM, the Commission issued its first trending letter.
During the past several years, the Comission and the staff have evaluated the issue of the advisability of centinuing the use of trending letters.
In an i
SRM dated February 14, 1994, the Comission requested feedback from the staff on the use of trending letters and a recommendation as to whether the practice should be continued, modified, or terminated.
During a periodic briefing for the Comission on operating reactors and fuel facilities in June 1994, the Comission expressed the concern that another performance category may "have i
been inadvertently created with the use of " followup trending letters.
The i
Commission was also concerned about how long a plant might be considered to be trending. The staff indicated that more experience was needed before a final recomendation could be made.
After gaining additional experience, the staff responded to the February 1994 SRM through issuance of SECY-95-085. "Recomendations on the Senior Management Meeting Process for the Continued Use of Trending Letters and for Plants that Remain on the Problem Plant List for an Extended Period." In SECY-95-085, the staff concluded that, based on the objectives of trending letters, the experience in implementing them, and the results obtained, the use of trending letters had achieved favorable results.
In the SRM that responded to SECY-95 085 the Comission directed the staff to use the trending letter only once, upon the detection of a declining trend, and discontinue the routine use of followup trending letters. The staff implemented changes in the SMM process to reflect the Comission guidance on use of trending letters. This change is reflected in the staff's current 3ractices, as described in Management Directive 8.14. " Senior Management
- 1eeting. "
2At that time, trending letters were reissued to plants for each SMM they were left in effect. These are referred to as " followup trending letters."
The Comissioners 4-i Following the January 29, 1997, Commission briefing on operating reactors, the Commission directed the staff (in SRM M970129A) to consider the advisability of reissuing adverse trending letters at each SHM, as well as issuing trending letters between SHMs (or some other defined evaluation interval) in warranted situations.
The results of the staff's evaluation are discussed below.
DISCUSSION OF THE USE OF TRENDING LETTERS:
l The staff has reviewed its current and past practices regarding the issuance of trending letters. As indicated above. the original intent of issuing trending letters was to provide an early warning to licensees identified by the NRC as trending towards placement on the Watch List.
Our experience indicates that the use of trending letters has been effective.
For those plants with trending letters currently in ef fect, licensees have begun to take actions to address the. issues identified.
Similarly, licensees that received trending letters previously have responded positively and initiated corrective actions.
In almost all instances, licensees that were
' issued trending letters reversed the negative trends, thereby preventing placement on the Watch List.
While the staff believes that the practice of issuing treading letters should be continued, there are no compelling arguments that would indicate that it is l
better to provide a single trending letter as is current practice, or to reissue trending letters after each SMM they are left in effect.
In addition, the use of trending letters will likely be impacted by the overall direction the NRC chooses as a result of the ongoing integrated review of the NRC assessment process.
Therefore, in the interim, the staff recomends that the current practice be maintained.
The staff also considered the advisability of issuing trending letters between SMMs in warranted situations.
In concept, issuing trending letters between SHMs. would allow the agency to provide earlier notification to licensees regarding adverse wrformance trends.
In deciding whether to issue trending letters between SMis, the following points must be considered:
The NRC currently has a variety of tools that are used to comunicate e
the concerns of NRC managers regarding plant performance to utility These include, but are not limited to, inspection reports, managers.
management meetings and enforcement conferences.
The tools ensure that the agency can take action to discuss emerging issues and trends with licensee s to ensure the earliest possible recognition and corrective action.
Historically, changes in plant performance have been gradual.
e The likelihood that a plant will experience a sudden adverse trend in licensee's performance that would warrant SMM action in the intervening 4-6 months before the next SMM is remote.
The issuance of trending letters is intended to be a SMM. action.
To issue trending letters between SMMs the staff would need to revise its existing assessment processes to identify possible candidates for this
The Commissioners 5-expedited attention based on routine / ongoing performance insights and provide opartunity for agerecy senior manager review, deliberation, and I
consensus afore a decis This process would be redundant to existing processes (e.g. ion is made.
, plant performance review, SMM screening and would result in an increased administrative burden on manageme. SHM) staff without commensurate benefit, nt and Changes to make the process less burdensome, such as permitting issuance o
of the trending letter by the regional administrator without pgency wide involvement achieved through processes like the SMM, would potentially result in greater inconsistency between regions and would potentially diminish the perceived impact of the letter.
Based on the above. the staff does not believe that issuance of trending letters between Sms is advisable.
As mentioned above, the. staff is conducting an integrated review of its various assessment processes.
improvements to the SM process may have implications for the future use ofT trending letters.
The staff will make additional recomendations regarding the tacific prac'. ice of issuing trending letters as part of the changes that will :e proposed to the Comission for approval.
L.
eph Callan Executive Director for Operations DISTRIBUTION:
Connissioners oGC oCAA oIG oPA oCA' ACRs CIo CFo EDo REGloNs sECY t
k