ML18123A319

From kanterella
Jump to navigation Jump to search
Memorandum Communicating Results of the CY2018 ROP Effectiveness Reviews - Ifrm Pilot and IMC 0307 Pilot Metric E-4
ML18123A319
Person / Time
Issue date: 05/09/2018
From: Mohammed Shuaibi
Division of Reactor Safety III
To: Chris Miller
Office of Nuclear Reactor Regulation
Garmoe A
References
Download: ML18123A319 (14)
v • d • e


Text

C. Miller UNITED STATES NUCLEAR REGULATORY COMMISSION WASHINGTON, D.C. 20555-0001 May 9, 2018 MEMORANDUM TO: Christopher G. Miller, Director Division of Inspection and Regional Support Office of Nuclear Reactor Regulation FROM: Mohammed A. Shuaibi, Deputy Director /RA/

Division of Reactor Safety Region III

SUBJECT:

RESULTS OF THE CALENDAR YEAR 2018 REACTOR OVERSIGHT PROCESS SELF-ASSESSMENT EFFECTIVENESS REVIEWS ON THE INSPECTION FINDING RESOLUTION MANAGEMENT PILOT PROGRAM AND INSPECTION MANUAL CHAPTER 0307 PILOT METRIC E-4

SUMMARY

This paper presents results from the calendar year (CY) 2018 Reactor Oversight Process (ROP) self-assessment effectiveness review of the inspection finding resolution management (IFRM) program trial period. A team comprised of staff from the Office of Nuclear Reactor Regulation (NRR), the Office of Enforcement (OE), and all four regions evaluated implementation of the IFRM program, the impact of IFRM program on the dispositioning of potentially greater-than-Green (GTG) findings, and recommended that the piloted IFRM process be adopted with some changes. Specific recommendations and changes are included in this paper.

This paper also discusses an effectiveness review of the 120-day performance deficiency (PD) pilot metric, metric E-4 in Inspection Manual Chapter (IMC) 0307, Appendix A, Reactor Oversight Process Self-Assessment Metrics. The 120-day metric is an inherent part of the IFRM process. This pilot metric was introduced in November 2015 as part of a comprehensive revision to the ROP self-assessment metrics to make them more objective and align with the Principles of Good Regulation. The effectiveness review of this metric recommended that the metric be made permanent with some enhancements. Specific recommendations and changes are included in this paper.

BACKGROUND:

Annual effectiveness reviews were added to the ROP self-assessment program as part of the November 23, 2015, revision to IMC 0307, Reactor Oversight Process Self-Assessment Program. Effectiveness reviews are used to assess recently implemented ROP changes to evaluate their effectiveness to ensure that the intended results have been realized and to evaluate any unintended consequences. This paper addresses two areas that were selected for effectiveness reviews: the IFRM process trial period and the 120-day PD pilot metric in IMC 0307, Appendix A.

CONTACT: Alex D. Garmoe, NRR/DIRS 301-415-3814

C. Miller DISCUSSION:

The IFRM and 120-day PD metric were developed in response to the staff requirements memorandum (SRM) for COMSECY-14-0030, Proposed Suspension of the Reactor Oversight Process Self-Assessment for Calendar Year 2014. In the SRM, the Commission stated, in part, As part of its efforts to address ROP improvement recommendations, the staff should work to streamline the Significance Determination Process and establish appropriate timeliness metrics for finalizing inspection findings.

On June 30, 2015, in response to SRM-COMSECY-14-0030, the staff issued a Commissioners Assistants (CA) note (Agencywide Documents Access and Management System (ADAMS)

Accession No. ML15180A105) outlining plans to carry out the work. In the CA note, the staff specifically discussed the development of a new timeliness metric to be included in IMC 0307, Appendix A, which would track the time from the date the agency is first made aware of an event or degraded condition to the date the associated inspection finding is described in a preliminary significance determination letter. The staff also discussed plans to take a broader and more holistic review of the significance determination process (SDP) to further improve the overall efficacy of the process. On August 18, 2015, the Commission issued SRM M150806B, Staff Requirements Memorandum - Strategic Programmatic Overview of the Operating Reactors Business Line, directing the staff to pilot the proposed revisions to the SDP and hold public meetings or workshops to clarify its approach to risk-informing the process.

When the staff issued the revised IMC 0307, Appendix A, which was effective as of January 2016 and included the new E-4 metric, a note was included that stated it was a pilot metric that would be evaluated through focused and effectiveness reviews. The approved wording of metric E-4 was:

The time from the identification date (i.e., the date the issue of concern was brought to the licensees attention by the NRC, the date the performance deficiency was self-revealed, or the date the licensee documented the condition resulting from the performance deficiency in the corrective action program) to the start date used for consideration of inspection findings in the assessment process (as defined by IMC 0305) is within 120 days.

The start date used for consideration of inspection findings in the assessment process is defined in IMC 0305 as the end of the inspection activities that designate the issue as an apparent violation (AV), violation (VIO), finding (FIN), or non-cited violation (NCV) in the RPS.

For quarterly integrated inspection reports, the last day of the quarter being assessed is the start date or the date of a re-exit if the finding disposition has changed since the original exit meeting. For all other inspection reports, the start date is the last day of onsite inspection activities in which the item was identified as an AV, FIN, VIO, or NCV. This date is often the date of the exit meeting or the date of a re-exit if the disposition of the finding or violation changed since the original exit meeting. So, effectively, IMC 0307, Appendix A, established a limit of 120 days from the time of identification of an issue to the date of the last exit on the issue.

In addition to establishing the 120-day metric, a multi-disciplinary team with representation from several NRC offices and regions examined ways to improve the efficiency and effectiveness of the SDP. The team reached out to internal stakeholders by conducting a series of meetings and held several public meetings to seek feedback from external stakeholders. In a CA note dated September 19, 2016 (ADAMS Accession No. ML16231A235), the staff provided an update on this effort. In the CA note, the staff outlined its areas of focus for streamlining the

C. Miller SDP and renamed the effort as the IFRM initiative. The staff documented that the following areas should receive additional focus for improving how the staff dispositions inspection findings of GTG safety significance:

1. management oversight and planning/execution of inspection findings
2. implementation of the SDP timeliness metric
3. interactions with licensees
4. conduct of the Significance and Enforcement Review Panel (SERP) process
5. integrated risk-informed approach to SDP decision-making The fifth item was separated into its own effort to revise Inspection Manual Chapter (IMC) 0609, Appendix M, Significance Determination Process Using Qualitative Criteria. The first four items were incorporated into the IFRM effort, which was described in revised process documents that were issued for trial use:
  • IMC 0609 Attachment 1TP, Significance and Enforcement Review Panel (SERP)

Process (ADAMS Accession No. ML16110A267); and

The September 2016 CA note also discussed how the new 120-day pilot metric and the IFRM process being developed were intended to work together to enhance efficiency and effectiveness in processing inspection findings without affecting the final decisions. Staff indicated that lessons learned would be evaluated to determine if permanent changes to the process should be made. The IFRM temporary procedures took effect in November 2016 and introduced several concepts to the overall dispositioning of inspection findings:

  • An Inspection Finding Review Board (IFRB) intended to ensure early alignment on the PD and approach to dispositioning the issue.
  • Introduction of a 255-day goal from the time of initial identification of an issue to issuance of the final significance determination. This goal is the sum of the IMC 0307 ROP metric of 120 days from issue identification to the date the issue is considered in the assessment process, 45 days to subsequently issue the inspection report, and the IMC 0307 ROP metric of 90 days to issue a final significance determination.
  • A call from the responsible Division Director to senior site management to discuss the outcome of the IFRB and plan for resolution of the issue.

The IFRM pilot period ran through calendar year 2017. As of the end of calendar year 2017, the staff had two years of 120-day metric data and one year of IFRM pilot run time data to evaluate.

The effectiveness review documented herein addressed both of these items.

Effectiveness Review of IFRM Pilot Period A review of historical ROP performance data revealed that the majority of timeliness issues occurred in the initiating events, mitigating systems, and barrier integrity cornerstones. This is not unexpected since issues in these three cornerstones are most often subject to a detailed risk evaluation (DRE) to determine the risk significance. The SDP guidance for the other cornerstones uses deterministic questions and flowcharts to arrive at a significance determination because emergency preparedness, radiation protection, and security issues typically cannot be characterized via core damage frequency. In many cases, arriving at a

C. Miller significance determination using these deterministic questions and flowcharts is straightforward, and consequently requires significantly less time than a DRE.

The IFRM temporary procedures discussed above were issued on October 28, 2016, and became effective on November 15, 2016. Training sessions and outreach were conducted to familiarize regional offices with changes reflected in the temporary procedures prior to commencement of the pilot period. All regional offices were also provided the opportunity to comment on the draft procedures prior to implementation. The comment resolution document is available in ADAMS under Accession No. ML16110A211.

Since the temporary procedures became effective on November 15, 2016, inspection issues with a start date on or after that date could have been dispositioned fully from start to finish using the IFRM program. Therefore, November 15, 2016, was used as the start date for the pilot. The pilot period ended December 31, 2017. The Director of the Division of Inspection and Regional Support in NRR communicated to the regions to continue using the temporary IFRM procedures until the effectiveness review team completed its work and revised procedures were developed. This was intended to preclude a period of uncertainty in which the IFRM pilot period was completed but the evaluation of which aspects to keep moving forward was not yet complete.

The primary objective of the IFRM program is to enhance the efficiency of the SDP with an emphasis on improved management oversight and planning for GTG inspection findings. As discussed above, the pilot covered the initiating events, mitigating systems, and barrier integrity cornerstones. Table 1 lists findings in these cornerstones with a start date on or after November 15, 2016, that were transmitted to the licensee as preliminarily GTG.

Table 1: Issues with start dates after November 15, 2016, in IE, MS, and BI cornerstones in which preliminary GTG determination was documented Report Final Issue Exit Date 255-Issuance Preliminary Issuance Final Plant Identification IFRB Date (120-day day (45-day Significance (90-day Significance (Start) Date metric) metric metric) metric)

Cooper: Bus Fault - High Energy Arc Flash 8/11/17 8/14/17 12/20/17 337 1/17/17 4/26/17 White Green (206 days) (3 days) (128 days) days Clinton: Division 1 EDG Relay Fan Failure to Start 8/3/17 8/14/17 11/27/17 263 3/9/17 5/11/17 White White (147 days) (11 days) (105 days) days Catawba 2: EDG Excitation Diodes Overtemperature 7/17/17 8/22/17 10/16/17 188 4/11/17 6/29/17 White White (97 days) (36 days) (55 days) days Farley 2: 2B EDG Jacket Water Leak 7/27/17 8/4/17 Apparent 10/27/17 189 4/21/17 6/12/17 Green (97 days) (8 days) Violation (84 days) days Clinton: Div 3 SX Pump Failure 12/28/17 1/26/18 2/22/18 252 6/15/17 10/26/17 White White (196 days) (29 days) (27 days) days Davis-Besse: AFW Wiped Bearing 1/31/18 3/9/18 4/13/20181 2121 9/13/17 11/20/17 White White (140 days) (37 days) (35 days) days 1

The final significance determination was issued on April 13, 2018, after the IFRM effectiveness review concluded, thus the 90-day and 255-day metrics are not included in this report. However, the 120-day and 45-day metric data is included since it was completed prior to the effectiveness review.

C. Miller Because one of the objectives of the IFRM effectiveness review team was to consider whether the IFRM should be expanded to the other cornerstones, Table 2 lists the findings in the four cornerstones not subject to the IFRM pilot with a start date on or after November 15, 2016, that were transmitted to the licensee as preliminarily GTG.

Table 2: Issues with start dates after November 15, 2016, in non-IFRM cornerstones in which preliminary GTG determination was documented Report Final Issue Exit Date Issuance Preliminary Issuance 255-day Final Plant Identification (120-day (45-day Significance (90-day metric Significance Date metric) metric) metric)

Browns Ferry: Security Issue 4/17/17 5/3/17 7/11/17 131 3/2/17 GTG GTG (46 days) (16 days) (69 days) days Sequoyah: Security Issue 9/20/17 10/10/17 12/13/17 181 6/15/17 GTG GTG (97 days) (20 days) (64 days) days Wolf Creek: Security Issue 10/4/17 11/2/17 12/20/17 131 8/11/17 GTG GTG (54 days) (29 days) (48 days) days One of the key activities introduced by the IFRM process is the IFRB. The IFRB is intended to ensure that regional management and staff are aligned on the PD, plan for assessing the significance, and a completion schedule and supporting action items. Table 3 lists IFRB occurrences during the pilot period.

Table 3: All IFRBs conducted during pilot period Discovery IFRB Plant Issue IFRB Conclusion Issue Final Outcome Date Date Region I 9/28/16 Pilgrim EDG right angle 1/24/17 Approved performance of Green finding issued gearbox oil leak a DRE; requested staff further refine PD 5/17/17 Pilgrim DRV solenoid coil 6/20/17 Further evaluate issue Green finding issued resistance Region II 8/21/16 St. Lucie Configuration control 12/5/16 Proceed with current SDP White finding issued issue resulted in result of White reactor trip 3/8/17 Vogtle Loss of EDG building 6/1/17 No PD No finding HVAC 3/18/17 Turkey Point High Energy Arc 6/1/17 Revised PD, agreed on Green finding issued Flash planning SERP and senior reactor analyst site visit to support DRE 4/11/17 Catawba EDG excitation 6/29/17 PD approved, plan for White finding issued diodes SERP overtemperature 4/21/17 Farley 2B EDG jacket water 6/12/17 PD approved, plan to Green finding issued leak perform DRE 6/12/17 Browns EDG building flood 8/23/17 PD approved, plan to Green finding issued Ferry seal gaps perform DRE Region III 5/10/16 Clinton Division 1 SX room 4/6/17 PD approved, plan to Licensee provided fan trip perform DRE then have additional info - no SERP DRE/SERP needed

C. Miller 11/6/16 Perry EDG failed diode 12/22/16 PD approved, plan to White finding issued perform DRE then have SERP 12/5/16 Clinton RHR C breaker 2/23/17 PD approved, plan to DRE concluded Green failure perform DRE 1/25/17 Duane CIV Failures 2/23/17 PD approved, plan to Additional info and Arnold perform DRE finding re-screened to Green 2/1/17 Quad Cities Control Room Bulb 4/6/17 PD approved with follow- Further inspection Fire up actions, plan for revealed no PD planning SERP and need for fire modeling expertise 3/8/17 Clinton Div 1 EDG relay fan 5/11/17 Plan to perform DRE and White finding issued failed to start have SERP 6/15/17 Clinton Div 3 SX pump 10/26/17 PD approved, plan to White finding issued failure perform DRE and have SERP 8/23/17 Perry Failure of 9/25/17 One PD with 2 violations DRE concluded Green HPCS/RCIC suction approved post-IFRB, plant transfer below 32F to perform DRE 9/13/17 Davis-Besse AFW pump failure 11/20/17 PD approved, plan to White finding issued perform DRE and have (see Footnote 1)

SERP Region IV 9/16/17 ANO EDG bearing failure 11/21/16 PD approved, perform White finding issued DRE 9/24/16 Grand Gulf ADHR issue 2/16/17 PD approved, perform Green finding issued DRE 12/22/16 ANO QCST to CST Swap 1/12/17 PD approved, additional Green finding issued Over inspection required to support DRE Toward the end of the pilot period, an effectiveness review team was selected to evaluate the pilot period and determine if changes to the process should be made. This team consisted of representatives from all four regional offices, NRRs Divisions of Risk Assessment and Inspection and Regional Support, and OE. Industry feedback was also solicited during several monthly ROP public meetings. The IFRM effectiveness review team met at NRC headquarters on March 6 and 7, 2018.

Industry feedback on IFRM during and following the pilot was generally focused on communication and concerns about unintended consequences of timeliness metrics. Early communication between NRC and senior site management was viewed as helpful and a positive development, and industry does not believe the post-IFRB phone call should be delegable from division management. Industry voiced some concern about possible unintended consequences of the additional timeliness metrics in IFRM, in particular that staff might be driven by the metrics to arrive at a premature conclusion rather than waiting for sufficient information to be available.

Effectiveness review team representatives reported that their regions/offices supported the overall philosophy of the IFRM; however, the actual implementation of the process varied. After considering all inputs, the team recommended incorporating the IFRM process into permanent guidance, with some changes described herein. The team acknowledged that the pilot period was short and the sample size of GTG findings dispositioned in accordance with the pilot process was relatively small. Nonetheless, there was general agreement that the IFRM process

C. Miller has driven increased early engagement, focus, and alignment on issues that are potentially GTG, and the effectiveness review team viewed this as appropriate.

Recommendation 1: IFRM should be continued with the incorporation of the recommendations in this effectiveness review report.

The effectiveness review team reviewed data from the IFRM pilot period. As compared to ROP experience prior to the IFRM pilot, in which many GTG findings in the three cornerstones covered by the IFRM pilot were finalized in greater than 255 days (40%) or greater than one year (30%), timeliness during the IFRM pilot period showed improvement. For GTG findings within the three cornerstones covered by the IFRM pilot, 40% were finalized in greater than 255 days after discovery but none were finalized more than one year after discovery (a substantial improvement). In addition, the average time from initial discovery to final dispositioning of GTG findings in the three IFRM cornerstones improved from 271 days in the three years preceding the IFRM pilot to 246 days for findings discovered after the IFRM pilot began, a roughly 10%

improvement.

While overall timeliness in dispositioning GTG inspection findings has shown improvement, the relatively small sample size during the IFRM pilot period and the differences in the way the four regions implemented the IFRM provided challenges to determining the effectiveness of the various elements of the IFRM program in driving timely decisions. Complete data is provided in Table 1. During the pilot period, for issues within the IFRM pilot cornerstones, the 120-day metric covering the period from issue discovery to the date the issue was considered in the assessment process was met in two out of six instances (33%), the 45-day metric to issue the inspection report was met in every instance (100%), and the 90-day metric to finalize the SDP result was met in three out of five instances (60%). For the purposes of this effectiveness review, the Davis-Besse auxiliary feedwater issue identified in September 2017 is being included in the 120-day and 45-day metrics even though the final significance was not known at the time of the review (see Footnote 1).

Recommendation 2: Perform a follow-up effectiveness review of the IFRM process once permanent procedure revisions have been implemented and the process has been provided at least three full years of run time.

The 120-day metric was a pilot metric added to IMC 0307, Appendix A, at the beginning of calendar year 2016, so the staff has limited comparable historical data to compare performance against. The 120-day metric was only met in two out of six instances for GTG issues that were within the IFRM pilot cornerstones and were discovered after the IFRM temporary procedures became effective. However, regional experience revealed that there was an increased focus on alignment and timeliness early in the process of dispositioning potentially GTG issues. The primary driver for this was viewed to be the introduction of the 120-day PD metric. Another insight is that regional experience generally showed that the 120-day metric resulted in a shifting of the majority of significance determination work to earlier in the timeframe for dispositioning issues than in the past. This was an unintended effect of the IFRM and it places added importance on the need to accurately characterize the PD early in the process. Changes to the PD later in the process can have a significant impact on the DRE.

The 90-day metric for findings finalized as GTG in calendar year 2017 (93% total and 88% in the IFRM pilot cornerstones) showed a decline from calendar year 2016 (100% total) but was improved over calendar year 2015 (88% total and 82% in the IFRM pilot cornerstones). of SECY-17-0049, Reactor Oversight Process Self-Assessment for Calendar Year 2016, noted that the 90-day SDP metric was met in all instances but also stated, However, the staff continued to experience challenges in timely completion of the GTG inspection findings

C. Miller from initial identification until a final decision on significance is made. There was one issue finalized as GTG in the IFRM pilot cornerstones in which the 90-day SDP metric was not met in calendar year 2017. The agency made a deliberate decision to ensure updated information from laboratory testing was considered before a final determination was made, which resulted in missing the 90-day SDP metric.

There was significant discussion amongst the effectiveness review team regarding the value of the three metrics that make up the overall IFRM 255-day timeliness goal, whether they are set appropriately, and how the IFRM metrics dont align with the internal OE goal to complete non-Office of Investigations cases within 120 days of the inspection exit. This 120-day OE goal starts at the same time the 45-day inspection report issuance goal starts, which means once the PD is considered a part of the ROP assessment program, the staff is working to meet both a 135 day ROP metric (45 days to issue the inspection report and 90 days to complete the final significance determination) and a 120 day OE metric to complete the case. The team concluded that a holistic review to align agency metrics would help staff efficiently work through agency processes by focusing on a single common end point.

Recommendation 3: There are numerous agency metrics related to the disposition of inspection findings. These metrics dont always align, causing staff to work toward different and competing timeliness goals. A holistic review of agency metrics in this area should be conducted to ensure they provide consistent goals.

While the team noted the general positive influence of the 120-day metric in driving increased early focus and alignment, one area of concern that the team does not believe has been sufficiently considered is the starting point for the metric. The starting point of the 120-day metric is defined in IMC 0307, Appendix A, metric E-4, Completion of Performance Deficiency Determinations, as, i.e., the date the issue of concern was brought to the licensees attention by the NRC, the date the performance deficiency was self-revealed, or the date the licensee documented the condition resulting from the performance deficiency in the corrective action program. The team found the first two criteria to be clear but discussed concerns with the third criteria in certain instances. In particular, the team noted that specialized inspectors (for example, radiation protection, security, inservice inspection, or fire protection) only periodically visit the site for focused inspection activities. By the time one of these inspectors visits a site and can fully assess condition reports that describe conditions that might, on the surface, appear insignificant, most or all of the 120-day metric time could potentially be exhausted.

Recommendation 4: The 120-day metric in IMC 0307 relies on prompt staff recognition of all potentially GTG issues, which may be an unreasonable expectation in some circumstances. Further review of when this metric is defined to begin is recommended.

The effectiveness review team noted differences in how IFRM was implemented in each region.

The team concluded that each regional office believed they were implementing the spirit of what IFRM intended, but in some cases did so in a way that dovetailed into existing region-specific processes. This appeared to be at least in part due to differences across the regional offices in the interpretation of what issues should be brought to an IFRB, when in the course of dispositioning an issue the IFRB should be held, and the level of detail expected to be brought to an IFRB. These differences are revealed in the data displayed in Table 3, which shows a large disparity in the number of IFRBs conducted among regional offices. The team assessed these implementation differences and noted that, while the existing IFRM guidance did contain sufficient detail to alleviate some of the differences in IFRB implementation, any permanent guidance would benefit from additional clarity and specificity. In addition, practices implemented

C. Miller by some of the regions related to early flagging and discussion of issues seemed to bring even more concerted attention to the 120-day metric. Since the team concluded that focus on the 120-day metric was a key driver to timely disposition of issues, it seems appropriate to explore best practices in this area.

Recommendation 5: Explore best regional practices in bringing early focus to potentially GTG issues.

The temporary IFRM procedures issued in support of the pilot period define the objectives of the IFRB as:

  • Ensure regional management and staff align on the licensee PD, the degraded condition, and how the PD is the proximate cause of the degraded condition.
  • Ensure there is early alignment on the scope, schedule, and involved resources to support an efficient and effective preliminary significance assessment on potentially GTG inspection findings.
  • Provide a mechanism to effectively communicate with licensee senior management the inspection finding, support needed from their staff in reaching the preliminary assessment decision, and the appropriate timeframe to provide information.

The effectiveness review team recommends that the IFRB objectives be clarified as follows:

  • Ensure regional management and staff align on the licensee PD, the degraded condition, and how the PD is the proximate cause of the degraded condition. The proposed violation can be discussed, but alignment is not necessary at this point.
  • Ensure there is early alignment on the scope, schedule, and involved resources to support an efficient and effective preliminary significance assessment.
  • Develop key messages to communicate to licensee senior management.

In addition to the specific objectives mentioned above, the guidance should be enhanced to clarify the following points:

  • IMC discussions on conduct of the IFRB should include that the IFRB should discuss if a planning SERP is needed. The IFRB form should include a box for determination of whether a planning SERP is necessary. The IFRB guidance should refer to existing guidance on when a planning SERP is needed.
  • It is beneficial to have as much certainty around the PD as possible at the IFRB since subsequent changes to the PD are likely to have resource implications. Specifically, changes to the PD would typically result in the need to make corresponding revisions to the DRE. However, it is also recognized that additional information might become available after the IFRB that warrants adjustments to the PD.

Recommendation 6: Refine and enhance the IFRB guidance to better define the objectives and purpose of the IFRB.

Another area that the team determined led to inconsistent implementation of the IFRM pilot surrounded the level of detail expected to be brought to the IFRB, specifically regarding the DRE. One of the modifications introduced with the IFRM process was a modification to the SERP worksheet. The process was designed to include one worksheet package, starting with IFRB preparations and culminating with completion of a SERP, in which any information documented to date would be carried forward in the same information package to support later steps in the overall process of dispositioning an inspection finding. This was intended to increase efficiency by requiring information to be documented one time then carried forward and

C. Miller built upon, but in practice this single information package created some confusion over what information was intended to be presented at the IFRB and what information was expected to be developed later in the process. In some cases, IFRBs were held early in the process, before any significant detailed risk work had been completed, and focused on the proposed PD and plan for assessing the significance. In other cases, the impression was that detailed risk information was necessary before holding an IFRB, and IFRB members tended to challenge aspects of the risk evaluation work that had not, and should not have, been developed yet at that point. The team recommends splitting the existing combined IFRB-SERP package into separate IFRB and SERP forms to help indicate that these are different steps in the process with different purposes, and that DREs are not necessary at the IFRB stage.

Recommendation 7: Simplify the IFRB form to align with the level of detail expected at an IFRB and split the IFRB and SERP forms into separate products.

In recommending to continue using the IFRM process and build it into permanent inspection finding dispositioning guidance, the effectiveness review team also considered whether the IFRM process, which was piloted only in the initiating events, mitigating systems, and barrier integrity cornerstones, should be expanded to the other four cornerstones: emergency preparedness, public radiation safety, occupational radiation safety, and security. The IFRM process was piloted in the three cornerstones in which a DRE is most likely to occur, and such findings have typically required the most time to disposition. Issues identified in the four cornerstones not subject to the IFRM pilot are assessed by SDP appendices that use flowcharts and qualitative questions to arrive at a significance determination since an impact on core damage frequency often cannot be assessed for such issues. Dispositioning issues through these SDP appendices has typically involved significantly less time and complexity. Because of this, there was a view from some of the regional offices that expanding the IFRM process, and more specifically the IFRB component, to these remaining cornerstones would be of little to no benefit while requiring additional resources. A contrasting view was also presented that there is value in dispositioning issues via a consistent process regardless of the cornerstone and the IFRB could still add value in instances where SDP questions cannot be clearly answered based on nuances of certain issues. After significant discussion on this issue, the team recommends that the IFRM be expanded to the remaining cornerstones. However, the IFRB entry criteria should reflect that an IFRB is not required if the issue can be clearly, simply, and quickly routed through a deterministic SDP. With regards to guidance on when an IFRB should be held, the team recommends the following:

  • The IFRB is a regional activity that is convened at the recommendation of the regional branch chief and senior reactor analyst (if involved in the issue). It is intended as a planning and alignment meeting for the dispositioning of complex issues.
  • For non-deterministic SDPs, the IFRB should be convened when inspection findings do not initially screen to Green. However, the IFRB is not necessary for issues that do not initially screen to Green but are straightforward issues that experience has shown will ultimately and quickly result in a Green determination.
  • For deterministic SDPs, the IFRB should be convened when inspection findings involve complexities such that the outcome of the deterministic SDP is not clear or straightforward. An IFRB is not necessary when the significance of the finding using deterministic SDP flowcharts appears to be clear and straightforward, regardless of proposed significance.
  • An IFRB can be held based on management discretion regardless of whether the entry criteria are met.

C. Miller The team also recommends that the guidance indicate the following:

  • Regions should use judgement in determining when to schedule the IFRB, balancing the desire to delay for purposes of seeking additional information with the ability and desire to disposition the issue in a timely manner.
  • It is beneficial to hold the IFRB promptly once it is known that the IFRB entry criteria are met to drive timely dispositioning of the issue.
  • A follow-up IFRB should be considered when it is expected that there may be significant departures from what was agreed upon at the initial IFRB. For example, a follow-up IFRB could be considered when:

o A change to the previously aligned-upon PD is proposed that may result in a significant change to the previously aligned-upon resources, schedule, and plan for assessing the significance o If the need for one was determined at a previous IFRB on the issue

  • Sensitivity should be given to the possible impact on the licensee of changes to resource and schedule plans, and whether an update call with licensee senior management is appropriate.

Recommendation 8: IFRM should be expanded to the remaining cornerstones with enhanced IFRB entry criteria and guidance.

The effectiveness review team discussed IFRB attendance expectations, noting that regional differences led to slight inconsistencies in who was expected to attend IFRBs. Attachment 5TP of IMC 0609 states that the IFRB should consist of the IFRB Chair (the Division Director or Deputy Director responsible for the issue); the lead inspector; the Branch Chief responsible for the inspection; the Division of Reactor Projects Branch Chief, if different from the inspection Branch Chief; and a regional enforcement specialist. An individual from headquarters was typically present by phone at IFRBs during the pilot period to assist in carrying out this new process. The team felt this attendance list was generally appropriate. Because the IFRB is held early in the process of dispositioning an issue and is intended to gain regional alignment, there were some views presented that headquarters staff should not be involved before a PD has been agreed upon. Others on the effectiveness review team noted the potential for increased efficiency if headquarters staff participated as observers, as they would gain an early understanding of the issue once it moved into the SERP process. The team ultimately recommends that headquarters participation at an IFRB not be an expectation, but be left up to regional discretion. The team also recommends that the guidance indicate that if a finding involves a Division of Reactor Safety (DRS) technical area, that the appropriate DRS Branch Chief attend the IFRB if they are not the inspection Branch Chief.

One of the main objectives of the IFRB is to enhance early communications with the licensee on issues that are potentially GTG. This early communication helps leadership in both NRC and licensee organizations gain a common understanding of resource and information needs as the issue is dispositioned. Because of this desire for common understanding at the leadership level, existing IFRM temporary procedures direct that this communication be made at the Division Director (or Deputy Director) level and be non-delegable. Some regions were of the view that this call could or should be handled at a lower level - the Branch Chief or Senior Resident Inspector level - since communications about the issue would be occurring at this level anyway. After much discussion, the team generally agreed with the value of a management level discussion to ensure both organizations are aware of the issue, the plan for dispositioning the issue, and any resource or information needs. As a result, the team

C. Miller recommends that this call occur following the IFRB at the SES and senior site management level, with the intent of the call being clearly stated to:

  • communicate that NRC has completed an IFRB,
  • discuss the PD,
  • discuss NRCs planned schedule,
  • discuss information needs, and
  • consider whether the concepts of best available information and/or proximate cause must also be discussed.

Recommendation 9: Modify IFRB attendance expectations and clarify the intent of the management call following an IFRB.

Focused Review of IMC 0307, Appendix A, Pilot Metric E-4 The IFRM effectiveness review team had extensive discussion about the 120-day metric since it is a fundamental aspect of the IFRM. This discussion lead to two recommendations about the 120-day metric documented in the preceding section of this report.

Recommendation 3: There are numerous agency metrics related to the disposition of inspection findings. These metrics dont always align, causing staff to work toward different and competing timeliness goals. A holistic review of agency metrics in this area should be conducted to ensure they provide consistent goals.

Recommendation 4: The 120-day metric in IMC 0307 relies on prompt staff recognition of all potentially GTG issues, which may be an unreasonable expectation in some circumstances. Further review of when this metric is defined to begin is recommended.

Staff from the ROP Assessment Branch (IRAB) in the Division of Inspection and Regional Support in NRR further assessed the 120-day pilot metric from an ROP self-assessment perspective. As currently worded, pilot metric E-4 includes a note stating, This is a pilot metric that will be evaluated through focus and effectiveness reviews, and only applies to those findings finalized as Greater-than-Green. The purpose of the metric is to track the front end timeliness in the overall process of dispositioning inspection findings. This metric starts at the point in which the agency becomes aware of an issue and concludes at the start date used for consideration of inspection findings in the assessment process. Because at this point in the process it is not known what the final significance of the issue will be, the IRAB staff believes the metric should be applicable to all findings in which a preliminary GTG significance was transmitted to the licensee, regardless of the final outcome of the issue.

Recommendation 10: Modify the 120-day metric so that it is applicable to all findings in which a preliminary GTG significance was transmitted to a licensee, regardless of the final significance.

In reviewing adherence to this metric in calendar years 2016 and 2017, the staff encountered a situation in which the agency became aware of an issue and held an exit meeting with a preliminary White finding in calendar year 2017, but the preliminary and final significance determination letters (both White) were not sent until calendar year 2018. There were differing views on whether this issue should count in the calendar year 2017 or 2018 ROP self-assessment and staff concluded that the existing guidance did not provide sufficient clarity on this point. Along with its recommendation that the metric apply to all issues in which a preliminary GTG significance determination is transmitted, the staff proposes that the metric clearly state that the date of this preliminary significance transmittal letter dictate the time period

C. Miller in which the 120-day metric for the issue be reported. Before transmittal of the preliminary significance determination letter, staff would not be certain that this metric is applicable to the issue.

Recommendation 11: Clarify that the 120-day metric should be reported in the timeframe commensurate with the date in which the preliminary GTG significance determination was transmitted in writing to the licensee.

CONCLUSION AND RECOMMENDATIONS:

Despite some instances of inconsistent implementation of the IFRM process during the pilot period, the IFRM effectiveness review team found broad support for the philosophy of IFRM and recommends that it become part of the permanent agency process for dispositioning potentially GTG inspection findings. Specifically, the IFRM process has driven increased early engagement, focus, and alignment on issues that are potentially GTG, enhanced communication between agency and licensee senior leadership, and the timeliness of dispositioning of such issues showed improvement. The team developed the following recommendations:

1. IFRM should be continued with the incorporation of process adjustments recommended in the effectiveness review report.
2. Perform a follow-up effectiveness review of the IFRM process once permanent procedure revisions have been implemented and the process has been provided at least three full years of run time.
3. There are numerous agency metrics related to the disposition of inspection findings.

These metrics dont always align, causing staff to work toward different and competing timeliness goals. A holistic review of agency metrics in this area should be conducted to ensure they provide consistent goals.

4. The 120-day metric in IMC 0307 relies on prompt staff recognition of all potentially GTG issues, which may be an unreasonable expectation in some circumstances. Further review of when this metric is defined to begin is recommended.
5. Explore best regional practices in bringing early focus to potentially GTG issues.
6. Refine and enhance the IFRB guidance to better define the objectives and purpose of the IFRB.
7. Simplify the IFRB form to align with the level of detail expected at an IFRB and split the IFRB and SERP forms into separate products.
8. IFRM should be expanded to the remaining cornerstones but the IFRB should not be mandated when the deterministic SDP outcome is clear and straightforward.
9. Modify IFRB attendance expectations and clarify the intent of the management call following an IFRB.

The IRAB staff evaluated the 120-day PD metric (metric E-4 of IMC 0307, Appendix A), which is a component of the IFRM and is currently in pilot status. The staff recognizes and agrees with Recommendations 2 and 3 from the IFRM effectiveness review, which deal with the 120-day metric. Based on its review, the IRAB staff also recommends the following:

10. Modify the 120-day metric so that it is applicable to all findings in which a preliminary GTG significance was transmitted to a licensee, regardless of the final significance.
11. Clarify that the 120-day metric should be reported in the timeframe commensurate with the date in which the preliminary GTG significance determination was transmitted in writing to the licensee.

C. Miller

SUBJECT:

RESULTS OF THE CALENDAR YEAR 2018 REACTOR OVERSIGHT PROCESS SELF-ASSESSMENT EFFECTIVENESS REVIEWS ON THE INSPECTION FINDING RESOLUTION MANAGEMENT PILOT PROGRAM AND INSPECTION MANUAL CHAPTER 0307 PILOT METRIC E-4. MAY 9, 2018 DISTRIBUTION:

RidsNrrDirs RidsNrrDirsIrab RidsNrrOd RidsRgnIVMailCenter RidsRgnIMailCenter RidsRgnIIMailCenter RidsRgnIIIMailCenter GGeorge CMiller MKing THipschman GBowman MShuaibi KOBrien PLouden JLara JYerokun BWelling DPelton MGray JMunday MFranke AGody MMiller AVegel JClark TPruett RLantz MFerdas SSandal AStone ADAMS ACCESSION NO: ML18123A319 *concurred via email OFFICE RI/AES* RII/DRP/RPB1* RIII/DRP/B1* RIV/DRS/PSB2*

NAME BBickett FEhrhardt KStoedter HGepford DATE 04/23/2018 04/13/2018 04/18/2018 04/23/2018 OFFICE OE/EB* DRA/APHB* NRR/DIRS/IRAB* NRR/DIRS/IRAB*

NAME MMarshfield MKichline AGarmoe GBowman DATE 04/20/2018 04/23/2018 04/23/2018 05/01/2018 OFFICE RIII/DRS*

NAME MShauibi DATE 05/09/2018 OFFICIAL RECORD COPY

Retrieved from "https://kanterella.com/w/index.php?title=ML18123A319&oldid=2406356"