ML15014A160
| ML15014A160 | |
| Person / Time | |
|---|---|
| Site: | Monticello, Browns Ferry, Palisades, Palo Verde, Fermi, Wolf Creek, Point Beach, Grand Gulf, Cooper, Prairie Island, Diablo Canyon, San Onofre, Comanche Peak, Fort Calhoun, Nuclear Energy Institute  |
| Issue date: | 01/14/2015 |
| From: | Nuclear Energy Institute |
| To: | NRC/NRR/DIRS/IPAB |
| Daniel Merzke, 415-8744 | |
| Shared Package | |
| ML15014a140 | List: |
| References | |
| SECY-99-007 | |
| Download: ML15014A160 (6) | |
Text
Attachment 2 Page 1 of 6 Working Group Recommendations to Revise the Substantive Cross
-Cutting Issue 1 Process 2 3 Background
- The NRC incorporated cross
-cutting areas into the original Reactor Oversight 4 Process (ROP) framework because the task group that developed the framework determined 5 that, "these themselves as the root causes of performance problems," as documented in 6 SECY-99-007, "Recommendations for Reactor Oversight Process Improvements," (ADAMS 7 Accession No. ML992740074). The NRC further developed the current substantive cross
-cutting 8 issue (SCCI) process because the Commission directed the staff to enhance the ROP treatment 9 of cross-cutting issues to more fully address safety culture in Staff Requirements Memorandum 10 (SRM) 04-0111, "Recommended Staff Actions Regarding Agency Guidance in the Areas of 11 Safety Conscious Work Environment and Safety Culture," (ADAMS Accession No. 12 ML042430661). The intended purpose of assigning an SCCI is to, "inform the licensee that the 13 NRC has a concern with the licensee's performance in the cross
-cutting area and to encourage 14 the licensee to take appropriate actions before more significant performance issues emerge," as 15 documented in Inspection Manual Chapter (IMC) 0305, "Operating Reactor Assessment 16 Program," (ADAMS Accession No. ML13178A032).
17 18 The staff held a public meeting with industry representatives on November 21, 2013, to discuss 19 the scope of issues to be addressed in the ROP Enhancement Project
- Assessment Program.
20 Industry representatives stated that their top concern was the assignment and closure of SCCIs
. 21 They challenged the premise that four findings with the same cross
-cutting aspect in a 12
-month 22 period is indicative of a potential performance problem, especially considering that the findings 23 are primarily of very low safety significance. Industry offered that licensee trending programs 24 should identify and initiate corrective actions for trends that might lead to declining performance.
25 The meeting summary can be found at ADAMS Accession No. ML13337A637.
26 27 Staff held an additional public meeting on February 5, 2014, to further discuss industry concerns 28 with the SCCI process. One concern noted was the significant resources required to address 29 and disposition the SCCIs for the perceived very low safety benefit. At this meeting, industry 30 provided an alternative model for oversight of licensee safety culture in lieu of SCCIs (ADAMS 31 Accession No. ML14056A299).
32 33 The ROP Independent Assessment Report, dated February 18, 2014, (ADAMS Accession No.
34 ML14035A567) recommended that the staff perform a comprehensive analysis to determine 35 whether the use of cross
-cutting issues and safety culture provides regulatory value in terms of 36 licensee safety performance for the resources expended.
The report further suggested that the 37 staff consider replacing the use of SCCIs and the current cross
-cutting aspects, components, 38 and areas with a process that uses the recently developed Nuclear Safety Culture Common 39 Language traits and attributes in a graded regulatory response.
40 41 An SCCI working group was assembled to review the process and develop recommendations.
42 The working group was comprised of staff from DIRS, safety culture experts from DRA and OE, 43 and staff from all four Regions.
44 45 The staff performed an effectiveness review of the SCCI process, and the conclusions were 46 documented in a memorandum dated April 23, 2014 (ADAMS Accession No. ML14099A171). 47 The staff concluded:
48 1. It is very difficult to prove that licensee corrective actions resulting from identification of 49 an SCCI prevented more significant performance issues, especially for those licensees 50 Attachment 2 Page 2 of 6 whose performance was steady before and after an SCCI was identified. Therefore, the 1 staff cannot say definitively that the SCCI process is either effective or ineffective.
2 2. SCCIs are not a leading indicator for declining licensee performance.
Licensees moved 3 right in the Action Matrix 86 times without identification of an SCCI
. (10 of those times a 4 cross-cutting theme was identified without issuing an SCCI.)
5 3. The resource cost for implementing the SCCI process is significant without an apparent 6 commensurate increase in the safety benefit.
7 8 The staff held another public meeting with industry on May 20, 2014, to further discuss the 9 industry proposed model to replace the SCCI process, relying on an industry voluntary initiative 10 with licensees monitoring their own safety culture.
(ML14155A082) 11 12 Discussion: The original problem statement was the level of effort to develop, open and close 13 SCCIs is not commensurate with the perceived value.
Stated differently, regulatory actions and 14 outcomes from identifying and monitoring SCCIs do not achieve gains in protecting public health 15 and safety that are commensurate with the resources expended.
The SCCI working group met 16 on July 22, 2014, to discuss the industry proposal, as well as options for revising the current 17 process. The staff aligned around several recommendations:
18 19 Reject the industry proposed model to replace the SCCI process. The staff 20 reviewed the industry proposal and determined it was not an acceptable alternative to 21 the SCCI process. After the staff reviewed the initial pilot implementation of the NEI 09
-22 07, "Fostering a Healthy Nuclear Safety Culture," process, there were concerns about 23 inconsistent implementation of the program with different licensees.
The Nuclear Energy 24 Institute (NEI) recently briefed the staff on Revision 1 to the NEI 0 9-07 process, which 25 provides several different options for licensees to implement the program, and 26 encourages licensees to tailor the safety culture monitoring program to each site's 27 specific circumstances. While this makes sense for the licensees, it further reduces 28 standardization of the process, and challenges the NRC's ability to inspect the 29 programs, since each licensee will likely have a unique implementation methodology.
30 31 Furthermore, the staff would need to develop an inspection procedure to inspect 32 licensee implementation. It was suggested that IP 71152 (PI&R) could be used to 33 inspect; however, the staff felt that most inspectors did not possess the skill set for 34 inspecting safety culture programs. The working group concluded that a separate 35 inspection procedure would likely need to be develope d, as well as a fairly extensive 36 training program. Additionally, the staff determined that enforcement would be a 37 challenge. As all licensees have adopted the NEI 09
-07 process as an industry 38 voluntary initiative, it is possible to identify performance deficiencies.
However, the 39 working group determined that in screening potential performance deficiencies in the 40 safety culture area, the staff would be challenged to screen them to be more-than-minor. 41 The working group could not readily identify acceptable escalated enforcement options.
42 43 The working group also had concerns with the lack of formalized training for members of 44 the Safety Culture Monitoring Panels. There was also a concern that the independent, 45 or third-party, assessment of a licensee's safety culture was removed from NEI 09
-07, 46 Rev. 0, and industry has not yet formalized that guidance.
An independent safety 47 culture assessment is a critical component of an effective program. 48 49 Another working group concern was the lack of transparency in the proposed model.
50 The SCCI process provides a formal means of communicating an NRC concern with a 51 Attachment 2 Page 3 of 6 licensee's performance in a cross
-cutting area in a public way through the semi
-annual 1 assessment letters. While there are several ways that the NRC can informally 2 communicate concerns to a licensee, most of those methods are not transparent to the 3 public. 4 5 Finally, the working group asserts that if a licensee effectively implements the NEI 09-07 6 safety culture monitoring process, the likelihood of being issued an SCCI is significantly 7 reduced. The WG noted that there would be value in giving licensees credit for having 8 an effective safety culture monitoring program and NEI 99
-07 process, particularly when 9 considering whether to assign an SCCI to a licensee.
10 11 The concern over the large resource burden without a commensurate safety benefit 12 could be addressed through a revision to the current SCCI process. The implementation 13 of the Safety Culture Common Language into the ROP should reduce challenges t o 14 binning findings, which may also reduce the resource burden.
Additional recommended 15 changes are described below.
16 17 Revise the terminology.
The word "substantive" has negative connotations that may or 18 may not be valid. It implies an issue with a higher significance than may be justified 19 considering the very low safety significance from which the SCCI usually derives.
20 Additionally, a licensee causal analysis may determine that the issue is not indicative of 21 a performance problem. Therefore, the working group recommends replacing the term 22 "substantive cross
-cutting issue" with "cross
-cutting issue."
23 24 The following recommendations are intended to reduce the level of resource effort expended on 25 the SCCI process
- 26 27 Increase the threshold for a cross
-cutting theme to six for all cross
-cutting 28 aspects, except for safety conscious work environment (SCWE).
The current 29 threshold of four findings with the same CCA was viewed as arbitrary and set too low to 30 indicate a trend. An increase in the threshold should reduce some of the resource 31 burden on industry and NRC and be more indicative of a trend. A higher threshold could 32 make SCCIs a better predictor of declining licensee performance. One potential 33 drawback is the public perception of relaxing regulatory oversight of licensees.
34 35 The following analysis is provided to support the recommended threshold:
36 37 CCAs were tabulated for each site over the course of 5 years (2009
-2013); each data 38 point represents one year of CCAs at one site (site
-year). This represents roughly 39 65X5=325 site
-years. 40 The mean is the average number of times a CCA was assigned in one year at any 41 site over the 5
-year period.
42 The median is 1.00 for any CCA assigned at least once in more than 50% of the site
-43 years. (No CCA had a median higher than 1; that is, there was no CCA assigned 2 or 44 more times in more than 50% of the site
-years). The CCAs with a median of 1 are 45 listed below.
46 The 95th% (97th%)(99th%) is the number of times a CCA had to be assigned such that 47 only 5% (3%)(1%) of site
-years met or exceeded that value.
48 49 50 Attachment 2 Page 4 of 6 H.1(b) H.2(c) H.4(a) H.4(b) P.1(c) P.1(d) Conservative decision-making Complete and accurate documentation Work practice
- human error prevention Follow procedures Problem evaluation Implement corrective Action Mean 1.28 1.39 0.94 1.01 1.49 0.76 Median 1.00 1.00 1.00 1.00 1.00 1.00 95th% 4.00 4.00 3.00 3.00 4.00 3.00 97th% 5.00 4.00 3.00 4.00 6.00 3.00 99th% 7.00 6.00 5.00 4.00 8.00 4.00 The highlighted boxes represent CCAs where the cross
-cutting theme threshold of 4 1 was met in at least 5% of site
-years (roughly speaking, 5% of plants would get a 2 cross-cutting theme in that CCA in a given year).
3 Raising the threshold to 6 (bold) would reduce the number of plants meeting the 4 cross-cutting theme threshold, but would still capture the sites with significant 5 problems. Licensees who would have still exceeded the new threshold based on 6 historical data include: Browns Ferry, Cooper, Diablo Canyon, Fort Calhoun, Fermi, 7 Monticello, Palisades, Palo Verde, Point Beach, Prairie Island, San Onofre, and Wolf 8 Creek. 9 10 The subjective questions should be revised to set more objective criteria for 11 opening an SCCI.
The staff recommends giving licensees credit for effective 12 implementation of the NEI 09
-07 process by removing the current subjective questions 13 for opening an SCCI.
14 15 For the first assessment cycle a cross-cutting theme is identified, no SCCI would be 16 issued. The theme would be identified in an assessment letter, and the Region would 17 review licensee actions with regards to a causal analysis and/or corrective actions for 18 that theme, potentially through a focused PI&R sample.
Findings contributing to the 19 theme would have been identified over previous 12 months.
20 21 The second consecutive assessment cycle where the licensee has the same theme, an 22 SCCI would still not be assigned. The Region would notify the licensee in the 23 assessment letter that this is the second consecutive assessment cycle with the same 24 cross-cutting theme, and the Region would consider the effectiveness of licensee 25 actions (e.g., additional findings with the same aspect during the last 6 months of the 26 assessment cycle) in determining whether or not to perform additional follow
-up of 27 licensee corrective actions. IP 71152 could be revised to address additional follow-up 28 activities.
Findings contributing to the second consecutive theme would have 29 accumulated over previous 18 months.
30 31 For the third consecutive assessment cycle with the same theme, an SCCI would be 32 automatic.
This would trigger an SCCI follow
-up inspection, similar to a 95001 33 inspection.
The issue would remain open pending successful completion of the follow
-34 up inspection.
Findings leading to an SCCI would have accumulated over a 24
-month 35 period. 36 37 Attachment 2 Page 5 of 6 Eliminating the subjective questions associated with opening an SCCI would make the 1 process more objective, transparent, and predictable to stakeholders, and should reduce 2 the resource burden in determining whether or not to open an SCCI during the semi-3 annual assessment meetings.
4 5 For Column 4 plants, all SCCIs would be closed out in the Confirmatory Action 6 Letter (CAL) closing out the 95003 inspection.
For plants in column 4, SCCIs would 7 still be identified based on the criteria being met.
However, closure of the CAL would 8 indicate that the licensee has addressed the significant programmatic issues which led 9 to the issuance of the large number of findings from which the SCCIs were derived.
10 11 The following recommendations are intended to increase the regulatory impact
- 12 13 Develop a backstop at the cross
-cutting area level.
The staff would develop a 14 maximum number of findings in a cross
-cutting area, above which the NRC would have 15 a concern with a licensee's performance in that cross
-cutting ar ea. Theoretically, 16 assuming the new threshold at the cross
-cutting aspect is adopted, a licensee could 17 have as many as 70 findings with human performance cross
-cutting aspects and not 18 meet the criteria for a cross
-cutting theme. Similarly, a licensee could have as many as 19 30 findings with PI&R cross
-cutting aspects without reaching the threshold. These 20 numbers are unacceptably high, and would be indicative of serious problems at a site.
21 22 Staff performed a data analysis based on numbers of finding in cross-cutting areas, with 23 the results shown below:
24 25 Using the same data used to analyze individual cross
-cutting aspects assigned by plant 26 year from 2009
-2013, this analysis was performed by cross
-cutting area. The average, 27 median, and 95 th (97th)(99th) percentile results are shown below.
28 29 Human Perf PI&R Average 7.88 3.60 Median (50%)
7 3 95% 18 9 97% 19.46 11 99% 26.64 18.92 30 The working group is recommending using the 97 th percentile results, setting a backstop 31 of 20 findings for human performance issues, and 12 for PI&R issues in a 12-month 32 period to trigger a cross
-cutting theme.
33 34 Under these criteria, licensees who would have tripped the threshold include Comanche 35 Peak, Cooper, Diablo Canyon, Fort Calhoun, Grand Gulf, Palo Verde, Palisades, Prairie 36 Island, SONGS, and Wolf Creek. Most of these licensees would have crossed the 37 threshold for 6 findings at the CCA level before hitting the backstop.
38 A licensee who trips the threshold for the backstop at the cross
-cutting area level would 39 meet the criterion for a cross
-cutting theme, which would trigger some follow
-up 40 inspection.
41 Attachment 2 Page 6 of 6 1 Develop standard SCCI closure criteria.
One industry concern with SCCIs is that they 2 often don't know what they need to do to close out the SCCI. IMC 0305 requires 3 regional offices to establish SCCI closure criteria, and for that criteria to be documented 4 in the assessment letter that opens the SCCI, or where an SCCI remained open. A 5 review of SCCI history identified some instances where the closure criteria were met, but 6 the SCCI remained open.
7 8 The working group recommends developing an SCCI follow
-up inspection procedure, 9 similar to IP 95001, to review a licensee's causal analysis and corrective actions, in 10 order to close out an SCCI. The inspection procedure could use elements of IP 95001, 11 IP 71152 for PI&R SCCIs, and IP 71841 for human performance SCCIs.
12 13 Additionally, it should be acceptable for a licensee to do a causal analysis on the cross
-14 cutting theme and find there is no common cause. Under the current process, licensees 15 are compelled to find a common cause and develop corrective actions, even if there is 16 no common cause, in order to satisfy the regulator and close out the concern.
17 Inspectors could still review the licensee causal analysis for rigor, and this could reduce 18 the industry resources spent addressing cross
-cutting themes that are not valid 19 concerns.
If a licensee is issued an SCCI, then the theme has existed for at least one 20 year, and should be addressed with a causal analysis and corrective actions.
21 22 Consider additional actions for licensees after the second consecutive 23 assessment cycle with the same SCCI. The working group recommends deleting the 24 terminology for a long
-standing SCCI. By the time an SCCI is assigned, the theme has 25 already existed for three consecutive assessment cycles. The licensee would be 26 expected to develop corrective actions, and the NRC would conduct an SCCI follow
-up 27 inspection. If the theme still exists in the next assessment cycle, or the licensee does 28 not successfully complete the follow
-up inspection, a second consecutive SCCI would be 29 assigned. The actions formerly associated with long
-standing SCCIs would apply. The 30 working group recommends adding additional options for Regions to consider when 31 dealing with licensees with a second consecutive SCCI. Those options would include 32 having Regional Administrators and/or the Director of NRR or the EDO meet with the 33 licensee's Board of Directors to discuss licensee performance. Additionally, the working 34 group recommends revising the criteria for which licensees should be discussed at the 35 Agency Action Review Meeting (AARM) to include licensees with consecutive SCCIs, 36 with those licensees potentially being recommended to meet with the Commission.
37 38