ML15014A160

Attachment 2 - NRC Working Group Recommendations to Revise the Substantive Cross-Cutting Issue Process

ML15014A160
Person / Time
Site:	Monticello, Browns Ferry, Palisades, Palo Verde, Fermi, Wolf Creek, Point Beach, Grand Gulf, Cooper, Prairie Island, Diablo Canyon, San Onofre, Comanche Peak, Fort Calhoun, Nuclear Energy Institute
Issue date:	01/14/2015
From:	Nuclear Energy Institute
To:	NRC/NRR/DIRS/IPAB
Daniel Merzke, 415-8744
Shared Package
ML15014a140	List: ML15014A160
References
SECY-99-007
Download: ML15014A160 (6)

Text

Attachment 2 1 Working Group Recommendations to Revise the Substantive Cross-Cutting Issue 2 Process 3

4

Background:

The NRC incorporated cross-cutting areas into the original Reactor Oversight 5 Process (ROP) framework because the task group that developed the framework determined 6 that, these themselves as the root causes of performance problems, as documented in 7 SECY-99-007, Recommendations for Reactor Oversight Process Improvements, (ADAMS 8 Accession No. ML992740074). The NRC further developed the current substantive cross-cutting 9 issue (SCCI) process because the Commission directed the staff to enhance the ROP treatment 10 of cross-cutting issues to more fully address safety culture in Staff Requirements Memorandum 11 (SRM) 04-0111, Recommended Staff Actions Regarding Agency Guidance in the Areas of 12 Safety Conscious Work Environment and Safety Culture, (ADAMS Accession No.

13 ML042430661). The intended purpose of assigning an SCCI is to, inform the licensee that the 14 NRC has a concern with the licensees performance in the cross-cutting area and to encourage 15 the licensee to take appropriate actions before more significant performance issues emerge, as 16 documented in Inspection Manual Chapter (IMC) 0305, Operating Reactor Assessment 17 Program, (ADAMS Accession No. ML13178A032).

18 19 The staff held a public meeting with industry representatives on November 21, 2013, to discuss 20 the scope of issues to be addressed in the ROP Enhancement Project - Assessment Program.

21 Industry representatives stated that their top concern was the assignment and closure of SCCIs.

22 They challenged the premise that four findings with the same cross-cutting aspect in a 12-month 23 period is indicative of a potential performance problem, especially considering that the findings 24 are primarily of very low safety significance. Industry offered that licensee trending programs 25 should identify and initiate corrective actions for trends that might lead to declining performance.

26 The meeting summary can be found at ADAMS Accession No. ML13337A637.

27 28 Staff held an additional public meeting on February 5, 2014, to further discuss industry concerns 29 with the SCCI process. One concern noted was the significant resources required to address 30 and disposition the SCCIs for the perceived very low safety benefit. At this meeting, industry 31 provided an alternative model for oversight of licensee safety culture in lieu of SCCIs (ADAMS 32 Accession No. ML14056A299).

33 34 The ROP Independent Assessment Report, dated February 18, 2014, (ADAMS Accession No.

35 ML14035A567) recommended that the staff perform a comprehensive analysis to determine 36 whether the use of cross-cutting issues and safety culture provides regulatory value in terms of 37 licensee safety performance for the resources expended. The report further suggested that the 38 staff consider replacing the use of SCCIs and the current cross-cutting aspects, components, 39 and areas with a process that uses the recently developed Nuclear Safety Culture Common 40 Language traits and attributes in a graded regulatory response.

41 42 An SCCI working group was assembled to review the process and develop recommendations.

43 The working group was comprised of staff from DIRS, safety culture experts from DRA and OE, 44 and staff from all four Regions.

45 46 The staff performed an effectiveness review of the SCCI process, and the conclusions were 47 documented in a memorandum dated April 23, 2014 (ADAMS Accession No. ML14099A171).

48 The staff concluded:

49 1. It is very difficult to prove that licensee corrective actions resulting from identification of 50 an SCCI prevented more significant performance issues, especially for those licensees Page 1 of 6

Attachment 2 1 whose performance was steady before and after an SCCI was identified. Therefore, the 2 staff cannot say definitively that the SCCI process is either effective or ineffective.

3 2. SCCIs are not a leading indicator for declining licensee performance. Licensees moved 4 right in the Action Matrix 86 times without identification of an SCCI. (10 of those times a 5 cross-cutting theme was identified without issuing an SCCI.)

6 3. The resource cost for implementing the SCCI process is significant without an apparent 7 commensurate increase in the safety benefit.

8 9 The staff held another public meeting with industry on May 20, 2014, to further discuss the 10 industry proposed model to replace the SCCI process, relying on an industry voluntary initiative 11 with licensees monitoring their own safety culture. (ML14155A082) 12 13 Discussion: The original problem statement was the level of effort to develop, open and close 14 SCCIs is not commensurate with the perceived value. Stated differently, regulatory actions and 15 outcomes from identifying and monitoring SCCIs do not achieve gains in protecting public health 16 and safety that are commensurate with the resources expended. The SCCI working group met 17 on July 22, 2014, to discuss the industry proposal, as well as options for revising the current 18 process. The staff aligned around several recommendations:

19 20

• Reject the industry proposed model to replace the SCCI process. The staff 21 reviewed the industry proposal and determined it was not an acceptable alternative to 22 the SCCI process. After the staff reviewed the initial pilot implementation of the NEI 09-23 07, Fostering a Healthy Nuclear Safety Culture, process, there were concerns about 24 inconsistent implementation of the program with different licensees. The Nuclear Energy 25 Institute (NEI) recently briefed the staff on Revision 1 to the NEI 09-07 process, which 26 provides several different options for licensees to implement the program, and 27 encourages licensees to tailor the safety culture monitoring program to each sites 28 specific circumstances. While this makes sense for the licensees, it further reduces 29 standardization of the process, and challenges the NRCs ability to inspect the 30 programs, since each licensee will likely have a unique implementation methodology.

31 32 Furthermore, the staff would need to develop an inspection procedure to inspect 33 licensee implementation. It was suggested that IP 71152 (PI&R) could be used to 34 inspect; however, the staff felt that most inspectors did not possess the skill set for 35 inspecting safety culture programs. The working group concluded that a separate 36 inspection procedure would likely need to be developed, as well as a fairly extensive 37 training program. Additionally, the staff determined that enforcement would be a 38 challenge. As all licensees have adopted the NEI 09-07 process as an industry 39 voluntary initiative, it is possible to identify performance deficiencies. However, the 40 working group determined that in screening potential performance deficiencies in the 41 safety culture area, the staff would be challenged to screen them to be more-than-minor.

42 The working group could not readily identify acceptable escalated enforcement options.

43 44 The working group also had concerns with the lack of formalized training for members of 45 the Safety Culture Monitoring Panels. There was also a concern that the independent, 46 or third-party, assessment of a licensees safety culture was removed from NEI 09-07, 47 Rev. 0, and industry has not yet formalized that guidance. An independent safety 48 culture assessment is a critical component of an effective program.

49 50 Another working group concern was the lack of transparency in the proposed model.

51 The SCCI process provides a formal means of communicating an NRC concern with a Page 2 of 6

Attachment 2 1 licensees performance in a cross-cutting area in a public way through the semi-annual 2 assessment letters. While there are several ways that the NRC can informally 3 communicate concerns to a licensee, most of those methods are not transparent to the 4 public.

5 6 Finally, the working group asserts that if a licensee effectively implements the NEI 09-07 7 safety culture monitoring process, the likelihood of being issued an SCCI is significantly 8 reduced. The WG noted that there would be value in giving licensees credit for having 9 an effective safety culture monitoring program and NEI 99-07 process, particularly when 10 considering whether to assign an SCCI to a licensee.

11 12 The concern over the large resource burden without a commensurate safety benefit 13 could be addressed through a revision to the current SCCI process. The implementation 14 of the Safety Culture Common Language into the ROP should reduce challenges to 15 binning findings, which may also reduce the resource burden. Additional recommended 16 changes are described below.

17 18

• Revise the terminology. The word substantive has negative connotations that may or 19 may not be valid. It implies an issue with a higher significance than may be justified 20 considering the very low safety significance from which the SCCI usually derives.

21 Additionally, a licensee causal analysis may determine that the issue is not indicative of 22 a performance problem. Therefore, the working group recommends replacing the term 23 substantive cross-cutting issue with cross-cutting issue.

24 25 The following recommendations are intended to reduce the level of resource effort expended on 26 the SCCI process:

27 28

• Increase the threshold for a cross-cutting theme to six for all cross-cutting 29 aspects, except for safety conscious work environment (SCWE). The current 30 threshold of four findings with the same CCA was viewed as arbitrary and set too low to 31 indicate a trend. An increase in the threshold should reduce some of the resource 32 burden on industry and NRC and be more indicative of a trend. A higher threshold could 33 make SCCIs a better predictor of declining licensee performance. One potential 34 drawback is the public perception of relaxing regulatory oversight of licensees.

35 36 The following analysis is provided to support the recommended threshold:

37 38 CCAs were tabulated for each site over the course of 5 years (2009-2013); each data 39 point represents one year of CCAs at one site (site-year). This represents roughly 40 65X5=325 site-years.

41

• The mean is the average number of times a CCA was assigned in one year at any 42 site over the 5-year period.

43

• The median is 1.00 for any CCA assigned at least once in more than 50% of the site-44 years. (No CCA had a median higher than 1; that is, there was no CCA assigned 2 or 45 more times in more than 50% of the site-years). The CCAs with a median of 1 are 46 listed below.

47

• The 95th% (97th%)(99th%) is the number of times a CCA had to be assigned such that 48 only 5% (3%)(1%) of site-years met or exceeded that value.

49 50 Page 3 of 6

Attachment 2 H.1(b) H.2(c) H.4(a) H.4(b) P.1(c) P.1(d)

Conservative Complete and Work Follow Problem Implement decision- accurate practice - procedures evaluation corrective making documentation human error Action prevention Mean 1.28 1.39 0.94 1.01 1.49 0.76 Median 1.00 1.00 1.00 1.00 1.00 1.00 95th% 4.00 4.00 3.00 3.00 4.00 3.00 97th% 5.00 4.00 3.00 4.00 6.00 3.00 99th% 7.00 6.00 5.00 4.00 8.00 4.00 1

• The highlighted boxes represent CCAs where the cross-cutting theme threshold of 4 2 was met in at least 5% of site-years (roughly speaking, 5% of plants would get a 3 cross-cutting theme in that CCA in a given year).

4

• Raising the threshold to 6 (bold) would reduce the number of plants meeting the 5 cross-cutting theme threshold, but would still capture the sites with significant 6 problems. Licensees who would have still exceeded the new threshold based on 7 historical data include: Browns Ferry, Cooper, Diablo Canyon, Fort Calhoun, Fermi, 8 Monticello, Palisades, Palo Verde, Point Beach, Prairie Island, San Onofre, and Wolf 9 Creek.

10 11

• The subjective questions should be revised to set more objective criteria for 12 opening an SCCI. The staff recommends giving licensees credit for effective 13 implementation of the NEI 09-07 process by removing the current subjective questions 14 for opening an SCCI.

15 16 For the first assessment cycle a cross-cutting theme is identified, no SCCI would be 17 issued. The theme would be identified in an assessment letter, and the Region would 18 review licensee actions with regards to a causal analysis and/or corrective actions for 19 that theme, potentially through a focused PI&R sample. Findings contributing to the 20 theme would have been identified over previous 12 months.

21 22 The second consecutive assessment cycle where the licensee has the same theme, an 23 SCCI would still not be assigned. The Region would notify the licensee in the 24 assessment letter that this is the second consecutive assessment cycle with the same 25 cross-cutting theme, and the Region would consider the effectiveness of licensee 26 actions (e.g., additional findings with the same aspect during the last 6 months of the 27 assessment cycle) in determining whether or not to perform additional follow-up of 28 licensee corrective actions. IP 71152 could be revised to address additional follow-up 29 activities. Findings contributing to the second consecutive theme would have 30 accumulated over previous 18 months.

31 32 For the third consecutive assessment cycle with the same theme, an SCCI would be 33 automatic. This would trigger an SCCI follow-up inspection, similar to a 95001 34 inspection. The issue would remain open pending successful completion of the follow-35 up inspection. Findings leading to an SCCI would have accumulated over a 24-month 36 period.

37 Page 4 of 6

Attachment 2 1 Eliminating the subjective questions associated with opening an SCCI would make the 2 process more objective, transparent, and predictable to stakeholders, and should reduce 3 the resource burden in determining whether or not to open an SCCI during the semi-4 annual assessment meetings.

5 6

• For Column 4 plants, all SCCIs would be closed out in the Confirmatory Action 7 Letter (CAL) closing out the 95003 inspection. For plants in column 4, SCCIs would 8 still be identified based on the criteria being met. However, closure of the CAL would 9 indicate that the licensee has addressed the significant programmatic issues which led 10 to the issuance of the large number of findings from which the SCCIs were derived.

11 12 The following recommendations are intended to increase the regulatory impact:

13 14

• Develop a backstop at the cross-cutting area level. The staff would develop a 15 maximum number of findings in a cross-cutting area, above which the NRC would have 16 a concern with a licensees performance in that cross-cutting area. Theoretically, 17 assuming the new threshold at the cross-cutting aspect is adopted, a licensee could 18 have as many as 70 findings with human performance cross-cutting aspects and not 19 meet the criteria for a cross-cutting theme. Similarly, a licensee could have as many as 20 30 findings with PI&R cross-cutting aspects without reaching the threshold. These 21 numbers are unacceptably high, and would be indicative of serious problems at a site.

22 23 Staff performed a data analysis based on numbers of finding in cross-cutting areas, with 24 the results shown below:

25 26 Using the same data used to analyze individual cross-cutting aspects assigned by plant 27 year from 2009-2013, this analysis was performed by cross-cutting area. The average, 28 median, and 95th (97th)(99th) percentile results are shown below.

29 Human Perf PI&R Average 7.88 3.60 Median (50%) 7 3 95% 18 9 97% 19.46 11 99% 26.64 18.92 30 31 The working group is recommending using the 97th percentile results, setting a backstop 32 of 20 findings for human performance issues, and 12 for PI&R issues in a 12-month 33 period to trigger a cross-cutting theme.

34 35 Under these criteria, licensees who would have tripped the threshold include Comanche 36 Peak, Cooper, Diablo Canyon, Fort Calhoun, Grand Gulf, Palo Verde, Palisades, Prairie 37 Island, SONGS, and Wolf Creek. Most of these licensees would have crossed the 38 threshold for 6 findings at the CCA level before hitting the backstop.

39 A licensee who trips the threshold for the backstop at the cross-cutting area level would 40 meet the criterion for a cross-cutting theme, which would trigger some follow-up 41 inspection.

Page 5 of 6

Attachment 2 1

2

• Develop standard SCCI closure criteria. One industry concern with SCCIs is that they 3 often dont know what they need to do to close out the SCCI. IMC 0305 requires 4 regional offices to establish SCCI closure criteria, and for that criteria to be documented 5 in the assessment letter that opens the SCCI, or where an SCCI remained open. A 6 review of SCCI history identified some instances where the closure criteria were met, but 7 the SCCI remained open.

8 9 The working group recommends developing an SCCI follow-up inspection procedure, 10 similar to IP 95001, to review a licensees causal analysis and corrective actions, in 11 order to close out an SCCI. The inspection procedure could use elements of IP 95001, 12 IP 71152 for PI&R SCCIs, and IP 71841 for human performance SCCIs.

13 14 Additionally, it should be acceptable for a licensee to do a causal analysis on the cross-15 cutting theme and find there is no common cause. Under the current process, licensees 16 are compelled to find a common cause and develop corrective actions, even if there is 17 no common cause, in order to satisfy the regulator and close out the concern.

18 Inspectors could still review the licensee causal analysis for rigor, and this could reduce 19 the industry resources spent addressing cross-cutting themes that are not valid 20 concerns. If a licensee is issued an SCCI, then the theme has existed for at least one 21 year, and should be addressed with a causal analysis and corrective actions.

22 23

• Consider additional actions for licensees after the second consecutive 24 assessment cycle with the same SCCI. The working group recommends deleting the 25 terminology for a long-standing SCCI. By the time an SCCI is assigned, the theme has 26 already existed for three consecutive assessment cycles. The licensee would be 27 expected to develop corrective actions, and the NRC would conduct an SCCI follow-up 28 inspection. If the theme still exists in the next assessment cycle, or the licensee does 29 not successfully complete the follow-up inspection, a second consecutive SCCI would be 30 assigned. The actions formerly associated with long-standing SCCIs would apply. The 31 working group recommends adding additional options for Regions to consider when 32 dealing with licensees with a second consecutive SCCI. Those options would include 33 having Regional Administrators and/or the Director of NRR or the EDO meet with the 34 licensees Board of Directors to discuss licensee performance. Additionally, the working 35 group recommends revising the criteria for which licensees should be discussed at the 36 Agency Action Review Meeting (AARM) to include licensees with consecutive SCCIs, 37 with those licensees potentially being recommended to meet with the Commission.

38 Page 6 of 6