ML19008A031

Preparatory Materials Provided to Interviewees

ML19008A031
Person / Time
Issue date:	01/30/2019
From:	Michael Orenak Plant Licensing Branch II
To:
Orenak M, 415-3229
References
Download: ML19008A031 (4)

Text

Preparatory Materials Provided to Interviewees Interview Questionnaire Note that this questionnaire is meant to be a guide for the interview and should be adapted and used to fit the particular interview subject. Interviewers should ask interviewees to provide examples to expand on their answers to the questions.

The preferred approach is a face-to-face interview for about 1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br />. This will be conducted by two team members, with the primary interviewer being the lead for the process, as indicated in Appendix 1. The other interviewer will be responsible for taking notes. The questionnaire, backgrounder, and risk insight table should be send to the interviewee in advance. In case face-to-face interviews are not possible, the interviewee can send written responses to the questionnaire or it could be done by email.

Interview Questions For the questions listed below, replace [process] with the specific process being discussed:

1. In [process], do you sometimes find that engineering judgement needs to be used in assessing the safety significance of a technical issue? Are there specific triggers that would lead you to this conclusion (e.g., certain language)?

2. In [process], do you have to make decisions about the scope of the technical review based on the potential effects of the proposed change, or is the scope well defined by the submittal or associated regulatory guidance?

3. In [process], do applicants ever submit information that makes use of risk-based justifications (whether qualitative or quantitative), regardless of whether it is labeled as such or in accordance with RG 1.174?

4. In [process], are there sometimes aspects of the review that may be handled more efficiently, without loss of focus on safety, by risk informing the level of effort? Do these aspects appear frequently in submittals, or are they outliers?

5. How familiar are you with risk-informed decision making? (Based on the answer to the above questions: provide any needed background. Tailor the answers to be as specific to the background of the interviewee as possible. What would be expected from that individual? How would these tools change how the work is done? What would the outcome look like compared to current work products?)

6. For each of the above, would these cases benefit from the use of off-the-shelf PRA tools and a standardized approach to apply quantitative risk insights? Can you think of any other cases that would benefit from risk insights?

Now, lets move on to a different set of considerations:

7. In [process], do cross-cutting issues arise that require involvement by more than one technical branch?

8. In [process], are there cases where a significant number of different branches are involved, without clear guidance regarding the scope and SE input for each branchs area?

9. In [process], do all staff working on a single project generally understand the roles of the other staff working on the project (especially their own)? Is this true even for relatively inexperienced staff members (assuming they are correctly using relevant guidance)?

10. In [process], do you believe that generation of the SE based on input from different reviewers sometimes causes challenges to the clarity and consistency of the message communicated by the NRC staff?

11. How familiar are you with the Integrated Review Team (IRT) concept? (Based on the answer to the above questions: provide any needed background. Tailor the answers to be as specific to the background of the interviewee as possible. What would be expected from that individual? How would these tools change how the work is done?

What would the outcome look like compared to current work products?)

12. For each of the above, would these cases benefit from the use of an IRT approach?

Can you think of any other cases that would benefit from the IRT approach?

Backgrounder on the RIDM Project and Risk Insights Table for Interviewees What is risk-informed decision making (RIDM)?

Risk is the combined answer to three questions that consider: (1) what can go wrong, (2) how likely it is, and (3) what its consequences might be.

Risk-informing a regulatory activity means combining risk information (for example, probabilistic risk assessment (PRA) results) with other factors (for example, engineering design features) to arrive at a decision. RIDM is an approach to regulatory decision making in which insights from PRA are considered with other engineering insights. A term often used incorrectly in place of risk-informed is risk-based; these terms are not synonyms. A risk-based approach to decision-making means that the decision is based only on risk information (e.g., risk results obtained from a PRA), whereas a risk-informed approach combines risk information with other factors to arrive at a decision.

How are we currently applying these concepts to existing NRC processes?

The NRC uses risk in a variety of applications. For example, when assessing licensee performance using the Reactor Oversight Process (ROP), the NRC uses risk to determine the safety significance of a licensee non-compliance so that the NRC can gauge the necessary level of regulatory oversight. The NRC also uses risk information to inform the development of regulations that focus on realistic events that have a significant impact on safety. For example, using PRAs, the NRC found that station blackouts significantly contributed to the likelihood of damage to the reactor core and backfitted licensees to meet a new station blackout regulation.

In licensing, the framework for risk-informing licensing actions is currently described in Regulatory Guide 1.174, which discusses how to integrate the consideration of regulatory compliance, defense-in-depth, safety margins, PRA, and performance monitoring when reviewing licensing actions. Licensee-provided PRA information has to be of a certain quality if it wishes to formally risk-inform its applications. Examples of such applications include the NFPA-805 fire protection programs and amendments that would allow licensees to risk-inform Technical Specification frequencies and completion times. However, the NRC is considering an additional framework that describes how the staff can use its own, rather than licensee-provided, risk insights, such as those from its own PRA models.

What is the integrated review team (IRT) approach?

NRC management has recently tasked the staff with increasing the use of RIDM, including expanding the use of review teams for licensing actions that team up risk analysts with technical

reviewers to collaboratively arrive at an integrated decision using a graded approach to risk. In response, the staff developed a process called the IRT process that has risk analysts and technical reviewers meeting more frequently during reviews of licensing actions. During the meetings, the reviewers would arrive at an understanding of their scopes of review and how their evaluations may be integrated and would work together to develop one consolidated safety evaluation instead of separate isolated inputs. The process also includes tools that assist the review teams with making a decision about whether to include risk insights, what types of risk insights to use, and how to document the use of those risk insights.

Management Direction:

NRR Office Director (OD) issued a memo in June 2017 in response to a Region 4 differing professional opinion (DPO) on the Palo Verde emergency amendment, where the staff used risk insights even though the application was not a RG 1.174 request (i.e.,

formally risk informed where the licensees PRA info has to meet certain standards, get peer reviewed, etc.). The memo listed tasks for the staff to accomplish regarding its use of risk in licensing actions.

Simultaneous Commission direction for staff to increase its use of risk in regulatory applications.

Resulting NRR RIDM Action Plan first issued in August 2017 (updated ~quarterly).

Phase 1 resulted in a findings and recommendations report issued June 2018 documenting the staffs recommendations related to the following NRR OD tasking:

o Task 1 -team up risk analysts and engineers so that their decisions and work products are better integrated.

o Task 2 - expand the definition of risk beyond CDF and LERF o Task 3 - develop a graded approach for considering risk in licensing reviews RIDM Phase 1 Summary:

Tasks 1 and 3 were essentially combined because the Task 1 project management framework would depend on the tools that Task 3 developed and vice versa. The combined effort describes: a framework where review team members meet more frequently (meeting goals are described in the process) to enable integrated decision making; the creation of consolidated review products (i.e., RAIs and SEs); and various checklists that instruct the team as to whether to use risk insights, what kind of risk insights can be offered, and how to document the staffs consideration of various levels and types of risk insights.

The combined framework has been called the integrated review team (IRT) process.

The IRT process has two aspects - the project management framework for the consolidated products, and the graded approach for using risk insights.

The approach was developed for a limited subset of licensing actions; hence, the Phase 1 recommendation to explore whether other processes can use either aspect of the IRT process.

RIDM Phase 2:

The draft IRT process is going through a trial period to get feedback about its feasibility.

Team E is exploring whether other NRR processes can use aspects of the IRT process.

Other teams are developing training, metrics, formal guidance, desktop guides, and other tools to facilitate the increased use of risk insights in licensing decision-making.

For more information, a training presentation (for DORL PMs), the Phase 1 Report, and the review team checklists/tools can be found at:

http://fusion.nrc.gov/nrr/team/dorl/RIDM/Integrated%20Review%20Team%20Toolkit/Forms/AllIt ems.aspx Leveraging Information from PRA Models to Risk-Inform Individual Responsibilities Information Derived from PRA Models Leverage Opportunities Relative importance of different initiating events (e.g.,

Loss of Offsite Power, Anticipated Transient Without Scram)

Determine impact of initiating event of concern Identify whether SSC of concern participates in mitigation of important initiating events (calibrate scope of review)

Risk importance of SSCs and operator actions Identify whether SSC of concern is an important contributor (calibrate scope of review)

Determine sensitivity to assumption of guaranteed failure of SSC of concern (calibrate scope of review)

Identify which operator actions are important; Determine if review is impacted by them or alters them (calibrate scope and focus of review)

Sequential failure of system and components leading to core damage (CD)

Identify the sequential failures that include the system and/or component of concern; Determine relative contribution of those sequential failures Determine if proposed compensatory actions make sense given important sequential failures Identify how many failures before and after the SSC of concern are necessary to lead to CD (another way of looking at defense-in-depth)

Impact on quantified risk from What If scenarios Determine impact of proposed change to SSC of concern by increasing or decreasing their failure probability (scope of review)

Determine impact of proposed change to SSC of concern on operator actions (scope of review)

Determine impact of proposed compensatory actions by changing their failure probability Determine impact of increased structural failure probability due to aging on seismic risk System dependencies Determine which system relies on other systems to work Identify support systems that are crucial for frontline systems to operate (e.g., service water for EDG operation)

Identify additional systems for consideration/review (e.g., during license renewal) (scope of review)

Consideration of testing and maintenance (T&M) on risk Identify any routine T&M that shows up as an important contributor Determine impact of changing T&M frequency (by changing the failure probability)

Contribution to quantified risk (CDF and LERF) from different hazards (e.g., accidents and transients, internal flooding, earthquakes, fires)

Determine relative impact of each hazard Identify additional considerations for external hazards (e.g.,

earthquakes, fires) that impact SSCs of concern (scope of review)