ML19002A184
| ML19002A184 | |
| Person / Time | |
|---|---|
| Issue date: | 02/14/2019 |
| From: | Office of Nuclear Reactor Regulation |
| To: | |
| Orenak M, | |
| Shared Package | |
| ML19002A177 | List: |
| References | |
| Download: ML19002A184 (82) | |
Text
Regional RIDM Online Survey Results Region I Risk-Informed Decision-Making Online Survey Question 1: What is your job function at the agency?
Other (please specify)
Health Physicist, DNMS Administrative Professional Senior Health Physicist Ops Examiner Senior Operations Engineer IT Specialist Enforcement Team Member Administrative Assistant Examiner AET Adminstrative Assistant health physicist materials Material Inspector/License reviewer Senior Health Physicist materials inspector and license reviewer. thanks for not listing us. or thnaks for sending us to a survey not applicablke.
Sr. HP Reactor Health Physics Inspector Materials Inspector NA Senior Security Inspector Admin Materials Inspector 0.00%
5.00%
10.00%
15.00%
20.00%
25.00%
30.00%
35.00%
What is your job function at the agency?
DRM or Secretary (OTHER) since the rest of the positions were listed specifically.
Question 2: Which of the following have you had review experience with (check all that apply):
50.65 50.69 NOED MD 8.3 and IMC 0309 IMC 0609 TSTF-425 Initiative 5b TSTF-505 Initiative 4b 0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
70.00%
80.00%
90.00%
100.00%
Which of the following have you had review experience with (check all that apply):
Question 3: The use of risk or risk insights is appropriate for inspection or inspection planning.
Strongly Disagree Neither Agree nor Disagree Strongly Agree 1
2 3
4 5
6 7
8 9
0 2
4 6
8 10 12 14 16 18 20 The use of risk or risk insights is appropriate for inspection or inspection planning.
Question 4: The use of risk or risk insights results in an unwarranted increase or decrease in the significance of inspection findings.
Please explain I believe safety/security are best served when we apply a risk-informed and performance-based approach to our analysis.
Risk insights tend to result in the correct finding significance determination.
Risk insights results in appropriate increase and decrease in significance of inspection findings.
Incorporating Risk can increase or decrease the significance of findings used in relation to SAFETY.
Characterizing findings based on risk seems appropriate, however, the scale seems tipped toward "low significance."
We inspect because nukes present a risk to health and safety. Risk is what determines if a finding means anything or is a waste of time.
It can go either way.
Risk insights can be very useful in determining significance of findings. However, they are not the only tool that should be used.
They are merely one tool in a larger toolbox.
Sometimes issues that seem like they should be more-than-minor or of a higher significance level, because of the egregiousness of the licensee's handling of the issue, are either not documented or are categorized as lower than they ought to be. There is little room for regulatory discretion in the enforcement process, because of the strict reliance on risk measures.
I don't feel the use of risk or risk insights results in an "unwarranted" increase or decrease at all I believe we should do away with Green findings and perhaps even White findings as they are both considered low risk.
Depends on its application. Some inspections findings need to have deterministic significance as not to not recognize low risk, but important issues.
Typically, if something has a low risk, then the resulting finding will have a low risk. This is not an "always" situation, but merely a typical outcome.
Strongly Agree Agree Neither Agree nor Disagree Disagree Strongly Disagree 0.00%
5.00%
10.00%
15.00%
20.00%
25.00%
30.00%
35.00%
40.00%
45.00%
The use of risk or risk insights results in an unwarranted increase or decrease in the significance of inspection findings.
I think it depends on reaching a shared understanding of what is meant by risk - especially when applying the term to inspections of materials licensees and decommissioning reactor sites. Some licensees use technologies with more inherent defense-in-depth, and the differences should be considered and understood when applying risk insights. Similarly, some reactor inspection areas involve risks that may be different than what the managers or inspectors (as we use more rotations) may be familiar with.
If decisions are made that are based on consideration of the risk from a shared view of knowledgeable staff, I think it can be beneficial. If not, it may result in improper outcomes.
Potentially. One could argue that the significance of inspection findings should be commensurate with the severity of the performance deficiency itself, rather than based on the risk/consequences of the event/issue. Some PDs may be relatively benign oversights that, due to the circumstances, can result in escalated enforcement. Meanwhile some blatant/gross PDs that shouldn't happen, by circumstance, may be dismissed as Green (or minor). Given thought and/or different circumstances, the blatant/gross PD may really be the behavior that warrants the more significant follow-up and correction.
A performance deficiency is a performance deficiency regardless of the process used to pick the sample. Sometimes it is easier to make a performance deficiency more than minor if it is related to a system that is more important to plant safety. I don't consider this to be "unwarranted." It's an efficient use of resources.
The use of risk does not result in unwarranted increase/decrease of significance of inspection findings because we want to understand what is really happening and what cold/would happen If risk informing our requirements makes some rules unnecessary or less important, it follows that less violations of significance will be issued. Enforcing rules that don't make sense or are unimportant doesn't make sense. We are not or should not be in the business of maintaining requirements that are not needed because our inspectors need something to do. Its not a proper growth mindset for an agency to have.
When used in the proper context the risk insight provides a delta in consequence. It helps remove the human element from disposition of a finding.
Risk insights, when used properly, shouldn't increase or decrease significance - it should pinpoint the significance properly.
Risk and risk insights appropriately guide issue screening of inspection findings. The value of the finding should be tied to the risk significance in my opinion.
I think use of risk results in appropriate significance of individual findings. I think we are missing out on being able to apply the appropriate level of additional oversight by allowing unlimited green findings to still be assessed as licensee response column.
Especially since making it more difficult to end up with a cross cutting theme. I suggest assessing the aggregate risk of all findings annually and setting a threshold for increasing oversight.
Using risk or risk insights helps appropriately determine the significance of the issue/event. Therefore, if properly designed and based, risk insights should identify if how much of a safety/security risk it was/is. The trick is designing a risk insight process that is straightforward to use and has predictable outcomes for certain issues of identified significance.
as long as the assessment is done in a technically rigorous manner and has supportable assumptions.
Question 5: "Risk" or "risk insights" are a form of engineering judgment.
Please explain Risk and risk insights are the result of the training, experience, and tools we provide to staff.
Risk insights and risk tend to reduce "judgement", but there is judgement involved with risk insights.
Through EXPERIENCE, the use in JUDGEMENT on all inspections is commonplace. Straightforward Black & White determinations are not as common.
Quantifying "risk" is subjective and therefore is an outcome of exercising engineering judgment.
Risk is a statistical calculation, a probability estimate based on assumptions. Judgement is necessary to evaluate the assumptions.
But not a foundation for operability calls...
Agree. Risk insights can shed a lot of light on the postulated outcome of an event for a particular plant equipment configuration.
In particular, relative comparisons of calculated CDF. (Qualitative vs. Quantitative).
Sometimes it seems to be the case, based on the ways that PRA parameters are "tweaked" during analysis.
I believe they are based more on assessments already linked to probability.
In some ways, yes, because who's to say what is an acceptable level or risk.
I agree, although I would note that we inspect areas that may not traditionally be thought of as "engineering," such as security, EP, health physics. But, these are all technical fields, so I see them as falling under the same umbrella.
When I pick samples, I consider the licensee's risk "pie chart," which factors in PRA. Though there are assumptions associated with a PRA, I wouldn't consider this to be engineering judgment.
We have risk models that help guide us to understanding the risk.
I would argue that if the facts support the PRA assessment less judgment is needed.
Strongly Agree Agree Neither Agree nor Disagree Disagree Strongly Disagree 0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
"Risk" or "risk insights" are a form of engineering judgment.
I think experience gives you judgment for risk. Insight however is dependent on PRA models or an otherwise calculated number.
This would then be a tool for decision making. Poor question can't properly answer.
Human performance/operator actions that are hard to quantify in risk may lead to using engineering judgement true for some materials applications, less so for others Risk can be quantified using PRA methods but often has a qualitative aspect as well. So therefore some professional and engineering judgments are involved in applying the regulations for each specific case under consideration.
They are a highly informed form of judgment, but still judgment. I believe them more than just an engineer's experience, but less than a natural law. I think this is justified as I have heard second hand that we as an agency have compared the number events actually seen at plants to the PRA predictions and have found a delta.
"Engineering judgement" would appear to be a very subjective term/process. I'm not an engineer, but I've heard it bandied about with varying levels of agreement on the same issue by engineers. Again, risk insights should be as predictable as possible for known issues of an assigned significance.
Question 6: The ROP and Enforcement guidance have resulted in unintended and unreliable outcomes in dispositioning issues of concern.
Other (please specify)
One of the goals of the ROP is to provide scrutable, repeatable outcomes. Admittedly, there have been some consistency issues with the ROP, but we've generally been able to work through them and clarify requirements and/or provide enhanced tools and training to staff to ensure better consistency.
Unintended (sometimes) but Unreliable (no).
The only area I would say the ROP/RISK does not value enough is dose reduction systems for offsite people/environment.
Standby gas should be important because it directly impacts the amount of radiation that is released. Containment isolation valves are also typically shunted to ground as Green when they have a valuable role in the accident sequence.
ROP guidance seems to have resulted in more reliable dispositioning of issues over the years.
The NRC seems to make an effort to be consistent.
If the standards are clear and ROP documents representative...we are good. This has not necessarily been the case with cyber. The standard has been handed over to NEI guidance documents.
Strongly Agree Agree Neither Agree nor Disagree Disagree Strongly Disagree 0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
The ROP and Enforcement guidance have resulted in unintended and unreliable outcomes in dispositioning issues of concern.
This can occur, and I'm sure licensees will say that it is occurring, but I actually see the amount of effort and painful consideration that goes into evaluating findings and violations, and I think we generally come out in the right place. People with less exposure to the processes probably have a different view, and those views should be evaluated to see if they hold any merit. We can always do better.
Some people are not following the ROP guidance and are using their own personal interpretation of the guidance.
I think most of the arguments center around minor/more-than-minor. However, that really doesn't matter IMO, as all that determines is whether a plant is Column 1 with three green findings or 25 Green findings (i.e., who cares? they are still column 1). Even though they do not change Columns with Green findings, I still think they are important, as it shows that we are accomplishing something as an Agency, it holds the licensee to task, and provides a means to trend issues. Additionally, it provides other regions/inspectors with items to consider reviewing during the inspection planning process. Finally, who knows how many Green/minor findings have prevented something more significant? This is impossible to measure, but is worth considering.
Core damage risk associated with PRA has generally provided reliable results. The areas of EP, RP, and security continue to be relatively unreliable and can/do result in GTG issues that are not commensurate with the equivalent in the BI and MS cornerstones.
While there is room for improvement the ROP provides a good foundation for consistency.
I haven't seen that probably not applicable to materials insepctions I have limited experience in this area for the occupational and pubic radiation safety cornerstones. I have not heard, seen, or experienced this to date.
I've only ever known the ROP, but I can't imagine outcomes were more reliable or consistent previously.
Partially disagree. Sometimes the ROP or more specifically the SDP do not produce reliable outcomes, but that is impossible to totally eliminate in a process that cannot encompass all possible scenarios. In general, most of the time, the correct outcome is reached although it can be painful, time consuming, and laborious. Most of the time enforcement has been flexible enough to accurately capture the appropriate disposition, however, I would give it lower marks than the ROP because frankly so many of the examples are outdated or not applicable to actual significance.
generally we are ok, however, probably a few issues along the way that may have been mis-characterized due to assumptions - most likely for issues at the margins between one significance and another.
Question 7: Inspection resources are focused on the most risk significant issues.
Strongly Disagree Neither Agree nor Disagree Strongly Agree 1
2 3
4 5
6 7
8 9
0 5
10 15 20 25 30 Inspection resources are focused on the most risk significant issues.
Question 8: I have concerns with expanding the use of "risk" or "risk insights" by the agency.
Strongly Disagree Neither Agree nor Disagree Strongly Agree Question 9: There is value in identifying minor and Green inspection findings.
Strongly Disagree Neither Agree nor Disagree Strongly Agree Question 10: Do you think the current guidance or training is sufficient for you to use risk or risk insights in your decision making?
1 2
3 4
5 6
7 8
9 0
2 4
6 8
10 12 14 I have concerns with expanding the use of "risk" or "risk insights" by the agency.
1 2
3 4
5 6
7 8
9 0
5 10 15 20 25 30 There is value in identifying minor and Green inspection findings.
Strongly Disagree Neither Agree nor Disagree Strongly Agree Question 11: Please explain your answer above.
Responses While our training on practical application of risk insights has greatly improved through the years, more needs to be done...particularly with risk informing our day-to-day decision making and for developing a shared vision of how risk insights can be used to best position the agency for the future.
Current guidance and training is good, but can always be improved upon.
I feel I have the necessary tools and understanding of risk and risk insights to use in my daily work.
Inspector qualification, continued on the job training, and various inspection tools or resources are sufficient to use risk and risk insights.
Current guidance in IPs is high level as to how to develop risk insights in SSC selection for inspection. For my vantage point I see a gap in how inspectors develop risk insights to select SSCS for review. I do not see inspectors availing themselves to risk tools, such as historical IPE/IPEEEs, old phase 2 notebooks, SPAR model or plant specific Sapphire models to gather risk insights in a systematic fashion. Only for DBAIs with SRA support do I observe a systematic and overt risk informed SSC selection process.
With the new transformation initiatives, and turnover of people over time - more training on RIDM is warranted. Especially at HQs. This is part of changing our mindset and culture at the Agency.
As an Admin Assistant, I have little to no "decision making" to do in any process.
There is plenty of available risk information to facilitate focusing on what is "most important" which, in turn, increases our likelihood of being successful at protecting the health and safety of the public.
SDPs work I think many of us have a basic understanding of using risk or risk insights but a better understanding in simple terms may be helpful.
Heavy reliance on SRAs, so current training is sufficient.
Yes. There is sufficient guidance for using risk insights for decision making. I do not feel more risk guidance is necessary.
1 2
3 4
5 6
7 8
9 0
2 4
6 8
10 12 14 Do you think the current guidance or training is sufficient for you to use risk or risk insights in your decision making?
From a inspection sample decision making standpoint, we are trained about the plants we are inspecting and can make risk insightful decisions based on that knowledge.
I'd like to comment on question 9 but no space was provided. There is VALUE in IDENTIFYING and presenting such issues to the licensee and have them correct them. HOWEVER, there is little to no value in expending the effort to document them in an inspection report. We should modify the 'NCV' documentation guidance to allow the inspector to NOT document a finding if the licensee has entered it into their corrective action program and are taking reasonable CA's iaw the safety significance.
Our guidance and training are very good, but we can do even better.
I am not sure what training this is referring to. The P-111 course was good.
We could do a better job with this. There should also be continuing training after full qualification.
As a prior SRI and also training as an SRA, that has provided me sufficient information for decision making. However, no withstanding that I would find it challenging to be informed enough to support risk-informed decision making. Managers are risk adverse unless it is clear they are making the best decision. That is not always the case to support.
In general, the current guidance is sufficient; however, there are changes to the regulation (such as 50.69 and risk-informed completion) which need new/enhanced guidance and additional training!
I think the guidance leaves a lot of room for staff and management subjectivity. I don't have a problem with that, because I don't know how prescriptive we can or want to get in all areas.
I think the guidance is available. It really depends on the team members selecting the samples and cross-referencing those selections with risk data.
Inspectors need more than initial SAPHIRE training or we are forced to rely on SRA expertise We have very little formal training in risk and have no guidance on using risk insights for decision making.
Most SDP's that I use are deterministic and have no risk insights built into them.
I think I have enough knowledge, plus I know where to go to get more, if needed. We have SRAs.
I have had some PRA training. More may be beneficial but inspectors don't need to know all of the details that are used to develop a PRA.
Yes. We received training on safety significance and risk significance and the use of PRA. It may be helpful to have an annual refresher, though.
There are varying interpretations associated with minor and MTM screening. Although by definition these are not GTG and therefore not safety significant, improved guidance and training could improve consistency.
The training helps, however inspectors rely on the licensee to provide ongoing risk assessment. There is no objected independent verification of the licensees methods. There is not sufficient tools available to the I've received some training. However, the use of risk within the Agency is being made more significant. As such, I do not know that my training is commensurate with the path the Agency is moving toward.
We don't train much at all on risk. Tell me what class Im required to take after I got qualified - ZERO All Inspectors are required to at least take P-105 PRA basics. Many take additional courses and are more thoroughly trained in this area. All inspectors are instructed and understand that the NRC is a risk informed agency. Many of the inspectors have years of experience in the fields they are inspecting. I believe that the inspectors are well equipped to use "risk" and/ or "risk insights" in their decision making.
Consider more guidance on connecting risk to safety significance not applicable I think more training could be good. Perhaps training on how to risk inform inspection activities that may not be related to PRAs.
Or more guidance on having more than one component out of service. RAW values are great for risk ranking, but technically are
only valid for one component out at a time. With multiple activities going on, or emergent issues having a better understanding on risk ranking a couple components deep would be good.
Risk and risk insights are routinely utilized in the inspection process. The importance is not to equate risk and risk insights to PRA and PRA numbers. A PRA based approach would not properly identify all the potentially risk significant issues at a site.
Risk insights need to be better incorporated into the continuing training and initial qualification process.
For the most part, our SDPs and the related training (for the Security Cornerstone) are sufficient for the use of risk or risk insights in decision making. However, there are some task forces looking at implementing risk informed regulatory standards (such as compensatory measures) that are worrying. I fear the current version will be so subjective that it will be difficult to have a substantive discussion with licensees as to the accuracy of their risk assessment. I see this also with the potential to give additional credit for off site resources. The devil will be in the details.
If you are asking about the Risk-Informed Training, I have been told that it is a little helpful, but it still needs some tweaks. That was almost a year ago and the tweaking might have already occurred. I find it difficult to answer your questions.
For me, I use risk insights for all my inspection activities.
The risk informed notebooks for each site are very valuable in determining inspection samples, however, I am not sure they are utilized as much as they should be.
I don't know enough to comment good foundation but can always do more - specifically in using the models to assess conditions in lieu of relying exclusively on the sras Question 12: Do you think there is a gap between management expectations and staff use of risk or risk insights?
Strongly Disagree Neither Agree nor Disagree Strongly Agree Question 13: Please explain your answer above Responses Risk insight isn't deterministic, based on my time at the site I will employ my insight differently then someone else who has been onsite longer/shorter and has different past experiences.
I think staff believe in the use of risk insights and often are more adept and more in tune with applying such insights to their work than the senior management team.
1 2
3 4
5 6
7 8
9 0
1 2
3 4
5 6
7 8
9 10 Do you think there is a gap between management expectations and staff use of risk or risk insights?
I haven't seen a gap in my experience. We are encouraged to use risk insights whenever possible.
I believe management is inconsistent with the use of risk insights. When it comes to findings or reactive inspections, they overweight risk. When it comes to day to day issues and events, they get buried in the details and don't step back and put a risk perspective on the issue.
In my view management expectations regarding available tools and best practices as to how to develop risk insights are not conveyed in a systematic and consistent manner.
Yes, management tends to support RIDM, where some staff definitely lag in accepting the new approach.
There will always be some sort of GAP because unless you are doing the inspection yourself, you could always expect someone else to somewhat les of a job due to experience and other "regulatory" insights.
Don't know.
We inspect per procedure. Results are assessed via the risk informed SDPs.
don't know Perhaps we need to continue training of what risk informed vs risk based regulation is and continue training in inspection prep tools using risk Don't know management expectations No gap. Management clearly encourages the use of risk and risk insights in decision making.
N/A Management will routinely ask questions that have little to no risk value and expect full detailed replies. However, issues that may result in GTG are not given significant attention or support from the Region.
Managers in Region I tend to get very spun up over issues that are of very low significance.
I think management expectations are not consistent between different levels of management and differentt regions.
I get asked all the time, what is the risk number or ball park number. It's not always that easy. Many factors come into play and takes time.
Not sure.
I honestly don't fully understand what management expects. I hear a lot of managers saying they want to use more risk insights, but I don't know what that means in a practical sense. I would very much like to know, specifically, what is desired from the staff in this area. And it isn't satisfying to me to be asked to figure out what I think it should mean.
Not really. I think management expects teams to inspect using risk insights. I think that the agency should clearly define risk and risk insights. There is a strong chance that most people think risk is just PRA. I think we should be understanding the consequences of failures and blend that into the equation. A vulnerability can have a low risk with a high consequence and would be screened out for review because the risk is misunderstood to be the PRA risk. There are opportunities to review "low" risk samples with high consequence which be a more balanced approach to selecting samples.
Do not know. It has not been a topic of discussion.
Most inspectors want violations so they can say they did something, even if they are minor in nature. Management is more focused on risk significance.
I'm not sure management at the director level and above (mainly outside the regions) understand that sometimes there are worthwhile samples to be had on items that may not be the most risk significant.
We are still a compliance agency and spend a lot of time on minor issues that we won't let go of.
The guidance provided to me is to use risk insights when choosing inspection samples.
No I believe management expects staff to provide risk insight or deltas from findings or other issues based upon policies and procedures. I'll note also it appears to me this is done after the fact. The first opportunity for regional staff to gain insight is after there is a finding or after the licensee has issued a LER.
The staff has expectations based on their training and job description that does contains aspects outside of risk. However, management seems to focus much more on only risk items.
Over the 4.5 years of my experience at the NRC, the management has been consistent with their expectations of inspectors using risk and/ or risk insights. Management has consistently taken the time to discuss findings and encouraged peer to peer discussion for context and perspective. Additionally, management took the time during inspection accompaniments to provide guidance on applying risk and risk insights.
Not enough quantitative guidance in place to be able to determine whether a gap exists I think we understand our use of risk in performing and evaluating inspecitons in MAterials I have never been told I have inappropriately used or didn't use risk insights, so I assume I am meeting expectations. If there is a gap it is invisible to me.
Risk/risk insights are insights/tools to focus inspection efforts. Not the end all. It is a tool. One input.
As a region based branch chief, there is a clear expectation to utilize risk and risk insights into ongoing inspection activities.
Risk and risk insights in many cases become talking points vice decision making tools.
Not for the most part and not in my branch. However, there are pockets of inspectors who do not seem to grasp the concept of taking risk into account when determining significance (NSIR/SPEB).
All I know is that Region I managers/supervisors are quite open about expectations, but I do not know if there is a gap at all when it comes to the staff use of "risk" or "risk insights."
My branch chief is aligned with risk insights.
Not sure if we have set management expectations on the use of risk insights.
management needs to understand what is risk significant equipment and not just bean counting green findings.
The use of risk and risk insights is used in varying degrees by inspectors and managers on a daily basis. Some inspectors are more risk based, and some are more defense in depth or deterministic based.
we haven't received enough explanation about the process think good communications in this area in a regional office Question 14: Can you briefly describe what you would consider barriers to incorporating the use of "risk" or "risk insights" in your daily work?
Responses The largest barrier IMO tends to be with pockets of senior leadership who tend to be risk averse and unwilling to adopt approaches to staffing, work hours, work locations, etc. (even those currently approved via policy, CBA, etc.). We will not be a truly nimble organization until we develop a shared vision that we all carry forward.
No barriers in my view.
I don't see any
None.
- 1. It is hard to disregard minor violations and move on to look for bigger problems.
The challenge between walking away from minor issues (violations) to save time for focusing on finding bigger problems. My nature is to ensure all deficiencies are identified to the licensee for entry into their CAP and resolution. Fixing small problems or process gaps can prevent big problems from occurring. However, this takes time.
The majority of our time is better spent on looking for more risk significant problems.
In my view there is uneven use of risk insights in agency processes. I have observed instances where hundreds of inspector hours were devoted to a mere TS compliance issue of little to no safety significance because risk insights were not adequately considered. On the other end of the spectrum I have observed acceptance of conditions under operability reviews that put all our eggs in one basket involving very low LOCA probabilities at the expense of mitigating equipment. I have questions regarding how well our passive failure rate assumptions will hold up as plants continue into their periods of extended operation. I do not see a clear feedback loop to PRA models in this regard.
Need more written guidance to staff and examples on how to apply RIDM.
For my daily functions, no I cannot since I am not an inspector.
None.
None Just not understanding enough about PRA models training....understanding for inspectors Little opportunity to do so.
Only one barrier comes to mind. A number of reactivity related issues have been discussed in recent years. Not much guidance or none at all in 0609 on things like fuel failures during full power operation, exceeding fuel assembly burnup limits, or mispositioning of fuel assemblies during outages. Typically refer to 0609 Appendix M (qualitative assessment) to disposition such issues. Perhaps more guidance could be added to 0609 Appendix A and 0609 Appendix G on reactivity related issues such as these.
The baseline inspection program generally calls for more samples than there are "risk significant" activities to observe and inspect. With surveillance frequency extensions and licensees attempting to perform less preventative maintenance, sometimes we are forced to inspect less risk significant activities to meet baseline.
The pre-established risk worksheets were an excellent tool for inspectors to evaluate risk significance quickly. The SPAR model is not user friendly and time intensive. The residents should be given better tools to screen issues more efficiently and effectively.
Reliance exclusively on SRAs to perform analyses. Residents could be better trained on use of SAPPHIRE and other tools.
Possibly some "old school" mindsets.
Mangers in Region I getting very spun up over issues that are of very low significance.
Clear set of tools and the flexibility to use them and not be buried in analysis paralysis.
Need to understand the safety classification of an SSC. Need to have an appropriate risk model with easy to understand results for SSC. Need to have a basis to select samples based on the new 50.69 regulation.
I guess a lack of framework for what use of risk or risk insights is meant to be.
Familiarity with risk models and the accuracy/applicability of those models to the sites.
The barrier is the mixed interpretation of the meaning of risk and risk insights. Operational Experience could be used for risk insights. I think the terms are subjective based on an inspector's experience and understanding.
See answer 11 above.
There is no guidance to use risk, when you are in a deterministic procedure.
My supervisor is generally supportive of the samples that I pick during inspections. The only recent example I could provide is having to do an external flooding sample at a site that was very unlikely to have an external flooding concern. We had to do one because the procedure said so and they removed the "out" that used to exist in past revisions.
Institutional and cultural issues.
For the most part we have to choose inspection based on the daily work schedules and issues that come up. So, we use risk as best we can, but in the end we are forced to choose ROP samples to complete a nominal baseline inspection program. So, even though there are times when we do not have a risk-worthy sample available we have to perform the inspection anyway to "complete the baseline" (as defined as performing samples to nominal, not minimum).
Lack of tools. I do not understand why NRC PRA software is not available to all staff, and a limited training and access to simplified modeling software cannot be utilized by PEs and residents.
up-to-date risk training Our job description is not only risk. Compliance is a major aspect, and should continue to be so.
No barriers Objectifying what could be a subjective area and ensuring that all inspectors have the same understanding/ perspective (uniformity) on applying risk and risk insights into an inspection.
Professional judgement and experience used in assessing risk are inhibited when procedures must be strictly adhered to probably no barriers Proficiency in running PRAs. That is partially my own fault though for not doing it often enough. As stated above, knowledge on how risk significance of equipment changes with more than one item OOS is limited. Additionally, risk insights with regards to human performance is a relative blind spot (at least for me). I feel like good or bad human performance in ops, maintenance, etc. can really change the risk of the plant, but other than a gut feel I don't have much to go on in that regards. Perhaps a tool that risk informs human performance behaviors. (e.g. proper use of STAR could feed into an initiating event frequency and or basic event probability, or place keeping could feed into the same, or some kind of guidance on assigning a risk insight to evolutions that are operator action intensive versus not operator action intensive, maybe pump A is more risk significant than pump B, but pump B Is more complicated to operate, which one do I choose?)
No significant barriers exist to incorporating risk or risk insights.
Differences in the licensee's use and assessment of risk and how we use and assess risk. Furthermore, we do not inspect the licensee models or have strong guidance on the interface of quantitative and qualitative risk.
First off, one could say that the minor/more than minor threshold is the first screen for significance and this has been fuzzy since the beginning. Security still has relatively few examples and the four questions are VERY subjective. Beyond that, when you get to the SDP I believe there are only two barriers to appropriately incorporating the use of risk: 1) The SDP doesn't appropriately address the particular issue or the mitigators associated with a similar issue because it wasn't anticipated; and 2) The mindset of some inspectors can be a barrier (specifically, some inspectors are stuck on a simple black and white interpretation of the SDP instead of looking beyond the black and white to what the intent was and being open minded).
In our work area, we always consider the ramifications of our words and actions. Perhaps I have always incorporated "risk" or "risk insights" in my daily performance.
Some samples that are required, in my opinion have little to do with risk insight. Those samples should be removed from the inspection samples.
When selecting samples at a site, the risk of daily activities may not be there, but samples still need to be performed. If we wait for only the high risk components, we will not complete the ROP baseline.
none. need to use more risk insights.
understanding of what is expected some limitations in modeling and engineering assumptions Question 15: Describe the difference between deterministic and risk-informed.
Responses Deterministic is running the risk model and only inspecting when CDF or LERF changes to yellow, or if a SSC is declared unavailable it will change to yellow or greater risk. Risk informed means using your knowledge of the plant, SSCs, personnel, and integrated risk to inspect. If a SSC is safety significant and past maintenance doesn't show success then an inspection may be warranted, that is risk informed.
"Deterministic" typically describes establishing a prescriptive set of requirements, based on test results and expert judgment, the outcome of which results in the establishment of layers of protection such as safety margins and other defense-in-depth approaches. Being "risk-informed" means applying risk (probability + consequence) insights to the decision making process to reasonably assure a desired outcome.
Deterministic is more black and white (If X, then Y). Risk-informed uses a numerical calculation, but allows use of judgment as well.
Deterministic is rigid and predetermined. Risk-informed allows for using the specifics of the situation to assess the finding or issue.
Deterministic is just based on whether a requirement is met or not. Risk informed is how significant an issue is and puts perspective on the issue.
Deterministic is performing an activity based on a list of specific requirements. This is somewhat like using a check list to ensure all critiria are met.
Risk-informed does not focus on a check list to ensure all criteria are met. It focuses the majority of resource on evaluating higher risk issues or criteria. Some low risk issues may be dropped from the inspection scope.
Deterministic involves classical chapter 15 accident analyses which assume initiating event boundaries such as LOPP/LOCA/bounding single failure or deterministic pipe break locations and assess results to explicit criteria in terms of plant fuel integrity, barriers and release limits. Deterministic approach utilized implicit risk insights. A risk informed approach gathers insights from PRA models that are built on data regarding active SSC failure rates and human error probabilities to develop insights on those SSCs and operators actions most important to safety.
deterministic = tied to licensing basis assumptions and defense in depth.
risk informed = use of PRA insights Deterministic is 2+3 = 5 and 5 is no good. Risk-Informed is 2+3 = 5 which may not be good, but there is little or minimal safety risk involved so 5 is not so bad...
I think: Deterministic is more "direct"... "if this, than that"... Risk-Informed provides more room to exercise judgment (supported by experience, etc.).
Deterministic: you have good statistics on component failure probabilities.
Risk - Informed: No stats, requires judgment, such as consequences of improper or inadequate training or FFD.
Deterministic has more to do with just a natural tendency for something to break or malfunction where risk-informed takes into account the probabilities and severity of consequences of that break or malfunction.
deterministic is judgment based, engineering opinion based, and not really based on any pure risk effects of an issue.....where risk informed gives some background in how a component affects core damage
Deterministic is defined, "will-happen" or "has-happened" events/statements. "Risk-informed" is based on probability and elevated chance for something to happen.
Risk informed involves calculating CDF values based on plant specific PRA studies. Deterministic does not involve statistical calculations.
Defense-in-depth vs. PRA Deterministic is a black and white requirement that either is met or not met. A licensee or NRC must determine whether the condition is fully met or fully not met. Risk informed allows us to see the spectrum of risk related to the issue and extract the risk significant portions of the issue/condition and focus on that directly. Operability is deterministic. Maintenance rule is risk informed.
A "deterministic" approach is based solely on what has gone wrong and what the known consequences are. A "risk-informed" approach takes a more integrated look at the issue in order to more precisely quantify the potential safety impacts and better gauge the seriousness of the issue.
Yes/No vs Maybe. For deterministic criteria, a component either meets it or it doesn't. If risk-informed several factors are considered to determine the likely impact.
Deterministic applies to cause and effect with specific aspects. Risk informed considers a range of possible outcomes with varying probabilities and consequences.
Deterministic tends to be more of a yes/no type of situation whereas risk-informed can be aided by quantitative data.
Deterministic: a pre-set of conditions that are met, certain actions are taken, or a determination is made. Trigger points.
Risk-informed: a condition that is viewed in the lens of its probability occurring...and decision or action based on that.
Is this a survey or a quiz? Deterministic is binary (e.g., safety related or not) while risk-informed is spectrum based on an assortment of inputs and has an associated uncertainty factor.
I think "deterministic" is more plug and chug and get an answer, while risk-informed is more subjective.
Deterministic relates to something that is objective. Pass/Fail. A deterministic method uses a calculated parameter and compares it to a set limit. Risk-information is used to inform the user about what aspects of a specific problem are most important to consider. It provides guidance in interpreting results/issues.
Deterministic should be very repeatable and not change over time. The outcome of an event should be very predictable. Risk informed is a fuzzy logic assessment. It utilizes many different inputs to assess a situation.
Deterministic means if 'A' occurs, then according to a table, your enforcement is 'B'.
Risk informed means if 'A' occurs determine the increase in risk quantitatively, if possible then perform enforcement action according to the increase in risk.
Deterministic is when the sdp says this, then it is a white finding. Risk informed would be where there is a calculation that tries to bound the risk of the issue.
Deterministic requirements tend to be very prescriptive (e.g., you must accomplish something via compliance with a specific set of requirements). Risk-informed is more performance-based, i.e., there is a goal, but there is flexibility in how that goal is achieved. There are some good examples in fire protection. Appendix R requirements are deterministic (i.e., meet these specific requirements and you are safe). NFPA 805 is performance-based, i.e., here are your end goals - you figure out how to do it.
Deterministic= rule based limit that is not based upon probability or consequence. Risk informed = limit is based upon probability and consequence.
Requirements were initially established using deterministic criteria which were to some degree risk informed. As such there isn't much difference. That said, today we have deterministic-based requirements of which not all represent significant risk. So, at times we risk-inform a decision to mean that we make a "different" decision than was originally made under deterministic criteria.
Determinist is a decision based on experience and judgment, for example ALARA principals. Risk informed is a calculated number that has little to do with experience or judgment.
qualitative or quantitative
I would have to look that up Deterministic is the absence of probability. Design basis accident is deterministic. It assumes something has occurred such as a double guillotine pipe break for a LOCA and then ensures safety systems, structures, and components can handle this postulated deterministic event. Risk informed uses PRA as an analytical tool to identify accident scenarios, estimate the likelihood of each accident scenario, and estimate the consequence of each accident scenario. It also can provide a confidence level to the PRA.
When used during inspections expected outcomes are better understood through deterministic programs vs. risk-informed which could be less defined deterministic isn't a term used much in materials; risk-informed performance-based inspection for us means focusing our inspections on the items that could lead to workers or public receiving uninitended dose/harm; compliance-based inspections means putting the same level of attention on administrivia issues in recordkeeping as onobserving the licensee's implementation of good radiation safety practices.
deterministic = predetermined, an IF THEN relationship. IF an EDG is out of service the other EDG is extra important therefore we should do a.04 sample on it.
risk informed - situational, generally quantitative, more of a set of tables and the table you pick from depends on what equipment is OOS overall. The a EDG is OOS, but the SBO is pre-aligned to the bus that EDG is normally aligned to, and there is a component cooling water heat exchanger that is OOS. Even though the EDG on its own is more risk significant, the component cooling heat exchanger that is in service may be more important because of the over all situation, but then the SBO EDG is found to be non-functional unexpectedly and now the operable EDG is once again the most important component.
Deterministic is the framework and the overall requirements imposed upon the licensees. Risk informed is utilizing risk information to better shape the regulatory framework, inspection, assessment, and enforcement actions to the most important areas. They work together to ensure public health and safety as the deterministic requirements often form the basis behind the risk information. For example, Fukushima had an incorrect deterministic design requirement for Tsunami height, this resulted in inaccurate risk information about the site.
A deterministic system is discretely event driven. The significance of the event is singularly associated with its occurrence.
A risk informed system is cumulatively opportunity driven. The significance of an opportunity can account for the likelihood of it to arise as well as the cost associated with its passing.
Deterministic: Fixed regulatory thresholds or for example in MD 8.3 criteria that need to be met to reach a certain significance, level, or action.
Risk Informed: Deterministic application which is informed by a risk evaluation of significance. You cross a threshold to get to a violation (deterministic), you use minor/more than minor and SDP to risk inform the final determination.
Deterministic is somewhat a fatalistic view of things - since this and that occurred before, it would occur again.
Risk-Informed is being completely aware of the "good and bad" about the issue at hand, which would help tackle the issue.
Am I close or are you laughing?
Deterministic is what happens to the object at hand.
Risk informed takes into account the effect on the system as a whole.
Deterministic does not involve computation of risk. In the IMC 0309 process, the deterministic criteria give us a quick insight as to whether a risk analysis should be performed such that the true risk of the event is known. Risk informed utilizes actual risk numbers to select samples for inspection/enforcement action.
deterministic - operable under all design basis conditions. risk informed - available for the functionality of the component if needed.
Deterministic and risk informed can be aligned or be very different, depending on the system and tech specs I feel deterministic is better defined and easier to implement. I think risk informed would be too vague to implement consistently
risk informed looks at the likelihood of an occurrence and consequences - deterministic is a little more precise Question 16: What does functional mean with respect to screening findings?
Responses "Functional" typically describes the ability of a system, structure, or component to operate (or be operated) consistent with established requirements (such as design and/or licensing requirements).
SSC may not be "operable" in terms of the licensing basis, but it may still be able to perform the mission.
Can the equipment do its basic design job.
An SSC is functional if it can performed it's intended function. If there is a finding but the outcome is that the SSC was functional, it would raise the question of whether the finding is minor because there was no impact on the ability to perform its function.
An SSC will perform its intended functions for a nominal mission time of 24 hours2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br />.
functional = not necessarily operable for Tech Spec purposes, but still able to perform specified function in licensing basis Will the safety equipment perform its intended mission for a reasonable amount of specified time if a supporting item is unavailable for that time period.
Not sure The device works, but does not fully satisfy surveillance requirements or does not meet meet seismic or EQ criteria.
Is a structure, system or component capable of performing what it was designed to do in the event of an accident?
achieves mitigating function however may not be operable...i.e. can start it manually Functional/non-functional is a non-safety related component designation.
Operable or not operable refers to safety related equipment. Functional or non-functional refers to equipment that is not safety related but still important to plant operation, such as station blackout diesel or Appendix R diesel.
Functionality pertains to systems/equipment not specifically referenced in TS but in other programs/documents. (UFSAR, TRM, EP, etc.) There is less significance from functionality assessments than operability determinations but findings associated with equipment that only meets functionality requirements can still be risk significant.
The system is capable of performing its function. Functional is not related to operability (operable) nor maintenance rule (availability). A component may be non-functional for a non-safety related function and still remain operable.
"Functional", with respect to screening findings, means that a system, structure, or component is capable of fulfilling its specified safety function.
A component is functional if it works but not at full required capability.
SSC can perform its function. Does not always equate to TS operability and qualification.
Is it practical or reasonable.
Functional means that the system or component was capable of performing its current licensing basis function Functional is the same as operable except it pertains to a non-safety related SSC.
I think it is referring to whether a safety system can still perform its safety function, even if it may not be considered operable.
I know that we assess if something is functional, but degraded which is to understand operability determinations. Functional, but degraded is used to assess More-than-minor criteria.
The equipment will still work, but a some reduced capacity below operable, if safety-related equipment. If not safety-related equipment, the equipment will work.
I know what functional is but have no idea how it would be used to screen a finding.
I usually associate 'functionality' with a non-safety related piece of equipment or equipment that is covered under the TRM. If it is functional, it is capable of performing its function.
Capable of meeting design function under most conditions.
It should mean is able to perform all functions in TS, safety-related design basis, PRA, MSPI, maintenance rule, and otherwise described in the FSAR functions. At times some use only one or a subset of these to screen functional.
I do not know.
Its like the term "operable" but for non TS, non safety-related equipment. Youre probably looking at an FIN if youre writing a finding about that, A degraded condition to the point where a safety related function is not being adequately performed or maintained.
I consider functional to mean whether findings can be identified with a practical outcome or how implementation can be determined in terms of significance not applicable functional means there is reasonable assurance that a piece of equipment would be able to perform its safety function if needed.
It may or may not meet all of the criteria to be considered operable, but may still be functional.
With respect to screening findings then it really depends what the standard or regulatory requirement is that is potentially being violated. If it is a potential TS violation then the functionality comes into the SDP, but potentially not the minor more than minor, if it is a 50.65 violation then functionality may play a big role in minor more than minor and the SDP.
Non-tech spec SSC are functional or non-functional.
Functional is a not a term used in the IMC 0612 screening process. It is used in IMC 0326 and IMC 0309 evaluations. For IMC 0326, a non-TS equipment has functionality when it is capable of performing its function(s) as set forth in the CLB. These CLB function(s) may include the capability to perform a necessary and related support function for an SSC(s) controlled by TS. In IMC 0609, again functional is not the most prevalent term utilized.
The description of the state of an SSC such that it is capable of performing a specific design basis function.
It doesn't come into play in security.
Not sure.
Functional means that the equipment will meet the design basis analysis. Therefore, if the equipment was functional, the equipment will perform during an accident and mitigate the consequences as analyzed.
if the piece of equipment can perform its specific function (available to mitigate an accident - does not have to be operable but available)
Functional is usually associated with the availability of non-safety related systems.
it will perform its intended function able to meet pra assumed function
Region II Risk-Informed Decision-Making Online Survey Question 1: What is your job function at the agency?
Other (please specify)
Operations Engineer Analyst operations Engineer Senior Fuel Facility Inspector Senior Health Physicist Administrative Specialist (Facility)
Construction Project Manager Senior Operations Engineer Construction Inspector Allegations Coordinator Senior Operations Engineer Administrative Assistanat Senior Fuel Facility Inspector construction inspector Operator License Examiner Operations Engineer 0.00%
5.00%
10.00%
15.00%
20.00%
25.00%
30.00%
What is your job function at the agency?
Question 2: Which of the following have you had review experience with (check all that apply):
Question 3: The use of risk or risk insights is appropriate for inspection or inspection planning.
Strongly Disagree Neither Agree nor Disagree Strongly Agree Question 4: The use of risk or risk insights results in an unwarranted increase or decrease in the significance of inspection findings.
50.65 50.69 NOED MD 8.3 and IMC 0309 IMC 0609 TSTF-425 Initiative 5b TSTF-505 Initiative 4b 0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
70.00%
80.00%
90.00%
100.00%
Which of the following have you had review experience with (check all that apply):
1 2
3 4
5 6
7 8
9 0
2 4
6 8
10 12 14 16 18 20 The use of risk or risk insights is appropriate for inspection or inspection planning.
Please explain We do not use risk insights in IMC 609 TFPI findings Based on what? Personal opinion. Are you asking if my opinion is better than the tools being used? Not sure how to answer this.
Risk or risk insights should be a factor in the significance.
Have seen were violations go away because they were deemed not risk significant enough.
Disagree because is does provide a justifiable method.
Done correctly the significance will match the change in risk.
As asked this is a weird question. Risk insight is used for inspection sample selection (e.g. you don't inspect the secondary plant or you may want to pick aux feed over containment spray IF you have to choose. Risk is used for severity of greater than green findings.
I have seen both, increase and decrease in the significance of findings. Risk insights are very helpful during inspection planning to inform the inspection samples.
The reduction in significance is warranted as moves the organization away from strict compliance and focuses on larger issues.
If risk results in a decrease in an inspection finding's significance, then that is warranted and focus should be elsewhere.
1st, poorly worded question, could be misleading. My take is that risk insights are to be communicated and analyzed, when necessary, to ensure an inspection finding is documented with the appropriate significance it seems to me risk is utilized appropriately The risk thresholds result in too many very low safety issues being pushed into higher risk categories (Greater than Green).
Better thresholds would provide for more accurate outcomes and better use of resources.
Question 5: "Risk" or "risk insights" are a form of engineering judgment.
Strongly Agree Agree Neither Agree nor Disagree Disagree Strongly Disagree 0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
The use of risk or risk insights results in an unwarranted increase or decrease in the significance of inspection findings.
Please explain Insights are subjective and interpertative While there may be some judgment used in the development of risk models and possibly the analysis of results, my general thought is the results of risk analysis should be repeatable independently by trained personnel and therefore not dependent upon engineering judgment.
Engineering judgment plays into the determination of developing the risk profile but plays a smaller role for sample selection.
Engineering judgment should not be used to override.
There are assumptions and a degree of judgment involved for the risk practitioner, similar to engineering judgment Agree, until politics come into play.
While scientific, the use of risk insights can still be subjective.
It is often uncertain rather or not an affected SSC can perform its function. Engineering judgment would be needed to decide.
Although the methodology used to apply risk values to each component could be viewed as engineering judgment (for example, which one was used and endorsed by the NRC), the application of the quantitative value derived from those methodologies eliminates the judgment or opinion associated with how each component is ranked.
Risk insights are as good as the assumptions that they are based on.
introduces the concept of probability or the likelihood of an event occurring over a specified period of time.
Risk models are technically developed. Engineering is a form of technical expertise. Applied to risk insights, engineering judgement is a great tool.
I work in DFFI, so the risk tools are fairly crude.
Again, this is a potentially misleading question that requires thought. Risk is different from risk insights. The latter is an input to the resultant risk. Some engineering judgement may be applied in some cases but typically the "insight" helps answer preestablished questions used to assign risk Strongly Agree Agree Neither Agree nor Disagree Disagree Strongly Disagree 0.00%
5.00%
10.00%
15.00%
20.00%
25.00%
30.00%
35.00%
40.00%
45.00%
50.00%
"Risk" or "risk insights" are a form of engineering judgment.
i don't know its based on a deep statistical resource and seems appropriate in circumstances where there are no concequences Everything requires some amount of judgment. Using risk is using informed judgment.
Question 6: The ROP and Enforcement guidance have resulted in unintended and unreliable outcomes in dispositioning issues of concern.
Other (please specify)
Enforcement severity level is somewhat subjective as every case has unique factors. As such severity level across similar cases can seem inconsistent.
Not enough personal experience to comment.
Based on what? Personal opinion. Are you asking if my opinion is better than the tools being used? Not sure how to answer this.
On the whole, they have aided in having a reliable program.
Differences between the Regions.
Changing interpretation and differing application of the use of ROP & TE together result in a cumbersome and unreliable enforcement process.
However, it has typically resulted in consistent agency response.
Don't believe that the ROP's deterministic approach has resulted in any unreliable outcomes.
ROP guidance doesn't provide a sufficient number of examples to guide inspectors. However, it is up to the inspectors to research past inspection violations/findings to maintain the regulatory consistency. By not doing so, the resulting outcomes do become unreliable and inconsistent.
Management meddling has resulted in unreliable outcomes.
Strongly Agree Agree Neither Agree nor Disagree Disagree Strongly Disagree 0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
The ROP and Enforcement guidance have resulted in unintended and unreliable outcomes in dispositioning issues of concern.
In practice, there is still a large amount of subjectivity in some of these conclusions, especially when developing a more than minor issue and how the issue can adversely affect a cornerstone objective.
Fuels guidance has come a long way to make enforcement predictable.
There will always be a few specific cases where this could be concluded but generally reliable outcomes are the result of the ROP and Enforcement guidance I don't use ROP it just seems that the timeline to make a determination is slow, not particularly faulted Because the two were never properly aligned, there are too many cases where both traditional and ROP processes are being used resulting in issuing violations of issues that would be minor under the ROP.
Question 7: Inspection resources are focused on the most risk significant issues.
Strongly Disagree Neither Agree nor Disagree Strongly Agree Question 8: I have concerns with expanding the use of "risk" or "risk insights" by the agency.
1 2
3 4
5 6
7 8
9 0
2 4
6 8
10 12 14 16 Inspection resources are focused on the most risk significant issues.
Strongly Disagree Neither Agree nor Disagree Strongly Agree Question 9: There is value in identifying minor and Green inspection findings.
Strongly Disagree Neither Agree nor Disagree Strongly Agree Question 10: Do you think the current guidance or training is sufficient for you to use risk or risk insights in your decision making?
1 2
3 4
5 6
7 8
9 0
2 4
6 8
10 12 14 I have concerns with expanding the use of "risk" or "risk insights" by the agency.
1 2
3 4
5 6
7 8
9 0
2 4
6 8
10 12 14 16 There is value in identifying minor and Green inspection findings.
Strongly Disagree Neither Agree nor Disagree Strongly Agree Question 11: Please explain your answer above.
Responses Training which provides a practical approach to applying risk insights in day to day decisions would be useful. I don't want to do detailed PRA analysis with risk numbers, but want to be able to apply risk principles in a practical way.
More specific examples of how the use of risk insights can be used to strengthen arguments for findings would be useful Don't understand the purpose.
generally agree Compared to what? I think we are good, what else is out there?
Understanding the risk involved from a global perspective helps us to proactively work towards decreasing risk and to address them before they adversely affect the agency operations.
The IPs and IMCs I use as a resident inspector are clear enough. More importantly, they allow me the leeway to do the samples I determine are most important.
Depends on position and past experience of individuals. Risk programs appears to assume that all maintenance and procedures are completed as written....Observations in past suggest that may not be a true assumption.
The program should more clearly allow an early off-ramp for compliance issues of little to no safety significance, to help maintain agency safety focus.
I think a lot of it comes down to experience and knowing what decisions will be accepted and which ones won't.
The initial training on risk and how to develop and use "risk insights" is minimal when considering how it is expected to be applied.
More emphasis on actual application would be beneficial for inspectors. Regular discussions, use, and demonstration of knowledge are needed.
While I can use risk insights to choose an item as an inspection sample, which automatically risk informs the inspection (we are looking at the most important risk contributors), that doesn't cover everything that is considered "decision-making". Decision making also involves deciding how much time to spend on items that are of low risk as well as dispositioning issues as violations or not.
1 2
3 4
5 6
7 8
9 0
2 4
6 8
10 12 Do you think the current guidance or training is sufficient for you to use risk or risk insights in your decision making?
Staff has been encouraged to selectively apply risk insights in pursuing compliance issues, and the staff has not been trained on how to do this.
We have sufficient training and tools available to evaluate risk and our processes are clear on the use of risk.
The guidance and training are sufficient for me to do my job as an inspector for both selecting inspection samples and for screening when issues are identified For the existing program yes. However, to expand the use of risk will require procedures, processes and training to be updated BEFORE being implemented to ensure consistency across the agency is maintained.
Yes when related to installed plant systems--No when related to plant programs, processes, and human decisions. How does poor training program relate to delta-CDF?
I believe the agency would benefit from a culture change in this area...not sure mandatory training would necessarily help.
the training is adequate however remember for the other less experienced inspectors it may be more difficult apply.
periodic training is not performed but would be worthwhile I am not aware of training that incorporates the use of spatial interaction or assesses the risk significance of passive structures who's reliability is assumed to be high.
Risk insights come from PRA and engineering judgment.
Management isn't clear if "decision making" means sample selection, or pursuing issues of concern, or final significance outcome.
Construction inspection activities should focus on quality assurance compliance; applying risk to that process is not well defined.
The fuel cycle program does not have sufficient tools to identify risk important inspection activities and largely depends on inspector knowledge, which is not consistent.
The use of Saphire and the SPAR models can be used in most cases to bound the "risk" of an issue. From those results, the inspectors can determine if it's worth pursuing and if so, the priority and resources necessary to close it.
Inspector judgement (based on risk and OpE) can be used for minor issues and even some Green findings.
We should consider incorporating risk and risk insights in the allegation process. This could help save resources for seemingly minor or Green issues that are strictly non-compliances with little or no safety impacts (ex. fatigue rule and work hour violations).
Since this concept is relatively new, the more guidance the agency puts forth, the better the residents can apply to their inspections at the sites.
We need better guidance with respect to what is acceptable from a risk standpoint. For example, how much safety margin is good enough.
The guidance is adequate. The training needs to catch up for people who have already qualified in their current position.
The guidance is designed to take those risk insights into account for noncompliance assessments.
Notwithstanding the use of risk, deterministic regulations often result in "having to" pursue issues that present themselves, leaving little room for risk-informing the issue beyond significance determination.
There needs to be more training (classroom training, requiring a classroom refresher on a periodic basis)
Continuous training is necessary for refreshing experience and new expansive ideas associated with risk. This agency must go this route to improve our efficiency and effectiveness. Risk equates directly to importance.
The PRA training we receive is a good starting point for risk based decisions. It must be used with the licensee's risk model to fully understand each site and what is risk significant to them.
I understand procedures that incorporate risk into the decision, e.g. MC0609, aspects of MD 8.3. I find it harder to understand issues that go to SRAs for a custom 'calculated decision.'
Fuels don't have well established guidance or training in the areas being discussed.
I am a very experienced and accomplished inspector. The answer to 10 above is probably dependent on inspector experience or the experience of the team around the inspector We were given just enough to be dangerous. There is too much reliance on computer modeling and without proper understanding of risk overly conservative assumptions are being used which results in artificially higher risk outcomes.
Staff feel compelled to document ALL regulatory infractions. There needs to be guidance to explicitly state this is not true....based on risk.
Question 12: Do you think there is a gap between management expectations and staff use of risk or risk insights?
Strongly Disagree Neither Agree nor Disagree Strongly Agree Question 13: Please explain your answer above Responses I am not sure what management expectations are regarding their use so I cannot judge the gap or delta?
Staff doesn't fully understand.
Not enough personal experience to comment.
A gap hasn't been voiced.
People are myopic and don't always understand how risks affect the agency outside of their scope of interest. Each individual should feel empowered evaluate areas and to actively assess the environment, the impact of our actions, as well as how best to improve in both critical and noncritical functioning areas of the NRC.
Management seems to be in line with resident inspectors. I have not encountered an issue related to risk or risk insights.
Procedures tell us to consider risk significant systems when inspecting...all I have to go by...are management expectations different?
1 2
3 4
5 6
7 8
9 0
2 4
6 8
10 12 Do you think there is a gap between management expectations and staff use of risk or risk insights?
no See #11. Program guidance should reflect agency expectations to ensure reliable application and maintenance of agency safety focus.
Don't know what management thinks. Seems to depend upon the individual potential finding/violation and what management decides to do with that specific potential finding/violation when brought to them.
Observing inspectors and hearing management speak indicates it is not being used as much as it appears management believes it is.
As explained above, there is a general preference for the inspectors to apply risk insights in deciding which compliance issues to pursue. Most issues of concern that an inspector comes across are based on the NRC's requirements, which are intended, in most cases, to be bounding, as there is no reason to write a separate requirement for every spectrum of design event if it can be bounded by a single "design event". An example is EVERYTHING that is safety-related/ covered by 10 CFR 50, Appendix B, is based on the bounding design basis events described in the station UFSAR. If the licensee meets the requirements for the bounding design basis event, then they should be in compliance for the less-stringent events. If we inspect to the requirement, then the bounding event is the one that is considered in the calculations/ station analyses and is the thing that TS operability is based on. If an issue comes up with the analysis, then there is a management perception that because the station analyses were based on a low-risk/ low-frequency event, then there is no to little risk involved in the issue of concern. However, the issue still may represent a violation and may have some unknown risk associated with it. In many cases, until further analysis is done, the impact of the violation and the risk remains unknown. Since the additional analysis often takes licensee effort, they have in many cases resorted to complaining to NRC management that the inspector is pursuing low risk items in order to avoid doing the additional work needed to characterize the risk involved in the issue. I believe this has led to a disconnect between management expectations and the staff's realistic use of risk insights.
Management and staff both agree on the use of risk or risk insights.
The agency is in a shift to be less burdensome on the licensees. With this comes management expectation for inspectors to us judgement and walk away from or downgrade issues based on "engineering judgment" rather than following the outcomes of our current guidance.
Once again the expansion of risk concepts to business practices is not consistency or fully understood across the agency.
Another way for management to ignore issues that are not related to plant hardware.
The transformational initiative has not reached staff at the inspector level.
No actual gap exists, but I believe there may be a perception that identifying violations is the best way to get good visibility.
In some cases management accepts findings or violations with minimal to no risk significance.
It's not clear what are management's expectations Management has not expressed an expectation for the use of risk or risk insights. The expectation is assumed through IP guidance.
Sometimes.
I think management needs to make it clear that they expect the staff to use risk and risk insight on sample selection for a given inspection procedure. As issues arise, like self-revealing ones, I don't think management should that expect we use risk or risk insight to discount them.
Use of the terms are common and generally convey a concept; however implementation by inspectors varies widely.
Not sure. More communications on management's expectations would be welcomed.
Somewhat. Management should elaborate more on their expectations for employing risk and risk insights towards inspection efforts.
I have no idea.
We need to do a better job understanding the underlying basis of inspection issues to ensure that we are on the same page with respect to risk insights.
The only real "gap" is just the continued reinforcement of standards to use risk and risk insights by management. However, not everything we do can be risk informed. There are literally dozens of NRC regulations and requirements that are not risk informed that have to be enforced.
Im not familiar with any divides but Im sure they exist especially with new managers.
See 11, above. It's difficult to tell staff to let go of compliance issues that they've uncovered, even if the risk associated with the issues is very low.
I think management is separated from the daily risk of each plant, and frequently over-reacts to items that may be risk-significant at some sites, but are not at all sites. Every plant is different, and they sometimes have difficulty remembering those differences (and trusting their staff to make those calls).
Recently, as a result of a significant industry event, NMSS/DFFI management provided expectations that inspectors should be inspecting less significant SSCs. I believe what they meant to say is that we should be looking at the licensee's evaluations of all SSCs over time to ensure the right risk significance has been applied.
Without knowing the specifics of the site, there is not a generic "risk" model you can apply. Each site is unique.
R2 management appears to want staff to overlook existing processes or offer informal enforcement discretion for some green issues.
The SDP process has risk informed and risk based decision making guidance as its cornerstone. We could increase the risk tool usage if determined to be necessary. What we should never forget is that observing, commenting and possibly even documenting less risk significant issues is important not just because of the overall agency assessment of licensees but also to identify key failure in licensee programs used to maintain BOTH risk significant and less risk significant SSCs Management expects inspectors to find violations, but when risk assessments are used properly it results in minor violations.
Management wants documented violations so they can be used for messaging to the licensees.
I think inspectors already have the flexibility to quit inspecting items of low risk, however, many feel compelled to continue inspecting if there is a chance at a finding.
Question 14: Can you briefly describe what you would consider barriers to incorporating the use of "risk" or "risk insights" in your daily work?
Responses Infrequent usage of the tool in daily work activity leads to a lack of familiariation Pulling the string on issues could be more difficult. If an issue of concern is not "risk significant" the residents might not look at it.
Sometimes these issues uncover larger issues that are risk significant.
No perceived barriers.
Time and tools that are easy to use by the lay-person.
Time to assess is the only immediate barrier I can see due to a cut in resources.
None None...I typically consider which system are being worked on and choose the risk significant ones (based on the PRA Pie Charts) to ask questions on.
tools and training Lack of consistency (in expectations) between branches - regions.
Easy to use tools - SAPHIRE did not work out like it was advertised
Picking inspection samples, deciding which inspection attributes to devote time to on an inspection, and deciding the priority/urgency with which to engage the licensee on a particular issue are things that inspectors should risk inform. If the inspector had to quantitatively risk inform each of these things, it would take an inordinate amount of time and resources. The amount of time allotted to inspection activities is not enough to specifically incorporate risk insights into everything that we do.
Inspectors will often have to rely on "gut feel" based on their experience to do most of this.
Adding risk insights to every day work could be burdensome.
Management expectations versus the written guidance Expectations and consistency. For the ROP we have guidance, procedures and training. For routine NRC business practices, I am not fully aware of risk and risk management practices. Therefore, in the business practice side, I do not have guidance, process or even a framework to ensure we are consistently thinking and implementing risk the same way.
Senior Management Will have to overcome the agency's history of how it does business.
management and lack of experienced inspectors lack of adequate training Amount of resources needed to determine risk values, inspections that require the coordination with the licensee to view activities don't lend themselves to allow selection based on risk, training and application of this strategy to maintain the level of knowledge to use it effectively, and SAFIRE isn't very intuitive/ easy to use.
The data we use for incorporating risk insights needs to be accurate.
no barriers, it is more of a mindset to focus your resources on areas of more risk.
Emergent issues sometimes drive sample selection at the site. I would be concerned about inappropriately triaging emergent issues and therefore not pursuing them because they are not "risky" enough.
Lack of appropriate tools for selecting to more risk important activities for fuel cycle inspections. The Integrated Safety Analysis does not generate risk measures (RAW, RRW, etc.). It is a list of safety features with no gradation on their importance.
Therefore everything in the ISA is important. Inspectors plan using "perceived risk" based upon their experience and level of knowledge. This has the potential to miss activities that are perceived to be low risk.
See above answer. If inspectors were more of aware of the expectations regarding the use of risk and/or risk insights, they may spend less time following up on non-safety related issues/events.
Several TS actions seem to be deterministic and not based on risk. Inspectors tend to focus a lot of resources on verifying TS compliance, even when there may be little to no risk.
The degree to which management enforces or lack thereof, the use of risk and risk insights in day to day inspection efforts.
None.
The mentality that we do not have enough flexibility within the existing framework to use risk insights when assessing findings using IMC 0609, 0616, and the enforcement policy.
The biggest barrier is that not all of the rules and regulations are risk informed. The residents are tasked with identifying all non-compliances, whether they are risk informed or not.
Not aware of any.
The plain language of many regulations. It won't be long before rulemaking efforts are required to fully incorporate risk into inspection.
The AP1000 is a unique design with a much lower baseline risk. There are numerous NRC staff (R2 Management, HQ, TTC) who have trouble with this even when it is explained over and over again. Incorporating risk would be a huge reduction in NRC's
oversight of the AP1000 units as they go online. The agency needs to be conscious of this and needs to prepare to explain it to the public. I think it is a good idea (we should give plants incentive to have lower risk, such as reduced oversight).
Lack of licensee risk models in fuel cycle plants.
I already incorporate risk insights in my daily work. It is how we determine what samples to select for our baseline inspection.
Complexity of decision making process involving risk.
Ambiguity of documented decision making process involving risk.
Fuels don't have well established tools, guidance, or training in the areas being discussed.
- 1) Lack of plant knowledge
- 2) Excess focus on non-safety related aspects of plant operation such as safety culture, proven with substantial margin programs like fire watches, control room manning, heavy load controls, reporting criteria, etc These programs have errors every day but really are not typically contributors to risk None.
Different inspectors have a range of perception with regards to risk. Some will try to say everything is minor while others are the exact opposite.
None Our lack of respect for the licensee and their ability to correct low level problems. Too many times I've heard branch chiefs say "We have to help them get better" or "They need to improve their performance."
a PRA model used that is actively used by the utility, not being SRA trained Less experienced inspectors. Guidance that is very clear.
Question 15: Describe the difference between deterministic and risk-informed.
Responses Deterministic methods apply actual or realistic/best estimate values in performing analyses. Risk-informed methods utilize probabilistic values in analyses, and consider both PRA and risk insights.
Deterministic establishes specific requirements but does no assess overqll impact, whereas risk informed uses insights that better focus attention on design and operational issues with regard to their importance to health and safety Deterministic decisions are physical engineering decisions. Risk-informed decisions rely on PRA.
Because of limited samples and/or the lack of data related to significant events, the results of risk analysis are not an end all (deterministic) but rather an input into final decisions (risk-informed).
Principles based on deterministic considerations, such as defense-in-depth, safety margin, redundancy, diversity, and independence, have served are key principles and useful concepts, and play an important role in keeping nuclear plant safety.
Risk insights can significantly help to make a more informed and robust decision making as compared to the decision making based on only deterministic analysis because of the added insights on plant safety as a result of the PRA where plant safety is thoroughly evaluated with explicit considerations of system interactions, common-cause failures, human errors, and so on.
A risk informed approach allows one to identify and assess the probabilities that can occur or implement necessary changes or approaches in the process that may reduce unnecessary requirements which stem from a deterministic approach. In a deterministic approach processes and procedures do not involve randomness therefore the outcomes will be the same because the inputs and relationships are known therefore they will generate the same results.
Without looking it up, I would say deterministic processes uses qualitative threshold criteria while risk-informed processes uses quantitative threshold criteria and limited qualitative criteria.
Deterministic = Algorithm, model, procedure, process, etc., whose resulting behavior is entirely determined by its initial state and inputs, and which is not random or stochastic.Processes or projects having only one outcome are said to be deterministic their outcome is 'pre-determined.' A deterministic algorithm, for example, if given the same input information will always produce the same output information.
Risk-informed decision making may take into account multiple sources of information not included specifically in the assessment of risk as inputs to the decision process in addition to risk information, while risk-based decision making uses the assessment of risk as the primary decision driver risk informed is an approach to regulatory decision making, in which insights from probabilistic risk assessment are considered with other engineering insights. Deterministic is based on if X happens the Y happens Deterministic is condition-based while risk-informed considers likelihood and impact Black & White, Go / No-Go (deterministic) vs. Shades of Gray (risk - informed).
Deterministic is criteria based and risk-informed uses insights from potential risk determinations to assess Deterministic is a pre-defined set of criteria that must be met in all cases.
Risk informed is a set of criteria to be met that can change depending on a particular configuration/ status of the plant.
Deterministic is a specific defined criteria (safety related/important to safety); risk informed would relate to the actual importance of a component or system during a postulated event combined with the expected frequency of the event.
deterministic uses a fixed criteria while risk-informed uses risk insights to make decisions calculated versus judgement In the reactor business line I consider DETERMINISTIC as defense in depth & diversity of methods use to protect. for example, redundant pumps that are motor driven and steam driven. RISK-INFORMED reviews a more realistic view of plant performance using multiple system to address and event. it is more flexible then the rigid deterministic view.
Depends on whom is speaking.
The difference is the use of risk insights to determine the overall impact to provide flexibility in making a conclusion.
'Deterministic' is rule-based, i.e., single failure or TS. 'Risk informed' is reality based, i.e., how likely is an accident.
qualitative vs quantitative Deterministic is sort of a go-no go test. Ex if a HPSI pump is OOS for 71 hours8.217593e-4 days <br />0.0197 hours <br />1.173942e-4 weeks <br />2.70155e-5 months <br /> (72 hours8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br />) this would be deemed acceptable However for risk informed analyses any OOS hours is considered a negative. Risk informed approaches can also sum up multiple systems to provide a more refined risk analysis Deterministic has a specified outcome and will always behave in that manner as the interaction is pre-determined. Risk-informed is applying variables that have a general probability/randomness associated with them such as spatial interaction.
Deterministic has limits - risk can be subjective deterministic is a prescribed list of requirements that must be followed. risk-informed is using PRA and engineering experience/knowledge to hone in on the more risky areas.
Deterministic to me is saying "it's a safety system and is fair game for inspecting" Risk informed means focusing inspections on the most important safety systems based on risk.
Deterministic evaluation is an "if-then" decision. If a criteria is met, then the conclusion is logically reached. Risk informed evaluation considers the severity of the consequence of an event and the likelihood of it happening
Deterministic issues are generally considered as compliance issues that maintain a large safety margin and defense-in depth regardless of the safety risk. Considered very conservative.
Risk informed issues are quantifiable based on industry approved risk models using approved methodologies and adequate assumptions. Assumptions are generally conservative, but can be re-evaluated as needed.
deterministic involves meeting certain criteria of event response that can occur at an operating plant versus risk informed or the likelihood of an adverse event occurring over a period of time.
Deterministic is not based on PRA.
cooking recipe vs. a buffet Deterministic refers to the use of discrete criteria to resolve the processing of an outcome for an issue. Risk informed refers to the use of a statistical probabilistic assessment to resolve an outcome for an issue.
Isnt deterministic a verbatim compliance and risk informed tries to weight the probability of an issue developing Deterministic is largely prescriptive and binary. Risk-informed allows for a more holistic look at requirements and outcomes.
Deterministic is basing the decision on the action, regardless of the outcome (or risk). It would be saying that RHR pump failure is a specific level finding at all plants, regardless of what plant it is at. Risk-informed is basing the decision on the increase of risk (it can be a percentage increase from "normal" or an increase over a threshold). It would be saying that RHR pump failure is certain level finding if that failure results in at double (100% increase) in risk or and increase in risk bringing the risk to above a certain number.
Deterministic is an objective view that a given cause results in a given consequence.
Risk-informed incorporates risk significance as an element of importance. If the consequence is not important/risk significant, why waste resources on it, both licensee and NRC.
Deterministic is set and cannot be changed. Risk-informed changes based on other supporting or non-supporting conditions.
Outcomes are pre-determined using objective thresholds vs outcome may involve some degree of variability based on risk.
Risk-informed incorporates quantitative risk information into a broader decision making framework. Deterministic implies the decision making is solely a function of the risk criteria.
Just words, Inspectors with good plant knowledge know what is risk significance. These words are so management and NRR can have debates Is this a test or a survey?
Deterministic falls cleanly into one bin while risk-informed is somewhat open to perception.
Deterministic refers to use of old criteria ie single failure and separation while risk informed addresses the role the component and system perform in the plants probabilistic models.
Deterministic safety thresholds are arbitrary; risk-informed is based on actual performance data and accepted risk assessment practices.
clear thresholds based on observable items, vice taking observations and making a calculation about the possible impact Deterministic is based on yes or no answers to questions. Risk-informed allows some use of engineering judgment or use of risk tools.
Question 16: What does functional mean with respect to screening findings?
Responses
There are degree's of what is considered functional with regards to systems structures and components performing their intended task that can be applied against findings that can magnify or lessen their significance with regard to impact on public health and safety Consider in the context of requirements (i.e. Technical Specifications). A system or component may not be TS Operable but non the less functional in that it "works" although it may be degraded in some manner.
The SSC can still satisfy the safety purpose.
Functional is the equivalent of operable for non-safety and important-to-safety SSCs. Not sure if I've seen the term used in screening findings other than for that.
Can the system / component perform its intended safety related function.
means it will meet the design function but is not in accordance with its design Does it perform intended function.
Able to perform SSC's safety design function.
When non-Tech-Spec equipment can meet their design operating requirements in the license.
Functional can be compared to fully operable and conforming. Functional means that the equipment would have accomplished its required functions given the ACTUAL PLANT CONDITIONS that existed since the deficiency was introduced to the component, including the current conditions present when the operability of the equipment was reviewed. If it was functional following the point at which the deficiency/ violation was introduced, then there was little to no risk and the issue should screen to green.
Fully operable means that the equipment would be able to perform its required functions UNDER ALL OF ITS WORST CASE DESIGN BASES CONDITIONS, which should bound all of the expected normal operating conditions, which are the initial conditions in the design bases analyses. If the component was not operable because of the deficiency, then functionality as described above would need to be reviewed to determine the risk involved.
Could the system/component perform a specific function (vice operable which has very prescribed definition). In SDP space the significance of a finding will depend on the functionality of a system/component. A system/component may be inoperable and thus a violation but functional and thus have a low risk significance.
Functional means a system is capable of performing its safety function Can the SSC still perform its intended function even if degraded or non-compliant.
Will the component be able to meet a minimum function such as pump 5000 gpm at 1150 psig, even though the component may not be able to meet EQ or design requirements imposed by the FSAR.
Non-Tech Spec SSC that can perform design basis job.
I believe this means taking into account whether equipment would/could perform its function.
Functional refers to a SSC 'realistically' accomplishing its design intent. Just because operators declare a SSC 'inoperable' does NOT necessarily make in non-functional.
ability to perform safety function.
Functional refers to no tech spec components whereas operability refers to tech spec components Able to meet its most design limiting intended function if called upon.
able to complete its intended task/action/function
Functional means the SSC will provide limited capability to do it's design function that may not meet surveillance criteria, but may still allow the overall safety function to be accomplished. For example, a failure of a diesel generator's control circuit which is limited to the synchronous mode may not affect it's ability to perform in the isochronous mode, and therefore although the surveillance can not be met, the EDG will perform it's safety function in the isochronous mode.
The component perform its safety function despite a nonconformance to certain quality assurance requirements or standards.
The equipment/train can still do what it needs to do when its called upon to do it for as long as it needs to.
functional implies that the system or SSC maintains its ability to perform its design bases purpose.
Functional basically means it will perform its function. An SSC may be functional although not operable by TS.
No idea.
Functional for safety related systems is whether or not they could have performed their design basis role in the event of an accident. Functional for non-safety systems is whether or not the system could perform its role.
Is the safety objective met degraded to the point of "inoperable," as we currently use the word.
This is a very complicated question, and I am not up to this explanation this early in the morning. My brain is spent from the last question.
If the finding does not impact a special activity, purpose, or task relating to the way in which a safety-related SSC/IROFS works or operates, then it is less risk significant.
If a piece of equipment was determined inoperable, but functional, the significance of the finding should be lower.
TS operability doesn't apply.
Component/system will achieve desired outcome even if degraded.
Not sure.
SSC capable of performing its function as described in a licensee UFSAR and other design documents (like calculations, etc)
Some will debate that "all" functions are not required for screening of findings and sometimes that is true. Need to treat function on a case - by - case basis.
If you discussing functional you could be discussing a non-safety related piece of equipment or a safety-related piece of equipment that has not been declared operational, but can still operate. Therefore, in the safety-related case, the risk of this piece of equipment not operated has been reduced.
It refers to whether the equipment still capable of operating/functioning. Operable refers to whether the equipment is capable of performing its safety related function.
Functional refers to the role the system and component play related to the risk functions described in the screening questions.
The equipment is able to perform its designated safety function; no impact on overall protection.
The component will perform its intended function when it needs to.
Region III Risk-Informed Decision-Making Online Survey Question 1: What is your job function at the agency?
Other (please specify)
ISFSI Inspector Health Physicist Senior Allegation Coordinator - I guess this means the survey is no longer anonymous for me Government Liaison Administrative Assistant admin staff administrative Physical security inspector anonymous Why did you send this to materials staff if we aren't even on the list of inspectors Materials Licensing Reviewer - HP Incident Response Staff Senior Operations Engineer Management Program Analyst I have been a resident inspector and am currently a reactor inspector Regional Government Liaison Health Physicist ROA Resident Office Assistant Administrative Assistant Question 2: Which of the following have you had review experience with (check all that apply):
0.00%
5.00%
10.00%
15.00%
20.00%
25.00%
30.00%
What is your job function at the agency?
Question 3: The use of risk or risk insights is appropriate for inspection or inspection planning.
Strongly Disagree Neither Agree nor Disagree Strongly Agree Question 4: The use of risk or risk insights results in an unwarranted increase or decrease in the significance of inspection findings.
50.65 50.69 NOED MD 8.3 and IMC 0309 IMC 0609 TSTF-425 Initiative 5b TSTF-505 Initiative 4b 0.00%
20.00%
40.00%
60.00%
80.00%
100.00%
120.00%
Which of the following have you had review experience with (check all that apply):
1 2
3 4
5 6
7 8
9 0
5 10 15 20 25 The use of risk or risk insights is appropriate for inspection or inspection planning.
Please explain significance of an issue is determined through IMC 0609. Our use of the terms "risk" or "risk insights" has no impact on the significance of findings.
While I believe "risk insights" are appropriate, "risk insights" are currently being used to ignore latent safety and compliance issues because they tend to be Green. Inspectors are encouraged to only pursue >Green issues and those require SSCs to be already failed.
Although the use of risk insights could result in an unwarranted increase or decrease in the significance of inspection findings, it is up to us as the regulator to make sure that the significance of inspection findings is appropriate to the the circumstances.
Use of risk insights is consistent with Agency policies and ROP.
The answer to this depends upon everyone having the same definition and use of risk and risk insights. Until this is defined, it is hard to say what the impact will be.
Risk or Risk Insights only works if the staff is willing to challenge the results based on their knowledge of the system in question.
The application of risk that is grounded in operating experience can strengthen the process of making a difficult decision by informing the process.
It gives room for opinions.
A finding can be driven to lower risk by luck rather than any overt action taken by the licensee (e.g., the same finding under a different set of circumstances would result in a higher risk significance). An example would be the difference between a PD that causes one train of a dual train system to be inop. vs a PD resulting in one train unknowingly inop while the licensee does maintenance on the redundant train. The latter is more significant because it resulted in two trains inop while the former tends to be looked at like the licensee was fortunate that they weren't doing maintenance on the other train although there wasnt anything but luck stopping them from doing such maintenance. The former tends to have more swift action taken by the licensee while the latter tends to follow the normal process because there is nothing driving it to move faster. When the resident office and the licensee dont see eye to eye on the corrective actions to be taken in the latter case, it can be difficult to motivate the licensee to move faster because the risk significance card isnt in our back pocket even though the licensee benefited from blind luck more than intentional actions.
Seems warranted Under our current regulatory framework safety is ensured via compliance.
Strongly Agree Agree Neither Agree nor Disagree Disagree Strongly Disagree 0.00%
5.00%
10.00%
15.00%
20.00%
25.00%
30.00%
35.00%
40.00%
45.00%
50.00%
The use of risk or risk insights results in an unwarranted increase or decrease in the significance of inspection findings.
The current approach of "risk", while important to the mission was not the basis for our regulatory approach. Hence, while considering risk is important it can significantly undercut the base assumptions that ensures safety if we use "risk insights" to ignore compliance.
The SDP is the method we use to determine the significance of findings and is, by definition, based on risk and risk insights.
Therefore, if risk increases then significance increases.
The ROP has been risk informed since inception. My sense is there hasn't been any change in the number or significance of findings documented due to "risk insights." The number and significance of findings does change, but there are other forces at work that are behind this -- mainly politics.
NRC has risk informed ISI LAR, IST LAR, NFPA 805, 10 CFR 50.69 LAR, Risk Informed Containment Leak Rate Test Frequency Extension, Point Beach Containment Dome Truss Risk Informed LAR. The NRC already includes/approves risk information and insights which change permanently the licensing basis of the plants. It is not clear what risk insights during inspection will change. Many plants will be submitting 10 CFR 50.69 LARs which have the potential if RISC-3 categorized to exempt the SSCs from specific regulations. This is significant and addresses risk insights into our inspection program.
Any changes would be warrented.
risk or risk insights should impact significance but I don't think the impact is unwarranted. Observations and PDs may seem important but risk should impact NRC's assessment of their "significance."
The changes in significance of findings are not generally unwarranted when risk is considered.
We still use the phrase "performance-based" when describing our processes. Some violations occur due to terrible performance, but do not result in a great amount of risk. These should not be minimized because of the risk numbers.
PD's and non-compliances come in many sizes and shapes; as evidenced by historical findings.
Minor issues or findings that are not risk-significant do not always receive attention until the issues become major problems.
NRC's use of "risk" or so-called "risk insights" is flawed and sometimes irresponsible. In fact, in some cases the emphasis on "risk" or "risk insights" send the wrong message regarding the importance of compliance. The majority of the data that the NRC uses to form it's "probability of occurrence" information has little to no pedigree. Licensees provide the NRC with data related to equipment failures and probabilities of failures and human error probabilities and that information is most of the time suspect at best. The inspectors have no true way to validate the information but the agency base regulatory and safety decisions on that information. Until the NRC has specific rulemaking on this topic and serious mandates to ensure that this information is independently validated, the "witch doctor" magic will continue.
The significance determination process integrates and balances a spectrum of factors, including legal basis and risk, among other elements comprising due diligence.
Risk insights play an important role in screening inspection findings.
Risk quantifications improve regulatory behaviors/decision reliability.
Occasionally it can be discouraging when you identify an issue that appears to be significant from a process or program perspective and then when it is worked through the SDP, it ends up screening to green or even minor. However, I realize that risk insights better informed than my gut feel.
Generally, risk and risk insights provide a reasoned and structured method to assess finding significance. I will also point out that the current process looks at the outcome of a PD. In some cases, numerous example of a PD (or similar PD) exist, but since individually each is minor or very low safety significance, less resources are applied to ensure correction of the underlying issues.
Question 5: "Risk" or "risk insights" are a form of engineering judgment.
Please explain Risk is a form of engineering judgment; however, expectations on the use of this judgment is needed. Also, typically judgment is only appropriate based on expertise. For example, an engineer with an EE background/expertise may not have the appropriate skills to provide engineering judgment to a vortexing issue.
risk may be an input to engineering judgment Inputs into risk models are often based on judgement, as sometimes data is lacking in robustness or is unknown for a particular system or situation. Also, determining when and how much to incorporate risk or risk insights is also based on judgement.
We are always using our professional judgment on inspections when determining what to spend our time on based on our understanding of risk. Our understanding of the risk comes from our professional experience in our area of expertise.
Risk or Risk Insights brings with it a formal assessment. Engineering judgment does not have the same rigor unless the staff is willing to challenge the results. this may be difficult since the risk assessment are the product of the SRA computer programs that are not available to the staff I believe risk sights are engineering judgment plus operating experience.
Risk and risk insights are the result of calculations based on how the plant operates and the systems available at the time of the calculation. Engineering judgment is less calculated and more "gut feel" or experience based.
It is all based on the Bayesian inference and having updated and accurate failure rates for all the components in the Risk pool.
Risk=Consequence x Probability Probability at the end of the day are only educated guesses. Hopefully, backed by some level of reasonable, repeatable observations (i.e. failure probabilities, etc.)
Enginering judgments are also educated guesses...ideally supported by engineering and technical principles.
I believe it can be a form of judgment at times because the determination of items such as HEPs involve assumptions such as complexity and stress levels which use some judgment.
Strongly Agree Agree Neither Agree nor Disagree Disagree Strongly Disagree 0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
"Risk" or "risk insights" are a form of engineering judgment.
Engineering judgment is when a determination is made based on some technical knowledge or understanding. Another knowledgeable engineer should be able to make the same conclusion. Knowledge of risk and risk insights are certainly a viable method for engineers to make decisions and conclusions based on.
Risk insights are a valuable in determining the significance of an issue of concern.
As a purist, I'm not sure I would call "risk insights" a form of engineering or engineering judgement.
Judgment is to risk as fact is to consequence.
Risk assessment and risk insights involve some judgment but rely primarily on calculation via a process. I think of "engineering judgment" as a less rigorous process involving experience and informed opinion but without a strong basis in specific calculation.
Risk analysis does require some technical judgment.
An engineer can identify components that are important to a system performing its function from drawings and procedures; think qualitative. Risk staff have methods of assigning quantitative risk importance numbers to a component. If both activities are done correctly, there should be a direct and proportional relationship between both activities.
"Engineering Judgment" is based on applying engineering principle derive from the application of physics and other scientific principles. "Risk" or "risk insight" is derive form the theory of "chance". Its just a rolling the dice or the witch doctor throwing the bones.
Risk or risk insights leverages operating experience, design, and technology to more efficiently inform due diligence.
Risk quantifications involve assumptions which are better informed by engineering judgment.
Risk evaluations are far from perfect, and while we can postulate what may happen, what actually happens is often never exactly what we postulate. Alternatively, engineering judgment is often far from perfect, but can oftentimes get you to what ends up to be the right answer.
PRA is complex and in development judgment must be used. Competent analysts can arrive at significantly different risk numbers despite both using applicable techniques.
Question 6: The ROP and Enforcement guidance have resulted in unintended and unreliable outcomes in dispositioning issues of concern.
Other (please specify)
Strongly Agree Agree Neither Agree nor Disagree Disagree Strongly Disagree 0.00%
5.00%
10.00%
15.00%
20.00%
25.00%
30.00%
35.00%
40.00%
The ROP and Enforcement guidance have resulted in unintended and unreliable outcomes in dispositioning issues of concern.
Not is used properly. ROP is intended for Risk outcomes, Enforcement for compliance.
Management's inappropriate encouragement to ignore guidance and policies have resulted in unreliable outcomes.
There appears to be a disconnect between Enforcement Policies and a risk informed oversight program. Issues of very low (none) risk significance continue to take much time and effort to resolve with no safety benefit. Enforcement manual and managers needs to reinforce the performance-based aspect of the ROP. It's not simply compliance.
We still struggle in consistency with minor/more than minor.
I have no knowledge of our guidance prompting the staff to reach a wrong conclusion The ROP has limitations because its a defined process that employs logic steps. I believe in today's environment the logic steps need to be informed through operating experience to broaden their application.
This is a mixed bag. In some cases, they may have... in others, not so much.
Non-risk significant issues can get more regulatory attention and treatment than warranted.
I'm not sure yet. But we need to be careful as we step away from RISK INFORMED and meander down the path of RISK BASED that we don't ignore defense in depth and design bases.
no.
While there is inconsistency in the more-than-minor threshold for ROP...in general the ROP and enforcement guidance processes are clear. The problem begins when individuals and/or managers try to DEVIATE from established processes. For example....one huge and recurring deviation is the desired to use "risk informed thinking" as an excuse to deviate from established processes.
I certainly believe that when evaluating issues of minor significance or very low safety significance, there is unreliable and inconsistent outcomes. The more than minor determination is very subjective. In another area, performance indicators in the ROP rarely (other than maybe SCRAMs and power changes) result in a color other than green. Are we getting what was originally intended from PIs?
Enforcement guidance sometimes leads to documenting violations of little significance. These efforts add little to the NRC mission to protect public health and safety, and distract NRC and licensee resources that could be put toward more risk significant issues.
NRC Management, through multiple vehicles, has failed to provide consistent guidance to inspectors. It is not isolated to the ROP or Enforcement Guidance.
Over the past two years we have had almost 10 withdrawn violations.-Indian Point and Salem Baffle bolt related to Appendix B Criteria V violation
-appealed Palisades EDG testing violation in Inspection report 05000255/2017007
-SEQUOYAH NUCLEAR PLANT - NRC EVALUATION OF CHANGES, TESTS, AND EXPERIMENTS AND PERMANENT PLANT MODIFICATIONS INSPECTION REPORT 05000327/2015007 AND 05000328/2015007 - WITHDRAWAL OF NON-CITED VIOLATION
-DIFFERING PROFESSIONAL OPINION PANEL REPORT ON COOPER PROBLEM IDENTIFICATION AND RESOLUTION INSPECTION REPORT (DPO-2017-010) will result in withdrawal of 4 NCVs
-Braidwood and Byron overturned backfit
-withdrawn violation documented in COLUMBIA GENERATING STATION - WITHDRAWAL OF NON-CITED VIOLATION IN NRC INSPECTION REPORT 05000397/2016009 (https://www.nrc.gov/docs/ML1728/ML17289A336.pdf)
-wthdrawn 50.59 VIOLATION DOCUMENTED IN EVALUATION OF CHANGES, TESTS, AND EXPERIMENTS BASELINE INSPECTION REPORT 05000454/2017009, 05000455/2017009).
-withdrawn Cooper fire protection NCV NCV 05000298/2018002-01, Failure to Set Performance Monitoring Action Levels to Ensure that the Assumptions in the Engineering Analysis Remain Valid, Inspector do not exercise the TIA process because it needs to be overhauled. The TIA needs to be efficient and effective in establishing agency positions and not provide informal positions for which some of our inspection staff act upon. We also don't want inspection staff setting agency policy.
Not been my experience.
The more-than-minor questions leave too much ambiguity because they are trying to be a one size fits all approach. There should be specific guidance based on the inspection procedure or the regulation of concern.
From time to time. No process is perfect. I would characterize the deviations in Green space as an indication that there is room for improvement with respect to alignment. I am not sure that all would agree with the selection of terms. I focus more on the distribution, how often does this approach serve us well? Would a change in approach make it better or just different? How do we know or figure this out?
A lot of time and resources are expended over arguing about significance.
If decision reversals are the indicator, these guidance elements, and the range of various office perspectives may produce inconsistent and other outcomes that do not survive scrutiny of the community.
We are all over the Map on minor vs more than minor. I believe that is because management is afraid that an issue will be contested if we call it more than minor so they push the staff to not document issues.
It is commonly discussed among reactor inspectors that findings can be crafted for significance or insignificance because the ROP and enforcement guidance is commonly viewed by inspectors as subjective.
I don't really have a perspective before the ROP to compare the reliability.
It is not clear to me that the guidance for ROP and Enforcement is an issue or if it is our interpretation and application of the guidance. ROP intended for enforcement to be an outcome of the assessment process but in practice it appears we continue to focus first and primarily on violations and second on plant degraded conditions.
I think the interpretation/implementation of the guidance by individual inspectors has resulted in unintended and unreliable outcomes.
Question 7: Inspection resources are focused on the most risk significant issues.
Strongly Disagree Neither Agree nor Disagree Strongly Agree Question 8: I have concerns with expanding the use of "risk" or "risk insights" by the agency.
Strongly Disagree Neither Agree nor Disagree Strongly Agree Question 9: There is value in identifying minor and Green inspection findings.
1 2
3 4
5 6
7 8
9 0
2 4
6 8
10 12 14 16 18 Inspection resources are focused on the most risk significant issues.
1 2
3 4
5 6
7 8
9 0
2 4
6 8
10 12 14 I have concerns with expanding the use of "risk" or "risk insights" by the agency.
Strongly Disagree Neither Agree nor Disagree Strongly Agree Question 10: Do you think the current guidance or training is sufficient for you to use risk or risk insights in your decision making?
Strongly Disagree Neither Agree nor Disagree Strongly Agree Question 11: Please explain your answer above.
Responses it is not clear if management has changed direction. If management is using the terms to tell me not to follow up on an item, it would be better to tell me not to follow up instead of hiding behind "be risk informed."
1 2
3 4
5 6
7 8
9 0
5 10 15 20 25 There is value in identifying minor and Green inspection findings.
1 2
3 4
5 6
7 8
9 0
2 4
6 8
10 12 14 16 Do you think the current guidance or training is sufficient for you to use risk or risk insights in your decision making?
While I personally try to use risk and significance in my decision making, there are not a lot of resources to apply risk and risk insights directly and clearly to ISFSI activities. Therefore I rely heavily on the experiences of others and our perception of risk to apply to decision making.
For how we currently use risk, the training we receive is fine. If the definition changes, maybe changes in training would be needed.
Staff and managers are unsure what risk informed thinking and risk informed decision-making looks like.
Without a training on where the "line-in-the-sand" is, regarding risk, inspectors and reviewers etc. will use personal judgement and bias to determing how much risk a given scenario entails, and how much risk is acceptable, including how frequently that is so.
I think the terms "risk" and "risk insights" have changed and continue to change. I don't think there is a common understanding of the terms.
not used in the allegation program I don't think using risk insights is anything new on how we were always supposed to do work I accordance with the ROP. I think a lot of the confusion and apprehension is related to the gap between expectations and implementation.
The thresholds and boundaries that govern the use of risk insights have not been well defined for application.
As an SRA the answer above must be "absolutely" yes because by definition I AM using the existing guidance and training to make risk informed decisions on a daily basis.
At my level in the organization, I don't know the extent to which I need more risk-related training... at least not "risk" in the PRA sense.
risk insights is a complex subject and time consuming to truly understand. the training received is great but sometimes important information can be lost in the details. my recommendation would be to make the training an online interactive training course Inspectors do not demonstrate consistent understanding of RIT, RIDM, agency policies, etc. Therefore, I conclude that guidance (written and oral) and/or training are not adequate.
We can always benefit from training and refresher training regarding risk assessments as part of our professional decision making I don't think we are all on the same page on how to risk inform uniformly throughout the NRC. A good example would be the head butting that occurred between the Region and NRR during the LaSalle Anchor Darling HPCI valve issue. The lessons learned report seemed to indicate that we need to be more risk informed in our decision making, yet the licensee did find the wedge pin sheared and damage to the valve stem when the region brow beat them into shutting down the opposite unit and inspecting the other injection valve. Had we been more risk informed in our decision making and considered industry OpE as NRR wanted, we would have made the wrong decision.
We have guidance but it could be more clear. Ideally we could have more refresher training (NOT COMPUTER BASED) like hearing more from John Hannah and Laura Kozak.
There is a recurring push from Senior Management and Senior Reactor Analysts (SRA) to be more "risk informed"...yet when asked to provide actionable guidance, procedures or processes to implement this desire they are coming up empty. When asked to provide examples of where we are incorrectly using our resources, they usually can't and those time examples are provided they usually require inspectors to ignore established procedures, enforcement guidance or regulatory requirements.
We have sufficient risk tools available to make proper decisions. More risk tools will only serve to complicate and potentially degrade our decision making process.
While we do get some specific training periodically, we only get in-depth training once and then expect everyone to remember what was taught. Also, not everyone has a firm grasp on differences between risk assumptions and design basis assumptions.
For example, you don't need to assume single failure when looking at things that could have risk significance.
I think we can use continuing training in this area. I honestly do not know where to find the guidance for risk "informing" our decisions. We should continue to stress the "informing" part and not move to a risk "based" decision making. We need not look farther back than the financial crisis of 2008 to see what happens when we put all of our faith into risk. Stay "informed" and keep the big picture in mind.
As stated previously, NRC Management has not been consistent in providing guidance to field inspectors.
RIII SRAs have provided excellent training at counterpart meetings and on line.
Think implicit vs explicit or venn diagrams. In some places our guidance is quite clear; selection of components/systems for inspection or enforcement as examples. Guidance does not exist at the interface of compliance and risk; when is non-compliance so insignificant that it need not be addressed? If the risk is very low, perhaps the licensee could exclude correction via an amendment process. The amendment process is expensive and takes time; can we align the need with the service? Can we allow licensee's to aggregate issues of this type for a future amendment? Risk of this type needs to be managed. If an amendment is not the solution, what type of transition process do we envision?
Current guidance helps but additional examples of practical applications in both training and guidance would help me. Suggest adding some to the inspector newsletter.
The agency will provide 10 CFR 50.69 training in preparation for IP 37060 inspections.
As in inform not base decisions.
As an inspector, there are guidance to use risk to select sample for inspection. However, the minor/more than minor questions are too subjective and no guidance to use risk insights at that point.
This is an area where the average inspector will need additional training to fully understand how SSC are evaluated from a risk standpoint and how to fully incorporate that into inspection planning and significance determination for findings.
Need more specific guidance on use of risk as it applies to actual inspections. NOT inspection planning since we already incorporate PRA numbers in inspection planning. Now we need cookbook for how to incorporate risk numbers when faced with an issue of concern.
These terms are well understood in theory, but more emphasis should be placed on how to apply them - supplied by multiple real-life examples.
The current guidance and training regarding inspection activities is not alignment with the agency's risk informed decision-making initiative especially when it come to documenting and following through issue that most likely will only result in Green inspection findings.
The current guidance is literally "be more risk informed." We have been provided the risk triplet, but that only helps informs what samples to spend time in, or what items to ask more questions about. Once an issue of concern has been identified, the current guidance does not provide any help because we have a process to follow. My decision is, and will always be, to follow the established process. If the process were enhanced to be more risk informed along the way, that would help.
Yes, training is fine.
I received training from INL during P-111 & P-105 & P-108 about risk-informing my inspection but management does little to encourage risk-informing (by quantifiable means) my inspection activities. It seems more like lip service to risk-inform NRC activities, like the NRC understands what sounds nice to the public, than actual changes involving PRA to improve inspection actions.
I believe the current guidance is adequate as well as the reinforcement by management to look at risk significant issues.
However, it has been a long time since I have taken initial inspector qualifications.
Risk is involved in my duties as an external communications resource only peripherally, at best.
I think my knowledge of risk and risk insights is reasonable for me to use them.
I believe my current guidance and training would help me to find "lists" of risk significant systems/components, but I don't think I truly understand why and in what ways each of those systems is risk significant to truly be able to use that information effectively.
While guidance and training can always be improved, I think more of the issue is resistance to the use of risk and the over emphasis on compliance issues (enforcement). As and SRA, risk is always in my decision-making, but it is not always at the forefront of an inspector's mind. I think some of the inspection procedures could be improved to be focus on not just risk significant SSCs, but risk significant functions of SSCs and more of an integrated operational approach rather than a focus on design and licensing basis.
Tools are available but they are not that easy to use for a non-SRA individual I think that overall, we need more training on dispositioning findings. We need more training on minor/more-than-minor determinations.
It is not clear whether I can ignore something I consider to be low risk.
Question 12: Do you think there is a gap between management expectations and staff use of risk or risk insights?
Strongly Disagree Neither Agree nor Disagree Strongly Agree Question 13: Please explain your answer above Responses Management couldn't come to an agreement on what is minor/MTM so the term "risk informed" is being thrown out so that inspectors will make that decision. Management didn't like the idea that some plants would be impacted by compliance issues (requiring shutdowns to fix) - they could not reach consensus or a decision on how to handle these issues so they tell inspectors to be "risk informed" and hope that the straight compliance issues will go away. Solid examples with answers in writing would be nice.
It sometimes seems that management expectations when using risk or risk insights are for the staff to minimize issues or findings. I believe that overall the agency's focus on safety should not be minimized for individual issues based only on a perception of low or lower risk issues.
I don't really know what the expectation is other than to continue how we had been using risk before.
Managers and branch chiefs need to reinforce and train inspectors on how risk insights and risk judgments are appropriate inputs into RIT and RIDM.
I believe that management is aware of the diversity of training and experience and philosophy of different inspectors. However, I believe management operates on a "hope' that inspectors will use risk sufficiently to avoid safety events, while inspectors have no clear guidance on the topic.
I don't think there is common understanding of the terms. I don't think this is limited between management and staff and think it occurs between staff to staff and even manager to manager.
there has been on discussion in risk information the enforcement program.
I think the gap rests in management's expectations vs what the staff believes should be done.
I believe the application of risk insights that are not guided by clear expectations results in this gap.
1 2
3 4
5 6
7 8
9 0
2 4
6 8
10 12 14 16 18 20 Do you think there is a gap between management expectations and staff use of risk or risk insights?
There is no agreed to definition of "risk informed" etc. and at times NRC staff deliberately mis-use the term for their individual agendas. Additionally many NRC staff have no interest or desire to be better educated regarding risk despite the fact it is the lingua franca of the domestic nuclear industry.
I think the staff receives different messages from management dependent upon which organization or philosophy their manager supports.
again risk insights or use of the new regulation is complex; management has attempted to explain the use of the new regulation since more licensee's are gravitating towards implementing it; in terms of selecting certain inspections however I believe staff are using the risk insights they have from being at their specific sites-the only issue in that sense is that sites are revising their risk databases so a system that was once risk significant is showing up to be as risk significant especially with the implementation of the fukishima modifications and procedures Some inspectors express lack of knowledge of agency policies and resist management expectations. This results in uneven application of risk-informed approaches and inconsistent treatment of issues.
I am not sure.
If the resident office determines that job #1 is not as risk significant as job #2 and chooses to focus inspection resources on the second job as a result, I'm not confident that the office won't get Monday morning quarterbacked if the first job goes south and turns into a more risk significant issue.
There will be a period of time where we are in training and being reminded to move more toward Risk Insights until we can meet management expectations. Perhaps we could use more guidance on exactly HOW we can get better at risk insights?
We are continually told to be more "risk informed" but when staff asks management they can't point to any established procedure or guidance other than the PRA policy statement. However, many of our established procedures, regulatory requirements and approved licensing actions were determistically based. As a result further use of "risk" would be contrary to established regulation and/or established processes. As a result management continues to preach an expectation that can't be applied by field inspectors because in many cases it would result in unofficial / illegal application of enforcement discretion and/or regulatory relief. While management has the authority to exercise this discretion/relief (yet rarely use it), field inspectors DO NOT have this authority, but inspectors are getting pressured to exercise an authority that is beyond their legal powers. This is becoming a recurring problem, quickly approaching the level where inspectors are being pressured to "look the other way" because some violations are "low risk".
Don't know? Specifically, how would you evaluate expectations vice use of risk in decision making?
Management keeps saying "risk insights" but hasn't defined what they really mean. It comes across to the staff as management asking the staff to "bring them a rock."
I neither agree nor disagree because I do not know what management's expectations are for using risk and risk insights... other than following the guidance in the IPs.
NRC Management relies too heavily on undocumented "expectations." There are cases where some of these "expectations" are in direct opposition to written guidance in the Inspection Manual, Enforcement Policy, etc. Field inspectors are getting too many examples of NRC Management talking out of both sides of the mouth, saying one thing yet not making any effort to revise the written guidance.
There is a gap between risk informed decision making expectations and enforcement expectations.
Think implicit vs explicit. If risk can be quantified, with error bars or uncertainty, how much is acceptable? Where is that written in the procedure? If it is a process, where is this process described? Is this prosecutorial discretion vs enforcement discretion?
Where or when do we discuss the difference between these positions? Do inspectors have this authority? Should training be provided if written guidance is not?
What are he management expectations?
I don't think we're all on the same page yet. Managers probably are not aligned with each other yet. So management clearly wants staff to use risk more, but is still working to be clear regarding how staff practices at the individual level should change.
Staff use of risk also likely varies more than we want.
Management may believe the reason for appealed or withdrawn violations over the past two years is because inspectors are not risk informed but that is not the case.
I do not have a metric for either expectations or use so it is assumed to be incongruent relationship.
I think the staff would like to have more guidance before they are on the same level as management in term of the use of risk insight.
A lot of the gap will exist in training. Managers want the 50,000 foot elevation vision that the agency is using risk more broadly, but the inspectors will want to know HOW to do this effectively.
I do not clearly know the management expectations at this time to be able to answer this question.
It's hard to tell at this stage - risk informed approach is still at its childhood.
Management expectation varies among managers so the staff use is inconsistent and in fact staff is often confused regarding the expectations.
My perception is that the management expectation is for inspectors to go outside of the established agency process to apply risk informed thinking to issues of concern that have been identified and developed. That is a huge gap from how risk is to be used within our inspection process, as established by agency policy and procedures.
Management in Region III does not want to talk about Green issues. If the licensee brings the 'contest' word up, management immediately wants to drop the issue and use risk as leverage for doing so.
The preceding addresses this.
In some cases, senior managers apply too much weight to PRA numbers and fail to recognize the errors inherent to PRA models. Although managers state decisions should be risk informed as opposed to risk based, behaviors suggest that is not the case.
I don't have any experience to form an opinion I do not know the answer. I would like to think the gap is minimal.
I think we have been "risk-informed" pretty effectively the past decade, however it sounds like the recent push is to move closer toward "risk-based." And it seems that as that happens, there are non-risk significant items we will stop looking at. I could see a potential that if issues occur to some of these non-risk significant items, it may be difficult to explain to the public why we weren't looking at them.
I don't think management has fully supported the use of risk or risk insights which allows (and sometimes encourages) focus on issues from a compliance perspective rather than a risk perspective.
N/A Many times you have to take what is available to complete the required ROP items without spending an inordinate amount of time (e.g. overtime) but management expects you sometimes to cover everything.
Yes I do because sometimes management will say one thing and then when you go in the field, a fellow inspector will do another thing. I think clearer guidance, better and more frequent training (with real-life examples) will help.
Management uses the buzzwords but cannot provide concrete examples.
You asked question 11.
Question 14: Can you briefly describe what you would consider barriers to incorporating the use of "risk" or "risk insights" in your daily work?
Responses I use risk insights in my daily work. However, my definition may be different from yours.
inappropriate use or understanding of risk to inflate the significance or reduce the significance of an issue or problem.
I think risk and risk insights are currently incorporated into my daily work. More specific guidance or processes on how to incorporate risk might be helpful, but are not specifically required.
The perception that if it is not PRA based (doesn't affect frequency or magnitude of an accident, then it isn't risk based.
Inspectors and supervisors view a risk-informed inspection effort as "sample selection" rather than a continuous thinking process.
Regulatory requirements Office of Enforcement Training shortfalls
- 1. Communication - I don't think there is definition of "risk" or "risk insights" that is common amongst NRC staff. Additionally, industry may have alternate definitions for these terms.
- 2. Resource Management - Periodic inspection program assessments use the number of findings per inspection effort (hours or something similar) to allocate the inspection scope and inspection hours. This short-cut approach for assessing effectiveness does not account for licensee maintaining an acceptable performance because the NRC reviews performance.
The one barrier I see is that there are issues that are considered low risk in PRA space, but have high impact due to public perception or consequences of the event that could be overlooked if we over rely in "risk" to make decisions.
The barriers are the communication of clear expectations of application and the acceptance by inspectors in the current culture.
No barriers for myself, but cultural barriers for others would exist.
The pervasiveness of a traditional, strict-compliance mentality within NRC is a key barrier; and that mentality exists within staff and management.
additional training is needed No specific barriers to incorporating the use of "risk" or "risk insights" that I would consider notable.
Staff resistance, inadequate guidance and training, internal disagreements, disagreements between NRC and external stakeholders. Slow pace or inability to make progress and move forward.
I am not concerned about barriers, just concerned that there is consistency across the agency in application of risk principles Deterministic requirements to perform a certain number of ROP baseline inspection samples. Sometimes I want to go look at an evolution "just because" but I have to find a way to shoe horn it into an inspection procedure lest I be accused of being an "industrial tourist". I feel that you can gain a lot of insight into how an organization functions by not just watching the things that they expect you to be watching, because those things are more safety/risk significant, but also watching how they perform evolutions perceived to be not safety/risk significant.
Currently doing a job, these considerations are added tasks, which will inevitably make staff do more work in the same amount of time.
Wanting to make sure i am also properly following inspection procedures and policies when issues arise. More training is needed and written expectations and guidance Simply:
- 1) Lack of guidance and approved processes 2)Inspectors can't use "risk insights" to ignore known violations, simply because they have been determined "low risk" Inspectors are incorporating risk insights everyday during their work and inspections (sample selection, areas to focus, significance determination, etc.). However, when the "barrier" is that what inspectors are being asked to do is contrary to regulation and or established process (i.e. ignore violations) then overcoming this barrier (without amending our processes or regulations) would be incorrect, unethical and open the inspector to Inspector General investigation and maybe even liable if a problem is later identified.
No substantive barriers exist. I/We appropriately apply risk insights in daily work.
I cannot think of an example where there is a barrier to incorporating the use of risk, unless I am misunderstanding the Q.
There is a DEFINITE tendency to point blame at the inspectors for issues that go unidentified, especially in the wake of an event.
This results in inspectors perceiving the need to go after everything, even items that are not so risk significant. No inspector wants to be the one in the aftermath of an event to try and say, "Well, we didn't look at that earlier because it just wan't risk significant." Every inspector knows that once something bad happens, all bets are off.
Getting to hung up on enforcement for minor issues.
We have to be able to draw this process as a process diagram.
If the answer is 'apply at all steps,' we need another document that discusses how to apply risk at each step. provide examples.
discuss successes and failures of the application of risk. If this can not be done, we are unlikely to achieve alignment.
When I've been acting RI, the inspection program incorporate risk to a substantial degree and the SRI and I considered risk (and local site performance) in choosing what systems to inspect and when to inspect them but the types and numbers of inspections required were rigid with only tiny +/- tolerances.
It is not clear what risk insights mean. The safety-significance of performance deficiency is based on operational risk or the likelihood a design basis event occurring. It does not include probabilistic structural reliability to understand the margin of failure of an SSC.
Technical capability to completely characterize any specific condition using computer models; confidence in model fidelity; variance in model assumptions.
I use risk insight from daily work. Some of our procedures, regulations or processes are not risk-informed so consideration of risk was not recommended.
Technical/training challenge.
It will be very difficult to get the technical staff all on the same page as far as how to implement risk insights. If the guidance is vague or not consistently reinforced, everyone will just do what seems right to them.
I do not know how my daily work will change at this time.
Inspectors sticking to their tradition of how to do things can be one barrier to making a shift in paradigm.
training and lack of detailed guidance associate with risk insight during the inspection phase and issue development portion of inspections.
There is little applicability to the use in risk/risk-insights in my day to day work, therefore I don't have much proficiency in dealing with risk models.
There are several barriers. The first is that as an inspector, I verify compliance with established requirements. While I may have an idea of which requirements are more important than others, I do not have the authority of granting exemptions from regulatory requirements just because an issue is perceived to be low of low risk. I have to trust that if a requirement exist, it is important to some degree and shall be complied with. Requirements can be changed through either rulemaking or a license amendment, but it is illegal to change requirements during an inspection just because compliance with a requirement is perceived to be of low risk. There's always something I don't know or wasn't privy to when the decision was made to impose a requirement on a licensee. The second barrier is that our current policy and procedures only allow a limited use of risk during inspections. After an Issue of Concern has been identified, there is a process I must follow. The IMC 0612 Appendix B process has not been changed. Therefore, that is limiting our ability to try to be more risk informed. I will not be coerced into not following our process.
That would be unethical and illegal.
I believe that we are moving away from being risk-informed to being risk-based. We need to ensure that defense-in-depth is maintained.
There are none.
What are the tools expected or designed for risk insights? Do I have sufficient control over such tools to use for my inspections?
Can these tools travel with me on-site for inspection? Risk-informing issues, decisions, etc. involves the regional SRA practically.
Other inspectors are not provided proper tooling to implement this idea of risk-informing.
Currently, use of the licensee computer programs is difficult because of the complexity of the program. The current Sapphire models are more usable, but may have inaccuracies.
N/A - Risk is involved in my duties as an external communications resource only peripherally, at best.
The agency's process much change to allow the usage of new methods.
The licensee often puts greater emphasis on the most risk significant items, so it is more difficult to identify issues with these items unless we expend much more effort in those areas, whereas less risk significant items may not get the same attention by the licensee, so they may be more prone to having issues, but we are spending all our resources on the most risk significant items.
Also, very few of us have the right knowledge to challenge the licensee's risk evaluation, so we are basically taking the licensee's word on what is risk significant or not.
Focus on compliance.
Distrust of risk at the inspector level.
Lack of management engagement in supporting the use of risk.
The view of risk as just the "numbers" The view of risk as "too uncertain for decision-making" N/A History, culture, and by some, aversion to more methodical methods.
A barrier would be how one interprets guidance. A barrier would be unclear guidance in inspection procedures.
Our regulations are prescriptive.
I already do to the extent allowed by guidance. Have never understood why risk is considered for NOED but not for operability.
If we did it thither way around, everyone could consider path forward in the light of day instead of at 2 AM.
Question 15: Describe the difference between deterministic and risk-informed.
Responses Best by an example. Both approaches deal with the operability, functionality, reliability, capability and availability of a SSC.
Deterministic equates all internal and external initiating events with the same importance/significance. That is, a valve is suppose to open during a seismic event or during a loss of feedwater. Both openings is important and have the same weight. Risk-informed looks at probability of the internal/external initiating events, the probability that the SSC isn't functional (routine testing, maintenance, etc.) and spits out the most importance function of the SSC (valve must open) with the predominant initiating event. In the case above, it would be more "risk-informed" to verify the valve would be able to open during a loss of feedwater event and not look at conditions which was cause it not to open during a seismic event.
Actual versus potential Deterministic looks at whether something will or won't work. (i.e., in an earthquake will this equipment work or not.) Risk-informed looks at how likely something is, and what the consequences will be. (i.e., how often will this event (meteor, airplane crash, etc.) occur and what will happen.)
I'm really getting frustrated that individuals assume that deterministic areas are not risk informed. Just because it doesn't have a number value (PRA) assigned, doesn't mean it isn't risk informed. For example, in the radiation protection area, significance is often tied to dose limits, which are risk-informed themselves.
Deterministic-decisions or judgments based upon pre-established criteria and rule-based decision-making process. Risk-informed - reliance upon risk information to help discern importance or relevance of information used in decision-making.
Deterministic is a yes/no question. Risk-informed may or may not result in a decision, based on information provided.
Deterministic is the rule of law interpretation. You either do it or you don't. Risk informed presents the likelihood of the event and the potential consequences of it not working.
Deterministic is concluded from specific issues that are well understood for occurrence. Risk-informed is concluded from issues that are understood from a broader perspective with some additional input that is applied generically.
Deterministic is a very limited, structured approach that exists in a theoretical construct and is easy to understand. Risk informed is not limited, requires creative thinking, has greater realism and is very challenging.
Deterministic factors are qualitative in nature. They're usually related to less tangible, non-quantifiable issues like human performance, corrective actions, or management concerns. Risk-informed factors - while not based solely on numerical calcs, do take into account the probability of occurrence and the associated consequence of whatever's being evaluated.
risk informed is based on one's knowledge associated with where an ssc ranks risk rise as determined by the licensee's pra or ipee; deterministic is a direct impact or results of something occurring Risk-informed - Analysis that incorporates insights from PRA together with engineering based determinations.
Determimistic - Analysis based on a well defined set of parameters.
Deterministic = free of consideration of safety and risk insights; compliance-based.
Risk-informed = considers what can go wrong, how likely is it, and what are the consequences.
Risk informed is probabilistic and deterministic is yes or no Deterministic: rule based. If the rules say "A" and you do "B", you're wrong.
Risk Informed: process based. The outcome changes based on what else is going on at the time.
Deterministic is a pre-solved discreet grouping which simplifies differentiating between levels of severity whereas risk informed is open-ended probing which leads to a quantifiable level of risk associated with the loss of a function of the set of SSCs that have impact on core damage or release probabilities.
Fail to follow a procedure is deterministic vs how significant of an error in terms of actual impact on CDF.
TS violation is deterministic vs what system, for how long, was the system available is more along the lines of risk informed Deterministic: there are specific numbers of needed equipment and specific assumptions (number or types of failures) combinations considered. (in general) regardless of reliability of equipment.
Risk informed: number of needed equipment, and types of failure vary significantly based on probabilities of initiating events and assumed equipment reliability.
Sliding scale with two polar/opposite ends. The more information we have on a subject, the more we can make a deterministic solution that does not depend on risk insights. The less information we have on a topic, the more we have to rely on risk insights to make a decision.
deterministic would mean a list of criteria/ truths/ facts in order to make a conclusion. So deterministic criteria for something may be "risk informed." I believe our screening in IMC 0609 is deterministic and based on risk. i.e. if we know something is a given (we only lost one division of a two division system), then we can make a conclusion of the safety significance of an issue - e.g.
Screen to GREEN.
Deterministic items involve decisions based on established engineering precedent. The fact that some event may not be risk significant is largely irrelevant with respect to some proposed regulatory action. Things like "politics" and "optics" and "public trust and confidence" can fall into this category and impact the way we regulate. Being risk-informed means looking at the cross product of probability and consequences of an event, and using that as a basis for the decision. Within the ROP we strive to integrate the two different, but not mutually exclusive, decision making philosophies.
Deterministic decision making follows a prescribed path to resolve an issue of concern. Risk informed decision making decision making uses PRA to arrive at a the significance of an issue of concern.
Deterministic: Processes having only one outcome are said to be deterministic, their outcome is 'pre-determined.' A deterministic algorithm, for example, if given the same input information will always produce the same output information.
Inform: to communicate knowledge, i.e., to inform a prisoner of his rights
Risk-inform: to communicate knowledge about risk.
So..., we have information about risk. What we need to figure-out is what to do with it. In most cases the licensee is not provided
'specific risk limits.'
Tornado is a great example. Licensees have identified equipment necessary to shutdown and maintain the unit shutdown. Lets say an inspector identifies that some of the equipment needed to support post tornado actions is not protected by tornado? what do we do? If the CLB does not include a risk model, think TORMIS, and we can all agree that the equipment is exposed to tornado forces or missiles; one deterministic answer is the equipment must be protected to be compliant with the CLB. OR submit an amendment that includes an approved risk model. [other examples exist] The real question is what process, procedure, or standard would allow an inspector to use risk as a basis for accepting a configuration other than protected? We could talk about this for days not hours.
Deterministic compares conditions to requirements. Risk-informed looks at safety significance. Favorite example: licensee must have EDG fuel for 7 days. W/ instrument error might only have 6.98 days. Seems like a big deal in deterministic view but risk informed recognizes the issue isn't significant. For another example deterministic criteria requires two trains of SI to be operable. Risk-informed looks at probably of event requiring SI and probability of each train being able to perform its safety function when considered operable.
Risk Informed incorporates general principles of RG 1.174 and deterministic does not.
Requirement versus guidance.
Deterministic criteria are those that are absolute without consideration of probability of event happening, such as those required by regulation. Risk informed would include consideration of probability of event happening.
Purely setting criteria which may or not may not have a quantitative risk assessment vs. considering risk from a quantitative standpoint when assessing an SSC.
Deterministic suggests a definite method and acceptance criteria for determining what is acceptable and what is not.
risk informed takes into account the degree of risk associated with the outcome.
Deterministic uses probability to arrive at a numerical value for a certain property or function and this probability is simply tied to repeatability of said value over a very large population such that said value is almost 100% likely to occur. The true value will either overshoot or undershoot this determined value. Engineering judgment is then used on top of "determined" said value to arrive at a margin or sufficient safety factor. Risk informed uses a population of these true values and fits these values into probability distributions to arrive at a spread that represents all the true values obtained and will also include the "determined" value above. These true values and the probability with which they occur are fed into scenarios involving failure using engineering judgment (is 75% occurrence of true value acceptable or need 92% occurrence to be reasonable?). In the end, Deterministic sets a much more conservative bar for "acceptable" values of a known property/function and Risk Informed sharpens the data analysis to arrive at a "Reasonably acceptable" value. However, BOTH approaches use PROBABILITY.
Deterministic means established thresholds must be crossed and is objective, whereas risk-informed is subjective and can vary among inspectors depending on their opinions, experiences, and preferences.
"deterministic" is applying the if happened scenario. "Risk informed" includes a qualitative assessment of the information.
Deterministic criteria are risk independent, where as risked informed criteria are based on the level of risk associated with not satisfying a given criteria.
Deterministic means there's an established set of criteria which must be met, independent of the specifics for each individual site.
The criteria are intended to ensure safety and therefore are imposed somewhat consistently to each site. For some sites, given the specifics, meeting some one criterion may be more important than meeting another criterion. Being risk informed means the difference in importance of the criteria to be met is considered. A risk informed inspection would focus on identifying which criteria are more important from a safety perspective, and then spending more time focused on compliance with that criteria.
Risk-informed incorporates risk insights and deterministic requirements.
risk-informed is both deterministic and risk Based deterministic has a well define cutoff
Deterministic involves no randomness and produces an output for a given input but risk-informed does involve randomness.
In a deterministic approach, the analysis assumes an event occurs and that safety systems respond as designed-subject to certain criteria (eg single failure). In a risk informed approach, risk insights are used based on PRA models, that assign probabilities of occurrence of an initiating event as well as probabilities that mitigating SSCs will function as designed.
In engineering, "...a deterministic system is a system in which no randomness is involved in the development of future states of the system." In enforcement "...An approach to regulation taken by the NRC, which incorporates an assessment of safety significance or relative risk."
Deterministic is based on history and expert evaluation while risk-informed is a statistical evaluation of probabilities and consequences of something.
Deterministic is basically defense in depth, but risk informed is using probability and consequences in addition to defense in depth.
Determinstic analysis in nuclear power is an approach to safety where certain events/failures are postulated to occur and outcome is evaluated (i.e., operable, meets ECCS criteria, etc.).
Risk-informed analysis in nuclear power looks at a wider spectrum of events and failures and realistically evaluates the likelihood of various outcomes.
N/A Deterministic is what we did in the early days and risk-informed is more based on the history of systems, components, and what their failure or operation contributes to the probability of hurting people and the environment.
Deterministic - an outcome is predicable based upon input criteria.
Risk informed - making a choice based upon received difference in risk significance.
deterministic = yes or no risk informed = probably Question 16: What does functional mean with respect to screening findings?
Responses Functional is lower than operability - meaning TS may require 200 gpm; however, if the pump can deliver 195 gpm and calcs say 190 gpm is needed, the pump is functional. I don't know how management wants us to apply this to seismic, tornado and other external events.
ability to perform its intended purpose as designed when called upon Functional means whether equipment will serve it's safety function.
That the equipment can perform its safety function.
Functional - ability of SSC operate in a manner to mitigate an event or transient regardless of TS Operability. An SSC could be degraded or inoperable but yet functional.
I do not know The SSC could perform the intended actions.
It means that even though it might not meet all of its design bases (example: does not meet the code) it can still perform when called upon.
Functional means an accepted level of performance defined by controlling inputs.
Can the component, operator, structure, etc. do its job when called upon?
It means: did the degraded piece of equipment maintain its ability to perform its intended function given the condition in which it was found.
functional is anything that is not in tech specs SSC is capable of performing its function in accordance with the current licensing basis.
Functional refers to area of expertise Will perform as intended when called upon.
Capability of an risk-related object to perform as is designed during and following a design basis accident.
Still met the design functions. Sometimes there are specs to bound a condition, but a licensee may be able to show that if one spec is not met, the system is still functional because there is more than the minimum volume or flow rate, etc. There may also be thinking that if the system can meet the function for a 24 hour2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br /> period, even if not full mission time of 30 days for example that it is still functional. Same with an EDG which has only 6.95 days of fuel oil in the tank; I would consider the EDG fully functional despite not meeting a 7 days TS reqt.
will the component or system be capable of performing a specified function (but not necessarily under all design or licensing bases conditions)
Functional should mean nothing with respect to screening findings. Screening should evaluate the potential risk consequences of not correcting a finding going forward and it should not be looking backwards.
It means can the SSC still perform its PRA function.
can an SSC perform some function it is designed to perform. for inspection findings it usually refers to a safety function (can the SSC do what is described in the USFAR Chapter 8 and 15?).
In the simplest terms, will an SSC work? There's a lot that goes into that (i.e., mission time, etc.), so it's more complex than a two sentence answer. For additional details, see IMC 0326.
Functional means that a SSC is able to cause or support another SSC in causing a trip, scram, or other actuation to prevent or mitigate the release of radiation to the public.
0612 does not use the term 'function' within the screening question. I would argue that the questions imply that the issue is more than minor if there is a loss of function, a degradation in performance [not complete loss of function] that has not been identified or evaluated as yet. An error that has the potential to lead to decreased performance or loss of function [safety-concern]. Affect communications with the agency in such a way as to impede our regulatory efforts.
Able to perform the safety function of the system under required conditions for the required time.
IMC 0326 defines functional.
Ability to perform designed task under designed conditions.
Can the equipment perform its specified function?
Would the SSC have been able to meet its design with whatever degraded condition existed.
ability to perform the intended design function - although the available safety margin may be reduced from the design basis requirements.
It means those SSCs important to safety (but not necessarily classified as safety related) are able to perform their function when called upon to function. It's synonymous to the term 'operable' for the SSCs classified as safety related. With respect to screening findings, whether an equipment is 'functional' or not seems to be the most dominant factor in determining the significance of the finding. It seems very rare that once the SSCs having issues still are determined to be functional, the finding will go beyond "Green."
"Functional" means equipment functions in a way that some measure of success will occur.
Capable of performing the specified function but not necessarily meeting specified operability requirements.
Functional means there's some realistic assurance that a component would function given whatever performance deficiency is identified. The component might not be able to perform per to its design basis requirements, but it still would be able to function to a certain extent to mitigate the consequences of an accident or a subset of potential accidents.
Functional means that the component can still perform its licensing basis required function.
I would need more info SSC is capable of meeting or performing its design function regardless of observed performance deficiencies.
Functional means the SSC is capable of performing as designed with respect to those attributes related to safety or regulatory compliance.
I don't know.
I do not know.
It performs it's function NOW with whatever supporting equipment is available NOW and whatever conditions are present NOW.
In other words, not assuming worst case design basis scenario, but more realistic and current assumptions.
Functional is the ability of the SSC to perform its PRA function, which may differ from operability. Operability and PRA functionality overlap but are not the same thing. In addition, non TS SSCs are modeled in a PRA and may or may not be "functional" with respect to screening findings.
N/A It will work but not maybe with all the bells and whistles that are supposedly built into nuclear grade components (but functional usually refer to non-safety related components/systems)
Something operates even though it does not necessary conform to all design requirements.
can be used to accomplish the credited function
Region IV Risk-Informed Decision-Making Online Survey Question 1: What is your job function at the agency?
Other (please specify) do not want to say AA Physical Security Inspector Sr. Health Physicist Reactor Inspector Enforcement Specialist Team Leader MATERIALS LICENSING Investigator Operations Engineer Regional State Agreements Officer Health Physicist DRA Admin Regional Opeations Officer/Senior Reactor Inspector Health Physicist AA Site AA threat analyst Question 2: Which of the following have you had review experience with (check all that apply):
0.00%
5.00%
10.00%
15.00%
20.00%
25.00%
30.00%
35.00%
What is your job function at the agency?
Question 3: The use of risk or risk insights is appropriate for inspection or inspection planning.
Strongly Disagree Neither Agree nor Disagree Strongly Agree Question 4: The use of risk or risk insights results in an unwarranted increase or decrease in the significance of inspection findings.
50.65 50.69 NOED MD 8.3 and IMC 0309 IMC 0609 TSTF-425 Initiative 5b TSTF-505 Initiative 4b 0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
70.00%
80.00%
90.00%
100.00%
Which of the following have you had review experience with (check all that apply):
1 2
3 4
5 6
7 8
9 0
2 4
6 8
10 12 14 16 18 The use of risk or risk insights is appropriate for inspection or inspection planning.
Please explain Risk insights are more likely to be used to decrease the significance of findings than they are to increase them. There appears to be bias in the application of risk to determine significance.
Looking at risk associated with just the one specific issue can cloud the real significance of an issue. There could be external factors that increase or decrease the risk of a specific item. Many small minor issues can aggregate to increase the risk to the reactor overall.
I believe it can be a decrease in significance of inspection findings due the modeling. It is had for me to verify, although the SRAs review.
The use of risk to account for actual plant conditions results in a realistic assessment of the risk. It does not decrease conservatism created through deterministic but allows us to assess what would or could be available.
Potential Greater-than-Green issues usually get pencil-whipped away once risk and PRA gets involved because of the low probabilities of LOOPs, LOCAs, tornadoes, etc. Inspectors use the IMCs and IPs to conduct inspections, then use the PRIB to see if an inspection finding is potentially Greater-than-Green, then ask a Senior Risk Analyst for a detailed risk assessment (probably Green). If still Greater-than-Green the region goes to a Reg Conference and then finalizes the issue (usually ends as less risk significant than where you started, most times the issue ends up Green).
It depends. We are supposed to be risk-informed (i.e. risk insights), but significance determinations tend to be risk-based (i.e.
focused on the quantitative risk as the only input).
Often we accept licensee testing in order to decrease significance when that testing is based on much less rigorous, less statistically significant information than the base PRA numbers.
For example, Waterford had a finding assessed as Yellow related to water intrusion into their EDG fuel oil. They ran a test on a diesel in Iowa that survived water intrusion, and we used that as our basis to reduce the significance of the finding to green.
However, that test doesn't really show that Waterford's diesels would survive water intrusion, or even that most EDGs would survive significant water intrusion most of the time. It just showed that one similar diesel in Iowa could survive it. Using that as a basis for the significance of a finding I think is contrary to the overall method of probabilistic risk assessment.
The terms 'Risk' & 'Risks Insights' are not clearly defined in each inspectable area to make a true assessment.
Strongly Agree Agree Neither Agree nor Disagree Disagree Strongly Disagree 0.00%
5.00%
10.00%
15.00%
20.00%
25.00%
30.00%
35.00%
40.00%
45.00%
50.00%
The use of risk or risk insights results in an unwarranted increase or decrease in the significance of inspection findings.
Sometimes it seems that risk or risk insights over complicate issues, which seems to me to have an impact on significance.
I don't understand "unwarranted".
The application of risk as described in established processes should result in the appropriate characterization of inspection findings. I do not believe that it is appropriate to use the term risk or risk insights as if they are two different and separate things.
Risk insights come from the risk tools that exist. To the extent that our risk tools provide the risk insights, the significance of inspection findings would also be appropriate. That being said, when an inspector identifies a violation of regulatory requirements, the licensee must take actions to restore compliance or request an exemption from the NRC.
Using risk helps to inform the decision, but is not used in the significance of the decision unless it is greater than Green. In these cases risk, I feel, is appropriately used.
The ROP has become risk "based" rather than risk "informed." The difference being that risk is now the only thing that matters.
this question makes no sense. seems like this should be split into two questions to make any sense out of the answers.
I'm not a qualified materials or reactor inspector so I cannot answer this question.
Case in point, Palo Verde diesel explosion and TS allowed outage time.....diesel generator isn't really really needed as long as you have a trailer mounted portable aux feed pump to drive down overall risk....that philosophy doesn't make much sense to me.
The use of risk or risk insights should determine how important an issue of concern should be etc.
Significance is a relative measurement metric. Compared to what? Risk is a good yardstick, as we don't use others anymore.
If used appropriately risk insights can be helpful. I am concerned that expanding risk insights will result in gamesmanship by NRC licensees.
Question 5: "Risk" or "risk insights" are a form of engineering judgment.
Please explain CDF is a mathematical construct we agreed to interpret as having some significance. It has no physical reference. The models that produce CDF numbers are extremely dependent on a series of engineering assumptions and values derived from other models.
Strongly Agree Agree Neither Agree nor Disagree Disagree Strongly Disagree 0.00%
5.00%
10.00%
15.00%
20.00%
25.00%
30.00%
35.00%
40.00%
45.00%
50.00%
"Risk" or "risk insights" are a form of engineering judgment.
risk and risk insights are a statistical evaluation using probability. An imaginative individual can make the numbers say anything they want. The upside is that the NRC gets to review and challenge the assumptions and values to some degree.
The licensee can always sharpen their pencil. I use risk plus my experience, logic, and gut feel.
It is different form of judgement when used by knowledgeable individuals.
If we continue to allow the use of risk by licensees and they credit the additional equipment to reduce regulatory impact and oversight then they need to ensure that the equipment being credited is being maintained and available The plant determines elevated and high risk activities per their procedure so each individual plant has different definitions of risk.
If risk INSIGHTS are used as one of many inputs, then yes it is a form of engineering judgment. However, if "risk" alone is the basis for a decision, then no.
See above.
The terms 'Risk' & 'Risks Insights' are not clearly defined in each inspectable area to make a true assessment. Whether we are speaking 'Deterministic' or Statistics' or 'Objective' or 'Subjective' Engineering Judgement may not be an appropriate term.
Unless a risk model is perfect--and there is no such risk model due to variables like event frequencies which are often gross estimates--risk and risk insights are a form of engineering judgment because judgment is used in the formulation of the risk model's assumptions.
The agency and licensees have different PRA models and use different numbers, yet the NRC does not inspect the licensee's PRA model to ensure the numbers used have a sound basis.
The assumptions used in PRA models are based, in many cases, on engineering judgement.
there is engineering judgment required to interpret and apply risk insights to a particular issue.
I'm not a qualified materials or reactor inspector so I cannot answer this question.
Anytime you have uncertainty in a measurement, judgments take place. It is a subjective value.
Engineering judgment means that a knowledgeable engineer can determine if something is safe or unsafe by comparing available data to a set of criteria derived from engineering principles. Inherent in the definition of risk is an "exposure to danger."
Risk insights connotate that there is an acceptable level of danger. Not the same.
True, but I am less than 100% confident with the underlying data used for baseline PRA models...so it's useful to inform inspection activities but it is not gospel.
Multiple assumptions which go into the risk analysis which impact the results.
Question 6: The ROP and Enforcement guidance have resulted in unintended and unreliable outcomes in dispositioning issues of concern.
Other (please specify)
For issues that are potentially risk significant, the process takes too long.
The Enforcement Guidance in many areas is often too limited or too vague to be useful. The logic tying examples together is often not explained. There are too few examples, especially at SL-IV and SL-III, and they don't bound the possible enforcement actions.
An issue was identified at a site that had significant impact on the reliability of the electrical systems. Because the site had not had an abnormal number of failures, and an actual incident had not occurred yet, the issue was evaluated as low safety significance. 2 years later, a fire occurred and the safety significance was determined to be RED. A contributing cause was the issue with the licensee's electrical program that had been identified 2 years ago.
The use of minor and more than minor criteria does help inspectors focus on more risk and safety significant issues.
Management must find a way to reward/acknowledge performance so that we can get "good" appraisals without counting the number of findings or acknowledging that those who have significant safety insights to eliminate human nature to "find" safety issues.
These are harder to identify as the industry matures There can be a mismatch between ROP and traditional enforcement outcomes.
Actual Consequences versus Potential Consequences. Minor vs More-than-Minor The ROP and Enforcement guidance are not perfect, but I disagree that they have resulted in unintended and unreliable outcomes. Sometimes, I would suggest, those implementing the guidance interpret the guidance differently, which can lead to different outcomes.
I'm not a qualified reactor inspection so I cannot answer this question.
There have been issues on both sides some we make a big deal out and are really not and some we don't make a big deal about and they are a big deal. Use od risk sand risk insights will help determine how important things to safety.
The different interpretations of the guidance have resulted in unintended and unreliable outcomes.
there is room for improvement, but be careful!
Strongly Agree Agree Neither Agree nor Disagree Disagree Strongly Disagree 0.00%
5.00%
10.00%
15.00%
20.00%
25.00%
30.00%
35.00%
The ROP and Enforcement guidance have resulted in unintended and unreliable outcomes in dispositioning issues of concern.
Question 7: Inspection resources are focused on the most risk significant issues.
Strongly Disagree Neither Agree nor Disagree Strongly Agree Question 8: I have concerns with expanding the use of "risk" or "risk insights" by the agency.
Strongly Disagree Neither Agree nor Disagree Strongly Agree Question 9: There is value in identifying minor and Green inspection findings.
1 2
3 4
5 6
7 8
9 0
2 4
6 8
10 12 Inspection resources are focused on the most risk significant issues.
1 2
3 4
5 6
7 8
9 0
1 2
3 4
5 6
7 8
9 I have concerns with expanding the use of "risk" or "risk insights" by the agency.
Strongly Disagree Neither Agree nor Disagree Strongly Agree Question 10: Do you think the current guidance or training is sufficient for you to use risk or risk insights in your decision making?
Strongly Disagree Neither Agree nor Disagree Strongly Agree Question 11: Please explain your answer above.
Responses it is ok We could use additional guidance for minor/more than minor violations.
1 2
3 4
5 6
7 8
9 0
2 4
6 8
10 12 14 16 18 20 There is value in identifying minor and Green inspection findings.
1 2
3 4
5 6
7 8
9 0
1 2
3 4
5 6
7 8
Do you think the current guidance or training is sufficient for you to use risk or risk insights in your decision making?
Not enough practical guidance to allow over-tasked resident inspectors to really risk-inform their work. Wide disparities exist between NRC and licensee risk models, making risk-informing sample selection, etc. a bit fuzzy.
There are too many different opinions associated with risk and risk insights for me to be comfortable using risk and risk insight.
Each "expert" has a different opinion on what the risk should be for a given piece of equipment or component, and that risk varies depending on conditions and circumstances in the plant at any given time. Risk is not static, and must be reassessed for each independent set of circumstances. Unless we use the worst case risk at the worst time, there are too many variables to easily assess the risk for any given issue.
We could use some more training, and when I talk about training I am talking about inspectors sharing insights to ensure that there is alignment. The inspectors in the field are applying, management should be trying to understand, assist, and guide, not dictate how to do it...they are not in the field using it.
The use of risk and risk insights is still subjective and there has not been agency wide policy or training on how to consistently apply them in varying areas of regulatory oversight.
Our current guidance is useful - we need to ensure that it is applied quickly so that all the information provided is not lost.
Another way to ensure retention is to have recurring training to reinforce the concepts. Think along the lines of emergency preparedness training.
The PRA course required for IMC 1245 inspector qualifications (P-111?) gives a good overview of risk, but the inspectors don't use SAPHIRE at all once they leave the class. A refresher or at least a risk tool geared for inspectors would be beneficial.
Too many inspectors and managers misuse the phrases risk, risk insights, risk-based, and risk-informed. Until the distinction is understood by all staff through additional training, it is risky (pun intended) to claim risk-informed decisions when they are actually risk-based.
I've personally witnessed many inspectors spend an inordinate amount of time on issues that would at most be minor. That seems to run contrary to the intent of risk-informed decision making and implies that training/guidance is insufficient.
Additionally, we spend a lot of time trying to determine minor vs green for PDs when, by definition, a green PD is of very low safety significance. I think that this is due, in part, to the target of what constitutes a more-than-minor PD changing over time. I don't think all BCs and both DRP and DRS are aligned with what exactly is a more than minor PD.
What's the training for each inspectable area regarding "Risk Insights" ??
Risk consideration is always a part of my decision making. I have adequate ability to apply risk through my experience and training received over the years.
Inspector training specifically focuses on risk, and the inspection guidance is clear about focusing inspections on risk significant systems.
I feel that the decisions that my group have made on how to allocate inspection resources and activities currently utilize risk insights to maintain a focus on safety. This includes lower level issues that can identify themes and demonstrate gaps in the licensees performance, that if left unaddressed, can lead to significant safety issues.
I talk to our SRA (David Loveless) regularly for his insights. We have a good relationship, and he helps us pick samples that offer the public a good return for our baseline inspection program. That is the best "guidance" or "training" that exists.
PRA training given for qualification is bare minimum to understand how risk is assessed and incorporated by the licensee and by the agency. With a shift towards risk insights, more training should be given to inspectors, or create more SRA positions to assist.
Very little "training" has been given for 50.65, 50.69, 4b, or 5b. More and more plants are adopting these and inspectors are ill prepared to be able to inspect because of the heavy reliance on PRA. Unless you are a "junior" SRA it is unlikely you have the skill set to really do an adequate inspection of the program. Program implementation is easy enough to review, but without validating that the original assumptions, peer review, etc, were appropriately corrected the program could be massively flawed and never identified.
Yes, it is sufficient but I am not an expert.
current training on risk does not focus on how to apply insights from risk to inspection or decision making other than the SDP tool. the process would be better if it allowed inspectors and decision-makers to ignore issues of concern that appear to have minor risk.
Since I'm not a qualified materials or reactor inspector, I can not answer the above question adequately.
More PRA/SRA guidance up front when planning inspection samples would be beneficial. Perhaps similar to how the CDBI teams would use a list of top cutsets to select inspection samples.
as a senior inspector with over 14 years of inspection, I believe we need to ensure when we are making a big deal about an item of concern it is risk significant. I believe my training has prepared me to use risk and risk insights to inspect at the correct level.
My training directs me to include the SRA whenever any finding involves a potential greater-than-green finding. As such, the training I have received is adequate for this task. If I will be required to make more sophisticated decisions related to risk, then I will need additional training.
Training and experience both matter since targets and goals are changed frequency As an inspector, you are trained in using risk insight in resolution of issues. So it is sufficient, however, more guidance is necessary to determine minor/Green significance.
The guidance is reasonable.
Refresher training would be helpful The basic training inspectors receive during qualification should be expanded and refresher training developed now that more decisions are being risk-informed.
We only really use risk insights in sample selections; "dispositioning" issues simply (i.e., mostly) uses the four more-than-minor questions. Answering those questions tends to be relative to the situation at hand rather than in terms of overall risk to the plant.
If a change is proposed, current guidance is likely not sufficient nor clear.
Question 12: Do you think there is a gap between management expectations and staff use of risk or risk insights?
Strongly Disagree Neither Agree nor Disagree Strongly Agree Question 13: Please explain your answer above Responses 1
2 3
4 5
6 7
8 9
0 1
2 3
4 5
6 7
8 Do you think there is a gap between management expectations and staff use of risk or risk insights?
no I don't think there's a gap.
Frankly, management doesn't talk much about this. No real expectations set except those prescribed in the process documents (IMCs, IPs, etc).
There are several examples of management asking for changes in the risk calculation to move the result either in the more-risk or less-risk direction to better fit their perception of the significance of an issue. There are also examples of management disregarding the SDP as written to lessen the significance of apparent violations and make them Green.
Management can be a little out of touch. Some managers are not open to feedback on how the use of risk is being implemented.
Some managers think that risk significance is the only thing that matters. If it is not risk significant, "move on" is the direction I have received in the past.
Both inspectors and management cite the use of risk insights in agency activities and still come to widely divergent conclusions.
There has not been sufficient guidance on their use.
See my answer on recurring training. Inspecting is an art/acquired skill. Providing recurring training and the use of risk in the expected manner will ensure proper application so long as those providing the expectations do it consistently.
Just because you gain the title of manager does not ensure that you provide the appropriate expectations. see following answer on a recommended solution Management expects the inspectors to use "risk" and "risk insights" in accordance with the inspection manual chapters and inspection procedures, which we do.
Headquarters view vs Regions view I rarely see disconnects between the use of risk between management and staff.
No response.
In Region IV there is a clear expectation to use current processes (MD 8.3, baseline planning + risk modeling) to prioritize inspection resources and activities to ensure we are focusing on risk significant items.
No comments.
Having participated in numerous inspection de briefs, I do not sense that there is a gap.
No. This is fairly well aligned.
I do not have any experience that would give me a strong opinion on this.
inspectors believe they have no freedom to walk away from an issue that appears to be low risk; they feel they have to disposition every issue of concern, even if it doesn't result in a documented finding.
Since I'm not a qualified materials or reactor inspector, I cannot answer the above question adequately.
In some cases yes and in some cases no.
We seem to allow small things that have very little risk impact to the public become a big thing because that is the way we have done it in the pass.
Not that I'm aware of, however I am not involved in DRP decisions so it may be.
We get a new definition of minor/more than minor every year.
There is a push from industry to lessen impact of inspection findings. Industry has persuaded NRC management that we should move forward with these initiatives to reduce the impact. This is fine. However, the true intentions of the agency have not been communicated to staff, at all. For example, What is Senior Management's thoughts on reducing impact of inspection findings?
Does management want less impact or the status quo? Management should make a decision and announce to the staff what direction the agency is moving instead of delaying the decision 10 years down the road.
Generally, mgmt. makes their desires known, and the staff adjusts...so no, I doubt there is a major gap.
Management expectations are not being clearly communicated down to the working level staff.
I'm not sure management has ever explained their expectations on this point.
Staff and mgmnt expect we follow the IPs and other associated guidance.
Question 14: Can you briefly describe what you would consider barriers to incorporating the use of "risk" or "risk insights" in your daily work?
Responses none The licensee's PRA.
The use of flex equipment.
Human Error Probability.
External Event Initiators.
Time spent arguing between Operable and Functional.
Mostly differences between what "risk insights" we should consider. there are many "risk" products out there - NRC models, licensee models, etc. Which one should we use, and how? It's undefined. To us, risk informing means picking smarter samples, etc. To the industry, it's clear that risk informing the program means less inspection and enforcement. The wide difference between NRC and industry in this area is a significant barrier to progress.
In my areas of specialization licensees have site-specific unique commitments related to the protection of plant employees under routine operating conditions and protecting the public under emergency conditions. Providing adequate protection in both areas requires a coherent program that is properly maintained. A focus on only the most consequential elements of licensee programs does not ensure the coherence of their overall programs and that licensees are maintaining their entire programs. Both worker and public safety could be degraded if these programs are not properly maintained.
Time and resources. We are trying to do more inspection, with fewer inspectors. We don't have the time to perform a detailed risk assessment and complete a proper inspection of the licensee performance. If we properly utilize risk and risk insight to evaluate issues we identify, the scope of our inspections will be significantly reduce and problems will remain unidentified until they become self revealing. This would result in more reactive response, rather than the NRC being a proactive regulator.
knowledge The absence of detailed risk information in specific areas such as security, EP, and RP. In the reactor safety area, the available risk tools are not detailed enough to address gaps in programs and human performance issues.
the lack of a consistent standard or application of the risk and risk insights causes different interpretations Ensuring our reward system finds a way to reward employee performance based on something other than the significance of findings or the number of findings Ensuring managers provide more consistent expectations related to use of risk More and likely different recurring training Clarify that the loss of function of a component/train is a reduction in safety but not a loss of the function Little to no knowledge of how to use SAPHIRE.
Limited number of Senior Risk Analysts to assist with inspection items.
Licensee PRA models vs. NRC PRA models. They never match up. This requires a lot of rework, time, and money. There
should be either 1 PRA model/system that the NRC and every plant should use OR the NRC should "certify" each licensee's PRA model and then use/inspect each model.
Some cornerstones (EP, occupational RS, public RS, security) are not amenable to incorporating risk insights beyond what has already been incorporated in the inspection program and, in some cases, the SDPs. In most cases, findings in these cornerstones have no direct tie to CDF or LERF and a comparable risk metric cannot be calculated. In addition, 3 of the 4 cornerstones have a direct impact to protecting and/or informing the public, providing another basis for inspection resources, inspection scope, and finding significance to be based on more than just risk or risk insights. Just because a discipline can be further risk-informed/risk-based does not mean it should.
I think the definition for "risk" is a little short-sighted. At a higher-performing plant, a lot of work and planning goes into high risk activities. With that in mind, I think it's reasonable for inspectors to sometimes focus on the lower-risk activities that do not get as much attention from licensees, because they are more likely to be the source of mistakes and errors.
I think an over-emphasis on higher risk work can be a barrier to catching and correcting lower-risk, but easy-miss items.
Understanding Risk awareness. It should always be in the conversation during the decision-making process.
As a Senior Resident Inspector I already incorporate risk insights into my daily work. For example, many inspection samples involve selection of a system/component test or issue for review under the baseline program. Each selection takes into account risk insights, and residents should be maintaining an understanding of risk significant systems at their sites.
some amount of judgment will always be necessary, even when high quality risk information is available. There is a need to help staff better understand how such judgment is applied to decision-making None. I believe the current use of "risk" or "risk insights" in the inspection program are appropriate and provide robust guidance to ensure risk is considered.
Our ROP is risk based. Regulations (by definition) is compliance based. The two are different, but similar enough to use together.
No comments.
There are still cornerstones that rely on deterministic methods to determine the significance of findings.
This is plant specific based on how much that particular plant is using risk in their day in and day out routine. From an inspection standpoint it is very easy to incorporate risk, without understanding, by simply using the PRIB notebook. More knowledge is needed in understanding the basis - the cutset sequences if you will. This provides the details to be able to understand how and why something is risk important.
None there does not appear to be anything in our process that gives inspectors the freedom to let go of issues that do not appear to have more than minor risk; inspectors need to be trained and encouraged to let go of insignificant issues and focus their efforts/resources on finding and dispositioning more significant issues.
Not applicable to my former or current position Biggest barrier we just have not used it when it concerns determining the significant of a violation. We use it to determine what to inspect but then we stop using when determining significant of the issue.
No real barriers other than older guidance.
Management misunderstanding You cant put a number on everything.
LERF (one of the risk measures) has been interpreted out of existence. Every plant has an e-plan; and if you can show that you can evacuate its not a "LERF", no matter what the risk numbers tell you.
Lack of clearly defined risk assessment tools. Management interpretation of existing risk assessment tools. Lack of risk assessement knowledge from Regional Management, Office of Enforcement, OGC, and NRR. Reluctance to use insights from agency risk analysts because of ego.
none.
The quality and availability of risk information varies widely between licensees.
A plant is essentially in compliance or not in compliance with something. We tend to spend most of our time arguing with licensees on that point rather than if the issue is risk significant or not (or how we screen an issue). If the goal is to reduce the amount of time spent arguing with licensees on potential issues that are of low significance, they need to accept our position once we've made it or we need to drop the issue because it's of low significance (whether we're right or wrong on the disposition).
I am not an inspector; I am a threat analyst. While we do not recommend policy, we have found that management pushes back considerably when our analyses/findings on real-world events and intelligence do not conform with the current Agency practices/regulations. There is a great deal of resistance to the idea of modifying our security regulations, for example, even when there is intelligence to suggest that doing so would be appropriate/prudent. It appears as if NSIR is concerned about the potential consequence of such a threat action more than the likelihood or risk that such an action would be employed by an adversary. This is a barrier to the Agency doing more risk-informed work.
Current processes would not allow, other processes are less clear. Clarification of the process must be made to consistently apply risk insights.
Question 15: Describe the difference between deterministic and risk-informed.
Responses different terms Inspectors could be more efficient if there were simpler "risk-informed, deterministic" decision trees for assessing significance.
Deterministic = informed by important factors other than risk (i.e. NRC values, public stakeholder confidence, compliance, etc).
Deterministic: based on well-defined and extensive rules and guidelines.
Risk-informed: uses a scientific evaluation of known or potential adverse effects resulting from exposure to hazards. This process emphasizes reliance on numerical expressions of risk, qualitative expressions of risk, and characterization of the attendant uncertainties. There are inherent assumptions that can effect the accuracy of this assessment and poor assumptions won't be exposed until a failure occurs.
We have many examples of poor assumptions in the past. TMI, Davis-Besse Deterministic---yes/no answers to discrete questions Risk informed---a continuium based on the specific plant conditions, PRA based (models that the NRC doesn't routinely inspect--
-This is important)
My understanding of "deterministic" in the context of regulation is a focus on whether a rule was followed without focusing on the specific consequences or risk.
"Risk informed" to me means assessing compliance in areas that have a higher potential consequence if the rules are not followed, and focusing less on areas where the consequences would be low. But it does not mean ignoring non-compliance in those areas.
Deterministic provide design requirements, maintenance requirements, operating standards that have built-in conservatism and redundancy to ensure availability Risk-informed uses bounding statistical numbers and/or qualitative judgement to identify the likelihood that the components will perform as expected.
Deterministic: assuming an event and potential consequences occur when making decisions.
Risk-informed: taking the probability of an event occurring and potential consequences of that event into consideration when making decisions.
Deterministic focuses on the quantitative value of the risk rather than using the number as one input.
Deterministic: set/prescribed Risk-informed: altered emphasis based on associated information Hard numbers versus Soft numbers Deterministic criteria is applied to events or issues to determine if a risk threshold is exceeded. Risk-informed considerations help determine the amount of risk involved to help gauge level of the necessary action or response.
Technical Specifications are deterministic. For example, a station's TSs may have a 72 hour8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br /> action statement to restore an out of service ECCS pump or shutdown. This is a deterministic criteria. IMC 0609 Appendix M is an example of a 'risk-informed' process--sensitivity studies, qualitative considerations, and other risk insights are considered in determining the significance of an Appendix M issue.
- deterministic is "fact-based" and prescriptive
- risk-informed is the use of available risk information (what can happen? how likely is it? what would be the consequences?) to make informed judgments.
Deterministic - the use of a specified set of factors to determine the required action(inspection activity) without consideration for the value (safety significance) risk-informed - utilizing the perceived risk via modeling or some other criterion to inform action (inspection activity) with great consideration for the potential value of the output (safety significance)
Regulations are deterministic, ROP is risk based/informed. Call me at 601-437-4620 for my insights (Neil Day, GG Resident Inspector).
defense in depth versus a model that takes into consideration the probability that events take place and the severity of its effects with specific equipment unavailable to perform its safety function.
Deterministic is more of a defense in depth where as risk informed is based on the importance to core damage relative to each component failure.
One is a pass or no pass compliance check. The other is a measure of the safety impact of an out of compliance issue.
deterministic relates to compliance and pre-determined criteria for assigning significance; risk-informed relates to evaluating the specific circumstances surrounding the issue that may mitigate or exacerbate the risk associated with the issue on a case-by-case basis.
Not applicable to my former or current position Deterministic is much more of an absolute, ie. rigidly defined criterion.
Deterministic is using criteria events may include significant unplanned degraded conditions. We are suppose to use risk insights to see what type of response we the agency should have for the event. However if a event is a repetitive failure or event and the risk is low we would still potentially launch a SIT or higher.
Risk-informed is a process for making a more refined decision for the use of resources. Use the resources in areas having the highest risk to public safety in a graded manner. Deterministic decisions are based on worst-case scenarios; all resources are allotted to all issues; the entire space burns up in a fire as opposed to only certain equipment...
The difference is more realistic guesswork and analysis goes into risk informed Deterministic is not meeting design basis descriptions with single failures. Risk informed means categorizing findings based on their probability.
deterministic ~ yes/no risk-informed ~ maybe/maybe not Deterministic means a known outcome for a given scenario; risk informed means a likelihood of a certain outcome for a given scenario.
Deterministic requires that all specified requirements applicable to structures, systems and components (SSCs) be met. Risk-informed allows deviations for the deterministic requirements to be accepted based on low risk.
Deterministic: something can happen.
Risk-informed: there's a probability that something can happen and it's either low or high (or somewhere in between).
deterministic - driven by hard facts, data, statistics.
risk-informed - influenced by deterministic results where available, but allows for non-deterministic factors that are not quantified Question 16: What does functional mean with respect to screening findings?
Responses it works The SSC is "gonna work" Functional = capable of performing a defined safety function from design basis, even if degradations exist. This distinction has a direct effect on significance screening, both in minor questions and in some 0609 appendices. If industry is successful in eliminating the "functional" distinction as proposed in the operability process changes, it will cloud the process of screening findings.
Not applicable to my inspection area.
A component or system is available and capable of performing its function, but may not meet all requirements to be considered operable to meet a TS LCO (i.e. be able to function under all design basis conditions).
system can still perform its intended job, even though it may be in a degraded condition.
Capable of operating, i.e., pump flowing water, but may not have all elements for operability verified or available. Example, pump maintenance complete, clearance removed, should be able to function, but awaiting a PMT to declare operable. So, therefore capable of performing function, in the example above, water would flow, so cooling would occur.
When a SSC is functional it means it would have been able to perform its design function under the conditions that exist or existed during the time frame of concern.
Functional - the ability to perform the function. So long as either train of a designed two-train safety system is operational then the system function can be completed (e.g., low pressure safety injection).
If one train of a two-train system is out of service than that train it is not functional.
Functional also applies to the ability of equipment not in technical specifications to perform their function Functional also means that a safety component is expected to be able to operate properly but all the checks and administrative controls have not been completed and the component placed back in service and considered operational.
Functional means that a structure, system, or component (SSC) is still able to do its safety-related or important to safety function.
Examples: A pump is still able to pump some flow through the system. A fire door has a dent in it, but can still provide the 3-hr rating.
Able to perform its intended function, possibly at diminished capacity.
Would the item still work under degraded conditions.
Functional is applied to issues of concern to consider whether structures, systems, or components would have been able to do their job to a minimal level of fulfilling its objective. In other words, it may not be fully operable but its working good enough to do its job.
It depends--I would need more information to give a clear answer here. Functional could mean Technical Requirements Manual functional, maintenance rule functional, functional with respect to availability, or operable and functional.
functional means that an SSC will operate under specific circumstances, not all of which may be consistent with its design and/or licensing basis.
The ability of an SSC to perform its specified safety function as defined in the current licensing basis.
Will protect the public (for a duration) for that particular cut set scenario.
can the equipment or related support equipment perform its intended safety function as outlined in the CLB when its called upon to do so.
Functional is available to be able to do something. Not necessarily able to meet design function but able to do something.
It is capable of operating but it may not meet the definition of operable yet. For example, the pmt has not been performed but there is no reason to believe that it would not operate.
it relates to the specified safety function as described in the FSAR and is distinct from the Tech Spec definitions of operability.
Not applicable to my former or current position Could the component still perform its safety function, even in its degraded or nonconforming state?
Functional is the equipment can perform its function but may not be operable for some reason, low oil levels so can meet mission time.
Good question. It is often a subjective call by the inspector if the "functional" call is enough to ensure safety or meet the requirements.
Functional means the licensee accepts the degraded condition and resident have to accept that It looks like today that functional means "not in a million pieces on the floor" or "can be found somewhere on site."
Functional means that the component will "work" although design margin will be gone or degraded An SSC is able to perform its function but with less reliability and/or effectiveness than required for full operability. I see it as a continuum:
operable -> operable DNC -> inop but functional, inop and non-functional.
The system or component would be capable of fulfilling its safety function although not all of the deterministic requirements are being met.
For all intents and purposes, it means operable. If something (with an issue) was considered capable of performing its safety function, you could use that screening question in Appendix A to say that the finding was Green (or at least not greater than Green in terms of significance).
Current condition of the equipment is such that if called on to operate NOW, it would be expected to operate and perform the function it was designed to perform (pump so many gpm, close in so many seconds, etc...)
Integrated Results All four regions are presented on each plot, and the results are binned such that responses 1-3 are binned as disagree, responses of 4-6 are binned as neutral, and responses of 7-9 are binned as agree.