ML20293A370

PRA and Risk-Informed Decision Making at the NRC: Some Trends and Challenges (NCSU Lecture)

ML20293A370
Person / Time
Issue date:	10/22/2020
From:	Nathan Siu NRC/RES/DRA
To:
Nathan Siu
References
Download: ML20293A370 (161)

Text

PRA and Risk-Informed Decision Making at the NRC:

Some Trends and Challenges*

Nathan Siu Senior Technical Advisor for PRA Office of Nuclear Regulatory Research Nuclear Engineering Research Seminar (Virtual)

North Carolina State University, Raleigh, NC October 22, 2020

• The views expressed in this presentation are not necessarily those of the U.S. Nuclear Regulatory Commission

Thats so cool Are we Outline there yet?

• A decision making challenge

• Use of risk information at NRC

• Some PRA technology challenges (audience participation)

• Closing thoughts

• Additional material 2

Acknowledgments I would like to thank many colleagues (especially Jing Xing, James Chang, Susan Cooper, Keith Compton, Tina Ghosh, Chris Hunter, Shivani Mehta, Stacey Rosenberg, Tom Wellock, Sunil Weerakkody, John Garrick, and Andreas Bye) for their assistance and discussions in developing material for this presentation. I would also like to thank Jorge Luis Hernandez and Shahen Poghosyan (IAEA) for organizing a recent international workshop that provided current perspectives on a number of key topics. Any errors or changes in emphasis in material are my own.

3

Just for fun DAEDALUS, ICARUS, AND RISK-INFORMED REGULATORY DECISION MAKING 4

Greetings Daedalus!

3000+ Years Ago 5

MinTRC Chronology Y-450: Thera explosion, earthquake, giant waves Y-200: Earthquake, conquest Y-100: Minoan Transportation Development Agency (MinTDA)

Minoan Ministry of Transportation (MinMoT)

Y-75: MinTDA =>

Minoan Transportation Regulatory Commission (MinTRC)

Y-5: Developer complaints to King Minos (impediments to innovation)

Today

Daedalus Proposal

• Design concept: human-powered wings (imitate birds)

• Construction: feathers attached by wax and thread/twine, assembly bent into slight curve

• Procedures: oral

- Dont fly too high (scorching) or too low (damp feathers)

- Stick together; dont navigate by stars or constellations

• Testing:

- Demonstration of principle

- Two-person flight north

A Risk-Informed Decision Making Problem Defense-in-

• Current regulations: none applicable depth Current regulations Safety margins

• Defense-in-depth: none

• Safety margins: unknown, heavy Integrated Decision reliance on Daedalus skill Making

• Risk assessment: possible scenarios recognized but incomplete (see next)

Monitoring Risk

• Performance monitoring: possible Adapted from RG 1.174

Risk Assessment Concerns Models Risk Metrics Scenarios

• Completeness of risk metrics Structural impacts

• • Additional failure Public healthDeity Scorching Known Action and safety

• Correctness of models for ?Unknowns Wax CALL failure (heat, Pilotcold) 1-800-HELP-RES Environment Inexperience

? Thread/twine failure Construction identified scenarios Post-accident

• Lift failure Long-Flight Logistics Dampness Culture External Events

• Other possible scenarios

• Unsanctioned

? Fatigue Conceivability uses of technology Mechanical Failure

• and Pilotpotential error consequences

• Peer review (Daedalus) Mid-Air Collision Flight parameters Navigation Design Error Irradiance vs Altitude Unknown 1400 Altitude Effects?

Irradiance (W/m2)

Unknowns 1300 1200 1100 Mt Ida Clear Air Turbulence Modern 1000 hang gliders, scenario identified by Daedalus Modern0 hang gliders, 5000 scenario 10000not identified Possible, scenario not identified Altitude (m)

It is of the highest importance in the art of detection decision making to be able to recognize, out of a number of facts, which are incidental and which vital. Otherwise your energy and attention must be dissipated instead of concentrated.

- With apologies to Sherlock Holmes (The Hound of the Baskervilles)

NRC USE OF RISK INFORMATION 10

NRC Use of Risk Information Regulatory guidance established Building Confidence Regulatory policy established Technology available Modern Applications Expansion Across Industry Full PRA desired but not feasible* Early PRAs Hanford to WASH-1400 1940 1950 1960 1970 1980 1990 2000 2010 2020

• See T.R. Wellock, A figure of merit: quantifying the probability of a nuclear reactor accident, Technology and Culture, 58, No. 3, 11 678-721, July 2017.

NRC Use of Risk Information Triplet Definition of Risk (Kaplan and Garrick, 1981)*

Features Risk {si , Ci , pi }

• Vector, not scalar

• Qualitative and

• What can go wrong? quantitative

• What are the consequences?

• Differences across

• How likely is it? accident spectrum

• Adopted by NRC. See:

- White Paper on Risk-Informed and Performance-Based Regulation (Revised), SRM to SECY-98-144, March 1, 1999

- Glossary of Risk-Related Terms in Support of Risk-Informed Decisionmaking, NUREG-2122, May 2013

- Probabilistic Risk Assessment and Regulatory Decisionmaking: Some Frequently Asked Questions, NUREG-2201, September 2016 12

NRC Use of Risk Information NRC Uses of Risk Information PRA Policy Statement (1995)

Not just design

• Increase use of PRA technology in all approval regulatory matters Regulations and Guidance - Consistent with PRA state-of-the-art

- Complement deterministic approach, support defense-in-depth philosophy Licensing

• Benefits:

Operational Decision Experience Support and (1) Considers broader set of potential challenges Certification (2) Helps prioritize challenges (3) Considers broader set of defenses U.S. Nuclear Regulatory Commission, Use of Probabilistic Oversight Risk Assessment Methods in Nuclear Activities; Final Policy Statement, Federal Register, 60, p. 42622 (60 FR 42622), August 16, 1995.

13

NRC Use of Risk Information Risk-Informed Regulatory a philosophy whereby risk Decision Making (RIDM) Recentare insights Application considered (2019) together In any licensing withreviewotherorfactorsother to establish regulatory requirements decision, the staff should Defense-in-depth better focusprinciples that risk-informed apply licenseewhen Current regulations Safety margins and regulatory strict, attentionofon prescriptive application design and criteria deterministic operationalsuch asissuesthe single failure criterion commensurate withis unnecessary their to provide for reasonable importance to publicassurancehealth Integrated Not just risk Decision Making of and adequate safety. protection

[Emphases of public information health and safety.

added]

Staff Requirements White Paper on-Risk-Informed SECY-19-0036 - Application and of the Single Failure Criterion toRegulation, Performance-Based NuScale PowerSECY-98-144, LLCs Inadvertent Actuation Block Valves, SRM-SECY-19-0036, July 2, 2019.

Monitoring Risk January 22, 1998.

Adapted from RG 1.174 Adapted from: U.S. Nuclear Regulatory Commission, An Approach for Using Probabilistic Risk Assessment in Risk-Informed 14 Decisions on Plant-Specific Changes to the Licensing Basis, Regulatory Guide 1.174, Revision 3, January 2018.

NRC Use of Risk Information Multiple Products and Uses Adapted from NUREG-2150 Not just immediate Risk Information decision support

• Results

• Insights

• Explanations

• Uncertainties

• Qualifications 15

NRC Use of Risk Information Some Trends

• Market forces

- Increased number of applications

- Increased credit for capabilities (e.g., FLEX)

- Greater role in design (e.g., LMP)

• Novel designs, technologies, and operational concepts Photo courtesy of NEA Halden Reactor Project

• Improving analysis capabilities

- Computational resources

- Smart technologies (e.g., content analytics)

Challenge to NRC:

• Changing workforce (KSAs, preferences) Be Ready!

16

it is incumbent upon the new industry and the Government to make every effort to recognize every possible event or series of events which could result in the release of unsafe amounts of radioactive material to the surroundings and to take all steps necessary to reduce to a reasonable minimum the probability that such events will occur in a manner causing serious overexposure to the public.

W. F. Libby (Acting Chairman, AEC) - March 14, 1956 response to Senator Hickenlooper

[from D. Okrent, Reactor Safety, U. Wisconsin Press, 1981]

Audience Participation SOME PRA TECHNOLOGY CHALLENGES*

• In this presentation Technology = {methods, models, tools, data}

17

PRA Technology Challenges Identifying Challenges: Many Perspectives Fukushima Review* IAEA Technical Meeting** Uncertainty Typology***

• PRA scope (2)

• Dynamic PSA (7)

• Parameter uncertainty (3)

• Feedback loops (1)

• Combinations of hazards (3)

• Model uncertainty (6)

• Game over modeling and

• Portable equipment (4)

• Completeness uncertainty (8) intentional conservatisms (4)

• Use of PSA in development of + Internal risk communication (9)

• Long duration scenarios (3) SAMGs (6)

• External hazards analysis (12)

• Level 3 PSA (4) Pick a topic

• Human reliability analysis (HRA) (5)

• Software reliability and

• Representation of uncertainty in modelling (4) phenomenological codes (1)

• Incorporation of ageing

• Searching (vs. screening) (5) aspects (5)

• N. Siu, et al., PSA technology challenges revealed by the Great East Japan Earthquake, PSAM Topical Conference in Light of the Fukushima Dai-Ichi Accident, Tokyo, Japan, April 15-17, 2013.

[ML13038A203 (paper), ML13099A347 (presentation)]

18 **IAEA Technical Meeting on the Enhancement of Methods, Approaches and Tools for Development and Application of Probabilistic Safety Assessments, September 29-October 2, 2020.

• • • M. Drouin, et al., Guidance on the Treatment of Uncertainties Associated with PRAs in Risk-Informed Decisionmaking, NUREG-1855, Rev. 1, 2017.

PRA Technology Challenges [Fukushima Review]

Topics

• PRA scope

• Feedback loops

• Game-over modeling

• Long-duration scenarios

• External hazards analysis

• Human reliability analysis (HRA)

• Representation of uncertainty in phenomenological models

• Searching (vs. screening)

Qualitative lessons from a PRA-oriented review PRA TECHNOLOGY INSIGHTS FROM 3/11 19

PRA Technology Challenges [Fukushima Review]

PRA Technology Insights from 3/11*

• Review: 2013, updated 2016

Purpose:

support ongoing activities (Level 3 PRA, R&D planning, international discussions)

• Scope: all affected plants

• Approach

- Literature review

- Event review

• Timeline-based

• PSA-topic based

• Results: PRA-technology Reminders, Challenges, and discussions of selected topics

• See:

1) N. Siu, et al., PSA technology challenges revealed by the Great East Japan Earthquake, PSAM Topical Conference in Light of the Fukushima Dai-Ichi Accident, 20 2)

Tokyo, Japan, April 15-17, 2013. [ML13038A203 (paper), ML13099A347 (presentation)] (used for this presentation)

N. Siu, et al., PSA technology reminders and challenges revealed by the Great East Japan Earthquake: 2016 update, 13th International Conference on Probabilistic Safety Assessment and Management (PSAM 13), Seoul, Korea, October 2-7, 2016. [ML16245A871 (paper), ML16270A522 (presentation)]

PRA Technology Challenges [Fukushima Review]

PRA Scope (1/2)

Dimension Typical U.S. (c. 2011) Observations (3/11/2011)

Space - Single unit (reactor) - Multiple reactors, SFP; multiple sites

- Frontline mitigating systems + support - Additional systems (e.g., security access)

Time - At power operation - Shutdown operations (incl. testing)

- Accident - Post-accident susceptibility Organization - Onsite staff - Offsite involvement (directions, requests for information)

Improve Damage posture control Permanently restore site Reduce vulnerabilities, plan, Stabilize Identify and promulgate Reduce vulnerabilities, plan, prepare for effective response site lessons prepare for effective response Warning Event time Preparedness Response Recovery Reconstitution Preparedness

+ Analysis 21

PRA Technology Challenges [Fukushima Review]

PRA Scope (2/2): Multi-Site Events March 11, 2011 August 14, 2003 22

PRA Technology Challenges [Fukushima Review]

Feedback Loops (1/1)

Onsite actions Containment venting inhibited by radiation, delayed until local explosion effects evacuation confirmed General Analysis Flow Analysis Refinements 23

PRA Technology Challenges [Fukushima Review]

Game Over Modeling (1/4)

• Plant Level: Loss of AC and DC Core

- Conventional PRA analysis: core Plant Damage Onset (hr) damage (if AC power is not Peach Bottom (NUREG/CR-7110)* 1.0 recovered) Fukushima Daiichi Unit 1 19

- Deterministic analysis: rapid Fukushima Daiichi Unit 2 89 onset of fuel damage Fukushima Daiichi Unit 3 52

• Unmitigated short-term station blackout (STSBO). See N. Bixler, et al.,

• System Level: Loss of DC State-of-the-Art Reactor Consequence Analyses Project Volume 1:

Peach Bottom Integrated Analysis, NUREG/CR-7110, Rev. 1, 2013.

- Isolation condenser, RCIC, and HPCI fail (unable to control)**

• • DC power is used to operate a number of control valves. Uncontrolled RCIC and/or HPCI could lead to RPV overfill, water in the RCIC/HPCI steam line, and 24 failure of the RCIC/HPCI turbine(s). During the accident, operators bypassed some flow through a mini-flow test line (which returns water to the Condensate Storage Tank) to reduce the possibility of RPV overfill.

PRA Technology Challenges [Fukushima Review]

Game Over Modeling (2/4): Loss of DC 25

PRA Technology Challenges [Fukushima Review]

Game Over Modeling (3/4): RIDM Implications Fire

• Useful simplification for applications focused on Internal Events total results Internal Flooding Seismic

• Concerns High Winds External Flooding

- Potential overemphasis on scenarios that are actually not as important as others (masking effect)  ?

• Training resources

• Establishing expectations (bias) Fire

- Strong constraints on mitigation actions considered Internal Events Internal Flooding as viable, worth emphasizing (e.g., through Seismic procedures and training) High Winds

- Loss of PRA model credibility to key stakeholders External Flooding 26

PRA Technology Challenges [Fukushima Review]

Game Over Modeling (4/4): Other Notes

• Assuming immediate failures is not necessarily conservative

- in reality, lacking omniscience, operators might spend time trying to implement a non-feasible path

• Other common game over modeling assumptions

- Lack of credit for recovery or repair

- Assumed loss of structure contents on failure of structure 27

PRA Technology Challenges [Fukushima Review]

Long Duration Scenarios (1/3): Fukushima Early Data from multiple sources, including:

1) International Atomic Energy Agency, The Fukushima Daiichi Accident: Report by the IAEA Director General, STI/PUB 1710, Vienna, Austria, 2015.

28 2) Government of Japan, Investigation Committee on the Accident at the Fukushima Nuclear Power Stations of Tokyo Electric Power Company, Interim Report.

December 26, 2011.

PRA Technology Challenges [Fukushima Review]

Long Duration Scenarios (2/3): Fukushima Late Data from multiple sources, including:

1) International Atomic Energy Agency, The Fukushima Daiichi Accident: Report by the IAEA Director General, STI/PUB 1710, Vienna, Austria, 2015.

29 2) Government of Japan, Investigation Committee on the Accident at the Fukushima Nuclear Power Stations of Tokyo Electric Power Company, Interim Report.

December 26, 2011.

PRA Technology Challenges [Fukushima Review]

Long Duration Scenarios (3/3): Modeling Challenges

• Recovery and repair

- Human reliability analysis (HRA)

- Site and equipment conditions (debris, roads, tools, spares, housing, )

• Non-binary behavior (e.g., intermittent and/or degraded performance)

• Offsite

- Conditions (site access, demands on emergency services, )

- Organizational response Yuriage - Before and After 3/11 Tsunami 30

PRA Technology Challenges [Fukushima Review]

External Hazards Analysis (1/12)

• 3/11/2011: Seismically-induced loss of offsite power, tsunami-induced loss of all power and multiple severe accidents

• Long-standing general approach, e.g.,

- Zion/Indian Point PRAs (1982)*

- PRA Procedures Guide (1983)*

• Typical practice

- General emphasis on internal events, earthquakes, internal fires and floods

- Other external hazards (including external floods) sometimes dismissed (pre-3/11)

• Typical results

- Important or even dominant contributor to risk

- Uncertainty driver: hazards analysis

• See:

1) B.J. Garrick, Lessons learned from 21 nuclear plant probabilistic risk assessments, Nuclear Technology, 84, No. 3, 319-339(1989) 31 2) American Nuclear Society and the Institute of Electrical and Electronics Engineers, PRA Procedures Guide, NUREG/CR-2300, 1983.

PRA Technology Challenges [Fukushima Review]

External Hazards Analysis (2/12): Past PRA Results An early study (c. 1980) NUREG-1150 (1990)

Note: Orders-of-magnitude uncertainties 32

PRA Technology Challenges [Fukushima Review]

External Hazards Analysis (3/12): Past PRA Results IPEEE vs IPE CDF External vs. Internal CDF (SAMA) 1.E-03 1.E-03 1.E-04 1.E-04 IPEEE CDF External 1.E-05 1.E-05 1.E-06 1.E-06 1.E-07 1.E-07 1.E-07 1.E-06 1.E-05 1.E-04 1.E-03 1.E-07 1.E-06 1.E-05 1.E-04 1.E-03 IPE CDF Internal Note: External includes internal fires 33

PRA Technology Challenges [Fukushima Review]

External Hazards Analysis (4/12): A Fukushima Precursor Le Blayais (December 27, 1999)

• Wind-driven waves + major storm surge

- Overtop and sweep around dike, damage dike, flood site

- Flood waters pass through penetrations, burst an internal fire door, and flood key areas

• System impacts

- Loss of offsite power (LOOP) at Units 2 and 4 E. de Fraguier, Lessons learned from 1999 Blayais flood: overview of EDF flood risk management

- Unit 1 service water degraded plan, NRC Regulatory Information Conference,

- Units 1 and 2 low-head safety injection and Rockville, MD, March 9-11, 2010.

containment spray pumps lost

- Site access lost 34

PRA Technology Challenges [Fukushima Review]

External Hazards Analysis (5/12): PFHA*

Background Potomac River (Little Falls, VA)*

• Parameter of interest: frequency of major flooding () Date Flood Height (ft) 5/14/1932 15.25

• Prior state-of-knowledge: minimal 2/27/1936 14.69

• Evidence: 12 major floods over 1932-2019 (87 years) 3/19/1936 28.10

, 0 4/28/1937 23.30

• Bayes Theorem: 1 , = 10/30/1937 15.62 0 , 0 10/17/1942 26.88

• Posterior state-of-knowledge: Poisson Non-informative 4/29/1952 14.17 8/20/1955 17.60 probability density 05 = 0.079/yr 6/24/1972 22.03 prior 50 = 0.13/yr 11/7/1985 17.99 95 = 0.21/yr 1/21/1996 19.29 posterior mean = 0.14/yr 9/8/1996 17.84 0.00 0.05 0.10 0.15 0.20 0.25 0.30 Major Flood Frequency (/yr)

• Notes:

1) PFHA = probabilistic flooding hazards analysis 35 2) Data from: https://water.weather.gov/ahps2/crests.php?wfo=lwx&gage=brkm2&crest_type=historic

3) Major Flood: height > 14 ft

PRA Technology Challenges [Fukushima Review]

External Hazards Analysis (6/12): PFHA Background Potomac River Flooding (Little Falls, VA) 30 28 70 26 60 24 Flood Height (ft) 50 22 40 20 Count 30 18 20 16 10 Major Flood 0 14 9 12 15 18 21 24 27 30 More Moderate Flood 12 Flood Height (ft) 10 1930 1940 1950 1960 1970 1980 1990 2000 2010 2020 Notes:

1) Data from: https://water.weather.gov/ahps2/crests.php?wfo=lwx&gage=brkm2&crest_type=historic 36 2) Major Flood: height > 14 ft; Moderate Flood: 12 ft < height < 14 ft

PRA Technology Challenges [Fukushima Review]

External Hazards Analysis (7/12): PFHA Challenge Potomac River Floods (Little Falls): 1932-2019 1.00 Weibull CCDF Moderate Flood 0.80 Weibull pdf (scaled)

Major Flood Gumbel CCDF 0.60 Gumbel pdf (scaled)

P{X > x} Empirical CCDF 0.40 0.20 Beyond historical experience:

0.00 5 10 15 20 25 30 how to estimate Flood Height (ft) for RIDM?

37

PRA Technology Challenges [Fukushima Review]

External Hazards (8/12): PFHA Challenge 38

PRA Technology Challenges [Fukushima Review]

External Hazards Analysis (9/12): PFHA Challenge 39

PRA Technology Challenges [Fukushima Review]

External Hazards Analysis (10/12): Lessons from Some Other Flood-Related Operational Events*

• Qualitative review of 5 floods and 5 storms

• Observations:

Confirmatory Less discussed

• Multiple hazards

• Multiple shocks

• Asymmetrical multi-unit impacts

• Scenario dynamics

• Less-than-extreme hazards

• Geographical extent and

• Hazard persistence potential for multi-site impacts

• Failure of mitigation SSCs

• Failure of implicitly considered SSCs

• Warning times and precautionary measures

• HRA and emergency response complexities

• See: N. Siu, et al., Qualitative PRA insights from operational events, 14th International Conference on Probabilistic Safety Assessment 40 and Management (PSAM 14), Los Angeles, CA, September 16-21, 2018. [paper (ML18135A109), presentation (ML18249A340)]

PRA Technology Challenges [Fukushima Review]

External Hazards Analysis (11/12): Lessons from Some Seismically-Initiated Operational Events*

• Qualitative review of 50 events

• Observations:

- Reported PGAs << max values considered in recent PRAs

- Other than offsite power, no direct damage to major mitigating systems due to ground motion; major effects due to induced hazards (fire, external flood)

- Some reactor trips/safety system actuations for events with very low onsite PGAs

- If one unit affected, typically all units onsite also affected

- Some events affected multiple sites: Impacts at Multiple Sites # Events

- Reactivity effects: Minimal response** or greater 10

• Flux-induced trips (Onagawa, 1993; North Anna, 2011)

Reactor trip 3

• Stuck control rod (Kashiwazaki-Kariwa, 2007)

Serious challenge 1

• See: N. Siu, et al., Qualitative PRA insights from seismic events, 25th Conference on Structural Mechanics in Reactor Technology (SMiRT-25), Charlotte, NC, August 4-9, 2019. [paper (ML19162A422), presentation (ML19210D835)]

41

PRA Technology Challenges [Fukushima Review]

External Hazards Analysis (12/12): Challenges Treatment of consecutive events

• Hazards analysis (and pre-conditioning)?

- Relevance of historical data

• Natural trends

• Man-made trends

- Need for knowledgeable experts

- Role of simulation

- Combination of hazards https://commons.wikimedia.org/wiki/File:Storms_Lothar_and_Martin_december_1999.png

- Technical cultures

• What is the hazard (varying points of view) Role of stochastic simulation?

• Buy-in for risk assessment (especially rare events)

• Fragility analysis

- Full range of hazards (dynamic loads, clogging, )

• Plant response analysis

- Human reliability analysis (HRA)

- Dynamics Hurricane tracks adapted from University of Wisconsin-Milwaukee (https://web.uwm.edu/hurricane-models/models/archive/)

Emergency response based on data from National Hurricane Center:

(https://www.nhc.noaa.gov/1992andrew.html) 42

PRA Technology Challenges [Fukushima Review]

HRA (1/5): Observations from 3/11

• Error of commission (isolation of 1F1 Isolation Condenser)

See next slides

• Psychological impacts

• External interventions in decision making

- Seawater injection

- Containment venting

• Uncertainty in plant conditions

- Loss of instrumentation

- Loss of access

- Loss of communication systems => messengers (with associated delays for transit, reporting)

• Evolving conditions (radiation, explosions, evacuating staff and contractors) affecting recovery actions 43

PRA Technology Challenges [Fukushima Review]

HRA (2/5): Current PRA and EOCs OPERATOR TERMINATES ISOLATION CONDENSER BUT OPERATION ISO-XHE-EOC-TERM WHY?

44

PRA Technology Challenges [Fukushima Review]

HRA (3/5): Adding EOCs (1) Cognitive Basis (2) Dynamic Context*

Understanding Decisionmaking Detection Action Execution Interteam Coordination

• Approaches to address context in a classical event tree/fault tree PRA framework include:

1) L. Podofillini, V.N. Dang, O. Nusbaumer, and D. Dres, A pilot study for errors of commission for a boiling water reactor using the CESA method, 45 2)

Reliability Engineering and System Safety, 109, 86-98 (January 2013).

C. Picoco and V. Rychkov, Advanced thermal-hydraulic simulations for HRA, IAEA Technical Meeting on Enhancement of Approaches and Tools for the Development and Application of Probabilistic Safety Assessments (Virtual), September 29-October 2, 2020.

PRA Technology Challenges [Fukushima Review]

HRA (4/5): The Human Dimension (3/11)

• Decision maker frustrations Yoshida was asked if he opened up the accident management manual and used it

- Limitations of available accident management as a reference. He said he never referred to it or even opened it up.

guidance He explained how ineffective measures

- Offsite organizational interventions thought up by people beforehand can be.

• Staff stressors Yoshida also explained that nuclear plants in Japan were designed with priority

- Progressive loss of situation awareness and control placed on internal factors leading to malfunctions. He went on to explain that

- Onsite conditions (aftershocks, tsunami warnings, no thought was given to malfunctions occurring simultaneously at a number of radiation, dark, debris, open manholes, ) plants due to external factors, such as

- ERC conditions (food, sleep, sanitation, ) tsunami, tornado, a plane crash or an act of terrorism.

- Offsite conditions - The Yoshida Testimony (2014)

The Yoshida Testimony: The Fukushima nuclear accident as told by plant manager Masao Yoshida, Asahi Shimbun, 2014.

46 (Available from: http://www.asahi.com/special/yoshida_report/en/)

PRA Technology Challenges [Fukushima Review]

HRA (5/5): Beyond Fukushima

• Other analysis concerns NRC, SPAR-H INL, SPAR-H

- Need for improved qualitative analysis (little stories)

• Basis for analysis assumptions

• Qualitative dimension of risk: what can go wrong

- Treatment of new situations

• Ex-MCR (particularly portable equipment) Same method, different teams

• Level 2

• Event and conditions assessment NRI, CREAM NRI, DT+ASEP

- Collection and use of empirical data

• Integrated Human Event Analysis System (IDHEAS)

• Scenario Authoring, Characterization, and Debriefing Application (SACADA) Same team, different methods

• A RIDM concern: recognition and treatment of model A Bye, et al., International HRA Empirical Study, NUREG/IA-0216, 2011.

uncertainty - more benchmarks?

Some IDHEAS and SACADA references:

1) Y.J. Chang and J. Xing, The general methodology of an Integrated Human Event Analysis System (IDHEAS) for human reliability analysis method development, 47 13th International Conference on Probabilistic Safety Assessment and Management (PSAM 13), Seoul, Korea, October 2-7, 2016. (ML16298A411)

2) Y.J. Chang, et al., The SACADA database for human reliability and human performance, Reliability Engineering & System Safety, 125, 117-133 (2014).

PRA Technology Challenges [Fukushima Review]

Representation of Uncertainties (1/1)

• Post-3/11 Fukushima Forensic study (SNL, ORNL):*

- Reconstructs accident progression at Units 1-3 and Unit 4 SFP

- Key challenge: accident data gaps and uncertainties

- Demonstrates that current tools (MELCOR, TRACE) and modeling approaches can reproduce general trends, with good quantitative agreement in portions of the results

• Questions

- How to incorporate findings into a PRA? Into RIDM? Uncertainty Frameworks and

- How to represent and communicate analysis uncertainties? Typologies

• Challenges

- Subject complexity

• Subjective (Bayesian) vs.

- Multiple purposes Objective (frequentist)

- Personal and discipline viewpoints, sometimes strongly held

• Aleatory/Epistemic

• Parameter/Model/Completeness Additional discussion on parameter, model, and completeness uncertainty

• Probabilistic vs. Non-Probabilistic

• R. Gauntt, et al., Fukushima Daiichi Accident Stuy (Status as of April 2012), SAND2012-6173, Sandia National 48 Laboratories, July 2012.

PRA Technology Challenges [Fukushima Review]

Searching (1/5): Active Supplement?

What is needed to cause a reactivity

• Typical PRA approach for identifying external hazards: excursion?

systematically generate possibilities, then screen

• Post-3/11 observations

- IPEEE guidance* allowed screening of external floods based on deterministic, design-basis considerations How can an earthquake cause a

- ASME/ANS PRA standard addenda (2009 and 2013) reactivity excursion?

allowed similar screening

- The Blayais flood (1999) can be viewed as a non-seismically induced precursor to the Fukushima Daiichi reactor accidents

• Active searches for hazards and hazard combinations (red teaming) might support efficient identification

- Logic-based approaches (e.g., Master Logic Diagram, Heat Balance Fault Tree, STAMP/STPA, )

- Functional classifications

- Operational experience

• U.S. Nuclear Regulatory Commission, Procedural and Submittal Guidance for the Individual Plant Examination of External Events (IPEEE) 49 for Severe Accident Vulnerabilities, Final Report, NUREG-1407, 1991.

PRA Technology Challenges [Fukushima Review]

Searching (2/5): Example Deductive Approach Heat Balance Fault Tree 50

PRA Technology Challenges [Fukushima Review]

Searching (3/5):

External Hazards Scenario-Based Classification Example 51

PRA Technology Challenges [Fukushima Review]

Searching (4/5): Empirical Experience Accidents Some Significant* U.S. Precursors Year Plant(s) Precursor? Year Plant(s) Notes 1979 TMI Davis-Besse (1977) 1975 Browns Ferry Worst precursor Fire => loss of U1 ECCS 1986 Chernobyl Leningrad (1975) 1978 Rancho Seco Next worst precursor 2011 Fukushima Blayais (1999) Human error (maintenance) => loss of NNI, LOFW 2002 Davis-Besse Most recent significant precursor Multiple human/organizational faults

=> RPV head corrosion

• Per Accident Sequence Precursor (ASP) program 52

PRA Technology Challenges [Fukushima Review]

Searching (5/5): Other Interesting Events Year Plant(s) Scenario Type Notes 1957 Windscale 1 (UK) Fire Graphite fire in core, release to environment.

Power cable fire, loss of main feedwater, pressurizer safety 1975 Greifswald 1 (East Germany) Fire valves fail to re-seat.

Partial loss of offsite power (LOOP) and subsequent loss of 1977 Gundremmingen A (East Germany) LOOP/LOCA cooling accident (LOCA) with internal flooding.

Turbine Building fire spreads into Main Control Room, collapses 1978 Beloyarsk 2 (Soviet Union) Fire Turbine Building roof.

1981 Hinkley Point A-1, A-2 (UK) External Flood; LOOP (weather) Severe weather LOOP and loss of ultimate heat sink (LOUHS).

1982 Armenia 1 (Soviet Union) Fire Fire-induced station blackout (SBO).

1989 Vandellos 1 (Spain) Fire Fire-induced internal flood.

1991 Chernobyl 2 (Soviet Union) Fire Fire-induced Turbine Building roof collapse.

1993 Narora 1 (India) Fire Fire-induced SBO.

1993 Onagawa 1 (Japan) Reactivity Excursion Seismically-induced reactivity excursion.

1999 Blayais 1, 2 (France) External Flood Severe weather LOOP and partial LOUHS.

2001 Maanshan 1 (Taiwan) LOOP (Weather); Fire (HEAF) Severe weather LOOP and subsequent SBO.

Pickering 4-8; Darlington 1, 2, and 4; Bruce 3, 4, and 6 (Canada);

2003 Fermi 2 , Fitzpatrick, Ginna, Indian Point 2 and 3, Nine Mile LOOP (weather) Northeast Blackout.

Point 1 and 2, Oyster Creek, Perry (U.S.)

2004 Madras 2 (India) External Flood Tsunami-induced LOUHS.

2009 Cruas 2-4 (France) External Flood LOUHS due to flood debris.

Fukushima Dai-ichi 5-6, Fukushima Dai-ni 1-4, Onagawa 1-3, Earthquake- and tsunami-induced incidents (in addition to 2011 Tokai Dai-ni, Higashidori 1-2 (Japan) External Flood accidents at Fukushima Dai-ichi 1-3).

53

PRA Technology Challenges [IAEA TM]

Topics

• Dynamic PSA

• Combinations of hazards

• Portable equipment

• Use of PSA in development of SAMGs

• Level 3 PSA

• Software reliability and modelling

• Incorporation of ageing aspects Perspectives on selected advanced PRA topics IAEA TECHNICAL MEETING (TM) ON PSA TECHNOLOGY ENHANCEMENT: TOPICS 54

PRA Technology Challenges [IAEA TM]

IAEA TM on PSA Technology Enhancement*

• Technical Meeting on the Enhancement of Methods, Approaches and Tools for Development and Application of Probabilistic Safety Assessments (virtual), September 29-October 2, 2020

• Objectives:

- Discuss current status of and Member State experience with advanced PSA methods, approaches and applications

- Update information on relevant topics in a draft technical report on advanced PSA approaches and applications.

• Highlighted topics (candidate areas for enhancements): Why?

- Dynamic PSA

- Combinations of hazards A:

Analyses Analyses (for application)

- Portable equipment

- Development of SAMGs T: Technology Technology (provides capability)

- Level 3 PSA

- Software reliability and modelling K: Knowledge Knowledge (provides context)

- Incorporation of ageing aspects 55 *Including state-of-practice as well as technology

PRA Technology Challenges [IAEA TM]

Dynamic PRA (1/7): Reality Local evacuation confirmed, 1st team dispatched Govt Start prep orders 2nd team dispatched, turned back (radiation) for venting venting Unsuccessful attempts to open AO-90 Open AO-72 1.0 manual venting of Pressure (MPa) wetwell Containment Venting:

• Prevents catastrophic 0.5 lower head failure failure pressurization from core steam dome

• Causes release to relocation to lower head drywell wetwell environment RPV-TEPCO steam line rupture DW-TEPCO WW-TEPCO 0.0 0 5 10 15 20 25 30 3/11/2011 Time (hr) 14:46 Adapted from: R. Gauntt, Fukushima Daiichi Accident Study: MELCOR Analyses and Results, OECD/NEA Fukushima Accident Analysis Workshop, Issy-les-Moulineaux, France, June 18-20, 2012.

56 See also R. Gauntt, et al., MELCOR Simulations of the Severe Accident at the Fukushima 1F1 Reactor, ANS Winter Meeting and Nuclear Technology Expo, San Diego, CA, November 11-15, 2012.

PRA Technology Challenges [IAEA TM]

Dynamic PRA (2/7): Classical Representation 57

PRA Technology Challenges [IAEA TM]

Dynamic PRA (3/7): Basics Dy*nam*ics, n. a branch of mechanics

• Risk {scenarios, consequences, likelihoods} that deals with forces and their relation primarily to the motion but sometimes also to the equilibrium of bodies

• PRA: likelihood expressed using probabilities Typical Modeling Approaches

• Dynamic PRA:

• State-transition (cell-to-cell)

• Dynamic event trees

- A simple view: PRA that explicitly models

• Direct simulation system dynamics (what not how) Related Names/Concepts

• Integrated Deterministic-Probabilistic

- Typically envisioned as a form of direct Safety Assessment (IDPSA) simulation but doesnt have to be

• Integrated Safety Assessment (ISA)

• Computational risk assessment (CRA)

- Not intended to address dynamically changing

• Integrated PRA (I-PRA)

• Simulation modeling (e.g., discrete PRAs (e.g., risk monitors) event simulation) 58

PRA Technology Challenges [IAEA TM]

Dynamic PSA (4/7): Benefits of Enhancement

• Analyses (anticipated, potential) Long history (starting with fast rx)

- Advanced reactor design approvals

- Operating reactor risk-informed applications (e.g.,

FLEX, security)

- External hazards scenarios (e.g., flooding)

- Severe accidents J.M. Lanore, et al, Progress in Methodology for Probabilistic Assessment of Accidents: Timing of

• Technology A. Amendola and G. Reina, Event Sequences and Consequence Spectrum: A Methodology for Probabilistic Accident Sequences," ANS/ENS Intl Mtg Probabilistic Transient Analysis," Nuclear Technology, 77, 297-315(1981). Risk Assessment, Port Chester, NY, 1981.

- Improved realism (fewer modelling approximations)

- Reduced completeness uncertainties (e.g., EOCs, passive systems)

- Improved synergy (other fields, educational trends)

• Knowledge

- Improved risk insights (margins, contributors)

See also draft white paper (ML19066A390) and presentation from 2019 IAEA workshop 59 (ML19248C656)

PRA Technology Challenges [IAEA TM]

Dynamic PSA (5/7): Status and Technical Challenges

• Current status

- U.S.: technology development, multiple demonstrations

- International: some applications

• Technical Challenges

- Phenomenological models (particularly operating crews)

- Data

- Computational requirements (for complex scenarios)

- Treatment of uncertainties

- Post-processing for insights 60

PRA Technology Challenges [IAEA TM]

Dynamic PSA (6/7): Modeling Operating Crews Based on data from A Bye, et al.,

International HRA Empirical Study, NUREG/IA-0216, 2011.

61

PRA Technology Challenges [IAEA TM]

Dynamic PSA (7/7): Implementation Challenge Not always complicated

• Challenge: transition from R&D to RIDM support

- Conflicting goals: technology advancement vs. problem solving Computational capabilities + investments (e.g.,

DOE, TSOs, universities) => time to reap benefits?

- Communication and perception T. J. McIntyre and N. Siu, "Electric Power Recovery at TMI-1, A Simulation Model," ANS/ENS Intl Topical Mtg Thermal Reactor Safety, San Diego, California, February 2-6, 1986.

- Investment requirements, TAF cost/benefit

- Cultural resistance 62

PRA Technology Challenges [IAEA TM]

Combination of Hazards (1/3): Background

• Historical treatment

- Hazards (earthquakes, fires, floods, high winds, aircraft crashes, offsite industrial accidents, ) typically one-at-a-time

- Specific combinations (seismically-induced fires and floods) recognized and addressed in PRA guidance

• Increased interest in broader consideration following the Fukushima Daiichi reactor accidents The Great Lisbon Earthquake (November 1, 1755)

(3/11/2011) https://commons.wikimedia.org/wiki/File:Lissabon-2.jpg American Nuclear Society and the Institute of Electrical and Electronics Engineers, PRA Procedures Guide, NUREG/CR-2300, 1983.

J.A. Lambright et al., "Fire Risk Scoping Study: Investigation of Nuclear Power Plant Fire Risk, Including Previously Unaddressed Issues, " NUREG/CR-5088, 1989.

63

PRA Technology Challenges [IAEA TM]

Combination of Hazards (2/3): Benefits of Enhancement

• Analyses (ongoing)

- Current risk-informed applications (specific combinations)

• Technology

- Improved realism (correlation of hazards)

- Reduced completeness uncertainties

- Improved synergy with natural hazards community

• Knowledge

- Additional/improved insights 64

PRA Technology Challenges [IAEA TM]

Combination of Hazards (3/3): Status and Challenges Treatment of consecutive events

• Current status (and pre-conditioning)?

- U.S.: some methods developed and used, included in PRA standards and guidance

- International: active development (focus on systematic, exhaustive approaches) https://commons.wikimedia.org/wiki/File:Storms_Lothar_and_Martin_december_1999.png

• Challenges Role of stochastic simulation?

- Efficient identification and prioritization

- Data augmentation (simulation?)

- Communication (discipline-specific frameworks - what is a hazard? Area of responsibility when performing analyses Hurricane tracks adapted from University of Wisconsin-Milwaukee (https://web.uwm.edu/hurricane-models/models/archive/)

Emergency response based on data from National Hurricane Center:

(https://www.nhc.noaa.gov/1992andrew.html) 65

PRA Technology Challenges [IAEA TM]

Portable Equipment (1/4): Early Perspectives

• McCullough, Mills, and Teller (1955): nuclear fire-fighters

• Indian Point 3 PRA (1983): alternatives to fixed measures (e.g., core catcher, filtered/vented containment)

• IPE/IPEEE (1988-2002) plant improvements:

- Portable pumps (e.g., isolation condenser makeup)

- Portable generators (battery chargers)

- Portable fans (room cooling, smoke removal)

• Severe accident management alternatives (SAMAs, 2002-2018) include similar ideas

• Reluctance to credit in PRA without incorporation in procedures and training See also presentation from 2019 IAEA workshop (ML19248C655) 66

PRA Technology Challenges [IAEA TM]

Portable Equipment (2/4): Some Pre-3/11 Events Onsite damage,

• Major External Events loss of site access,

- Hurricane Andrew/Turkey Point 3&4 (1992) offsite damage;

- Winter Storm Martin/Blayais 1&2 (1999) portable fire pumps, debris

• Major Internal Fires removal

- Greifswald 1 (1975) Loss of power and control, smoke, explosions (A);

- Armenia 1&2 (1982) temporary cables Turkey Point Turbine Deck

• Lesser events Effect of Hurricane Andrew on the Turkey Point Nuclear Generating Station from August 20-30, 1992, NUREG-1474, March 1993 (ML063550235)

- San Onofre 1 (1982): submersible pump for intake structure

- Diablo Canyon (2000): generator for switchyard battery charger

• Non-Nuclear Events Facility and infrastructure damage,

- Northridge Earthquake, M 6.7 (1994) fires, emergency service demands;

- Kobe Earthquake, M 6.9 (1995) portable generators, pre-planning, workarounds 67

PRA Technology Challenges [IAEA TM]

Portable Equipment (3/4): Benefits of Enhancement Task-Based Simulation: Old Idea, Recent Applications

• Analyses (ongoing)

- U.S.: RIDM applications (FLEX, other non-safety related equipment)

- International: PSA updates (PSR),

EOP/SAMG improvement, margins assessments

• Technology M.T. Lawless, K.R. Laughery, and J.J. Persensky, Using Micro Saint to Predict Performance in a Nuclear Power Plant Control Room: A Test of Validity and Feasibility, NUREG/CR-6159, 1995.

- Improved HRA for ex-MCR activities (possibly including task-based simulation)

- Improvements (practice, methods?) in constructing informed priors (potential)

• Knowledge

- Improved insights S. Fogarty, Approaches and Tools to Quantifying Facility Security Risk, INMM Workshop on Risk-Informed Security , Stone Mountain, GA, February 11-12, 2014.

68

PRA Technology Challenges [IAEA TM]

Portable Equipment (4/4): Status and Challenges

• Current status

- Strong interest U.S. and abroad

- Many applications (e.g., NOED, CRM, SDP, licensing actions, NTTF 2.1 seismic PRA) To SG

- Further applications being developed Mobile Pumps

- Improved HRA demonstration (IDHEAS)

• Challenges Diesel Fuel Supply To Core

- Reliability data (higher failure rates) Adapted from T. Hong, Application of Portable Equipment in PSA Model for WH-type Nuclear Power Plant, IAEA Technical Meeting on the Enhancement of

- HRA (e.g., granularity/aggregation, Methods, Approaches and Tools for Development and Application of quantification, credit for non- Probabilistic Safety Assessments, September 29-October 2, 2020.

proceduralized actions) 69

PRA Technology Challenges [IAEA TM]

Use of PSA in Development of SAMGs (1/6): Background Severe Accident

• SAMG/SAMGs = Severe Accident An accident more severe than design basis Management Guidance/Guidelines accidents; involves substantial damage to reactor core regardless of offsite consequences.*

• Severe Accident Policy Statement*

- [T]he commitment of utility management to the pursuit of excellence in risk management is of critical importance.

- 1983 draft criticized for perceived over-reliance on PRA

• SECY 88-147: NRC plan for closing severe accident issues**

• NEI 91-04, Rev. 1 (1994): industry commitment to implement SAMGs; no NRC regulation***

• U.S. Nuclear Regulatory Commission, Policy Statement on Severe Reactor Accidents Regarding Future Designs and Existing Plants, 50 FR 3218, August 8, 1985.

• • U.S. Nuclear Regulatory Commission, "Integration Plan for Closure of Severe Accident Issues," SECY 88-147, May 25, 1988.

70 ***Nuclear Energy Institute, Severe Accident Issue Closure Guidelines, NEI 91-04, Rev. 1, (formerly NUMARC 91-04), December 1994. (ML072850981)

PRA Technology Challenges [IAEA TM]

Development of SAMGs (2/6): Responding in extremis

[Before TMI] core damage was never never land

- R. Bari1 The NPS ERC [Emergency Response Center] received reports that the nuclear reactors were successively losing their power supplies and Units 1, 2 and 4 in particular had lost all of their power sources. Everyone at the NPS ERC was lost for words at the ongoing unpredicted and devastated state.

- Investigation Committee Interim Report2

[Site Superintendent] Yoshida was asked if he opened up the accident management manual and used it as a reference. He said he never referred to it or even opened it up. He explained how ineffective measures thought up by people beforehand can be.

- The Yoshida Testimony3

1) Plenary Panel: Perspectives on Nuclear Safety Since the Three Mile Island Event, ANS Intl Mtg Probabilistic Safety Assessment (PSA 2019), Charleston, SC, 2019.

2) Government of Japan, Interim Report (Main Text), Government of Japan Investigation Committee on the Accident at Fukushima Nuclear Power Stations of Tokyo 71 3) Electric Power Company), Tokyo, Japan, 2011.

The Yoshida Testimony: The Fukushima nuclear accident as told by plant manager Masao Yoshida, Asahi Shimbun, 2014. (Available from:

http://www.asahi.com/special/yoshida_report/en/)

PRA Technology Challenges [IAEA TM]

Development of SAMGs (3/6): Responding in extremis The Fukushima Daiichi accident extended over multiple days and imposed severe mental and physical fatigue on control room operators, field staff, and personnel in the plants emergency response center. Control room operators and field personnel were also exposed to physical stressors (e.g., loss of lighting and high radiation) as well as psychological stressors associated with risk to their lives and those of their co-workers and families.

- National Research Council1 we never had enough time, so the pump--the fire engine--ran out of fuel, and it could no longer pump water in when it was time to do so when reactor pressure had fallen. That gave us another letdown, and we talked about sending (workers) to pump in (water). That was when I thought we were coming to the end.

- M. Yoshida, The Yoshida Testimony2

1) National Research Council, Lessons Learned from the Fukushima Nuclear Accident for Improving Safety of U.S. Nuclear Plants, National Academies Press, Washington, DC, 2014.

72 2) The Yoshida Testimony: The Fukushima nuclear accident as told by plant manager Masao Yoshida, Asahi Shimbun, 2014. (Available from:

http://www.asahi.com/special/yoshida_report/en/)

PRA Technology Challenges [IAEA TM]

Development of SAMGs (4/6): Forms and Implications Tactical

• Forms IF [condition(s)]

- Tactical direction (procedure-like) THEN [specific action(s)]

- Strategic guidance Strategic

• PRA considerations

• Identify available means to perform function

- Scenario development (e.g., RCS (e.g., reducing containment release) conditions, site conditions)

• Identify preferred strategy to perform function

- Instrumentation survivability, Systems and lineups trustworthiness Detection means for negative impacts Limitations on uses of means

- Crew factors Special parameters to monitor

• PSFs/PIFs and effect on performance

• Direct Control Room to implement strategy

- Cognition, decision making

• Verify strategy implementation

- Execution, coordination

• Determine if challenge is being mitigated

• Crew-to-crew variability 73

PRA Technology Challenges [IAEA TM]

Development of SAMGs (5/6): Benefits of Enhancement

• Analyses

- Improved SAMGs and implementation (e.g.,

new scenarios, prioritization for training)

• Technology

- Improved HRA for post-core damage, guidance-oriented responses

- Improved phenomenological models (e.g., for severe-accident induced cascading failures)

• Knowledge

- Additional/improved insights (e.g., safety margins, priorities for severe accident R&D)

- Improved realism => improved acceptance and appreciation of Level 2 PRA 74

PRA Technology Challenges [IAEA TM]

Development of SAMGs (6/6): Status and Challenges SOARCA Assessment of B.5.b Measures

• Current status (use of PRA)

- U.S.

• SOARCA analyses of benefit

• Some changes identified during FLEX implementation

- International: widespread

• Challenges

- Level 2 PRA uncertainties

• HRA

• Severe accident phenomenology

- Scope: which scenarios/possibilities

- Appropriate realism

• Dependencies (e.g., multi-source)

• Use of Game Over modeling

- Low likelihood of event, incentives for full Level 2 PRA D. Collins, et al., Modeling Potential Reactor Accident Consequences, NUREG/BR-0359 Rev. 1, 2012. (ML12347A049) 75

PRA Technology Challenges [IAEA TM]

Level 3 PSA (1/4): Background

• Includes analysis of offsite consequences; some terminology ambiguity

- Initiating event to offsite consequences

- Release category to offsite consequences

• Scope of early PRAs

- NRC: WASH-1400 (1975), NUREG-1150 (1990)

- Industry: Oyster Creek (1979), Zion (1981), Indian Point (1982), Limerick (1982), Millstone (1983),

Seabrook (1983), Oconee (1984)

• Vulnerability analyses (IPE, IPEEE) and later:

focus on core damage and large early release 76

PRA Technology Challenges [IAEA TM]

Level 3 PSA (2/4): Probabilistic Consequence Assessment

• Key elements

- Weather and dispersion (atmospheric)

- Response (sheltering, evacuation, interdiction)

- Consequences

• Dose (individual, societal)

• Health effects (LNT, other)

• Economic

• Tools

- MACCS (U.S.)

- COSYMA (EU)

- PACE (UK)

- OSCAR (Japan) 77

PRA Technology Challenges [IAEA TM]

Level 3 PSA (3/4): Benefits of Enhancement

• Analyses

- Demonstration of Licensing Modernization Program (LMP) for operating plants (pilot)

- Regulatory analysis guidance revisions (potential)

- Applications to Emergency Planning Zone (EPZ) sizing (potential)

• Technology

- Developments in selected areas (e.g., multi-source PRA)

- Assessment of impact of more detailed models (e.g.,

dispersion)

• Knowledge

- Improved insights regarding safety margins

- Improved insights for performing analyses (risk, feasibility of and benefits from future Level 3 studies) "Risk-Informed Performance-Based Technology-Inclusive Guidance for

- Improved staff capabilities for performing and reviewing Non-Light Water Reactors," NEI 18-04, Rev. 1, August 29, 2019.

PRAs 78

PRA Technology Challenges [IAEA TM]

Level 3 PSA (4/4): Status and Challenges

• Status Level 3 PRA Project Scope

- U.S.: reference plant state-of-practice study ongoing

- International: some RIDM applications, e.g.,

• Generic design assessments (UK)

• Safety margins (Korea)

• Challenges

- Technical

• Current elements (dispersion, etc.)

• Unaddressed elements (e.g., aquatic pathways, non-radiological consequences)

• Uncertainties

- Programmatic (scope and resources) 79

PRA Technology Challenges [IAEA TM]

Software Reliability and Modeling (1/4): I&C Example

• Passive isolation condenser: flow (and cooling) controlled by opening/closing DC motor-operated isolation valve

• Possible control approaches

- Analog (relays)

- Digital

- Manual (operators) Criteria

- Combination

• Possible I&C failure modes include Power Command Hardware Signals

- Loss of signal(s) DC C Software Wetware AC power,

- Incorrect signal(s) dTRCS/dt,

- Incorrect (for situation) criteria LIC,

- Incorrect decision (signal/criteria processing)

- Incorrect command (decision implementation) 80

PRA Technology Challenges [IAEA TM]

Software Reliability (2/4): Diversity and Commonality

• Diverse views

- Software doesnt fail randomly like hardware; P{XlC,H}

important software failures are due to erroneous/inadequate requirements => improper to subjective knowledge model in classical PRA framework proposition conditions

- Software is part of an overall X-ware system (hardware, software, wetware) where similar arguments can be made about other system components => OK to model as a component at [A]ll models are wrong, but some conventional PRA level of abstraction are useful.

• Diverse problems - G.E.P. Box

- Certification of I&C systems

- Risk-informed plant design, operation, and oversight G.E.P. Box and N.R. Draper, Empirical Model-Building and Response Surfaces, John Wiley and

• Common current view: Too many items to address Sons, 1987.

deterministically => risk-informed approaches are needed 81

PRA Technology Challenges [IAEA TM]

Software Reliability (3/4): Benefits of Enhancement

• Analyses

- Licensing digital upgrades

- Approving new designs

- General risk-informed applications

• Technology

- Improved hazard identification

- Reduced completeness uncertainties

- Improved synergy with I&C community

• Knowledge

- Improved insights 82

PRA Technology Challenges [IAEA TM]

Software Reliability (4/4): Status and Challenges WGRISK Benchmarking Study

• Current status

- Conventional fault tree analyses (AP600, Sizewell

)

- U.S.: Technology development (e.g., integration of STAMP/STPA with conventional PRA)

- International: multiple activities

• Individual countries (e.g., Finland, France, Korea)

• WGRISK benchmarking study (DIGMAP)

• IAEA review

• Challenges*

- Technical

• Software CCF

• Data

- Implementation: standards and guidance M. Porthin, et al., Comparative application of digital I&C modeling approaches for PSA, International Topical Meeting on Probabilistic Safety Assessment (PSA 2019), Charleston, SC, April 28-May 3, 2019.

- Cultural

• See also S. Arndt, E. Thornsbury, and N. Siu, What PRA needs from a digital systems analysis, 6th International Conference on 83 Probabilistic Safety Assessment and Management (PSAM 6), San Juan, Puerto Rico, June 23-28, 2002.

PRA Technology Challenges [IAEA TM]

Incorporation of Ageing Effects (1/5): Background

• Conventional PRA models

- Failures are memoryless

• Fail on demand: Bernoulli process (binomial distribution)

• Failure during operation: Poisson process (Poisson and exponential distributions)

- Failure rates can be adjusted to reflect ageing

• More general model: time-dependent failure rates

- Burn-in, steady-state, ageing (degradation) => bathtub curve As good as new

= 1 0

- Different aging and repair/replacement for different SSCs

=> more complex model (Renewal Theory) 84

PRA Technology Challenges [IAEA TM]

Ageing Effects (2/5): Experiences

• Active components

- Subject to testing and renewal

- Large uncertainty bands

• Passive components

- More difficult to inspect and renew

- Subject to phenomena potentially amenable to mechanistic modeling and analysis

- Famous example: Davis-Besse (2002)

Davis-Besse Reactor Pressure Vessel Head Degradation Adapted from NUREG/BR-0353, Rev. 1 85

PRA Technology Challenges [IAEA TM]

Ageing Effects (3/5): Modeling Approaches

• Statistical

- Parametric models for failure rate (e.g., linear)

- Quantification via operational experience data

- Challenges

• Data collection (current systems are insufficient)

• Data characterization (failure? rectifiability?)

• Mechanistic Known Unknowns

- First principles causal models for SSCs Unknown Unknowns

- Challenges

• Completeness (e.g., unexpected mechanisms, combinations and M (Model of the World):

synergies; detection and response) Scope, structure

• Treatment of uncertainties i: Parameters

• Compatibility with conventional PRA framework  : Universe 86

PRA Technology Challenges [IAEA TM]

Ageing Effects (4/5): Benefits of Enhancement

• Analyses

- Risk-informed treatment of other degradation mechanisms (potential)

• Technology

- Capabilities should regulatory need arise (e.g.,

subsequent license renewal)

- Improved perspectives and approaches to mechanistic modeling (including the treatment of uncertainties)

- Improved synergy with non-PRA technical communities

• Knowledge

- Improved insights supporting awareness and prioritization of mechanisms/scenarios/mitigation measures (as compared with other risk contributors)

- Improved understanding of modeled mechanisms 87

PRA Technology Challenges [IAEA TM]

Ageing Effects (5/5): Status and Challenges

• Current status (U.S.)

- U.S.: long history of R&D, PTS application* Blending mechanistic and statistical perspectives

- International: research with demonstration changes PRA approach?

applications (European Union Aging PSA Network)

• Challenges

- Data

- Physics of failure modeling

- Recognition and treatment of other trends, e.g.,

• Technology (NDE, prognostics, )

• Workforce

• Fleet (unique reactors)

- Implementation D. Rudland and C. Harrington, xLPR Pilot Study Report, NUREG-2110, 2012. (ML12145A470)

• Separating advocacy wants from RIDM needs

• Incorporation in RIDM standards and guidance)

M. EricksonKirk, et al., Technical Basis for Revision Of the Pressurized Thermal Shock (PTS) Screening Limit in the 88 PTS Rule (10 CFR 50.61): Summary Report, NUREG-1806, 2006.

PRA Technology Challenges [Uncertainty Typology]

Topics

• Parameter Uncertainty

• Model Uncertainty

• Completeness Uncertainty

• Internal Risk Communication Perspectives on the analysis and communication of uncertainties for RIDM TREATMENT OF PARAMETER, MODEL, AND COMPLETENESS UNCERTAINTY 89

PRA Technology Challenges [Uncertainty Typology]

Parameter, Model, and Completeness Uncertainty mod*el, n. a M (Model of the World): representation of reality created with a specific Scope, structure objective in mind.

i: Parameters A. Mosleh, N. Siu, C. Smidts, and C. Lui, Model Uncertainty: Its Characterization and

Universe Quantification, Center for Reliability Engineering, University of Maryland, College Park, MD, 1995. (Also NUREG/CP-0138, 1994)

PRA models for NPPs

• Distinctions are not necessarily crisp

• Regardless of allocation to categories, need to consider Known Unknowns in characterization of Unknown Unknowns uncertainties See:

• M. Drouin, et al., Guidance on the Treatment of Uncertainties Associated with PRAs in Risk-Informed Decisionmaking, NUREG-1855, Rev. 1, 2017.

90

• U.S. Nuclear Regulatory Commission, An Approach for Using Probabilistic Risk Assessment in Risk-Informed Decisions on Plant-Specific Changes to the Licensing Basis, Regulatory Guide 1.174, Revision 3, January 2018.

PRA Technology Challenges [Uncertainty Typology]

Parameter Uncertainty (1/3): Current Practice Routinely estimated (Bayesian inference) and propagated (e.g.,

direct Monte Carlo, Latin Hypercube) 91

PRA Technology Challenges [Uncertainty Typology]

Parameter Uncertainty (2/3): Bayesian Example Potomac River (Harpers Ferry, VA)*

• Parameter of interest: frequency of major flooding () Date Flood Height (ft)

• Prior state-of-knowledge: minimal 3/19/1936 36.5

• Evidence: 10 events over 1877-2017 (140 years) 6/1/1889 34.8

, 0 10/16/1942 33.8

• Bayes Theorem: 1 , =

10/1/1896 33.0 0 , 0 11/6/1985 30.1

• Posterior state-of-knowledge: Poisson Non-informative 9/8/1996 29.8 1/21/1996 29.4 05 = 0.040/yr Probability Density prior 11/25/1877 29.2 50 = 0.069/yr 95 = 0.11/yr 4/27/1937 29.0 posterior mean = 0.071/yr 6/23/1972 27.7 return period = 12 yr 0.00 0.05 0.10 0.15 0.20 0.25 0.30

Major Flood Frequency (/yr) 1880 1900 1920 1940 1960 1980 2000

• Notes:

92 1) Data from: https://water.weather.gov/ahps2/crests.php?wfo=lwx&gage=hfew2&crest_type=historic

2) Major Flood: height > 24 ft

PRA Technology Challenges [Uncertainty Typology]

Parameter Uncertainty (3/3): Challenges

• Data pre-processing Runtime Failures (Motor-Driven Pumps)

- Selection 1.00 Probability Density Function

- Interpretation 0.80 0.60

• Effect of analysis shortcuts (Normalized) 0.40

- Standard (e.g., non-informative) 0.20 prior distributions 0.00 1.00E-09 1.00E-08 1.00E-07 1.00E-06 1.00E-05 1.00E-04 1.00E-03

- Simplified expert elicitation Failure Rate (/hr)

- Independence assumption Service Water Normally Running Standby

• Ensuring correspondence with

• 2015 Industry-wide estimates from: https://nrcoe.inl.gov/resultsdb/AvgPerf/

state-of-knowledge

• Service Water Pumps: 2 failures in 16,292,670 hours0.00775 days <br />0.186 hours <br />0.00111 weeks <br />2.54935e-4 months <br /> Normally Running Pumps: 225 failures in 59,582,350 hours0.00405 days <br />0.0972 hours <br />5.787037e-4 weeks <br />1.33175e-4 months <br />

- Basic events (micro view)

• Standby Pumps (1st hour operation): 48 failures in 437,647 hours0.00749 days <br />0.18 hours <br />0.00107 weeks <br />2.461835e-4 months <br />

- Overall results (macro view) 93

PRA Technology Challenges [Uncertainty Typology]

Model Uncertainty (1/6):

Hurricane Example Hurricane Andrew: 8/22/1992, 1200 UTC (about 2 days before FL landfall)

Hurricane tracks adapted from University of Wisconsin-Milwaukee: (https://web.uwm.edu/hurricane-models/models/archive/)

Emergency response based on data from National Hurricane Center: (https://www.nhc.noaa.gov/1992andrew.html) 94

PRA Technology Challenges [Uncertainty Typology]

Model Uncertainty (2/6):

Hurricane Example Hurricane Irma: 9/8/2017, 0000 UTC (about 2 days before FL landfall)

Hurricane tracks adapted from University of Wisconsin-Milwaukee: (https://web.uwm.edu/hurricane-models/models/archive/)

Emergency response based on data from National Hurricane Center: (https://www.nhc.noaa.gov/1992andrew.html) 95

PRA Technology Challenges [Uncertainty Typology]

Model Uncertainty (3/6): HRA Example NRC, SPAR-H INL, SPAR-H Same method, different teams NRI, CREAM NRI, DT+ASEP All teams, all methods Same team, different methods A Bye, et al., International HRA Empirical Study, NUREG/IA-0216, August 2011.

96

PRA Technology Challenges [Uncertainty Typology]

Model Uncertainty (4/6): HRA Example 1.0E+0 ASEP Team 1 1.0E-1 ASEP Team 2 Human Error Probability (HEP)

SPAR-H Team 1 SPAR-H Team 2 1.0E-2 CBDT & HCR/ORE Team 1 CBDT & HCR/ORE Team 2 1.0E-3 CBDT & HCR/ORE Team 3 ATHEANA Team 1 1.0E-4 ATHEANA Team 2 Empirical 95th Percentile Empirical 5th Percentile 1.0E-5 HFE 2A HFE 1C HFE 1A HFE 3A HFE 1B Decreasing difficulty HEPs by HFE (All Methods)

Adapted from NUREG-2156 97

PRA Technology Challenges [Uncertainty Typology]

Model Uncertainty (5/6): Current Approaches

• Important to acknowledge and treat (in context of decision)

• Standards and guidance:

characterize

• Alternatives

- Consensus model Adapted from V.M. Andersen, Seismic Probabilistic Risk Assessment Implementation Guide, EPRI 3002000709, Electric Power Research

- Sensitivity analysis Institute, Palo Alto, CA, December 2013

- Weighted alternatives (e.g., SSHAC)

- Output uncertainties 98

PRA Technology Challenges [Uncertainty Typology]

Model Uncertainty (6/6): Challenges

• Different technical points of view on treatment:

- Competition between models vs. multiple (correlated) sources of evidence

- Quantify vs. characterize

- Include or exclude user effects

• Methods to quantify model output uncertainty exist;* challenges include

- Uncertainties in unmeasured parameters M.H. Salley and A. Lindeman, Verification and Validation of Selected

- Sub-model limits of applicability Fire Models for Nuclear Power Plant Applications, NUREG-1824 Supplement 1/EPRI 3002002182, November 2016.

- Representativeness of computed results

• See, for example, E. Droguett and Ali Mosleh, Bayesian methodology for model uncertainty using 99 model performance data, Risk Analysis, 28, No. 5, 1457-1476, 2008.

PRA Technology Challenges [Uncertainty Typology]

Completeness Uncertainty (1/8)

It would cease to be a

• Sources danger if we could define it.

- Known gaps (missing scope) - Sherlock Holmes

- Unknown gaps (The Adventure of the Copper Beeches)

• Concerns Car Wont Start

- Excessive amplification (Fear of the dark)

- Excessive discounting (availability heuristic: Battery Charge Fuel System Other Engine All Other Out of sight, out of mind) Insufficient Defective Problems Problems Starting System Ignition System Mischievous Acts Defective Defective Of Vandalism Adapted from B. Fischhoff, P. Slovic, S. Lichtenstein, Fault trees:

Sensitivity of estimated failure probabilities to problem representation, Journal of Experimental Psychology: Human Perception and Performance, 4(2), May 1978, 330-344.

100

PRA Technology Challenges [Uncertainty Typology]

Completeness Uncertainty (2/8): Terminology Model

• Explicit or implicit?

• Extent of coverage? Viewpoint Known Precise classification is Gaps important only if it affects:

(Known Unknowns)

• Understanding

• Known by whom?

• Known when?

• Communication

• Time from idea to theory

• Decision making Unknown Gaps to PRA implementation?

(Unknown Unknowns) 101

PRA Technology Challenges [Uncertainty Typology]

Completeness Uncertainty (3/8): Known Gaps*

• Broad scenario categories Rationale Common Example(s)

Out of scope security/sabotage, operation outside approved limits Low significance (pre-analysis judgment) external floods (many plants pre-Fukushima)

Appropriate PRA technology* unavailable management and organizational factors PRA not appropriate software, security

• Contributors within categories Category Example(s)

External hazards multiple coincident or sequential hazards Human reliability errors of commission, non-proceduralized recovery Passive systems thermal-hydraulic reliability 102

• aka Known Unknowns

PRA Technology Challenges [Uncertainty Typology]

Multiple Hurricanes: A Known Unknown 103 https://en.wikipedia.org/wiki/Hurricane_Irma#/media/File:Irma,_Jose_and_Katia_2017-09-07.png

PRA Technology Challenges [Uncertainty Typology]

Completeness Uncertainty (5/8): Unknown Gaps*

Then (a surprise?)

Now (treated in current PRAs?)

Browns Ferry fire (1975) - a long-recognized hazard; not in draft Model WASH-1400 but routinely treated now TMI (1979) - precursors include Davis-Besse (1977); operator EOCs not in models; current recognition and some explorations Chernobyl (1986) - precursor at Leningrad (1975); non-routine test Known Gaps during shutdown in any LPSD analyses?

Blayais flood (1999) - external floods often screened at time; current recognition, multi-hazard under development Maanshan HEAF/SBO (2001) - HEAF phenomenon known, in any PRAs at time? Now included as an initiator; smoke effect?

Davis-Besse RPV corrosion (2002) - RPV failure analyses focused on Unknown Gaps crack propagation; M&O failure not in PRAs Fukushima Daiichi (2011) - precursors: Blayais (1999), Indian Ocean (2004), hazard under review at time; PRA models under development 104

• aka Unknown Unknowns

PRA Technology Challenges [Uncertainty Typology]

Completeness Uncertainty (6/8): Current Approaches

• Mind the gap

- Analysis guidance (NUREG-1855)

• Progressive analysis (screening, bounding, conservative, detailed)

• Change scope of risk-informed application

- Risk-informed decisionmaking (RG 1.174)

• Fill (or at least reduce) the gap (R&D) 105

PRA Technology Challenges [Uncertainty Typology]

Completeness Uncertainty (7/8): Role of R&D

• Continue to develop technology to address Event (NUREG/CR-4839), 1992 known gaps Aircraft impact Avalanche

- Risk-informed prioritization Coastal erosion

- Fully engage appropriate disciplines Drought External flooding

- Take advantage of general computational and Extreme winds and tornadoes methodological developments Fire

• Facilitate re-emphasis on searching Fog Forest fire

- Demonstrate efficiency and effectiveness with Frost Hail current tools (e.g., MLD, HBFT) vs. High tide, high lake level, or high checklist/screening river stage

- Develop improved tools (including OpE mining) 106

PRA Technology Challenges [Uncertainty Typology]

Completeness Uncertainty (8/8):

From Lampposts to Search Beacons Wheres the goat???

107

PRA Technology Challenges [Uncertainty Typology]

Internal Risk Communication (1/9): Context Adapted from NUREG-2150 With To Other Considerations

• Current regulations

• Safety margins

• Defense-in-depth

• Monitoring Quantitative Qualitative 108

PRA Technology Challenges [Uncertainty Typology]

Internal Risk Communication (2/9): Reminder

• Mathematically defined probability density function Mean

• Affected by tail

• Does not correspond to 0

50th (Median) = 3.9 x 10-5 /yr a specific percentile Mean = 7.6 x 10-5 /yr 95th = 2.6 x 10-4 /yr frequency (/yr) 109

PRA Technology Challenges [Uncertainty Typology]

Internal Risk Communication (3/9): Current Practice

• Often implicit (focus on mean values)

• Various graphic displays

• Includes story as well as numbers Likelihood Class 5 (10-5/yr) 4 (10-4/yr) 3 (10-3/yr) 2 (10-2/yr) 1 (10-1/yr)

A Marginal Undesirable Undesirable Critical Critical Documents and Interactive Severity Class Presentations Discussion B Marginal Marginal Undesirable Undesirable Critical (Flatland) (Storytelling) C No Action Marginal Marginal Undesirable Undesirable D No Action No Action Marginal Marginal Undesirable E No Action No Action No Action Marginal Marginal 110

PRA Technology Challenges [Uncertainty Typology]

Internal Risk Communication (4/9): Breakdowns*

• Differences in perception of information

- Relevance

- Consistency with prior beliefs

• Lack of understanding of underlying science

• Conflicting agendas

• Failure to listen

• Trust

• Sources of breakdowns with public, also at least partially relevant for internal risk communication. J.L. Marble, N. Siu, and K. Coyne, 111 Risk communication within a risk-informed regulatory decision-making environment, International Conference on Probabilistic Safety and Assessment (PSAM 11/ESREL 2012), Helsinki, Finland, June 25-29, 2012. (ADAMS ML120480139)

PRA Technology Challenges [Uncertainty Typology]

Internal Risk Communication (5/9): Information Complexity

• Hyperdimensional

- Scenarios

- Likelihood Uhh, we seem to have

- Multiple consequence measures a TEP vulnerability,

• Heterogeneous maybe, I think

- Qualitative and quantitative

- Multiple technical disciplines Thermal

• Dynamic Exhaust

- System changes (e.g., different Port operational modes, effects of decisions)

- Changing information (learning, adding/discounting data)

- New applications (and contexts)

• Uncertain

- Sparse or non-existent data

- Outside range of personal experience 112

PRA Technology Challenges [Uncertainty Typology]

Internal Risk Communication (6/9): More Complexities

• Individual user differences, e.g.,

- Knowledge Will somebody find me a

- Preferences/heuristics one-handed scientist?!

• Social factors, e.g., - Senator Edmund Muskie

- Trust (Concorde hearings, 1976)

- Decision and group dynamics I. Flatow, Truth, Deception, and the Myth of the One-Handed Scientist, October 18, 2012. Available from:

https://thehumanist.com/magazine/november-december-

• Situational context, e.g.,

2012/features/truth-deception-and-the-myth-of-the-one-handed-scientist

- Available time

- Decision support vs. informational 113

PRA Technology Challenges [Uncertainty Typology]

Internal Risk Communication (7/9): Stakeholder Trends

• Experiences, knowledge

• Information content and delivery preferences

• Comfort with analytics, risk, probability

• Mobility Language is not merely a tool for human communication; language is itself a means by which the realities of the world are divided and viewed.

- P.S. Dull, 1978 Source: https://www.nrc.gov/reading-rm/doc-collections/commission/slides/2019/20190618/staff-20190618.pdf 114 P.S. Dull, A Battle History of the Imperial Japanese Navy (1941-1945), Naval Institute Press, Annapolis, MD, 1978

PRA Technology Challenges [Uncertainty Typology]

Internal Risk Communication (8/9): Solution Trends

• Tufte model: escape Flatland using rich displays and Continuing Challenges reports, encourage user to explore

- Promotes active involvement of decision maker

• Target audience(s)

- Increases general trust? - Heterogeneous

- Changing

• A graduated technical approach to assist? - Constrained resources Interface Interaction Mode

• Schema

- Hyperlinked dashboards, reports - Manual - No standards:

currently an art Time - Video - AI assist - Solutions being

- Visual immersion developed intuitively; no scientific testing

- Multisensory immersion 115

PRA Technology Challenges [Uncertainty Typology]

Internal Risk Communication (9/9): The Future?

M. Korsnick, Risk Informing the Commercial Nuclear Enterprise, Promise of a Discipline: Reliability and Risk in Theory and in Practice, University of Maryland, April 2, 2014. Graphic adapted from https://www.flickr.com/photos/83823904@N00/64156219/

(permission CC-BY-2.0) 116

Closing Thoughts And what if the bird wont sing?

Nobunaga: Make it sing.

Hideyoshi: Make it want to sing.

Tokugawa: Wait.

- Eiji Yoshikawa (Taik)

CLOSING THOUGHTS 117

Closing Thoughts Is Winter Coming?

Anyone submitting a PRA for use in the LWR regulatory process should feel that his long-term technical reputation is on the line.

- D. Okrent (1981) 100 Reactors Increasing realism / Reducing conservatism 118

Closing Thoughts Challenge to NRC/RES and Opportunities To increase effectiveness and efficiency

• [Enterprise] risk-informed prioritization

• Consider new technical approaches

• Better target available resources (e.g., university grant funds)

• Leverage other programs

- Observe (learn, provide feedback)

- Cooperate

- Collaborate Dial 1-800-CALL-RES

• Good ideas are welcome!

119

ADDITIONAL SLIDES 120

Additional Slides PRA HISTORY: TREATMENT OF UNCERTAINTIES 121

Additional Slides [PRA History: Treatment of Uncertainties]

From Hanford to WASH-1400 Technical Challenges: 1) Quantifying accident probability

2) Means to communicate risk WASH-740 Hanford AEC/NRC Credible Accident UKAEA Estimates:

not in the generation

- OpE (pessimistic) of the ACRS members - Decomposition present (optimistic)

Recommend: Farmer Curve WASH-1400 accident System chain System reliability reliability SGHWR analysis studies studies analysis 1950 Windscale 1960 1970 TMI-2 1980 For more information: T.R. Wellock, A Figure of Merit: Quantifying the Probability of a Nuclear Reactor Accident, 122 Technology and Culture, 58, No. 3, July 2017, pp. 678-721.

Additional Slides [PRA History: Treatment of Uncertainties]

Early Views on Completeness

• W. F. Libby (Acting Chairman, AEC) - March 14, 1956 response to Senator Hickenlooper: it is incumbent upon the new industry and the Government to make every effort to recognize every possible event or series of events which could result in the release of unsafe amounts of radioactive material to the surroundings and to take all steps necessary to reduce to a reasonable minimum the probability that such events will occur in a manner causing serious overexposure to the public. [Emphasis added]

• L. Silverman (Chairman, ACRS) - October 22, 1960 letter to AEC Chairman John A. McCone: We believe that a searching analysis which is necessary at this stage [reactor siting approval] should be done independently by the owner of the reactor [Emphases added]

123

Additional Slides [PRA History: Treatment of Uncertainties]

WASH-1400 Uncertainties (Level 1)

WASH-1400: it is reasonable to believe that the WASH-1400 Uncertainties (Estimated*)

core melt probability of about 5x10-5 per reactor-year predicted by this study should not be significantly larger and would almost certainly not exceed the value value of 3x10-4 which has been estimated as the upper 5th 50th 95th Surry mean bound for core melt probability.

Peach Bottom Risk Assessment Review Group (NUREG/CR-0400):

We are unable to define whether the overall 1.E-05 1.E-04 1.E-03 CDF (/ry) probability of a core melt given in WASH-1400 is high or low, but we are certain that the error bands are *Based on data from Tables V 3-14 (PWR) and 3-16 (BWR) of Appendix V, assuming distributions are lognormal; median values are somewhat higher understated. We cannot say by how much. than reported in Section 7.3.1 of the Main Report.

124

Additional Slides [PRA History: Treatment of Uncertainties]

ACRS Concerns with WASH-1400 Methodology*

ACRS Concern Example Events[1] Post-WASH-1400 Accident initiator quantification Extensive treatment: fires, earthquakes Fukushima (Presumably external events) Inconsistent treatment: floods Atypical reactors Fermi 1 [2] Multiple PRAs for non-LWRs Many design and operational improvements identified Design errors [3] by PRAs; database includes events involving design problems Multiple methods emphasizing importance of context; Operator error quantification TMI-2 still an active area of development Consequence modeling Chernobyl, Fukushima Continuing, evolutionary improvements (MACCS)

Improved hardware database; fits and starts with Data Many HRA; extreme natural hazards a continuing challenge

• ACRS letter to Congressman Udall re: adequacy for estimating likelihood of low probability/high consequence events (Dec. 16, 1976)

Table Notes:

1. Events whose key characteristics (for the given topic) might not have been captured by a WASH-1400 vintage analysis.

125 2.

3.

Fermi 1 had limited fuel melting. However, without an analysis, it isnt clear if a WASH-1400 vintage analysis would have captured this scenario.

Design weaknesses have played a role in multiple events. More detailed review is needed to determine if: a) these are errors, and b) if they would have been missed by a WASH-1400 vintage analysis.

Additional Slides [PRA History: Treatment of Uncertainties]

Some Early Developments and PRAs Challenges: 1) Filling known gaps (completeness uncertainty)

2) Clarifying meaning: models and results Biblis Sizewell

(+aircraft)

(+DI&C) USDOE Clinch River Oyster Creek NRC (LMFBR)

Indian Point

(+seismic)

(full scope)

US Industry AIPA Forsmark International Limerick (HTGR) Koeberg Zion Millstone Other Notable

(~WASH-1400) (full scope)

Seabrook Super (full scope)

Phénix RSSMAP/IREP (FBR DHR) TMI-1 Oconee (full scope)

Apostolakis Kaplan/ (full scope)

Fleming (subjective Garrick EC/JRC Benchmarks

(-factor) probability) (risk) NUREG/CR-2300 (systems, CCF, HRA) 1975 TMI-2 1980 1985 Chernobyl 126

Additional Slides [PRA History: Treatment of Uncertainties]

Sample Level 1 Results Display 127

Additional Slides [PRA History: Treatment of Uncertainties]

Sample Results - Sub-Model Uncertainty Effect Effects of fire model (COMPBRN) uncertainty on fire growth time N. Siu, "Modeling Issues in Nuclear Plant Fire Risk Analysis," in EPRI Workshop on Fire Protection in Nuclear Power Plants, EPRI NP-6476, J.-P. Sursock, ed., August 1989, pp. 14-1 through 14-16.

128

Additional Slides [PRA History: Treatment of Uncertainties]

Sample Results - Model Uncertainty (User Effect)

Early core melt, containment cooling Early core melt, no containment cooling Damage State Frequency (/yr), Review 10-4 Late core melt, containment cooling Late core melt, no containment cooling Containment bypass Steam generator tube rupture Direct containment failure 10-6 Internal Events External Events 10-8 1.E-03 1.E-03 1.E-04 1.E-04 1.E-05 1.E-05 1.E-06 Review 1.E-06 Review 1.E-07 1.E-07 10-10 1.E-08 1.E-08 1.E-09 1.E-09 1.E-10 1.E-10 1.E-11 1.E-11 1.E-11 1.E-10 1.E-09 1.E-08 1.E-07 1.E-06 1.E-05 1.E-04 1.E-03 10-10 10-8 10-6 10-4 1.E-11 1.E-10 1.E-09 1.E-08 1.E-07 1.E-06 1.E-05 1.E-04 1.E-03 Original Original Damage State Frequency (/yr), Original Data source: G.J. Kolb, et al., Review and Evaluation of the Indian Point Probabilistic Safety Study, 129 NUREG/CR-2934, December 1982. (ML091540534)

Additional Slides [PRA History: Treatment of Uncertainties]

Severe Expansion Across Industry (US)

Accident Policy Technical challenges: 1) Characterizing the fleet (variability)

Statement 2) Developing confidence for mainstreaming RIDM Safety Goal PRA Policy NRC Policy Statement Statement US Industry GL 88-20 GL 88-20 Supplement 4 NUREG-1560 NUREG-1742 NUREG-1150 NUREG-1150 (draft) (final) 1982 ASP Plant Class Models SPAR Models IPEEEs IPEs 1985 Chernobyl 1990 1995 2000 9/11 130

Additional Slides [PRA History: Treatment of Uncertainties]

NUREG-1150 Estimated* Uncertainties (Level 1)

Model Uncertainty Model Uncertainty

• Notes: totals shown are estimated.

1) NUREG-1150 does not aggregate the hazard-specific results. The totals shown are rough estimates assuming that the NUREG-1150 distributions are lognormal.

131 2) The WASH-1400 distributions are based on data from Tables V 3-14 (PWR) and 3-16 (BWR) of Appendix V, assuming that the distributions are lognormal. The median values are somewhat higher than reported in Section 7.3.1 of the Main Report

Additional Slides [PRA History: Treatment of Uncertainties]

Parameter Uncertainties:

Industry Studies Industry results from: Garrick, B.J., Lessons learned from 21 nuclear plant probabilistic risk assessments, Nuclear Technology, 84, No. 3, 319-339(1989).

132

Additional Slides [PRA History: Treatment of Uncertainties]

Parameter Uncertainties: Logarithmic vs Linear 133

Additional Slides [PRA History: Treatment of Uncertainties]

IPE/IPEEE - Variability Across Fleet Internal Events + Internal Floods Total 40 40 BWR BWR PWR PWR 30 30 Number Number 20 20 10 10 0 0 1x10-6 3x10-6 1x10-5 3x10-5 1x10-4 3x10-4 1x10-3 1x10-6 3x10-6 1x10-5 3x10-5 1x10-4 3x10-4 1x10-3 CDF (/ry) CDF (/ry) 134

Additional Slides [PRA History: Treatment of Uncertainties]

IPE/IPEEE - Contribution of External Events IPEEE vs IPE CDF 1.E-03 1.E-04 IPEEE CDF 1.E-05 1.E-06 1.E-07 1.E-07 1.E-06 1.E-05 1.E-04 1.E-03 IPE CDF 135

Additional Slides [PRA History: Treatment of Uncertainties]

The Modern Era (US)

Technical challenges: 1) RIDM issues (e.g., realism, heterogeneity, aggregation)

SECY-98-144 2) Post-Fukushima issues (e.g., external hazards)

3) New/advanced reactors (e.g., conduct of operations)

RG 1.174 NUREG-2150 ASME PRA NRC Risk-Standard NTTF Request US Industry Informed for Information ROP NUREG-1855 (Reevaluations) 10 CFR 50.48(c)

NFPA 805 (Fire Protection) NFPA 805 LARs (Fire Protection)

SAMAs (Life Extension)

Risk-Informed License Amendment Requests (LARs)

SPAR Models 2000 9/11 2005 2010 Fukushima 2015 2020 136

Additional Slides [PRA History: Treatment of Uncertainties]

SAMA - Contribution of External Events External vs. Internal CDF (SAMA) 1.E-03 1.E-04 External 1.E-05 1.E-06 1.E-07 1.E-07 1.E-06 1.E-05 1.E-04 1.E-03 Internal Note: External includes internal fires 137

Additional Slides [PRA History: Treatment of Uncertainties]

Variability in Recent Results (Level 1) 0.35 0.30 Population Mean:

4.7x10-5 0.25 Fraction of Plants 0.20 0.15 0.10 Lowest Highest Reported: Reported:

0.05 3.5x10-6 1.3x10-4 0.00

-6.0 -5.5 -5.0 -4.5 -4.0 -3.5 -3.0 1E-6 1E-5 1E-4 1E-3 CDF (per reactor year) 138

Additional Slides [PRA History: Treatment of Uncertainties]

Variability in Results - Comparison with IPE/IPEEE 1E-3 0.001 0.50 NFPA 805 Total CDF (IPE + IPEEE) 0.40 Fraction of PRAs IPE/IPEEE 0.30 1E-4 0.0001 0.20 0.10 0.00 1 2 3 4 5 6 7 8 9 10 0.01 0.1 1 10 100 1000 1E-5 0.00001 1E-5 1.00E-05 1E-4 1.00E-04 1E-3 1.00E-03 Fire CDF/Internal Events CDF Total CDF (Recent LARs) 139

Additional Slides DRIVE TO RIDM AND TRENDS 140

Additional Slides [Drive to RIDM and Trends]

RIDM and NRCs Principles of Good Regulation Readily Efficiency Logical Defense- Independence Understood

• Acceptable In-Depth Risk Safety Best

• Openness Margins Integrated Information

• Efficiency Decision Reliability Openness Coherent

• Clarity Current Making Performance

• Reliability Practical Monitoring Regulations U.S. Nuclear Regulatory Independence Candid Public Commission, Principles of Good Highest Clarity Competence Regulation (ADAMS ML14135A076)

Standards 141

Additional Slides [Drive to RIDM and Trends]

Drive to RIDM: Back to the Future SECY-19-0036, April 11, 2019 (ML19060A081):

• Early years: progressive evolution of the staff is seeking Commission affirmation that protection considering maximum credible the most damaging single active failure of safety-accident related equipment is required to be considered in

- Remote siting performing design, and transient and accident

- Containment analyses, unless such a failure can be shown with

- Engineered safeguards, single failure criterion high confidence to not be credible.

SRM-SECY-19-0036, July 19, 2019 (ML19183A408): In any licensing review or other regulatory decision, the staff should apply risk-informed principles when strict, prescriptive application of deterministic criteria such as the

• Current: engineering solutions considered single failure criterion is unnecessary to provide

- Single failure for reasonable assurance of adequate protection

- Containment? of public health and safety.

142

Additional Slides [Drive to RIDM and Trends]

Looking Ahead: Possible Futures U.S. Nuclear Regulatory Commission, The Dynamic Futures for NRC Mission Areas, 2019. (ADAMS ML19022A178) 143

Additional Slides [Drive to RIDM and Trends]

Drive to RIDM: Transformation

• Evolving situation (market forces, new nuclear technologies, new analytical methods and data, new professionals)

• Vision: make safe use of nuclear technology possible

• Continuing standard: reasonable assurance of adequate protection

• Attitude: recognize potentially different ways of achievement - embrace change Applying the Principles of Good Regulation as a Risk-Informed Regulator, 144 October 15, 2019 (ADAMS ML19260E683)

Additional Slides [Drive to RIDM and Trends]

Drive to RIDM: Effect of Market Forces Operating Rx - More use of PRA models New Rx - Early use of PRA in design Risk-Informed LARS Received*

50 40 Miscellaneous Risk Insights TMRE 30 Number Fire Seismic GSI-191 20 EPU 50.69 10 TSTF-XXX RI TS Comp Time RI-ISI 0 ILRT FY-17 FY-18 FY-19 FY-20 Fiscal Year *As of June 8, 2020 "Risk-Informed Performance-Based Technology-Inclusive Guidance for Non-Light Water Reactors," NEI 18-04, Rev. 1, August 29, 2019.

145

Additional Slides [Drive to RIDM and Trends]

Drive to RIDM: New Technologies Im worried about the mission, Dave.

Cmon HAL, open the pod bay door Photo courtesy of NEA Halden Reactor Project

• New designs

• Smart Reactor Systems

• New operational concepts

• Improved Analysis 146

Additional Slides [Drive to RIDM and Trends]

Drive to RIDM: New Professionals Changing

• Experiences, knowledge

• Information content and delivery preferences

• Comfort with analytics, risk, probability

• Mobility Source: https://www.nrc.gov/reading-rm/doc-collections/commission/slides/2019/20190618/staff-20190618.pdf 147

Additional Slides [Drive to RIDM and Trends]

Trends and Impacts: A Two-Way Street Decision Making

• Issue Identification

• Option Identification Trends

• Analysis

• Increasing # RI-applications

• Deliberation

• New licensing approaches

• Implementation

• New designs

• Monitoring Challenge to NRC:

New operational concepts

• New technologies Be Ready!

• New analytical methods

• New professionals PRA Technology

• Methods

• Models

• Tools

• Data 148

Additional Slides [Drive to RIDM and Trends]

NRC/RES/DRA: The Cardinal Questions Who? What? When?

Policy/Decision Makers Users

- Needs DRA - Applications Suppliers

- Contractors

- Grantees Non-NRC Near Term Long Term R&D

- Collaborators (User Need) (Future-Focused)

Where? How? Why?

149

Additional Slides [Drive to RIDM and Trends]

PRA/RIDM:

Technical Area Topic Area Technical Area Topic Area Reactors Level 1 internal events at power Special Topics HRA Level 2 Ageing Topic Areas Level 3 Passive components Passive systems Low power and shutdown (LPSD) for Potential Digital systems Operational data CCF Event analysis Design and construction Generic safety issues (GSI)

R&D Fire Performance indicators and thresholds Seismic New reactors (evolutionary) Other external events Advanced reactors Security-related events EP&R Research and test reactors General Systems PRA tools Non-Reactor Geologic repositories Analysis Facilities and Methods and Uncertainty and sensitivity analysis methods and tools Activities High-level waste (HLW)

Tools Ive got a Advanced computational methods little list Low-level waste/decommissioning Advanced modeling methods (e.g., simulation)

Fuel cycle facilities Elicitation methods Transportation Sources Implementation PRA quality (e.g., guidance, standards) and Application Risk-informed regulation infrastructure Risk-informed regulation applications Risk perception and communication 150

Additional Slides [Drive to RIDM and Trends]

Need for Focus NRC Research Budget (FY 1976 - FY 2019) 700 50 45 600 Actual ($M)

Inflation Adjusted ($M) 40

% NRC Total 500 35 Contracting Budget ($M) 30 400

% NRC Total 25 300 20 200 15 10 100 5

0 1975 1976 1977 1978 1979 0

1980 1981 1982 1983 1984 1985 1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 Budget data from NUREG-1350 (NRC Information Digest) Year 151

Additional Slides RIDM APPLICATION EXAMPLES 152

Additional Slides [RIDM Application Examples]

Risk Info Uses - Regulations Example (Risk-Informed Fire Protection)

• Browns Ferry Nuclear Power Plant fire (3/22/75) Adapted from NUREG-0050

• Candle ignited foam penetration seal, initiated cable tray fire; water suppression delayed; complicated shutdown 11.5m 8.5m

• Second-most challenging event in U.S.

nuclear power plant operating history TVA File Photo

• Spurred changes in requirements and analysis 3m 153

Additional Slides [RIDM Application Examples]

Risk Info Uses - Regulations Example (Risk-Informed Fire Protection)

• Post-Browns Ferry deterministic fire protection (10 CFR Part 50, Appendix R) hour fire barrier, OR

- 20 feet separation with detectors and auto suppression, OR hour fire barrier with detectors and auto suppression

• Risk-informed, performance-based fire protection (10 CFR 50.48(c), NFPA 805)

- Voluntary alternative to Appendix R

- Deterministic and performance-based elements

- Changes can be made without prior approval; risk must be acceptable

- More than 1/3 U.S. fleet has completed transition

• Methods adopted by international organizations From Cline, D.D., et al., Investigation of Twenty-Foot Separation Distance as a Fire Protection Method as Specified in 10 CFR 50, Appendix R, NUREG/CR-3192, 1983.

154

Additional Slides [RIDM Application Examples]

Risk Info Uses - Licensing Example (Changes in plant licensing basis - RG 1.174)

• Voluntary changes: licensee requests, NRC reviews

• Small risk increases may be acceptable

• Change requests may be combined

• Decisions are risk-informed 155

Additional Slides [RIDM Application Examples]

Risk Info Uses - Oversight Example (Reactor Oversight Program)

• Inspection planning CDF < 1E-6 LERF < 1E-7

• Determining significance of findings

- Characterize performance deficiency 1E-6 < CDF < 1E-5 1E-7 < LERF < 1E-6

- Use review panel (if required)

- Obtain licensee perspective 1E-5 < CDF < 1E-4

- Finalize 1E-6 < LERF < 1E-5

• Performance indicators CDF > 1E-4 LERF > 1E-5 156

Additional Slides [RIDM Application Examples]

Risk Info Uses - OpE Example (Accident Sequence Precursor Program)

• Program recommended by WASH-1400 review group (1978) significant precursor

• Provides risk-informed view of nuclear plant operating experience

- Conditional core damage probability (events) precursor

- Increase in core damage probability (conditions)

Licensee Event Reports 1969-2018

• Supported by plant-specific Standardized (No significant precursors since 2002)

Plant Analysis Risk models 157

Additional Slides [RIDM Application Examples]

Risk Info Uses - Decision Support Example Decision (Research) re*search, n. diligent and systematic inquiry or investigation in order to discover or revise facts, theories, applications, etc.

Specific Analyses Typical products (regulatory research)

• Ways to look at and/or approach problems (e.g.,

Methods, Models, frameworks, methodologies)

Tools, Databases, R&D

• Points of comparison (e.g., reference Standards, calculations, experimental results)

Guidance,

• Job aids (e.g., computational tools, databases, standards, guidance: best practices, procedures)

• Problem-specific information (e.g., results, Foundational Knowledge insights, uncertainties)

Side benefits

• Education/training of workforce Regulatory Decision Support

• Networking with technical community 158

Additional Slides [RIDM Application Examples]

Risk Info Uses - Decision Support Example (Research: Frameworks/Methodologies)

NRC-sponsored Fire PRA Technology Neutral R&D (universities) Framework

• Started after Browns

• Explored use of risk Ferry fire (1975) metrics to identify

• Developed fire PRA licensing basis events approach first used in

• Inspiration and part industry Zion and basis for current Indian Point PRAs Licensing (early 80s), same basic Modernization approach today Program

• Started path leading to risk-informed fire protection (NFPA 805) 159

Additional Slides [RIDM Application Examples]

Risk Info Uses - Decision Support Example (Research: Reference Points)

NUREG-1150 SOARCA

• Continuing point of

• Detailed analysis of comparison for potential severe Level 1, 2, 3 results accidents and offsite

• Expectations consequences (ballpark)

• Updated insights on

• Basis for regulatory margins to QHOs Peach Bottom analysis (backfitting, generic issue resolution)

NUREG-1150 (Surry)

Surry Sequoyah 160

Additional Slides [RIDM Application Examples]

Risk Info Uses - Decision Support Example (Research: Methods/Models/Tools)

SPAR IDHEAS-G

• Independent plant- IDHEAS is coming.

• Improved support for specific models Resistance is futile! qualitative analysis (generic data)

• Explicit ties with cognitive

• All-hazards (many) science (models, data)

• Support SDP, MD 8.3,

• General framework for ASP, GSI, SSC studies developing focused

• Adaptable for specific applications (e.g., IDHEAS-circumstances ECA)

SAPHIRE

• Benefits from NPP simulator studies

• General purpose

• Consistent with current model-building tool HRA good practices

• Multiple user guidance (NUREG-1792) interfaces From https://en.wikipedia.org/wiki/SAPHIRE 161