ML21138A792

RES Seminar Part 2 - Short Takes

ML21138A792
Person / Time
Issue date:	05/13/2021
From:	Nathan Siu NRC/RES/DRA
To:
Siu, Nathan - 301 415 0744
Shared Package
ML21138A647	List: ML21138A778 ML21138A779 ML21138A781 ML21138A792 ML21138A793 ML21138A805 ML21138A810
References
Download: ML21138A792 (133)

Text

Short Takes:

Snippets on Some PRA Topics*

Nathan Siu Senior Technical Adviser for PRA Analysis Office of Nuclear Regulatory Research Division of Risk Analysis RES Staff Technical Seminar (Virtual) - Part 2 May 13, 2021 (3:004:00)

• The views expressed in this presentation are not necessarily those of the U.S. Nuclear Regulatory Commission.

The Menu

• Dynamic PRA Notes

1) Each snippet provides a 1520 minute talk on the

• Identifying Scenarios - Weird Stuff and the subject.

Importance of Active Searching 2) Most snippets are accompanied by extra slides providing additional details.

• Internal Risk Communication 3) Links to additional presentations (pdf versions) are

• A Brief History: PRA and the Characterization provided in the Additional Resources portion of this slide set.

of Uncertainties 4) PowerPoint presentations (with fullresolution

• PRA Lessons from NPP Accidents and graphics and fullfunctioning navigation links) will be uploaded into an ADAMS package.

Incidents a. Slides from this seminar

• RiskRelated Regulatory R&D (R4&D) b. Presentations identified in Additional Resources

• Treatment of Uncertainties c. Miscellaneous additional presentations, snippets, and notes1

• Additional Resources 1Extended and illustrated notes on a subject not intended as an actual presentation but provided in PowerPoint form for convenience.

2

DYNAMIC PRA

• What is it?

• Why do we care?

• Where are things now?

3

Fukushima Daiichi 1, 3/11/2011: Static Description

{Loss of Power} CORE DAMAGE AND {Loss of Isolation Condenser} CD AND {Failure of Alternate Cooling} LOSS OF DECAY HEAT REMOVAL LODHR

= {Core Damage}

LOSS OF ALL LOSS OF ISOLATION FAILURE OF AC AND DC POWER CONDENSER ALT COOLING LOP Ext LOIC Ext FALTC Ext 4

Adding Time, Motion (Kinematics)

Emergency Isolation Actions to Actions to Offsite LOOP EDG LongTerm Power Condenser Extend Shed Power (Seismic) Recovery Cooling (EDGs) (IC) IC Ops DC Loads Recovery Earthquake and LOOP (T = 0:00)

LOOPEQ EPS ISO EXT DCL OPR DGR LTC

• Tsunami (T+0:40) 1 2 CD

• Loss of all power (T+0:50) 3 4 CD 5

• IC outboard valve closed (T+3:40)* 12 hr 12 hr 6

7 CD CD

• Core damage (T+4:00, estimated 8 9 CD postaccident) 8 hr 10 11 CD 8 hr 12 CD 13 14 CD

• What but not Why 4 hr 15 16 CD

- Closure of isolation condenser 4 hr 17 CD 18

- Delay in implementing alternate 19 CD 20 cooling (fire pumps) 1 hr 1 hr 21 CD 22 CD 5 *Manual action that had little actual effect; inboard valves already closed

Dynamic Interactions => Context => Why Time Hazard Systems Indications Operators/Workers ERC/ER team EP Time 14:46 0:00 Earthquake Scram MSIVs close, turbine trips, EDGs 14:47 0:01 Rx level drops start and load RV pressure decreases; RV level 14:52 0:06 ICs start automatically in normal range 40 minutes between Cooldown earthquake rate exceeding techand tsunami; 15:03 0:17 ICs removed from service Manually remove IC from service transition fromspec confident limits control to disbelief Disaster HQ established in TEPCO 15:06 0:20 Tokyo Determine only 1 train IC 15:10 0:24 needed; cycle A train First tsunami 15:27 0:41 arrives Second tsunami 15:35 0:49 arrives 15:37 0:51 Loss of AC Degradation and failure over time, 15371550: Gradual loss of instrumentation, indications gradually affecting operator 15:37 0:51 Determine HPCI unavailable (including IC valve status, RV information and ability to control level), alarms, MCR main lighting TEPCO enters emergency plan 15:42 0:56 (loss of AC power); ERC established D/DFP indicator lamp indicates 16:35 1:49 "halted" Review accident management Cannot determine RV level or Review accident management procedures, start developing injection status; work to restore procedures, start developing Declared emergency (inability to 16:36 1:50 procedure to open containment level indication; do not put IC in procedure to open containment determine level or injection) vent valves without power service vent valves without power 6

Dynamic PRA - What Is It?

• Dynamic PRA PRA that explicitly treats interac ons among system elements and resulting motions (including rates of change), e.g.,

- Hardware component transitions (e.g., available to unavailable, or even intermediate states)

- Changes in operating crew situation awareness Plant I&C Crew

- Changes in plant thermalhydraulic state

• Degree of treatment of dynamics => continuum of analyses, e.g.,

Environment

- Current PRA (phenomenological submodel; some direct Complexity dependencies, e.g., support systems)

- Taskoriented network models and simulations

- Largely mechanistic simulations with stochastic elements Frequent conceptualization of dynamic PRA 7

Dynamic PRA - Potential Benefits

• Additional insights (suggesting alternative risk management strategies), e.g.,

- Untreated mechanisms (e.g., feedback loops)

- Timing of key events

- Causes of key events

• Fewer intermediate and often conservativelyoriented simplifications (e.g., discretization, success criteria)

- More realistic results

- Improved use of available evidence (what we know) => improved DM confidence

• Directly supportive of phenomenological whatif and optimization analyses, e.g.,

- Assessing effect of different parameter values (e.g., ATF properties, arrival times for offsite aid)

- Identifying potentially troublesome ranges of parameter values (cliffedge effects)1

• Modeling in disciplinespecific terms (native language)

- Reduced chance of translation errors

- Increased stakeholder involvement and buyin

• Engineering trends (integrated simulation) 1Analysis requires coupling of dynamic PRA model with appropriate mathematical searching and optimization tools.

8

Dynamic PRA - Where Are We?

• Strong interest: academia, international

• NPPs: tools, demonstrations

• Nonnuclear: decision support applications (e.g., aerospace, hydropower)

Methodologies for Current PSA (Phased Mission, Dynamic PSA HighFidelity, Competing Risks, Tools and SimulationBased Level 3 PSA) Toolboxes Dynamic PSA Late Intermediate Early (Mature, Stable) (Adolescent, Developing) (Infancy, Emerging)

Developmental Stage 9

Dynamic PRA - Concluding Remarks

• All NPP accidents have involved significant dynamic interactions among system elements

• Explicit treatment of these interactions can benefit PRA studies and the PRA enterprise

• Work (particularly decision support applications) is needed to achieve these benefits 10

Dynamic PRA - Extra Slides 11

Indicators of Technology Maturity1 Early Intermediate Late (Infancy, Emerging) (Adolescent, Developing) (Mature, Stable)

Many welltrained and experienced practitioners Small research community Larger number of practitioners Recognize limits of applicability of Small number of practitioners Larger number of experienced Practitioners Strong personality influences, researchers methods Can adapt methods to new situations competing schools of thought Can work with researchers to identify important issues New practicedriven research problems Most research driven by needs of Driven by perceived needs Some consensus positions for some practice Research Problem selection affected by personal broadly defined problem areas More abstract research addresses Agenda choice (e.g., due to ease of formulation Some unproductive research lines needs clearly identifiable by all or solution) abandoned concerned Incomplete coverage of topics Fast growth Local applications (addressing small Vocabulary has evolved Developing vocabulary Applications parts of larger problems) General framework exists Optimistic views on new methods; No broader framework Little selling of area limitations not well understood 1 Adapted from: Cornell, C.A., Structural safety: some historical evidence that it is a healthy adolescent, Proceedings of Third 12 International Conference on Structural Safety and Reliability (ICOSSAR 81), Trondheim, Norway, June 2325, 1981.

U2: start depressurization (stuck RV, then continue) Browns Ferry 1 & 2 U2: D DG tripped, multiple boards lost 19750322 U2: control panel malfunctions, scram, turbine trip, FW trip, MSIVs close U2: conditions U2: shutdown stabilized cooling established U1: start depressurization U1: FW tripped, HPCI and RCIC stopped, use CRD pump U1: enter RB U1: scram, 2/3 FW pumps to assess SSC U1: RV control tripped, multiple boards lost conditions restored U1: spurious alarms, actuations U1: loss of operating relief valves U1: shutdown cooling established Fire reported to U1/U2 MCR U1: prepare for RHR cooling (15 hr, 50 min)

OFD notified TVA notified Fire out CSR CO2 discharge CSR fire out, resume Start using water RB firefighting Smoke, CO2 enter MCR Fire start 0 2 4 6 8 10 12 13 Time from Start (hr)

Loss of EFW (burned cable Start laying temporary Greifswald 1 to 2nd EFP) cable to power EFPs 19751207 EFW restored Trial and error fault diagnosis (power to EFPs) actions => more failures (including instrumentation) Close primary hot leg main gate valves, start DGs start, power to forced circulation 1/2 emergency buses Natural circulation: MCR power restored; Turbine use SVs and cold FW pressurizer SVs open, 2/6 fail to reclose; Stable trip to control primary emergency cooling pump started cooling Start firefighting Corridor Fire Fire ventilation alarm out restored Heavy smoke, need respirators Fire start, spread 0 2 4 6 8 10 12 14 Time from Start (hr)

Start laying temporary power cable from Armenia 1&2 U2 DG to U1 emergency makeup pump 19821015 Station U1: only instrumentation is Feedwater makeup to Blackout primary pressure (local station) SGs (temporary cable)

Loss of main coolant pumps, MCR Power to U1 emergency lights, readouts, alarms, phones, makeup pump from DG power, normal and emergency makeup Manual SG SRV Operators manually open SG dump valves MCR power trip U1&2 opened in upper TB (wearing breathing masks) restored Offsite Break cable spreading Fire out FBs arrive room wall to access fire FB arrives, open MCR TB, transformer fires Fire hatch to spray vault under control controlled H2, transformer explosions Fire start, Smoke MCR smoke spread in MCR unbearable 0 2 4 6 8 10 12 15 Time from Start (hr)

Blayais 14 U1: shutdown 19991227 U4: 400 kV restored Level 2 Emergency Plan activated for U1; utility U1: Train A ESWS and regulator national emergency teams pumps submerged activated; agree to SG cooldown strategy U2: 400 kV restored U1 & U2: LHSI and CSS Regulator Walkdown discovers U1 Train A pump rooms flooded informed of U1 & ESWS pumps submerged U2 status and SG U2 & U4: Loss of 400 kV cooldown Use fire engines to assist 225 kV power (grid instability), scram strategy in pump floodwaters restored (U1U U1U4: Loss of 225 kV Level 1 Emergency Plan activated: onsite pumps power (fallen trees) for floodwaters, recover submerged equipment U4: High Site access regained; needed tide alarm offsite workers can arrive Floodwater pumping (continues to ~50 hr)

Flood overtops dyke, site access lost 0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 16 Time from Start (hr) 19:00 0:00 0:00 12/27 12/28 12/29

Dynamic PRA Fukushima Daiichi 1 20110311 Relative Time Hazard Systems Indications Operators/Workers ERC/ER team EP Time 14:46 0:00 Earthquake Scram MSIVs close, turbine trips, Rx level drops (1 of 3) 14:47 14:52 0:01 0:06 EDGs start and load ICs start automatically RV pressure decreases; RV level in normal range 15:03 0:17 40 minutes between earthquake and tsunami; ICs removed from service Cooldown rate exceeding Manually remove IC from tech spec limits service transition from confident control to disbelief Disaster HQ established in 15:06 0:20 TEPCO Tokyo Determine only 1 train IC 15:10 0:24 needed; cycle A train First tsunami 15:27 0:41 arrives Second tsunami 15:35 0:49 arrives 15:37 0:51 Loss of AC 15371550: Gradual loss of Determine HPCI instrumentation, Degradation unavailable and failure over time, 15:37 0:51 indications (including IC gradually affecting operator valve status, RV level),

alarms, MCR main lighting information and ability to control TEPCO enters emergency 15:42 0:56 plan (loss of AC power);

ERC established D/DFP indicator lamp 16:35 1:49 indicates "halted" Review accident Cannot determine RV level Review accident Declared emergency management procedures, or injection status; work to management procedures, (inability to determine start developing restore level indication; do start developing level or injection) 16:36 1:50 procedure to open not put IC in service procedure to open containment vent valves containment vent valves without power without power 17 17

Dynamic PRA Fukushima Daiichi 1 20110311 Time Relative Hazard Systems Indications Operators/Workers ERC/ER team EP Time (2 of 3) 16:45 16:55 1:59 2:09 Tsunami alert Determine RV level Workers on way to check Emergency cancelled D/DFP had to turn back Lose ability to determine Reentered emergency plan 17:07 2:21 External influence RV level or injection status Site superintendent directs 17:12 2:26 triggering work investigation of using fire protection to inject water 17:15 2:29 stoppage, temporary Estimated core uncovery in 1 hr Tsunami alert evacuation, 17:19 2:33 cleared accountability Dieseldriven fire pump Pressure above 100 psi Manually open valves (in started and left to idle dark) from fire protection system to core spray 17:30 2:44 system; take turns holding D/DFP switch to keep in standby Error3:32 18:18 of commission (disabling DC power partially returned MO3A and MO2A indicate closed passive safety system) possibly MO3A and MO2A Open IC valves MO3A and 18:18 3:32 opened 2A. Steam from condenser based on assumed low inventory observed MO3A closed Remove IC from service (usage) (concerned about failing lines). Entered R/B and T/B to manually open MOV for 18:25 3:39 FP lineup. Hard time finding valve, had wrong key, hard to operate hand wheel. Long time.

18

Fukushima Daiichi 1 20110311 Time Relative Time Hazard Systems Core damage (45 hr Indications Operators/Workers ERC/ER team EP 18:50 4:00 (3 of 3) 19:00 4:14 after trip)

Close valves for broken outdoor FP pipes. Broke Ask Tokyo for more fire engines lock to allow passage between Units 2 and 3.

Govt. declares nuclear 19:03 4:17 emergency InNohindsight, core damage pressure indication in MCR; Reactor pressure =

20:07 5:21 Game 6.89 MPa (1000Over psi) local for 1F1; indication Small portable generatorcontinuing 1F1 MCR recovery has temporary lighting 20:49 6:03 installed 20:50 6:04 activities and events impact Local authorities order evacuation within 2 km other units Level indication (1F2 and 1F3 core restored; 21:19 6:33 level = 0.20 m (8) above uncovery TAF on 3/14)

Prime minister orders 21:23 6:37 evacuation within 3 km; sheltering out to 10 km MO3A opened Place IC in service; steam 21:30 6:44 observed Access to RB restricted due 21:51 7:05 to dose rates - indirect indication of core uncovery Level = 0.55 m (21.7) 22:00 7:14 above TAF Drywell pressure = 0.50 Restoration team from 23:50 9:04 MPa (87 psi) above design ERC enables reading Offsite power supply 23:59 9:13 trucks arrive by midnight 19

Earthquake Fukushima Daiichi 16 2nd Tsunami 20110311 Request: Suspend Order: Vent Seawater Injection U1 and U2 Order: Confirmed:

Local Evac. Local Evac.

U5 Rx U2 Cont. U5 Level =

Depressurizing Venting TAF + 0.95m SBO (U1U5) U1 Cont. U3 Cont. U2 Core Loss of DC (U1U4) Venting Venting Uncovery U1 Core U1 RB U3 Core U3 RB U4 RB Damage (est.) Explosion Uncovery Explosion Explosion 3/11 3/12 3/13 3/14 3/15 20

Fukushima Daiichi 16 20110311 U5 SFP Cooling Restored U6 SFP SDF Truck Spray Cooling Restored U4 SFP Earthquake U2 Core U4 SFP Lev Uncovery <0.5m above 2nd Tsunami U5 Level = U4 RB 4/

SBO (U1U5) TAF + 0.95m Explosion U1 Core U1 RB U3 Core U3 RB Damage (est.) Explosion Uncovery Explosion 3/11 3/12 3/13 3/14 3/15 3/16 3/17 3/18 3/19 3/20 3/21 21

Local evacuation confirmed, 1st team dispatched Govt Start prep orders 2nd team dispatched, turned back (radiation) for venting venting Unsuccessful attempts to open AO90 Open AO72 1.0 manual venting of Pressure (MPa) wetwell Containment Venting:

• Prevents catastrophic 0.5 lower head failure failure pressurization from core steam dome

• Causes release to relocation to lower head drywell wetwell environment RPVTEPCO steam line rupture DWTEPCO WWTEPCO 0.0 0 5 10 15 20 25 30 3/11/2011 Time (hr) 14:46 Adapted from: R. Gauntt, Fukushima Daiichi Accident Study: MELCOR Analyses and Results, OECD/NEA Fukushima Accident Analysis Workshop, IssylesMoulineaux, France, June 1820, 2012.

See also R. Gauntt, et al., MELCOR Simulations of the Severe Accident at the Fukushima 1F1 Reactor, ANS Winter Meeting and Nuclear Technology Expo, San Diego, CA, November 1115, 2012.

22

INTERESTING Clickbait IDENTIFYING SCENARIOS - WEIRD STUFF AND THE IMPORTANCE OF ACTIVE SEARCHING

• What is the concern?

• What tools are available?

• How might we do better?

23

Reminders

1) Risk = {si,Ci,pi}

Scenarios: what can go wrong? (qualitative)

2) All models are wrong, but some are useful.1

• What isnt in the PRA model wont be quantified

• What isnt conceived of might not be addressed in a riskinformed decision 1 G.E.P. Box and N.R. Draper, Empirical ModelBuilding and Response Surfaces, John Wiley and Sons, 1987. See the Wikipedia article All 24 models are wrong for background.

Youre analyzing a floating NPP. Have you thought of this one?

Chazhma Bay (August 10, 1985)1

• Echo II class submarine K431 is nearly done refueling. Fresh fuel has been loaded, workers are preparing to reattach 12ton reactor vessel head which has control rods attached.

• Workers see seal is not tight (there are leaks), decide to lift head using refueling ship crane. (Decision is against regulations and made without consulting supervisor. Did not drain primary loop to ensure no moderation, did not detach lattice used to keep control rods in place.)

• Passing torpedo boat creates large wake, rocks refueling ship; crane pulls control rods out of the core.

• Reactivity excursion causes steam explosion which blows head and fuel assemblies out of the reactor compartment, destroys the submarine pressure hull.

1See M. Takano, V. Romanova, H. Yamazawa, Y. Sivintsev, K. Compton, V. Novikov, and F. Parker, Reactivity Accident of Nuclear Submarine 25 near Vladivostok, Journal of Nuclear Science and Technology, Vol. 38, No. 2, pp. 143157 (February 2001).

What Can Go Wrong?

• PRA scenarios need a starting point (initiating hazard or event)

• Complementary methods to identify starting point:

- Inductive (e.g., FMEA, HAZOP)

- Deductive (e.g., Master Logic Diagram, Heat Balance Fault Tree)

- Lists (e.g., possible hazards, actual events, other PRAs)

• Notes:

- Conventional focus on postinitiator scenario can blur or even miss important factors in preinitiator buildup

- Real events can involve unanticipated mechanisms and sequences of events that appear perfectly reasonable in hindsight. Click here for more examples.

26

Checklists can be useful but Aircraft impact Local intense precipitation

• Might not actually be exhaustive Avalanche Biological events Low lake or river water level Low winter temperature

• Can be confusing (e.g., overlapping Coastal erosion Drought Meteor or satellite strike Onsite chemical release categories) External fire External flooding Pipeline accident River diversion

• Can promote oneatatime consideration Extreme winds and tornadoes Fog Sandstorm Seiche (actual events can involve multiple Forest fire Frost Seismic activity Severe temperatures categories) Hail Snow High summer temperature Soil shrinkswell

• Can be inefficient (e.g., excessive attention High tide Hurricane Space weather Storm surge on ultimately unimportant categories) Ice cover Transportation accident Industrial/military facility accident Tsunami

• Lengthy lists might trigger impulse to screen Internal flooding Landslide Turbinegenerated missiles Volcanic activity rather than explore Lightning 27

Active Searching

• Searching emphasized in the early days it is incumbent upon the new industry and of nuclear power the Government to make every effort to recognize every possible event or series of

• A fundamental first principles attitude: events which could result in the release of using understanding of system, look for unsafe amounts of radioactive material to potential problems (rather than expect the surroundings and to take all steps them to be revealed by some analytical necessary to reduce to a reasonable minimum the probability that such events process) will occur in a manner causing serious

• Potentially valuable for new/novel overexposure to the public.

situations where operational experience W.F. Libby (1956)1 is weak or entirely lacking 1W. F. Libby (Acting Chairman, AEC) - March 14, 1956 response to Senator Hickenlooper [See D. Okrent, Reactor Safety, University of 28 Wisconsin Press, 1981. (NRC Technical Library TK9152 .O35, multiple copies)]

Hazard Identification Example: Checklist vs Active Search Checklist Active Search (aka Red Teaming)

General Process Stepping through list Looking at undesired state (e.g., failure of

1) Ask what each hazard might do key components), ask

2) Screen or retain for further analysis using 1) What conditions might cause this established criteria undesired state

2) What hazards or hazard combinations might create these conditions

3) If there are protective barriers preventing the undesired conditions, what might fail these barriers Advantages More complete More direct Methodical, easy to document Less restricted by categorization Engages imagination Challenges Not wasting time on unimportant categories Tempering imagination with plausibility Avoiding urge to screen (to finish the job) Ensuring reasonable completeness 29

Active Search >> Drawing Fault Tree

• Need to identify plausible mechanisms

- Possible failures can always be added to a fault tree

- Reasonable causality needed for retention and quantification

• Examples

- Operator disabling of safety systems (errors of commission)

- Seismicallyinduced reactivity transients 30

Example: Disabling a BWR Isolation Condenser OPERATOR TERMINATES Possible ISOLATION CONDENSER OPERATION but what ISOXHEEOCTERM reason?

31

Example: SeismicallyInduced Reactivity Excursion

• Observations

- Global operational experience: at least 4 (perhaps 5)

North Anna Nuclear Generating Station earthquakes causing fluxinduced trips at 7 (perhaps 9) reactors1

- Some reactor designs have unstable operating regimes

- Systems with timedelayed feedback (e.g., restorative forces) can oscillate, even resonate

• Q: Can a seismic event induce a resonance leading to a runaway reaction? Under what conditions?

Adapted from: https://earthquake.usgs.gov/earthquakes/

1Ground motion trips either not available (e.g., power loss) or not triggered (e.g., accelerations are too low) 32

Example: SeismicallyInduced Reactivity Excursion Seismic Hazard Controls Neutronics Plausible?

Structures Operational Experience Thermal Hydraulics Systems Integration

• Movement? Bowing?

• Reactivity effects?

• Feedback?

• Resonance? Perhaps not, but

• Fluid flow & density effects?

• Time scales?

• Excursion?

• Heat transfer effects?

ask the question 33

Looking Forward: OpE + Advanced Technology

• Empirical evidence: strong argument for plausibility

• Challenges

- Enormous and growing database (not just nuclear)

- Unstructured, natural language and heterogeneous (content, form, quality) data

- Inferencing

• Exploratory study: advanced technology Adapted from:

1) 2)

https://str.llnl.gov/str/March02/March50th.html https://en.wikipedia.org/wiki/History_of_supercomputing#/media/File:Supercomputershistory.svg (AI/ML, Big Data) can help1 3) https://www.top500.org/news/japancapturestop500crownarmpoweredsupercomputer/

1See, for example

• N. Siu, K. Coyne, and F. Gonzalez, Knowledge Management and Engineering at a Riskinformed Regulatory Agency: Challenges And Suggestions, white paper, U.S. Nuclear Regulatory Commission, 2017. (ML17089A538) 34

• F. Gonzalez and N. Siu, Accident Sequence Precursors: Current Analyses, Challenges, and Future Research, WGRISK Annual Meeting, NEA HQ, BoulogneBillancourt, France, March 2022, 2019. (ML19071A160)

Identifying Scenarios - Concluding Remarks

• A longstanding and continuing PRA goal: ensuring completeness

• An important mindset: active searching (especially when dealing with new/novel situations)

• Currently a variety of tools and resources to support searching; advanced technology (e.g., AI/ML) can lead to further improvements 35

Extra Slides - Examples of RealWorld Events and Mechanisms 36

External flooding: obvious now but back then?

• Fukushima Daiichi (1990s)

- Added EDGs to supplement existing units (SAM modification)

- Aircooled EDGs1 installed at Units 2, 4, 6; crossties provided with Units 1, 3, 5

- All watercooled EDGs in building basements

- Aircooled EDGs installed on ground floor, metalclad switchgear in basement New 2011 EDG

• Great East Japan Earthquake (March 11, 2011)

- Earthquake => LOOP New M/C 2009 DB 10m

- Tsunami => SBO for Units 14 (W/C EDGs, M/C switchgear) Switchgear 1972 DB

- Unit 6 EDG supplies Units 5 and 6; air louver ~1m above tsunami height 1Not affected by loss of service water, e.g., due to tsunami. (Pumps are at elevation O.P.+4m.) Per IAEA Director Generals report, 37 choice of aircooled is due to current service water loads; unclear if diversity was a major factor.

Some Other Accidents Accident Notable Mechanisms/Events Sodium Reactor Reactor coolant pump organic coolant leaks into the primary circuit, causes flow blockages, Experiment (1959) higher fuel temperatures, interaction with cladding and formation of a lowmelting temperature alloy, coolant channel blockage, fuel damage, and release of radioactive gases and some volatiles into the sodium coolant and eventually the environment.

Fermi 1 (1966) Segments of zirconium sheets (installed late in construction as a safety barrier) tear loose during power ascension, blocking coolant flow. Two fuel assemblies melt. Following radiation alarms, reactor is manually scrammed.

Chernobyl 4 (1986) Interruption of a planned test due to offsite grid needs leads to Xenon poisoning, inability to achieve planned test conditions. Crew decides to proceed with the test despite the plant being in an unstable operating regime and disables an automatic scram to facilitate testing. A plant computer signal dictating immediate shutdown is ignored. The test initiates a positive reactivity excursion with a catastrophic steam explosion and core destruction some 44 seconds later.

38

Some Interesting Incidents Incident Notable Mechanisms/Events Vogtle 1 (1988) Smoke detector actuation (burnoff in a duct heater) led to pressurization of a preaction deluge system in cable spreading room, water discharge through leakoff valves (as designed), water seepage through a floor penetration into the main control room, and spurious opening of a PORV at power. Floor penetration design was faulty - assumed sealwelding (of embedded seal angles and upper angle iron assembly) would be watertight. See LER 424/88016.

Indian Point 3 Activation of an outdoor deluge system (in response to a transformer explosion and fire) led to (2015) bleed off water in a valve room adjacent to a vital 480V switchgear room. Due to insufficient drain system capacity, water backed up into the switchgear room. [Note: Although the water was not high enough to affect the switchgear, it constituted a potential industrial hazard that could have inhibited operator access to that room.] See Special Inspection Report ML15204A499.

39

Some RealWorld Mechanisms (1 of 4)

Mechanism Plant (Year) Description Unexpected U.S. plant

• EDG oil fire due to fatigue cracking of undocumented instrumentation line.

/Unusual

• Failure occurred during followup examination of a reported small oil leak; line was Loadings moved slightly [cause?]

Nogent

• Unit 2 condenser circulating water system leak causes p between Turbine (2006) Building foundation and floor, lifts floor, fails manhole.

• Water floods Unit 1 Turbine Building, enters ESW system gallery through penetrations, CCW pump room through drains.

Inadequate Forsmark 1

• Offsite switchyard twophase short circuit during maintenance causes LOOP Protective (2006)

• Inverters failed on overvoltage, causing loss of 2/4 trains of AC and DC power Systems 40

Some RealWorld Mechanisms (2 of 4)

Mechanism Plant (Year) Description Secondary Maanshan 1

• Salt spray caused LOOP; electrical fault caused highenergy arcing fault (HEAF), loss Hazards (2001) of faulted safety bus

• Heavy smoke from HEAF delayed access to switchgear room to restore power to undamaged safety bus => 2 hour2.314815e-5 days <br />5.555556e-4 hours <br />3.306878e-6 weeks <br />7.61e-7 months <br /> station blackout Cruas 24

• Flood management actions lead to vegetation debris downstream, clogging of (2009) service water intake

• Total loss of service water for Unit 4, partial loss for Units 2 and 3 Declared Blayais 12

• During site flooding, rooms containing Unit 1 and Unit 2 lowhead safety injection Inoperability (1999) and containment spray pumps partially flooded

• Pumps declared inoperable LaSalle 12

• Foreign material (injectable sealant foam) found on floor of service water tunnel (1996)

• Core standby cooling system, emergency core cooling system, and diesel generators declared inoperable, both units shutdown 41

Some RealWorld Mechanisms (3 of 4)

Mechanism Plant (Year) Description Worker Point Beach

• Communications lost with diver working in Unit 2 (shutdown) circulating water Safety (2000) pump house Concerns

• Manual shutdown of Unit 1 U.S. plant

• Oil fire near reactor coolant pump

• Spurious evacuation alarm (smoke clogged radiation monitor)

• Reactor building evacuated Operator Greifswald 1 During a severe power cable fire triggered by an electrician (performing a Choices (1975) demonstration for a trainee), operators manipulated switchgear to find intact cables for power (trial and error problem solving) but these actions caused additional failures TMI2 During a loss of feedwater event, operators throttled high pressure makeup in the (1979) mistaken belief that the reactor coolant system was going solid 42

Some RealWorld Mechanisms (4 of 4)

Mechanism Plant (Year) Description Operator DavisBesse

• During a loss of feedwater transient, the shift supervisor did not implement Choices (1985) operating procedures for feed and bleed cooling (which would contaminate (cont.) containment), counting (correctly) on timely restoration of auxiliary feedwater

• In haste to enter the auxiliary feedwater pump room (accessed via a locked grate),

an equipment operator tossed keys to another ten feet ahead Vandellos During a Turbine Building fire (hydrogen deflagration, cascading burning oil),

(1989) operators (using breathing apparatus) entered dark, smoke filled areas to perform recovery actions Fukushima Operators isolate the isolation condenser in the mistaken belief that it was close to Daiichi 1 drying out and failing (which would provide a direct release path to the environment)

(2011)

Maintenance Rancho A maintenance worker dropped a lightbulb into a cabinet, shorting out nonnuclear Error Seco (1978) instrumentation. Propagating faults led to a scenario that could easily have resulted in an outcome as serious as that of the accident at Three Mile Island a year later 43

Some Resources

1. Fukushima Daiichi (2011): International Atomic Energy Agency, The Fukushima Daiichi Accident, Director General Report, Vienna, Austria, 2015.

2. Sodium Reactor Experiment (1959): P. Pickard, Sodium Reactor Experiment Accident July 1959, Sandia National Laboratories, August 29, 2009. (Available from:

https://www.etec.energy.gov/Library/Main/Pickard%20SRE%20presentation.pdf)

3. Fermi 1 (1966): Fermi Fuel Melt Accident, Nuclepedia.

4. Chernobyl 4 (1989): U.S. Department of Energy, Electric Power Research Institute, Environmental Protection Agency, Federal Emergency Management Agency, Institute of Nuclear Power Operations, and the U.S. Nuclear Regulatory Commission, Report on the Accident at the Chernobyl Nuclear Power Station, NUREG1250, January 1987.

5. Vogtle 1 (1988): Water Leakage into Control Room/Potential Exists for a Safety System Failure, Licensee Event Report 424/88016, November 22, 1988.

6. Indian Point 3 (2015): U.S. Nuclear Regulatory Commission, Indian Point Nuclear Generating - Special Inspection Report 05000286/2015010, July 23, 2015.

7. Nogent (2006): U.S. Nuclear Regulatory Commission, ConstructionRelated Experience with Flood Protection Features, IN 200906, July 21, 2009. (ML090300546)

8. Forsmark 1 (2006: U.S. Nuclear Regulatory Commission, Significant Loss of SafetyRelated Electrical Power at Forsmark, Unit 1, in Sweden, IN 200618, August 17, 2006.

9. Maanshan 1 (2001): Atomic Energy Council, Taiwan, The Station Blackout Incident of the Maanshan NPP Unit 1, April 18, 2001. (Available from:

https://www.aec.gov.tw/webpage/control/report/safety/safety_04_002.pdf)

10. Cruas 24 (2009): P. Dupuy, G. Georgescu, and F. Corenwinder, Treatment of the loss of ultimate heat sink initiating events in the IRSN Level 1 PSA, NEA/CSNI/R(2014)9, Nuclear Energy Agency, BoulogneBillancourt, France, 2014.

11. Blayais 12 (1999): Blayais Flood, Nuclepedia.

12. LaSalle 12 (1996): Foreign Material Injected Into Service Water Tunnel Causes Dual Unit Shutdown Due to Inadequate Work Control, Licensee Event Report 373/96008R01, November 25, 1996.

13. Point Beach (2000): Manual Reactor Trip Due to Concerns for Diver Safety, Point Beach Nuclear Plant Unit 1, Licensee Event Report 266/00010R00, November 22, 2000.

14. Greifswald 1 (1975): M. Rwekamp and E. Gelfort, Sicherheitsrelevanter Kabeltrassenbrand im Kernkraftwerk Greifswald Beschreibung und Einschtzung, GRSVSR 24491, Gesellschaft für Anlagen und Reaktorsicherheit (GRS) mbH, Kln, Germany, June 2004.

15. TMI2 (1979): D. Marksberry, F. Gonzalez, and K. Hamburger, Three Mile Island Accident of 1979 Knowledge Management Digest, Overview, NUREG/KM0001, rev. 1, U.S. Nuclear Regulatory Commission, June 2016.

16. DavisBesse (1985): U.S. Nuclear Regulatory Commission, Loss of Main and Auxiliary Feedwater Event at the DavisBesse Plant on June 9, 1985, NUREG1154, July 1985

17. Vandellos (1989): S.P. Nowlen, M. Kazarians, and F. Wyant, Risk Methods Insights Gained from Fire Incidents, NUREG/CR6738, U.S. Nuclear Regulatory Commission, September 2001.

18. Rancho Seco (1978): R.M. Bernero and F.H. Rowsome, Single Failure Potentially Leading to Core Damage, memorandum to H.R. Denton and C. Michelson, U.S. Nuclear Regulatory Commission, March 14, 1980. (ML19323J370) 44

INTERNAL RISK COMMUNICATION

• What is it?

• Why is it hard?

• How might we improve?

45

Internal Risk Communication: Support RIDM Adapted from NUREG2150 With To Other Considerations

• Current regulations

• Safety margins

• Defenseindepth

• Monitoring Quantitative

+

Quantitative 46

Risk Information: Not Just for Current Decisions Prior (foundational) information affects DM processing of new information Specific Analyses

• Recognition

• Interpretation Methods, Models,

• Judging/Weighting Tools, Databases, Standards, Guidance, Foundational Knowledge 47

Risk Information: Inherently Complex

• Low likelihood => beyond personal Other Complications experience, intuition

• Heterogeneous

• Hyperdimensional o Qualitative and quantitative o Multiple views

- Scenarios (organizations, disciplines)

- Likelihood

• Dynamic

- Multiple consequence measures o System changes (e.g.,

different operational modes,

• Uncertain effects of decisions)

- Sparse or nonexistent data o New applications (and

- Multiple models contexts)

- Partial coverage 48

Other Challenges

• Individual user differences, e.g.,

- Knowledge

- Preferences/heuristics

• Social factors, e.g.,

- Trust

- Decision and group dynamics

• Situational context, e.g.,

- Available time to consider

- Decision support vs. informational Source: https://www.nrc.gov/readingrm/doccollections/commission/slides/2019/20190618/staff20190618.pdf 49

How to Ensure Message Capture and Retention?

External Flooding Fire

• Intrinsic value of content High Winds

- Risk level (absolute, relative) Seismic

- Risk importance (absolute, relative) Internal Flooding Internal Events

- Surprise

• Communication process

- Message formulation

- Delivery method

- Tools 50

Current Mechanisms Documents and Interactive Presentations Discussion (Flatland) (Storytelling) 51

Can We Do Better? Different Documents? Graphic Elements Small Font Questions Sidebars Embedded Graphics Conventional TwoColumn, Conversational Graphical 52

Can We Escape Flatland?

• Tufte model: use rich displays and reports, encourage user to explore

- Promotes active involvement of decision maker

- Increases general trust?

• A graduated technical approach to assist?

Interface Interaction Mode Hyperlinked dashboards, reports Manual Time Video (with sound?) AI assist Visual immersion Multisensory immersion 53

From Static to Interactive Dashboard. Then to SciFi?

M. Korsnick, Risk Informing the Commercial Nuclear Enterprise, Promise of a Discipline: Reliability and Risk in Theory and in Practice, University of Maryland, April 2, 2014. Graphic adapted from https://www.flickr.com/photos/83823904@N00/64156219/

(permission CCBY2.0) 54

Internal Risk Communication - Concluding Remarks Risk Communication Technical Communication Communication

• General communication good practices are helpful but not sufficient: special characteristics of risk information pose additional challenges

• Intuitively better approaches are being developed; scientific testing could be helpful

• Communication involves people: one risk communication solution may not work for all actors 55

Internal Risk Communication - Extra Slides 56

Risk Information: Qualitative + Quantitative*

Risk { i , i, i}

• What can go wrong?

• What are the consequences?

• How likely is it?

• Kaplan/Garrick triplet definition has been adopted by NRC. See:

White Paper on RiskInformed and PerformanceBased Regulation (Revised), SRM to SECY98144, March 1, 1999 57 Glossary of RiskRelated Terms in Support of RiskInformed Decisionmaking, NUREG2122, May 2013 Probabilistic Risk Assessment and Regulatory Decisionmaking: Some Frequently Asked Questions, NUREG2201, September 2016

Sources of Breakdowns: Risk Communication Between Risk Managers and Public*

• Differences in perception of information

- Relevance

- Consistency with prior beliefs

• Lack of understanding of underlying science

• Conflicting agendas

• Failure to listen

• Trust

• J.L. Marble, N. Siu, and K. Coyne, Risk communication within a riskinformed regulatory decisionmaking environment, International 58 Conference on Probabilistic Safety and Assessment (PSAM 11/ESREL 2012), Helsinki, Finland, June 2529, 2012. (ADAMS ML120480139)

Differences in Perspective (Example)

Our tendency is to focus on things that are interesting and Decision make them important. The thing that we have to do is focus Makers on what really is important Ron Rivera, 2020 is (developer)

Whats interesting might be (practitioner) important Practitioners Developers isnt* (decision maker)

The PRA/RIDM Community

• Or, at least, isnt necessarily - interesting and important are independence.

59

External Flooding Preference: Avoid Chart Junk Fire High Winds Seismic

• Visual effects (e.g., noninformative 3D with perspective) can add Internal pop but distract from or even distort messages. Flooding Internal Events

• Advanced animation tools can be even stronger attention grabbers with even greater distraction potential

- Focus attention on effects rather than message

- Saturate audiences, leading to the need for even stronger effects in future presentations to grab attention

• Use effects with moderation (if at all), recognizing that your audience External

- has preferences that vary from person to person and over time (maybe High Flooding Fire Winds they prefer 3D charts!)

Seismic

- is likely subject to many presentations besides yours (imagine the clamor of highly animated presentations seeking attention to their Internal specific messages) Flooding Internal Events 60

Spatial Information - Underused Resource?

• Common practice in everyday risk communication

• Going beyond - add changes over time?

61

An OftIgnored External Risk Communication Lesson:

Comparisons Dont Work for Everybody U.S. Annual Deaths, Various Causes (20102019) 700,000 600,000 500,000 400,000 Deaths (2020) 300,000 200,000 100,000 0

Flu Auto Guns Cancer COVID19 62

One Size Doesnt Fit All, Part II:

1,000 words (a story) > a picture?

On the evening of June 25, a freshly graduated high school Drunk Driving Accident Fatalities (2018) star QB was going over 100 mph on a neighborhood road, 1,800 trying to go fast enough to avoid speed camera detection

("whipping"). Out of control on a sweeping curve, the car hit a fence and two trees, and flipped. Two unbelted passengers No Alcohol were ejected and died at the scene. The QB and the front 10,600 seat passenger were seriously injured. All four were BAC > 0.08 g/DL teenagers. All had just left an underage drinking party and 0.01 < BAC < 0.07 g/DL were drunk. The QB was indicted on counts of vehicular 24,100 manslaughter, alcohol related vehicular homicide and causing a lifethreatening injury while driving under the influence of alcohol. The parent of the girl hosting the party, who was present and knowledgeable, pled guilty to two criminal Data from "Traffic Safety Facts 2018 Data: State AlcoholImpaired citations for allowing underage drinking at his home and was Driving Estimates," DOT HS 812 917, June 2020. (Available from:

ordered to pay $5,000 in fines. https://crashstats.nhtsa.dot.gov/#!/DocumentTypeList/11) 63

A BRIEF HISTORY: PRA AND THE CHARACTERIZATION OF UNCERTAINTIES

• What drove us to where we are now?

• What are some of the major milestones?

64

PRA History: Challenges and Responses RIDM issues (e.g., realism, heterogeneity, aggregation)

PostFukushima issues (e.g., external hazards)

New/advanced reactors (e.g., conduct of operations)

Modern Applications Characterizing the fleet (variability) Expansion Across Developing confidence for mainstreaming RIDM Industry Filling known gaps (completeness uncertainty) Early Clarifying meaning: models and results PRAs Quantifying accident probability Means to communicate risk Hanford to WASH1400 1940 1950 1960 1970 1980 1990 2000 2010 2020 65

From Hanford to WASH1400 Technical Challenges: 1) Quantifying accident probability

2) Means to communicate risk WASH740 Hanford AEC/NRC Credible Accident UKAEA Estimates:

not in the generation OpE (pessimistic) of the ACRS members Decomposition present (optimistic)

Recommend: Farmer Curve WASH1400 accident System chain System reliability reliability SGHWR analysis studies studies analysis 1950 Windscale 1960 1970 TMI2 1980 For more information: T.R. Wellock, A Figure of Merit: Quantifying the Probability of a Nuclear Reactor Accident, 66 Technology and Culture, 58, No. 3, July 2017, pp. 678721.

WASH1400 Uncertainties (Level 1)

WASH1400: it is reasonable to believe that the WASH1400 Uncertainties (Estimated*)

core melt probability of about 5x105 per reactoryear predicted by this study should not be significantly larger and would almost certainly not exceed the value 5th 50th 95th Surry of 3x104 which has been estimated as the upper mean bound for core melt probability.

Peach Bottom Risk Assessment Review Group (NUREG/CR0400):

1.E05 1.E04 1.E03 We are unable to define whether the overall CDF (/ry) probability of a core melt given in WASH1400 is high or low, but we are certain that the error bands are *Based on data from Tables V 314 (PWR) and 316 (BWR) of Appendix V, assuming distributions are lognormal; median values are somewhat higher than reported in Section 7.3.1 of the Main Report.

understated. We cannot say by how much.

67

Some Early Developments and PRAs Challenges: 1) Filling known gaps (completeness uncertainty)

2) Clarifying meaning: models and results Biblis Sizewell

(+aircraft)

(+DI&C) USDOE Clinch River Oyster Creek NRC Indian Point (LMFBR) (+seismic)

(full scope)

US Industry AIPA Forsmark International Limerick (HTGR) Koeberg Zion Millstone Other Notable

(~WASH1400) (full scope)

Seabrook Super (full scope)

Phénix RSSMAP/IREP (FBR DHR) TMI1 Oconee (full scope)

Apostolakis Kaplan/ (full scope)

Fleming (subjective Garrick EC/JRC Benchmarks (factor) probability) (risk) NUREG/CR2300 (systems, CCF, HRA) 1975 TMI2 1980 1985 Chernobyl 68

Sample Level 1 Results Display 69

Sample Results - SubModel Uncertainty Effect Effects of fire model (COMPBRN) uncertainty on fire growth time N. Siu, "Modeling Issues in Nuclear Plant Fire Risk Analysis," in EPRI Workshop on Fire Protection in Nuclear Power Plants, EPRI NP 70 6476, J.P. Sursock, ed., August 1989, pp. 141 through 1416.

Sample Results - Model Uncertainty (User Effect)

Early core melt, containment cooling Early core melt, no containment cooling Damage State Frequency (/yr), Review 104 Late core melt, containment cooling Late core melt, no containment cooling Containment bypass Steam generator tube rupture Direct containment failure 106 Internal Events External Events 108 1.E03 1.E03 1.E04 1.E04 1.E05 1.E05 Review 1.E06 1.E06 Review 1.E07 1.E07 1010 1.E08 1.E08 1.E09 1.E09 1.E10 1.E10 1.E11 1.E11 1.E11 1.E10 1.E09 1.E08 1.E07 1.E06 1.E05 1.E04 1.E03 1.E11 1.E10 1.E09 1.E08 1.E07 1.E06 1.E05 1.E04 1.E03 1010 108 106 104 Original Original Damage State Frequency (/yr), Original Data source: G.J. Kolb, et al., Review and Evaluation of the Indian Point Probabilistic Safety Study, NUREG/CR2934, December 1982.

71 (ML091540534)

Severe Expansion Across Industry (US)

Accident Policy Technical challenges: 1) Characterizing the fleet (variability)

Statement 2) Developing confidence for mainstreaming RIDM Safety Goal PRA Policy NRC Policy Statement Statement US Industry GL 8820 GL 8820 Supplement 4 NUREG1560 NUREG1742 NUREG1150 NUREG1150 (draft) (final) 1982 ASP Plant Class Models SPAR Models IPEEEs IPEs 1985 Chernobyl 1990 1995 2000 9/11 72

NUREG1150 Estimated* Uncertainties (Level 1)

Model Uncertainty Model Uncertainty

• Notes: totals shown in this

1) NUREG1150 does not aggregate the hazardspecific results. The totals shown are rough estimates assuming that the NUREG1150 distributions are lognormal.

73 2) The WASH1400 distributions are based on data from Tables V 314 (PWR) and 316 (BWR) of Appendix V, assuming that the distributions are lognormal. The median values are somewhat higher than reported in Section 7.3.1 of the Main Report

IPE/IPEEE - Variability Across Fleet Internal Events + Internal Floods Total 40 40 BWR BWR PWR PWR 30 30 Number Number 20 20 10 10 0 0 1x106 3x106 1x105 3x105 1x104 3x104 1x103 1x106 3x106 1x105 3x105 1x104 3x104 1x103 CDF (/ry) CDF (/ry) 74

The Modern Era (US)

Technical challenges: 1) RIDM issues (e.g., realism, heterogeneity, aggregation)

SECY98144 2) PostFukushima issues (e.g., external hazards)

3) New/advanced reactors (e.g., conduct of operations)

RG 1.174 NUREG2150 ASME PRA NRC Risk Standard NTTF Request US Industry Informed for Information ROP NUREG1855 (Reevaluations) 10 CFR 50.48(c)

NFPA 805 (Fire Protection) NFPA 805 LARs (Fire Protection)

SAMAs (Life Extension)

RiskInformed License Amendment Requests (LARs)

SPAR Models 2000 9/11 2005 2010 Fukushima 2015 2020 75

Variability in Recent Results (Level 1) 0.35 0.30 Population Mean:

4.7x105 0.25 Fraction of Plants 0.20 0.15 0.10 Lowest Highest Reported: Reported:

0.05 3.5x106 1.3x104 0.00 6.0 5.5 5.0 4.5 4.0 3.5 3.0 1E6 1E5 1E4 1E3 CDF (per reactor year) 76

Variability in Results - Comparison with IPE/IPEEE 1E3 0.001 0.50 Total CDF (IPE + IPEEE)

NFPA 805 Fraction of PRAs 0.40 IPE/IPEEE 0.30 1E4 0.0001 0.20 0.10 0.00 1 2 3 4 5 6 7 8 9 10 0.01 0.1 1 10 100 1000 1E5 0.00001 1E5 1.00E05 1E4 1.00E04 1E3 1.00E03 Fire CDF/Internal Events CDF Total CDF (Recent LARs) 77

Parameter, Model, and Completeness Uncertainty:

A Practical Categorization mod*el, n. a M (Model of the World): representation of reality created with a specific Scope, structure objective in mind.

i: Parameters A. Mosleh, N. Siu, C. Smidts, and C. Lui, Model

Universe Uncertainty: Its Characterization and Quantification, Center for Reliability Engineering, University of Maryland, College Park, MD, 1995. (Also NUREG/CP0138, 1994)

PRA models for NPPs

• Typically an assemblage of sub models with parameters

• Implicitly include issues considered but not explicitly Known Unknowns quantified Unknown Unknowns 78 For more discussion, see snippet on Treatment of Uncertainties

PRA History - Concluding Remarks NPP PRA:

• Has decades of experience with analyses and decision support applications

• Is strongly advocated and widely used internationally

• Has evolved in response to theoretical and practical challenges and will likely continue to do so with new challenges 79

PRA LESSONS FROM OPERATIONAL EXPERIENCE

• How can information from operational experience help PRA?

• How has this been explored and what has been learned?

• What might we do next?

See N. Siu, Nuclear Power Accidents and Incidents: Lessons for PRA, Research Seminar, University of Illinois UrbanaChampaign 80 (virtual), February 2, 2021 (ML20339A570) for a full seminar slide set.

OpE Input to Risk Assessment Operational Experience

(> statistics)

Adapted from NUREG2150 Other Considerations

• Current regulations

• Safety margins

• Defenseindepth

• Monitoring Quantitative Qualitative 81

Some Reactor Fuel Damage Accidents and Incidents*

Windscale 1 TMI 2 Fukushima Daiichi 13 Graphite Pile, UK PWR, US BWRs, Japan UMetal Fire Loss of Feedwater EQ + Tsunami, Loss of Power Fermi 1 Chernobyl 4 LMR, US RBMK, Ukraine Flow Blockage Reactivity Accident Leningrad 1 RBMK, Russia Reactivity Accident St. Laurent 1 Bohunice A1 Paks 2 GCR, France HWGCR, Slovak Republic VVER, Hungary Fuel Misload Fuel Loading Accidents Spent Fuel Pool Accident 1950 1960 1970 1980 1990 2000 2010 2020 82 *Events involving fuel damage at power and/or production reactors

And Some Other Rancho Seco PWR, US Madras 2 PHWR, India Maintenance Error Tsunami Serious Incidents* LOFW, TMI precursor LOUHS Gundremmingen A Turkey Point 3 & 4 H.B. Robinson B/F Bleed and feed cooling VVER, East Germany PWR, US PWR, US LOCA Loss of coolant accident Training Error Storm (Hurricane) Bus Fire (Arc)

LOFW Loss of feedwater Partial LOOP, RV LOCA LOOP RCP Seal Challenge LOMCR Loss of main control room LOOP Loss of offsite power Browns Ferry 1 & 2 Narora Maanshan Fukushima Daiichi 5 LOUHS Loss of ultimate heat sink RV Relief valve BWR, US DavisBesse PHWR, India PWR, Taiwan BWR, Japan SBO Station blackout (loss of AC power) Cable Fire PWR, US Turbine Fire Storm (Spray) EQ + Tsunami Complicated Trip LOFW, no B/F SBO, LOMCR SBO Loss of all power LaCrosse Armenia Blayais 1 & 2 Cruas 24 Duane Arnold BWR, US VVER, Armenia PWR, France PWR, France BWR, US Switchyard Fire Cable Fire Storm (Wind + Flood) Flood (Debris) Storm (Wind)

Partial Uncovery SBO LOOP, Degraded UHS LOUHS LOOP 1950 1960 1970 1980 1990 2000 2010 2020 83 *Selected nonfuel damage events with challenges to core cooling

NPP OpE Narratives

• Incident databases ETH = Eidgenssische Technische Hochschule

- Many public (e.g., LERs, ETH) and nonpublic (e.g., IAEA IRS, IRS = Incident Reporting System ICES = INPO Consolidated Event System INPO ICES) sources

- Varying purposes (affecting fields, entry criteria), degrees of Selected Reports on Fukushima:

coverage Cumulative Pages

- All contain narratives (unstructured text) 12000 10000

• OpE narratives 8000

- Content: subjective but potentially rich; can stimulate AND Pages 6000 4000 temper imagination (possible mechanisms and scenarios) 2000

- Volume: ranges from terse (passing mentions) to overwhelming 0

- Perspectives and usefulness for PRA: varied 03/11/2011 03/10/2012 03/10/2013 03/11/2014 03/11/2015 03/11/2016 03/11/2017 03/11/2018 Date 84

Text Mining Cautions The big issue was the hydrogen bubble...

• Be aware of 2020 hindsight, a.k.a.

- MMQB (Monday Morning Quarterbacking)

- I knew it all along syndrome as a barrier to learning Wasnt there

• Factual information is often uncertain, limitations can a major persist in later records human error?

- Simplifications

- Inconsistencies

- Factual errors

• Postevent judgments are subject to normal human biases

- Confirmation bias

- Underestimation/undervaluation of uncertainty

• Reviews

- Often reflect technical discipline perspectives

- Often used to assess blame rather than identify opportunities for improvement 85

Some OpE Mining Case Studies

• PRAoriented reviews of

- 30 fire events*

- Great East Japan Earthquake and Tsunami (2013, 2016)

- Selected storm and flood events (2018)

- Selected seismic events (20192020

• General Objectives

- Develop insights (observed mechanisms, scenarios) to support PRA technology development

- Support staff learning (familiarization with events, PRA Last two case studies approaches)

- Support future activities (e.g., smart tool development) 86

Insights Relevant to PRA Technology Case studies :

• Strengthened basis for many previously recognized messages (e.g., potential importance of external hazards, errors of commission)

• Identified instances where (depending on the decision problem) PRA scope might need to be extended (e.g., multisite events, longduration events)

• Identified mechanisms/scenarios needing multidisciplinary attention (e.g., multiple shocks, induced hazards, scenario dynamics)

• Identified phenomena potentially warranting PRA community attention (seismicallyinduced reactivity excursions, seismicallyinduced HEAFs*)

• Identified previously unrecognized/underpublicized precursors to Fukushima (Hinkley Point, Turkey Point, Blayais)

• Identified potential need for supplementary measures/means to highlight incidents (boost the signal) for PRA community attention

• The possibility of a seismicallyinduced HEAF has been recognized due to the 2007 KashiwazakiKariwa (station transformer) and the 2011 Onagawa (nonsafety switchgear) events. The insights are: a) generating mechanisms for observed nonseismically induced HEAFS 87 might be activated by a seismic event, and consequentially b) seismicallyinduced HEAFs might be risk significant (based on the impact of the Maanshan 2001 nonseismic HEAF).

Knowledge Management and Knowledge Engineering Tool Insights Connect the dots

• Knowledge Management

- Useful learning experience for all participants

- Demonstrated value of multidisciplinary perspectives

- Would have benefitted from increased team interactions

• Knowledge Engineering Tools Where does it say ?

12000

- Still need deep subject matter expert (SME) expertise to 10000 connect the dots, develop insights (not yet just analytics) 8000 Pages 6000

- Tools need to deal with enormous, heterogeneous database 4000

- With humanintheloop implementation, could use improved 2000 0

tools for screening documents, prioritizing remainder for further examination 03/11/2011 03/10/2012 03/10/2013 03/11/2014 03/11/2015 03/11/2016 03/11/2017 03/11/2018 Date 88

PRA Lessons from OpE - Concluding Remarks

• Not many NPP accidents and serious incidents, but perhaps more than realized

• Events illustrate how things can fail, sometimes by unexpected pathways and mechanisms

• Review of events

- Can inform PRA modeling (identification of possible scenarios)

- Can broaden knowledge base of reviewer

- Can support development of smart tools for data mining 89

Lessons From OpE - Extra Slides 90

Closing Remarks Reminder: Accidents are a real possibility Windscale 1 TMI 2 Fukushima Daiichi 13 Graphite Pile, UK PWR, US BWRs, Japan UMetal Fire Loss of Feedwater EQ + Tsunami, Loss of Power Fermi 1 Chernobyl 4 LMR, US RBMK, Ukraine Flow Blockage Reactivity Accident

[Before TMI] core damage was never never land Leningrad 1 RBMK, Russia Reactivity Accident R. Bari*

St. Laurent 1 Bohunice A1 Paks 2 GCR, France HWGCR, Slovak Republic VVER, Hungary Fuel Misload Fuel Loading Accidents Spent Fuel Pool Accident 1950 1960 1970 1980 1990 2000 2010 2020

• Plenary Panel: Perspectives on Nuclear Safety Since the Three Mile Island Event, ANS Intl Mtg Probabilistic Safety Assessment (PSA 2019), Charleston, SC, 2019.

91

Reminder: Accidents [often] have precursors Hinkley Point Blayais Fukushima Unpublicized a French problem Madras Unpublicized Leningrad Chernobyl Unconfirmed until 1990 TMI Rancho Seco TMI similarity recognized 1980*

1950 1960 1970 1980 1990 2000 2010 2020

• a twoyear old incident that could easily have resulted in an outcome as serious as that of the accident at Three 92 Mile Island. [R.M. Bernero and F.H. Rowsome, Single Failure Potentially Leading to Core Damage, memorandum to H.R. Denton and C. Michelson, U.S. Nuclear Regulatory Commission, March 14, 1980. (ML19323J370)]

Closing Remarks Reminder: Increasing Realism / Reducing Conserva sm

• Known gaps* in broad scenario categories Rationale Common Example(s)

Out of scope security/sabotage, operation outside approved limits Low significance (preanalysis judgment) external floods (many plants preFukushima)

Appropriate PRA technology unavailable management and organizational factors PRA not appropriate software, security

• Known gaps in treatment of contributors within categories Category Example(s)

External hazards multiple coincident or sequential hazards Human reliability errors of commission, nonproceduralized recovery Passive systems thermalhydraulic reliability

• Terminology of Guidance on the Treatment of Uncertainties Associated with PRAs in RiskInformed Decision Making, NUREG1855 Rev. 1, March 2017; 93 a.k.a. known unknowns

References

• N. Siu, Nuclear Power Accidents and Incidents: Lessons for PRA, Research Seminar, University of Illinois UrbanaChampaign (virtual),

February 2, 2021. (ML20339A570)

• S.P. Nowlen, M. Kazarians, and F. Wyant, Risk Methods Insights Gained from Fire Incidents, NUREG/CR6738, September 2001.

• N. Siu, D. Marksberry, S. Cooper, K. Coyne, and M. Stutzke, PSA technology challenges revealed by the Great East Japan Earthquake, Proceedings of PSAM Topical Conference in Light of the Fukushima DaiIchi Accident, Tokyo, Japan, April 1517, 2013. (Paper:

ML13038A203, Presentation: ML13099A347)

• N. Siu, K. Compton, S. Cooper, K. Coyne, F. Ferrante, D. Helton, D. Marksberry, and J. Xing, PSA technology reminders and challenges revealed by the Great East Japan Earthquake: 2016 update, Proceedings of 13th International Conference on Probabilistic Safety Assessment and Management (PSAM 13), Seoul, Korea, October 27, 2016. (Paper: ML16245A871, Presentation: ML16270A522)

• N. Siu, I. Gifford, Z. Wang, M. Carr, and J. Kanney, Qualitative PRA insights from operational events, Proceedings of 14th International Conference on Probabilistic Safety Assessment and Management (PSAM 14), Los Angeles, CA, September 1621, 2018. (Paper:

ML18135A109, Presentation: ML18249A340), NonPublic Report: ML18248A117)

• N. Siu, J. Xing, N. Melly, F. Sock, and J. Pires, Qualitative PRA Insights from Seismic Events, Proceedings 25th Conference on Structural Mechanics in Reactor Technology (SMiRT25), Charlotte, NC, August 49, 2019. (Paper: ML19162A422, Presentation: ML19210D835),

NonPublic Report: ML20309A718)

Note: Expanded versions of the PSAM 14 paper (storms and floods) and SMiRT25 paper (seismic events) can be found in nonpublic staff reports and public versions of these reports (ML21081A038 and ML21081A040, respectively) 94

RISKRELATED REGULATORY R&D (R4&D)

• What is the purpose of R4&D?

• How has R4&D supported NRCs riskinformed activities?

• Why can it be difficult to assess the potential benefits of R4&D?

95

NRC Uses of Risk Information PRA Policy Statement (1995)

Regulations

• Increase use of PRA technology in all and Guidance regulatory matters

- Consistent with PRA stateoftheart

- Complement deterministic approach, R&D support defenseindepth philosophy Operational Decision Licensing

• Benefits:

and (1) Considers broader set of potential challenges Experience Support Certification (2) Helps prioritize challenges (3) Considers broader set of defenses U.S. Nuclear Regulatory Commission, Use of Probabilistic Risk Assessment Methods in Nuclear Activities; Final Policy Statement, Federal Register, 60, p. 42622 (60 FR Oversight 42622), August 16, 1995.

96

Regulatory R&D in Decision Support Decision Typical products (regulatory research)

• Detailed

• Ways to look at and/or approach

• Problemdriven problems (e.g., frameworks, Specific

• Need it now Analyses methodologies)

• Points of comparison (e.g., reference Methods, Models, calculations, experimental results)

Tools, Databases, R&D

• Job aids (e.g., computational tools, Standards, databases, standards, guidance: best Guidance, practices, procedures)

• Broad

• Busy people => limited

• Problemspecific information (e.g.,

time for nonurgent results, insights, uncertainties) communication Foundational Knowledge Side benefits

• Potential future uses =>

• Education/training of workforce needs to persist

• Networking with technical community Regulatory Decision Support 97

R4&D Product Examples - Frameworks/Methodologies Dynamic PRA1 Model uncertainty -

• Inspired by accident quantification2 experience (TMI2,

• Focused on Chernobyl) between model

• NRCsponsored output and reality exploratory R&D

• Bayesian estimation (universities, labs)

• Includes user

• International effect as well as interest (WGRISK, fundamental IAEA) Time (s) Experiment (K) DRM (K) model/tool errors

• Futurefocused 180 360 400 465 450 510 research 720 530 560 840 550 565 1 N. Siu, Dynamic PRA for Nuclear Power Plants: Not If But When? U.S. Nuclear Regulatory Commission, March 2019. (ML19066A390; see also slides in this presentation) 98 2 E. Droguett and Ali Mosleh, Bayesian methodology for model uncertainty using model performance data, Risk Analysis, 28, No. 5, 1457 1476, 2008.

R4&D Product Examples - Reference Points Fire PRA maturity PRA lessons from and realism1 accidents/Incidents2

• Opinion papers

• Reviews of sparked by ongoing accidents and debate incidents for real Maturity vs. world scenarios and Realism mechanisms Available evidence 3/11/2011

• Quantitative and Fire events qualitative analyses Storms and floods

• Basis for later Earthquakes WGRISK Technical

• Also KM and KE Opinion Paper benefits 1 N. Siu, K. Coyne, and N. Melly, Fire PRA Maturity and Realism: A Technical Evaluation, U.S. Nuclear Regulatory Commission, March 2017. (ML17089A537; see also ML15035A678 and NEA 7417) 2 N. Siu, D. Marksberry, S. Cooper, K. Coyne, and M. Stutzke, PSA technology challenges revealed by the Great East Japan Earthquake, 99 Proceedings of PSAM Topical Conference in Light of the Fukushima DaiIchi Accident, Tokyo, Japan, April 1517, 2013. (ML13038A203; see also this presentations slides on OpE lessons)

R4&D Product Examples - Job Aids COMBPRN1 Content Analytics2

• Developed with

• Exploratory study of NRC support for Watson tech NPP fire PRA

• Unstructured (NUREG/CR2258) database (corpus)

• Zone model Identify and Time to target characterize multi (cable) damage unit events Uncertainty Current CDFs analysis

• Basis for input to

• Used in multiple current NRC AI/ML industry PRAs initiatives 1 N. Siu, "Probabilistic Models for the Behavior of Compartment Fires," NUREG/CR2269, 1981.

2 N. Siu, K. Coyne, and F. Gonzalez, Knowledge Management and Knowledge Engineering at a RiskInformed Regulatory Agency:

100 Challenges and Suggestions, U.S. Nuclear Regulatory Commission, March 2017. (ML17089A538; see also ML16355A373)

R4&D Product Examples - Decision Support Post9/11 studies Pressurized Thermal

• Shortterm analyses Shock1 to support orders

• Technical basis for

• Longerterm revision PTS confirmatory screening limit (10 analyses CFR 50.61)

• Advanced

• Quantitative communication Event sequence Video analysis Nonlinear T/H analysis (PT) papers PFM analysis (TWCF)

• Qualitative (Level 2) 1 M. EricksonKirk, et al, Technical Basis for Revision of the Pressurized Thermal Shock (PTS) Screening Limit in the PTS Rule (10CFR50.61):

Summary Report, NUREG1806, 2006.

101

From R4&D to RIDM SECY98144 RG 1.174 Safety Goal PRA Revised NUREG NUREG Policy Policy ROP 1860 2150 NUREG/CR ASME/ANS 2300 PRA Standard Indian Point PRA IPE/IPEEE NEI 1804 WASH NUREG Level 3 1400 1150 PRA 1970 1980 1990 2000 2010 2020 102

R4&D - From NearTerm to Blue Sky Activity Short Title DoB Notes Now Blue Sky Description

1. Already in use by some organizations; unknown effectiveness.

Automatic PRA 2. Topic previously suggested by FSR, accepted conceptually by DRA/PRAB Characterize current Model Light 3. Not requested by user offices, resisted by PRA old guard technologies 4. Challenges: developing understanding of technologies, obtaining information from Construction users (international, private)

Identify and 1. Widely recognized but vaguely characterized issue in move towards riskinformed Treatment of prioritize gaps and regulation; activity goes beyond current practices

2. Concept likely to have broad support Uncertainty in potential Light 3. Challenges: prioritizing gaps considering ability to do something (quantifying model PRA improvement uncertainty, reducing completeness uncertainty, improving communication of activities uncertainties, )

Prepare for future 1. Old research concept enabled by improved computational capabilities, external (U.S.

& international) R&D investments.

applications of more Light Dynamic PRA 2. Likely a feature of some advanced reactor applications.

simulationoriented Moderate 3. Pushed by R&D community, interesting to nonPRA types, resisted by PRA old guard PRA 4. Challenge: demonstrating sufficient value

1. Not a new concept, but tools are better; could involve application of NRCs Level 3 Characterize value PRA model.

2. Could examine some fundamental concepts (e.g., single failure criterion, Risk Impact of of regulatory Moderate containment)

Regulation requirements by risk 3. Likely to be resisted by some staff.

impact 4. Challenges: defining risk metrics, treatment of uncertainties, extending lessons DoB = f{technological readiness, beyond single plant.

clarity of application, Piloting Explore advanced 1. Current approaches involve flatland displays (possibly animated) and storytelling.

user skepticism} Potential for advanced technologies (AR, VR, multisensory inputs) not yet discussed.

Through technologies for risk Strong 2. Possible resistance from decision makers Hyperspace communication 3. Challenge: completely unknown potential benefits 103

Concluding Remarks

• R4&D is an essential element of NRCs continuing efforts to increase its use of risk information in regulatory decision making

• R4&D has many purposes, longterm as well as shortterm

- Ways to look at and/or approach problems

- Points of comparison

- Job aids

- Problemspecific information

• The benefits of R4&D can be disruptive, but also unforeseen and delayed

• Blue Sky proposals are welcome: submit to the NRCs FutureFocused Research Program 104

R4&D - EXTRA SLIDES 105

Degrees of Blue - More R4&D Examples Natural Hazards Decision Making SimulationBased AIassisted RIDM Extreme Hazards Advanced techniques for Climate Change risk communication Correlated Hazards Advanced metrics for RIDM DoB = f{technological readiness, Blue User clarity of application, Sky Needs user skepticism}

Automatic model construction AIbased data mining Dynamic PRA Errors of Commission Autonomous Reactors Full simulationbased PRA Org Factors in PRA Human/Org Factors Computational Methods 106

More Product Examples - Frameworks/Methodologies NRCsponsored Fire PRA Technology Neutral R&D (universities) Framework

• Started after Browns

• Explored use of risk Ferry fire (1975) metrics to identify

• Developed fire PRA licensing basis events approach first used in

• Inspiration and part Zion and Indian Point basis for current PRAs (early 80s), same Licensing general framework Modernization used today Program

• Started path leading to riskinformed fire protection (NFPA 805) 107

More Product Examples - Reference Points NUREG1150 SOARCA

• Continuing point of

• Detailed analysis of comparison for potential severe Level 1, 2, 3 results accidents and offsite

• Expectations consequences (ballpark)

• Updated insights on

• Basis for regulatory margins to QHOs Peach Bottom analysis (backfitting, generic issue resolution)

NUREG1150 (Surry)

Surry Sequoyah 108

More Product Examples - Methods/Models/Tools SPAR IDHEASG

• Independent plant

• Improved support for specific models qualitative analysis (generic data)

• Explicit ties with cognitive

• Allhazards (many) science (models, data)

• Support SDP, MD 8.3,

• General framework for ASP, GSI, SSC studies developing focused

• Adaptable for specific applications (e.g., IDHEAS circumstances ECA)

SAPHIRE

• Benefits from NPP simulator studies

• General purpose

• Consistent with current modelbuilding tool HRA good practices

• Multiple user guidance (NUREG1792) interfaces From https://en.wikipedia.org/wiki/SAPHIRE 109

From R4&D to RiskInformed Fire Protection RG 1.205 NUREG/CR NUREG 10 CFR 50.48(c) 2258, 2269 1150 ASME/ANS 10 CFR 50.48 PRA Standard Appendix R NUREG/CR6850 BTP 9.51 EPRI 1011989 RG 1.75 NFPA 805 Indian IPEEE NFPA 805 LARs Point PRA Browns 1970 Ferry 1980 1990 2000 2010 2020 110

TREATMENT OF UNCERTAINTIES

• What does treatment mean?

• What are the current approaches? Challenges?

• Can we do better? How?

111

Decision Making Under Uncertainty Safety Security p11 Environment

• Uncertainties C11 Cost p12 Reputation

- About outcome of alternative C12

- Conditioned on situation, A1 p1M 11 state of knowledge C1M

• Aim: treat uncertainties p21 C21 to ensure1 p12 A2 C22

- Effectiveness (best alternative) p2N

- Efficiency C2N

- Stakeholder confidence pij = P{outcome Cijlsituation, knowledge}

1Alternatively, can bounce approaches against the Principles of Good Regulation: independence, openness, efficiency, clarity, reliability 112

Treatment > Characterization Use in Decision Characterization Communication Making 113

Characterizing Uncertainties - A Pragmatic Framework M (Model of the World):

Scope, structure i: Parameters

Universe Known Unknowns Unknown Unknowns 114

Characterization Challenges Early core melt, containment cooling Early core melt, no containment cooling

• Parameter Uncertainties Late core melt, containment cooling Late core melt, no containment cooling

- Raw data preprocessing (selection and Containment bypass Steam generator tube rupture interpretation) Direct containment failure

- Potentially nonintuitive Bayesian updating Data source: G.J. Kolb, et al., Review and Evaluation of the Indian Point results Probabilistic Safety Study, NUREG/CR 2934, December 1982. (ML091540534)

- Stateofknowledge dependencies

- Appropriate simplification: expert elicitation Opinions

• Model Uncertainties 1) Failure to use uncertainty characterization

- Serious consideration of alternative models best or even good practices provides

- Mainstreaming of quantitative approaches an easy target for critics, can affect

• Completeness Uncertainties stakeholder confidence.

2) Pro forma, cookbook analyses can miss

- Systematic identification of gaps potentially useful insights.

- Serious efforts to reduce (transition to model) 115

Communicating Uncertainties Will somebody find me a onehanded scientist?!

Senator Edmund Muskie

• Content/format depend on audience and expected use (Concorde hearings, 1976)

- Different (and changing?) levels of comfort with

• Uncertainty

• Formal frameworks (parameter/model/completeness; aleatory/epistemic; probabilistic/nonprobabilistic) and displays

- Different decisions => different information

• Fundamental questions

- How confident am I (the analysis team) in the key results, insights, and implications?

- Why should you (the decision maker) be confident in my characterization?

Quote from I. Flatow, Truth, Deception, and the Myth of the OneHanded Scientist, October 18, 2012. Available from:

116 https://thehumanist.com/magazine/novemberdecember2012/features/truthdeceptionandthemythoftheonehandedscientist

Communication Challenges

• Simpler than risk communication (fewer dimensions, perhaps less visceral reaction) Hurricane Model Warning:

• Meeting the Goldilocks Principle Useful Advice or Just Venting?

- Are different types (per characterization frameworks) important to the decision? If anything on these products causes confusion, ignore the

- Does too much uncertainty information cause a loss of attention? entire product.

Reduce salience of key results, insights, implications?

- Does too little uncertainty information breed overconfidence or even a suboptimal decision?

• Designing to increase cognitive engagement

- Likely important for major, nonroutine decisions

- Active learning => reduce package polish to encourage audience internal processing and dialog with team?1 1 Of course this possible (and untested) approach requires a willing audience.

117

Using Uncertainty Information in Decision Making (An Outsiders View)

• Deliberative vs. Naturalistic Decision Making

• Structured approaches

- MultiAttribute Utility Theory (late 1960s,1 used Adapted from NUREG2150 by ASCE?)

- Simplifications (e.g., Analytic Hierarchy Process,2 early 1980s)

- Nowadays?

• Technical and social influences

- Needs of problem

- Views on uncertainty information (e.g., useful or confusing or even obfuscating?) From NUREG2114

- Heuristics and biases 1 See for example H. Raiffa, Decision Analysis: Introductory Lectures on Choices under Uncertainty, AddisonWesley, New York, 1968. (NRC Technical Library HD69.D4 R13 c.1) 118 2 See for example T.L. Saaty, Decision Making for Leaders: The Analytical Hierarchy Approach for Decisions in a Complex World, Lifetime Learning, Belmont, CA, 1982. (HD30.23 .S24 c.1)

Challenges in Using Uncertainty Information (An Outsiders View)

• Demonstrating value of/creating demand for beyond pro forma treatment

• Balancing

- Rulebased (repeatable, transparent)

- Knowledgebased (optimal use of evidence)

• Effective communication

- With providers (what is the question)

- With stakeholders (basis for decision) 119

Treatment of Uncertainties - Concluding Remarks

• Treatment covers characterization, communication, and use

• A longstanding concern for RIDM with

- Accepted practices

- Remaining challenges

• Improved methods and tools for treatment

- Are feasible

- Will provide better support for agency transformation

- May need culture change for investment and use 120

Treatment of Uncertainties - Extra Slides 121

Uncertainties and Decision Making (Two Days Before Landfall)

Andrew (1992) Irma (2017)

Hurricane Warning Hurricane Watch Evacuated Hurricane tracks adapted from University of WisconsinMilwaukee (https://web.uwm.edu/hurricanemodels/models/archive/)

122 Emergency response based on data from National Hurricane Center:

(https://www.nhc.noaa.gov/1992andrew.html)

Parameter Uncertainty: Current Practice

• Treatment involves Estimation (including expert elicitation, Bayesian updating)

Propagation

• Straightforward mathematics and mechanics

• Some practical challenges 123

Parameter Uncertainty Challenges

• Data preprocessing Runtime Failures (MotorDriven Pumps)

- Selection 1.00

- Interpretation Probability Density Function 0.80 0.60

• Effect of analysis shortcuts 0.40 (Normalized)

- Standard (e.g., noninformative) 0.20 prior distributions 0.00 1.00E09 1.00E08 1.00E07 1.00E06 1.00E05 1.00E04 1.00E03

- Simplified expert elicitation Failure Rate (/hr)

- Independence assumption Service Water Normally Running Standby

• Ensuring correspondence with

• 2015 Industrywide estimates from: https://nrcoe.inl.gov/resultsdb/AvgPerf/

stateofknowledge

• Service Water Pumps: 2 failures in 16,292,670 hours0.00775 days <br />0.186 hours <br />0.00111 weeks <br />2.54935e-4 months <br /> Normally Running Pumps: 225 failures in 59,582,350 hours0.00405 days <br />0.0972 hours <br />5.787037e-4 weeks <br />1.33175e-4 months <br />

- Basic events (micro view)

• Standby Pumps (1st hour operation): 48 failures in 437,647 hours0.00749 days <br />0.18 hours <br />0.00107 weeks <br />2.461835e-4 months <br />

- Overall results (macro view) 124

Model Uncertainty: Hurricane Andrew 8/22/1992, 1200 UTC Current Practice Adapted from University of Wisconsin Milwaukee (https://web.uwm.edu/hurricane models/models/archive/)

• Important to acknowledge and treat (in context of decision)

• Multiple approaches

- Consensus model

- Sensitivity analysis

- Weighted alternatives (e.g., SSHAC)

- Output uncertainties Adapted from V.M. Andersen, Seismic Probabilistic Risk Assessment Implementation Guide, EPRI 3002000709, Electric Power Research Institute, M.H. Salley and A. Lindeman, Verification and Palo Alto, CA, December 2013 Validation of Selected Fire Models for Nuclear Power Plant Applications, NUREG1824 Supplement 1/EPRI 3002002182, November 2016.

125

Quantification of Model Output Uncertainty Time (s) Experiment (K) DRM (K)

• Bayesian methods 180 400 450

- Framework consistent with overall PRA Data 360 465 510 720 530 560

- Early approaches used in past PRAs 840 550 565

- Can address practical issues (e.g., non Temperature (K) homogeneous data)* Assume Assume Non

• Challenges include Percentile Homogeneous Homogeneous Output Uncertainty Data Data

- Uncertainties in unmeasured parameters 1st 415.2 372.8

- Submodel limits of applicability 5th 437.5 400.7

- Representativeness of computed results 50th 457.1 470.5 95th 479.7 559.4

- Use in actual decision making 99th 509.1 608.7

• See E. Droguett and Ali Mosleh, Bayesian methodology for model uncertainty using model performance data, 126 Risk Analysis, 28, No. 5, 14571476, 2008.

Model Uncertainty Commentary

• Model uncertainties can be large; importance depends on decision

• Some practical NPP RIDM approaches (e.g., Hurricane Irma: 9/8/2017, 0000 UTC (about 2 days before FL landfall) consensus models, deterministic screening) can understate uncertainties

• Ensemble approaches (with SMEdetermined Outer best estimate) used by other disciplines prediction is closest

• Subjective probability framework => to actual course

- Need to consider user effect

/ Plot adapted from University of WisconsinMilwaukee (https://web.uwm.edu/hurricanemodels/models/archive/)

- Raises question regarding fundamental meaning of weighted average approaches 127

Completeness Uncertainty: NUREG1855 Rev. 1 (2017)

Current Practice Options:

• Progressive analysis (screening, bounding,

• Recognized concerns conservative, detailed)

- Known gaps (missing scope)

• Change scope of risk

• Scenario categories informed application

• Contributors within categories RG 1.174 Rev. 3 (2018)

- Unknown gaps

• Treatment (Mind the Gap)

- Analysis guidance

- Additional analysis/R&D

- Riskinformed decision making 128

NPP PRA Known Gaps1

• Broad scenario categories Rationale Common Example(s)

Out of scope security/sabotage, operation outside approved limits Low significance (preanalysis judgment) external floods (many plants preFukushima)

Appropriate PRA technology* unavailable management and organizational factors PRA not appropriate software, security

• Contributors within categories Category Example(s)

External hazards multiple coincident or sequential hazards Human reliability errors of commission, nonproceduralized recovery Passive systems thermalhydraulic reliability 1aka Known Unknowns 129

Completeness Uncertainty: Possible R&D

• Continue to develop technology to address Event (NUREG/CR4839), 1992 known gaps Aircraft impact Avalanche

- Riskinformed prioritization Coastal erosion

- Fully engage appropriate disciplines Drought External flooding

- Take advantage of general computational and Extreme winds and tornadoes methodological developments Fire

• Facilitate reemphasis on searching Fog Forest fire

- Demonstrate efficiency and effectiveness with Frost Hail current tools (e.g., MLD, HBFT) vs. High tide, high lake level, or high checklist/screening river stage

- Develop improved tools (including OpE mining) 130

Different Perspectives: Logarithmic vs Linear Displays 131

ADDITIONAL RESOURCES 132

Selected Lectures, Seminars, and Talks1,2

• Nuclear Power Accidents and Incidents: Lessons for PRA, Research Seminar, University of Illinois UrbanaChampaign (virtual),

February 2, 2021. (ML20339A570)

• PRA and RiskInformed Decision Making at the NRC: Some Trends and Challenges, Nuclear Engineering Research Seminar (Virtual),

North Carolina State University, Raleigh, NC, October 22, 2020. (ML20293A370)

• PRA and RiskInformed Decision Making at the NRC: Some Trends and Challenges, Modeling, Experimentation, and Validation Summer School (Virtual), Idaho National Laboratory, July 27, 2020. (ML20195B157)

• Technology for the Treatment of Uncertainties: History, Status, Commentary and Challenges, prepared for CRIEPI/NRRC and OECD/NEA Workshop on the Proper Treatment of Uncertainties in Safety Analyses, Tokyo, Japan, May 2627, 2020 (postponed, new date TBD). (ML20080N774)

• PRA and RiskInformed Decisionmaking at the NRC: Some Trends and Challenges, B.J. Garrick Institute for Risk Sciences, University of California, Los Angeles, February 21, 2020. (ML20035G249)

• Research and Integrated Decision Making (IDM): A Personal Perspective, Workshop: Integrated and RiskInformed Decisionmaking Forum for Managers, U.S. Nuclear Regulatory Commission, November 13, 2019. (ML19310F243)

• Dynamic PRA: Not If But When? Invited Talk, IAEA Workshop on Advanced PSA Approaches and Applications, Alkmaar, The Netherlands. September 913, 2019. (ML19248C656)

• Nuclear Power Plant Probabilistic Risk Assessment (PRA) and RiskInformed Decision Making (RIDM), Independent Activities Period, Massachusetts Institute of Technology, January 1623, 2019.3 (ML19011A416)

• Advanced Knowledge Engineering Tools to Support Probabilistic Risk Assessment (PRA) Activities A Whole New World? NRC Knowledge Management (KNOWvember) Webinar, November 21, 2017. (Webinar Video)

• PRA R&D - Changing the Way We Do Business? Invited Plenary Lecture, ANS International Topical Meeting on Probabilistic Safety Assessment (PSA 2017), Pittsburgh, PA, September 2428, 2017. (ML17292A552) 1The ML numbers refer to pdf versions. PowerPoint versions (with fullresolution graphics) will be provided in a separate ADAMS package.

133 2Although some of the talk titles are duplicative, the material has been tailored to the different audiences and venues.

3Lectures, workshops, and reference material for a 1week intensive course (meant to cover material normally provided in a semester).