ML21138A792: Difference between revisions

From kanterella
Jump to navigation Jump to search
← Older editNewer edit →
VisualWikitext
StriderTol Bot change
StriderTol Bot change
Line 16: Line 16:


=Text=
=Text=
{{#Wiki_filter:}}
{{#Wiki_filter:Short Takes:
Snippets on Some PRA Topics*
Nathan Siu Senior Technical Adviser for PRA Analysis Office of Nuclear Regulatory Research Division of Risk Analysis RES Staff Technical Seminar (Virtual) - Part 2 May 13, 2021 (3:004:00)
* The views expressed in this presentation are not necessarily those of the U.S. Nuclear Regulatory Commission.
 
The Menu
* Dynamic PRA                                                                                                                    Notes
: 1) Each snippet provides a 1520 minute talk on the
* Identifying Scenarios - Weird Stuff and the                                                          subject.
Importance of Active Searching                                                                  2) Most snippets are accompanied by extra slides providing additional details.
* Internal Risk Communication                                                                    3) Links to additional presentations (pdf versions) are
* A Brief History: PRA and the Characterization                                                        provided in the Additional Resources portion of this slide set.
of Uncertainties                                                                                4) PowerPoint presentations (with fullresolution
* PRA Lessons from NPP Accidents and                                                                    graphics and fullfunctioning navigation links) will be uploaded into an ADAMS package.
Incidents                                                                                            a. Slides from this seminar
* RiskRelated Regulatory R&D (R4&D)                                                                    b. Presentations identified in Additional Resources
* Treatment of Uncertainties                                                                            c. Miscellaneous additional presentations, snippets, and notes1
* Additional Resources 1Extended and illustrated notes on a subject not intended as an actual presentation but provided in PowerPoint form for convenience.
2
 
DYNAMIC PRA
* What is it?
* Why do we care?
* Where are things now?
3
 
Fukushima Daiichi 1, 3/11/2011: Static Description
{Loss of Power}                                                                CORE DAMAGE AND {Loss of Isolation Condenser}                                  CD AND {Failure of Alternate Cooling}                        LOSS OF DECAY HEAT REMOVAL LODHR
= {Core Damage}
LOSS OF ALL      LOSS OF ISOLATION            FAILURE OF AC AND DC POWER          CONDENSER                ALT COOLING LOP              Ext LOIC              Ext  FALTC            Ext 4
 
Adding Time, Motion (Kinematics)
Emergency  Isolation Actions to Actions to      Offsite LOOP                                                                  EDG  LongTerm Power  Condenser  Extend      Shed          Power (Seismic)                                                            Recovery  Cooling (EDGs)      (IC)    IC Ops    DC Loads    Recovery Earthquake and LOOP (T = 0:00)
LOOPEQ      EPS        ISO      EXT        DCL          OPR        DGR        LTC
* Tsunami (T+0:40)                                                                                                                            1 2 CD
* Loss of all power (T+0:50)                                                                                                                  3 4 CD 5
* IC outboard valve closed (T+3:40)*                                                                      12 hr 12 hr 6
7 CD CD
* Core damage (T+4:00, estimated                                                                                                              8 9 CD postaccident)                                                                                          8 hr 10 11 CD 8 hr 12 CD 13 14 CD
* What but not Why                                                                                    4 hr 15 16 CD
  - Closure of isolation condenser                                                                                    4 hr 17 CD 18
  - Delay in implementing alternate                                                                                                          19 CD 20 cooling (fire pumps)                                                                                1 hr 1 hr 21 CD 22 CD 5  *Manual action that had little actual effect; inboard valves already closed
 
Dynamic Interactions => Context => Why Time            Hazard                Systems                      Indications              Operators/Workers                      ERC/ER team                            EP Time 14:46  0:00 Earthquake    Scram MSIVs close, turbine trips, EDGs 14:47  0:01                                                  Rx level drops start and load RV pressure decreases; RV level 14:52  0:06                ICs start automatically in normal range 40 minutes between    Cooldown earthquake rate exceeding techand tsunami; 15:03  0:17                ICs removed from service                                          Manually remove IC from service transition fromspec    confident limits        control to disbelief Disaster HQ established in TEPCO 15:06  0:20 Tokyo Determine only 1 train IC 15:10  0:24 needed; cycle A train First tsunami 15:27  0:41 arrives Second tsunami 15:35  0:49 arrives 15:37  0:51                Loss of AC                                              Degradation and failure over time, 15371550: Gradual loss of instrumentation, indications gradually affecting operator 15:37  0:51                                                                                  Determine HPCI unavailable (including IC valve status, RV                          information        and ability to control level), alarms, MCR main lighting TEPCO enters emergency plan 15:42  0:56                                                                                                                                                    (loss of AC power); ERC established D/DFP indicator lamp indicates 16:35  1:49 "halted" Review accident management      Cannot determine RV level or      Review accident management procedures, start developing    injection status; work to restore  procedures, start developing  Declared emergency (inability to 16:36  1:50 procedure to open containment    level indication; do not put IC in procedure to open containment determine level or injection) vent valves without power        service                            vent valves without power 6
 
Dynamic PRA - What Is It?
* Dynamic PRA  PRA that explicitly treats interac ons among system elements and resulting motions (including rates of change), e.g.,
            - Hardware component transitions (e.g., available to unavailable, or even intermediate states)
            - Changes in operating crew situation awareness                        Plant        I&C        Crew
            - Changes in plant thermalhydraulic state
* Degree of treatment of dynamics => continuum of analyses, e.g.,
Environment
            - Current PRA (phenomenological submodel; some direct Complexity dependencies, e.g., support systems)
            - Taskoriented network models and simulations
            - Largely mechanistic simulations with stochastic elements  Frequent conceptualization of dynamic PRA 7
 
Dynamic PRA - Potential Benefits
* Additional insights (suggesting alternative risk management strategies), e.g.,
  - Untreated mechanisms (e.g., feedback loops)
  - Timing of key events
  - Causes of key events
* Fewer intermediate and often conservativelyoriented simplifications (e.g., discretization, success criteria)
  - More realistic results
  - Improved use of available evidence (what we know) => improved DM confidence
* Directly supportive of phenomenological whatif and optimization analyses, e.g.,
  - Assessing effect of different parameter values (e.g., ATF properties, arrival times for offsite aid)
  - Identifying potentially troublesome ranges of parameter values (cliffedge effects)1
* Modeling in disciplinespecific terms (native language)
  - Reduced chance of translation errors
  - Increased stakeholder involvement and buyin
* Engineering trends (integrated simulation) 1Analysis requires coupling of dynamic PRA model with appropriate mathematical searching and optimization tools.
8
 
Dynamic PRA - Where Are We?
* Strong interest: academia, international
* NPPs: tools, demonstrations
* Nonnuclear: decision support applications (e.g., aerospace, hydropower)
Methodologies for Current PSA (Phased Mission,        Dynamic PSA                              HighFidelity, Competing Risks,          Tools and                            SimulationBased Level 3 PSA)            Toolboxes                              Dynamic PSA Late                              Intermediate                              Early (Mature, Stable)                    (Adolescent, Developing)                (Infancy, Emerging)
Developmental Stage 9
 
Dynamic PRA - Concluding Remarks
* All NPP accidents have involved significant dynamic interactions among system elements
* Explicit treatment of these interactions can benefit PRA studies and the PRA enterprise
* Work (particularly decision support applications) is needed to achieve these benefits 10
 
Dynamic PRA - Extra Slides 11
 
Indicators of Technology Maturity1 Early                                              Intermediate                                                  Late (Infancy, Emerging)                                (Adolescent, Developing)                                        (Mature, Stable)
Many welltrained and experienced practitioners Small research community                                Larger number of practitioners Recognize limits of applicability of Small number of practitioners                          Larger number of experienced Practitioners          Strong personality influences,                          researchers methods Can adapt methods to new situations competing schools of thought Can work with researchers to identify important issues New practicedriven research problems Most research driven by needs of Driven by perceived needs                              Some consensus positions for some practice Research              Problem selection affected by personal                  broadly defined problem areas More abstract research addresses Agenda                  choice (e.g., due to ease of formulation                Some unproductive research lines needs clearly identifiable by all or solution)                                            abandoned concerned Incomplete coverage of topics Fast growth Local applications (addressing small                                                                              Vocabulary has evolved Developing vocabulary Applications            parts of larger problems)                                                                                        General framework exists Optimistic views on new methods; No broader framework                                                                                              Little selling of area limitations not well understood 1 Adapted from: Cornell, C.A., Structural safety: some historical evidence that it is a healthy adolescent, Proceedings of Third 12      International Conference on Structural Safety and Reliability (ICOSSAR 81), Trondheim, Norway, June 2325, 1981.
 
U2: start depressurization (stuck RV, then continue)                                                  Browns Ferry 1 & 2 U2: D DG tripped, multiple boards lost 19750322 U2: control panel malfunctions, scram, turbine trip, FW trip, MSIVs close                              U2: conditions                          U2: shutdown stabilized                              cooling established U1: start depressurization U1: FW tripped, HPCI and RCIC stopped, use CRD pump                                                      U1: enter RB U1: scram, 2/3 FW pumps                                                          to assess SSC    U1: RV control tripped, multiple boards lost                                                    conditions      restored U1: spurious alarms, actuations                          U1: loss of operating relief valves                    U1: shutdown cooling established Fire reported to U1/U2 MCR                    U1: prepare for RHR cooling                                        (15 hr, 50 min)
OFD notified      TVA notified                                                Fire out CSR CO2 discharge                          CSR fire out, resume            Start using water RB firefighting Smoke, CO2 enter MCR Fire start 0                        2                    4                      6                      8              10                    12 13 Time from Start (hr)
 
Loss of EFW (burned cable          Start laying temporary Greifswald 1 to 2nd EFP)            cable to power EFPs                19751207                    EFW restored Trial and error fault diagnosis                                                          (power to EFPs) actions => more failures (including instrumentation)                                                Close primary hot leg main gate valves, start DGs start, power to                                                              forced circulation 1/2 emergency buses Natural circulation:    MCR power restored; Turbine                          use SVs and cold FW    pressurizer SVs open, 2/6 fail to reclose;                      Stable trip                            to control primary      emergency cooling pump started                                  cooling Start firefighting Corridor Fire                          Fire                                                                              ventilation alarm                          out                                                                              restored Heavy smoke, need respirators Fire start, spread 0                        2                        4                6                      8                    10                12 14 Time from Start (hr)
 
Start laying temporary power cable from                                  Armenia 1&2 U2 DG to U1 emergency makeup pump 19821015 Station          U1: only instrumentation is                              Feedwater makeup to Blackout          primary pressure (local station)                          SGs (temporary cable)
Loss of main coolant pumps, MCR Power to U1 emergency lights, readouts, alarms, phones, makeup pump from DG power, normal and emergency makeup Manual            SG SRV            Operators manually open SG dump valves                                    MCR power trip U1&2          opened            in upper TB (wearing breathing masks)                                      restored Offsite                                  Break cable spreading Fire out FBs arrive                                room wall to access fire FB arrives, open MCR                TB, transformer fires          Fire hatch to spray vault                under control                  controlled H2, transformer explosions Fire start,    Smoke              MCR smoke spread        in MCR            unbearable 0                        2                      4                    6                      8              10            12 15 Time from Start (hr)
 
Blayais 14 U1: shutdown                                                                          19991227 U4: 400 kV restored Level 2 Emergency Plan activated for U1; utility U1: Train A ESWS                                    and regulator national emergency teams pumps submerged                                      activated; agree to SG cooldown strategy U2: 400 kV restored U1 & U2: LHSI and CSS      Regulator          Walkdown discovers U1 Train A pump rooms flooded          informed of U1 &    ESWS pumps submerged U2 status and SG U2 & U4: Loss of 400 kV            cooldown            Use fire engines to assist                                            225 kV power (grid instability), scram          strategy            in pump floodwaters                                                  restored (U1U U1U4: Loss of 225 kV                Level 1 Emergency Plan activated: onsite pumps power (fallen trees)                  for floodwaters, recover submerged equipment U4: High          Site access regained; needed tide alarm        offsite workers can arrive                                              Floodwater pumping (continues to ~50 hr)
Flood overtops dyke, site access lost 0        2        4          6      8        10      12      14        16      18        20    22        24      26      28        30 16                                                    Time from Start (hr) 19:00                    0:00                                                                                                        0:00 12/27                    12/28                                                                                                      12/29
 
Dynamic PRA Fukushima Daiichi 1 20110311 Relative Time                Hazard              Systems                  Indications          Operators/Workers              ERC/ER team                    EP Time 14:46  0:00    Earthquake    Scram MSIVs close, turbine trips, Rx level drops (1 of 3)      14:47 14:52 0:01 0:06 EDGs start and load ICs start automatically    RV pressure decreases; RV level in normal range 15:03  0:17 40 minutes between earthquake and tsunami; ICs removed from service Cooldown rate exceeding Manually remove IC from tech spec limits          service transition from confident control to disbelief                                                Disaster HQ established in 15:06  0:20 TEPCO Tokyo Determine only 1 train IC 15:10  0:24 needed; cycle A train First tsunami 15:27  0:41 arrives Second tsunami 15:35  0:49 arrives 15:37  0:51                  Loss of AC 15371550: Gradual loss of                            Determine HPCI instrumentation,                                    Degradation unavailable          and failure over time, 15:37  0:51                  indications (including IC                          gradually affecting operator valve status, RV level),
alarms, MCR main lighting                          information and ability to control TEPCO enters emergency 15:42  0:56                                                                                                                              plan (loss of AC power);
ERC established D/DFP indicator lamp 16:35  1:49 indicates "halted" Review accident          Cannot determine RV level Review accident            Declared emergency management procedures,    or injection status; work to management procedures, (inability to determine start developing          restore level indication; do start developing        level or injection) 16:36  1:50                                              procedure to open        not put IC in service        procedure to open containment vent valves                                containment vent valves without power                                          without power 17                                                                                                                                                                              17
 
Dynamic PRA Fukushima Daiichi 1 20110311    Time Relative Hazard              Systems                Indications        Operators/Workers                ERC/ER team                    EP Time (2 of 3)      16:45 16:55 1:59 2:09 Tsunami alert Determine RV level Workers on way to check Emergency cancelled D/DFP had to turn back Lose ability to determine                                Reentered emergency plan 17:07  2:21 External influence                    RV level or injection status Site superintendent directs 17:12  2:26                          triggering work                                                    investigation of using fire protection to inject water 17:15  2:29 stoppage, temporary                                                  Estimated core uncovery in 1 hr Tsunami alert evacuation, 17:19  2:33 cleared                accountability Dieseldriven fire pump  Pressure above 100 psi Manually open valves (in started and left to idle                        dark) from fire protection system to core spray 17:30  2:44 system; take turns holding D/DFP switch to keep in standby Error3:32 18:18  of commission            (disabling DC power partially returned MO3A and MO2A indicate closed passive safety system) possibly MO3A and MO2A                                Open IC valves MO3A and 18:18  3:32                  opened                                          2A. Steam from condenser based on assumed low inventory                                            observed MO3A closed                                    Remove IC from service (usage)                                                                    (concerned about failing lines). Entered R/B and T/B to manually open MOV for 18:25  3:39 FP lineup. Hard time finding valve, had wrong key, hard to operate hand wheel. Long time.
18
 
Fukushima Daiichi 1 20110311    Time Relative Time Hazard          Systems Core damage (45 hr Indications            Operators/Workers              ERC/ER team                    EP 18:50  4:00 (3 of 3)      19:00  4:14 after trip)
Close valves for broken outdoor FP pipes. Broke Ask Tokyo for more fire engines lock to allow passage between Units 2 and 3.
Govt. declares nuclear 19:03  4:17 emergency InNohindsight,            core damage pressure indication in MCR; Reactor pressure =
20:07  5:21 Game 6.89 MPa (1000Over psi) local for 1F1; indication Small portable generatorcontinuing 1F1 MCR              recovery has temporary lighting 20:49  6:03 installed 20:50  6:04                                  activities and events impact                                                          Local authorities order evacuation within 2 km other        units Level indication      (1F2 and 1F3 core restored; 21:19  6:33                                    level = 0.20 m (8) above uncovery TAF              on 3/14)
Prime minister orders 21:23  6:37                                                                                                                        evacuation within 3 km; sheltering out to 10 km MO3A opened                                          Place IC in service; steam 21:30  6:44 observed Access to RB restricted due 21:51  7:05                                                                  to dose rates - indirect indication of core uncovery Level = 0.55 m (21.7) 22:00  7:14 above TAF Drywell pressure = 0.50                                  Restoration team from 23:50  9:04 MPa (87 psi) above design                                ERC enables reading Offsite power supply 23:59  9:13 trucks arrive by midnight 19
 
Earthquake Fukushima Daiichi 16 2nd Tsunami                                                        20110311 Request: Suspend      Order: Vent Seawater Injection      U1 and U2 Order:      Confirmed:
Local Evac. Local Evac.
U5 Rx              U2 Cont.          U5 Level =
Depressurizing          Venting          TAF + 0.95m SBO (U1U5)            U1 Cont. U3 Cont.                          U2 Core Loss of DC (U1U4)          Venting      Venting                          Uncovery U1 Core              U1 RB                U3 Core            U3 RB          U4 RB Damage (est.)          Explosion              Uncovery          Explosion      Explosion 3/11                3/12                  3/13                  3/14                  3/15 20
 
Fukushima Daiichi 16 20110311 U5 SFP Cooling Restored U6 SFP                                          SDF Truck Spray Cooling Restored                                        U4 SFP Earthquake                                U2 Core                                                    U4 SFP Lev Uncovery                                                  <0.5m above 2nd Tsunami U5 Level =        U4 RB 4/
SBO (U1U5)                    TAF + 0.95m      Explosion U1 Core      U1 RB    U3 Core        U3 RB Damage (est.) Explosion  Uncovery      Explosion 3/11        3/12        3/13      3/14          3/15        3/16 3/17  3/18  3/19    3/20          3/21 21
 
Local evacuation confirmed, 1st team dispatched Govt Start prep                            orders              2nd team dispatched, turned back (radiation) for venting                            venting              Unsuccessful attempts to open AO90 Open AO72 1.0 manual venting of Pressure (MPa) wetwell Containment Venting:
* Prevents catastrophic 0.5                                                                                    lower head failure failure pressurization from core                                steam dome
* Causes release to relocation to lower head                                drywell wetwell                                            environment RPVTEPCO steam line rupture                                          DWTEPCO WWTEPCO 0.0 0                    5                      10                        15                          20                        25                        30 3/11/2011                                                              Time (hr) 14:46 Adapted from: R. Gauntt, Fukushima Daiichi Accident Study: MELCOR Analyses and Results, OECD/NEA Fukushima Accident Analysis Workshop, IssylesMoulineaux, France, June 1820, 2012.
See also R. Gauntt, et al., MELCOR Simulations of the Severe Accident at the Fukushima 1F1 Reactor, ANS Winter Meeting and Nuclear Technology Expo, San Diego, CA, November 1115, 2012.
22
 
INTERESTING Clickbait IDENTIFYING SCENARIOS - WEIRD STUFF AND THE IMPORTANCE OF ACTIVE SEARCHING
* What is the concern?
* What tools are available?
* How might we do better?
23
 
Reminders
: 1) Risk = {si,Ci,pi}
Scenarios: what can go wrong? (qualitative)
: 2) All models are wrong, but some are useful.1
* What isnt in the PRA model wont be quantified
* What isnt conceived of might not be addressed in a riskinformed decision 1 G.E.P. Box and N.R. Draper, Empirical ModelBuilding and Response Surfaces, John Wiley and Sons, 1987. See the Wikipedia article All 24  models are wrong for background.
 
Youre analyzing a floating NPP. Have you thought of this one?
Chazhma Bay (August 10, 1985)1
* Echo II class submarine K431 is nearly done refueling. Fresh fuel has been loaded, workers are preparing to reattach 12ton reactor vessel head which has control rods attached.
* Workers see seal is not tight (there are leaks), decide to lift head using refueling ship crane. (Decision is against regulations and made without consulting supervisor. Did not drain primary loop to ensure no moderation, did not detach lattice used to keep control rods in place.)
* Passing torpedo boat creates large wake, rocks refueling ship; crane pulls control rods out of the core.
* Reactivity excursion causes steam explosion which blows head and fuel assemblies out of the reactor compartment, destroys the submarine pressure hull.
1See M. Takano, V. Romanova, H. Yamazawa, Y. Sivintsev, K. Compton, V. Novikov, and F. Parker, Reactivity Accident of Nuclear Submarine 25 near Vladivostok, Journal of Nuclear Science and Technology, Vol. 38, No. 2, pp. 143157 (February 2001).
 
What Can Go Wrong?
* PRA scenarios need a starting point (initiating hazard or event)
* Complementary methods to identify starting point:
  - Inductive (e.g., FMEA, HAZOP)
  - Deductive (e.g., Master Logic Diagram, Heat Balance Fault Tree)
  - Lists (e.g., possible hazards, actual events, other PRAs)
* Notes:
  - Conventional focus on postinitiator scenario can blur or even miss important factors in preinitiator buildup
  - Real events can involve unanticipated mechanisms and sequences of events that appear perfectly reasonable in hindsight. Click here for more examples.
26
 
Checklists can be useful but Aircraft impact              Local intense precipitation
* Might not actually be exhaustive                          Avalanche Biological events Low lake or river water level Low winter temperature
* Can be confusing (e.g., overlapping                    Coastal erosion Drought Meteor or satellite strike Onsite chemical release categories)                                            External fire External flooding Pipeline accident River diversion
* Can promote oneatatime consideration        Extreme winds and tornadoes Fog Sandstorm Seiche (actual events can involve multiple                      Forest fire Frost Seismic activity Severe temperatures categories)                                                  Hail                            Snow High summer temperature                  Soil shrinkswell
* Can be inefficient (e.g., excessive attention              High tide Hurricane Space weather Storm surge on ultimately unimportant categories)                    Ice cover                  Transportation accident Industrial/military facility accident            Tsunami
* Lengthy lists might trigger impulse to screen          Internal flooding Landslide Turbinegenerated missiles Volcanic activity rather than explore                                      Lightning 27
 
Active Searching
* Searching emphasized in the early days                                                        it is incumbent upon the new industry and of nuclear power                                                                            the Government to make every effort to recognize every possible event or series of
* A fundamental first principles attitude:                                                    events which could result in the release of using understanding of system, look for                                                      unsafe amounts of radioactive material to potential problems (rather than expect                                                      the surroundings and to take all steps them to be revealed by some analytical                                                      necessary to reduce to a reasonable minimum the probability that such events process) will occur in a manner causing serious
* Potentially valuable for new/novel                                                            overexposure to the public.
situations where operational experience                                                                                                W.F. Libby (1956)1 is weak or entirely lacking 1W. F. Libby (Acting Chairman, AEC) - March 14, 1956 response to Senator Hickenlooper [See D. Okrent, Reactor Safety, University of 28    Wisconsin Press, 1981. (NRC Technical Library TK9152 .O35, multiple copies)]
 
Hazard Identification Example: Checklist vs Active Search Checklist                                      Active Search (aka Red Teaming)
General Process Stepping through list                          Looking at undesired state (e.g., failure of
: 1) Ask what each hazard might do              key components), ask
: 2) Screen or retain for further analysis using 1) What conditions might cause this established criteria                            undesired state
: 2) What hazards or hazard combinations might create these conditions
: 3) If there are protective barriers preventing the undesired conditions, what might fail these barriers Advantages      More complete                                  More direct Methodical, easy to document                  Less restricted by categorization Engages imagination Challenges      Not wasting time on unimportant categories    Tempering imagination with plausibility Avoiding urge to screen (to finish the job)    Ensuring reasonable completeness 29
 
Active Search >> Drawing Fault Tree
* Need to identify plausible mechanisms
  - Possible failures can always be added to a fault tree
  - Reasonable causality needed for retention and quantification
* Examples
  - Operator disabling of safety systems (errors of commission)
  - Seismicallyinduced reactivity transients 30
 
Example: Disabling a BWR Isolation Condenser OPERATOR TERMINATES Possible ISOLATION CONDENSER OPERATION      but what ISOXHEEOCTERM reason?
31
 
Example: SeismicallyInduced Reactivity Excursion
* Observations
    - Global operational experience: at least 4 (perhaps 5)
North Anna Nuclear Generating Station earthquakes causing fluxinduced trips at 7 (perhaps 9) reactors1
    - Some reactor designs have unstable operating regimes
    - Systems with timedelayed feedback (e.g., restorative forces) can oscillate, even resonate
* Q: Can a seismic event induce a resonance leading to a runaway reaction? Under what conditions?
Adapted from: https://earthquake.usgs.gov/earthquakes/
1Ground motion trips either not available (e.g., power loss) or not triggered (e.g., accelerations are too low) 32
 
Example: SeismicallyInduced Reactivity Excursion Seismic Hazard Controls Neutronics Plausible?
Structures Operational Experience Thermal Hydraulics Systems Integration
* Movement? Bowing?
* Reactivity effects?
* Feedback?
* Resonance?        Perhaps not, but
* Fluid flow & density effects?
* Time scales?
* Excursion?
* Heat transfer effects?
ask the question 33
 
Looking Forward: OpE + Advanced Technology
* Empirical evidence: strong argument for plausibility
* Challenges
    - Enormous and growing database (not just nuclear)
    - Unstructured, natural language and heterogeneous (content, form, quality) data
    - Inferencing
* Exploratory study: advanced technology                                                            Adapted from:
1) 2)
https://str.llnl.gov/str/March02/March50th.html https://en.wikipedia.org/wiki/History_of_supercomputing#/media/File:Supercomputershistory.svg (AI/ML, Big Data) can help1                                                                    3)    https://www.top500.org/news/japancapturestop500crownarmpoweredsupercomputer/
1See, for example
* N. Siu, K. Coyne, and F. Gonzalez, Knowledge Management and Engineering at a Riskinformed Regulatory Agency: Challenges And Suggestions, white paper, U.S. Nuclear Regulatory Commission, 2017. (ML17089A538) 34
* F. Gonzalez and N. Siu, Accident Sequence Precursors: Current Analyses, Challenges, and Future Research, WGRISK Annual Meeting, NEA HQ, BoulogneBillancourt, France, March 2022, 2019. (ML19071A160)
 
Identifying Scenarios - Concluding Remarks
* A longstanding and continuing PRA goal: ensuring completeness
* An important mindset: active searching (especially when dealing with new/novel situations)
* Currently a variety of tools and resources to support searching; advanced technology (e.g., AI/ML) can lead to further improvements 35
 
Extra Slides - Examples of RealWorld Events and Mechanisms 36
 
External flooding: obvious now but back then?
* Fukushima Daiichi (1990s)
  - Added EDGs to supplement existing units (SAM modification)
  - Aircooled EDGs1 installed at Units 2, 4, 6; crossties provided with Units 1, 3, 5
  - All watercooled EDGs in building basements
  - Aircooled EDGs installed on ground floor, metalclad switchgear in basement                                                                                                      New          2011 EDG
* Great East Japan Earthquake (March 11, 2011)
  - Earthquake => LOOP                                                                                                    New M/C            2009 DB 10m
  - Tsunami => SBO for Units 14 (W/C EDGs, M/C switchgear)                                                              Switchgear        1972 DB
  - Unit 6 EDG supplies Units 5 and 6; air louver ~1m above tsunami height 1Not affected by loss of service water, e.g., due to tsunami. (Pumps are at elevation O.P.+4m.) Per IAEA Director Generals report, 37    choice of aircooled is due to current service water loads; unclear if diversity was a major factor.
 
Some Other Accidents Accident          Notable Mechanisms/Events Sodium Reactor    Reactor coolant pump organic coolant leaks into the primary circuit, causes flow blockages, Experiment (1959)  higher fuel temperatures, interaction with cladding and formation of a lowmelting temperature alloy, coolant channel blockage, fuel damage, and release of radioactive gases and some volatiles into the sodium coolant and eventually the environment.
Fermi 1 (1966)    Segments of zirconium sheets (installed late in construction as a safety barrier) tear loose during power ascension, blocking coolant flow. Two fuel assemblies melt. Following radiation alarms, reactor is manually scrammed.
Chernobyl 4 (1986) Interruption of a planned test due to offsite grid needs leads to Xenon poisoning, inability to achieve planned test conditions. Crew decides to proceed with the test despite the plant being in an unstable operating regime and disables an automatic scram to facilitate testing. A plant computer signal dictating immediate shutdown is ignored. The test initiates a positive reactivity excursion with a catastrophic steam explosion and core destruction some 44 seconds later.
38
 
Some Interesting Incidents Incident        Notable Mechanisms/Events Vogtle 1 (1988) Smoke detector actuation (burnoff in a duct heater) led to pressurization of a preaction deluge system in cable spreading room, water discharge through leakoff valves (as designed), water seepage through a floor penetration into the main control room, and spurious opening of a PORV at power. Floor penetration design was faulty - assumed sealwelding (of embedded seal angles and upper angle iron assembly) would be watertight. See LER 424/88016.
Indian Point 3  Activation of an outdoor deluge system (in response to a transformer explosion and fire) led to (2015)          bleed off water in a valve room adjacent to a vital 480V switchgear room. Due to insufficient drain system capacity, water backed up into the switchgear room. [Note: Although the water was not high enough to affect the switchgear, it constituted a potential industrial hazard that could have inhibited operator access to that room.] See Special Inspection Report ML15204A499.
39
 
Some RealWorld Mechanisms (1 of 4)
Mechanism  Plant (Year) Description Unexpected U.S. plant
* EDG oil fire due to fatigue cracking of undocumented instrumentation line.
/Unusual
* Failure occurred during followup examination of a reported small oil leak; line was Loadings                  moved slightly [cause?]
Nogent
* Unit 2 condenser circulating water system leak causes p between Turbine (2006)        Building foundation and floor, lifts floor, fails manhole.
* Water floods Unit 1 Turbine Building, enters ESW system gallery through penetrations, CCW pump room through drains.
Inadequate Forsmark 1
* Offsite switchyard twophase short circuit during maintenance causes LOOP Protective (2006)
* Inverters failed on overvoltage, causing loss of 2/4 trains of AC and DC power Systems 40
 
Some RealWorld Mechanisms (2 of 4)
Mechanism    Plant (Year) Description Secondary  Maanshan 1
* Salt spray caused LOOP; electrical fault caused highenergy arcing fault (HEAF), loss Hazards      (2001)        of faulted safety bus
* Heavy smoke from HEAF delayed access to switchgear room to restore power to undamaged safety bus => 2 hour station blackout Cruas 24
* Flood management actions lead to vegetation debris downstream, clogging of (2009)        service water intake
* Total loss of service water for Unit 4, partial loss for Units 2 and 3 Declared      Blayais 12
* During site flooding, rooms containing Unit 1 and Unit 2 lowhead safety injection Inoperability (1999)        and containment spray pumps partially flooded
* Pumps declared inoperable LaSalle 12
* Foreign material (injectable sealant foam) found on floor of service water tunnel (1996)
* Core standby cooling system, emergency core cooling system, and diesel generators declared inoperable, both units shutdown 41
 
Some RealWorld Mechanisms (3 of 4)
Mechanism Plant (Year) Description Worker    Point Beach
* Communications lost with diver working in Unit 2 (shutdown) circulating water Safety    (2000)          pump house Concerns
* Manual shutdown of Unit 1 U.S. plant
* Oil fire near reactor coolant pump
* Spurious evacuation alarm (smoke clogged radiation monitor)
* Reactor building evacuated Operator  Greifswald 1 During a severe power cable fire triggered by an electrician (performing a Choices  (1975)      demonstration for a trainee), operators manipulated switchgear to find intact cables for power (trial and error problem solving) but these actions caused additional failures TMI2        During a loss of feedwater event, operators throttled high pressure makeup in the (1979)      mistaken belief that the reactor coolant system was going solid 42
 
Some RealWorld Mechanisms (4 of 4)
Mechanism  Plant (Year) Description Operator    DavisBesse
* During a loss of feedwater transient, the shift supervisor did not implement Choices    (1985)          operating procedures for feed and bleed cooling (which would contaminate (cont.)                      containment), counting (correctly) on timely restoration of auxiliary feedwater
* In haste to enter the auxiliary feedwater pump room (accessed via a locked grate),
an equipment operator tossed keys to another ten feet ahead Vandellos    During a Turbine Building fire (hydrogen deflagration, cascading burning oil),
(1989)      operators (using breathing apparatus) entered dark, smoke filled areas to perform recovery actions Fukushima    Operators isolate the isolation condenser in the mistaken belief that it was close to Daiichi 1    drying out and failing (which would provide a direct release path to the environment)
(2011)
Maintenance Rancho      A maintenance worker dropped a lightbulb into a cabinet, shorting out nonnuclear Error      Seco (1978)  instrumentation. Propagating faults led to a scenario that could easily have resulted in an outcome as serious as that of the accident at Three Mile Island a year later 43
 
Some Resources
: 1. Fukushima Daiichi (2011): International Atomic Energy Agency, The Fukushima Daiichi Accident, Director General Report, Vienna, Austria, 2015.
: 2. Sodium Reactor Experiment (1959): P. Pickard, Sodium Reactor Experiment Accident July 1959, Sandia National Laboratories, August 29, 2009. (Available from:
https://www.etec.energy.gov/Library/Main/Pickard%20SRE%20presentation.pdf)
: 3. Fermi 1 (1966): Fermi Fuel Melt Accident, Nuclepedia.
: 4. Chernobyl 4 (1989): U.S. Department of Energy, Electric Power Research Institute, Environmental Protection Agency, Federal Emergency Management Agency, Institute of Nuclear Power Operations, and the U.S. Nuclear Regulatory Commission, Report on the Accident at the Chernobyl Nuclear Power Station, NUREG1250, January 1987.
: 5. Vogtle 1 (1988): Water Leakage into Control Room/Potential Exists for a Safety System Failure, Licensee Event Report 424/88016, November 22, 1988.
: 6. Indian Point 3 (2015): U.S. Nuclear Regulatory Commission, Indian Point Nuclear Generating - Special Inspection Report 05000286/2015010, July 23, 2015.
: 7. Nogent (2006): U.S. Nuclear Regulatory Commission, ConstructionRelated Experience with Flood Protection Features, IN 200906, July 21, 2009. (ML090300546)
: 8. Forsmark 1 (2006: U.S. Nuclear Regulatory Commission, Significant Loss of SafetyRelated Electrical Power at Forsmark, Unit 1, in Sweden, IN 200618, August 17, 2006.
: 9. Maanshan 1 (2001): Atomic Energy Council, Taiwan, The Station Blackout Incident of the Maanshan NPP Unit 1, April 18, 2001. (Available from:
https://www.aec.gov.tw/webpage/control/report/safety/safety_04_002.pdf)
: 10. Cruas 24 (2009): P. Dupuy, G. Georgescu, and F. Corenwinder, Treatment of the loss of ultimate heat sink initiating events in the IRSN Level 1 PSA, NEA/CSNI/R(2014)9, Nuclear Energy Agency, BoulogneBillancourt, France, 2014.
: 11. Blayais 12 (1999): Blayais Flood, Nuclepedia.
: 12. LaSalle 12 (1996): Foreign Material Injected Into Service Water Tunnel Causes Dual Unit Shutdown Due to Inadequate Work Control, Licensee Event Report 373/96008R01, November 25, 1996.
: 13. Point Beach (2000): Manual Reactor Trip Due to Concerns for Diver Safety, Point Beach Nuclear Plant Unit 1, Licensee Event Report 266/00010R00, November 22, 2000.
: 14. Greifswald 1 (1975): M. Rwekamp and E. Gelfort, Sicherheitsrelevanter Kabeltrassenbrand im Kernkraftwerk Greifswald  Beschreibung und Einschtzung, GRSVSR 24491, Gesellschaft f&#xfc;r Anlagen und Reaktorsicherheit (GRS) mbH, Kln, Germany, June 2004.
: 15. TMI2 (1979): D. Marksberry, F. Gonzalez, and K. Hamburger, Three Mile Island Accident of 1979 Knowledge Management Digest, Overview, NUREG/KM0001, rev. 1, U.S. Nuclear Regulatory Commission, June 2016.
: 16. DavisBesse (1985): U.S. Nuclear Regulatory Commission, Loss of Main and Auxiliary Feedwater Event at the DavisBesse Plant on June 9, 1985, NUREG1154, July 1985
: 17. Vandellos (1989): S.P. Nowlen, M. Kazarians, and F. Wyant, Risk Methods Insights Gained from Fire Incidents, NUREG/CR6738, U.S. Nuclear Regulatory Commission, September 2001.
: 18. Rancho Seco (1978): R.M. Bernero and F.H. Rowsome, Single Failure Potentially Leading to Core Damage, memorandum to H.R. Denton and C. Michelson, U.S. Nuclear Regulatory Commission, March 14, 1980. (ML19323J370) 44
 
INTERNAL RISK COMMUNICATION
* What is it?
* Why is it hard?
* How might we improve?
45
 
Internal Risk Communication: Support RIDM Adapted from NUREG2150 With To Other Considerations
* Current regulations
* Safety margins
* Defenseindepth
* Monitoring Quantitative
                                  +
Quantitative 46
 
Risk Information: Not Just for Current Decisions Prior (foundational) information affects DM processing of new information Specific Analyses
* Recognition
* Interpretation Methods, Models,
* Judging/Weighting Tools, Databases, Standards, Guidance, Foundational Knowledge 47
 
Risk Information: Inherently Complex
* Low likelihood => beyond personal Other Complications experience, intuition
* Heterogeneous
* Hyperdimensional                    o Qualitative and quantitative o Multiple views
    - Scenarios                          (organizations, disciplines)
    - Likelihood
* Dynamic
    - Multiple consequence measures    o System changes (e.g.,
different operational modes,
* Uncertain                              effects of decisions)
    - Sparse or nonexistent data      o New applications (and
    - Multiple models                    contexts)
    - Partial coverage 48
 
Other Challenges
* Individual user differences, e.g.,
  - Knowledge
  - Preferences/heuristics
* Social factors, e.g.,
  - Trust
  - Decision and group dynamics
* Situational context, e.g.,
  - Available time to consider
  - Decision support vs. informational Source: https://www.nrc.gov/readingrm/doccollections/commission/slides/2019/20190618/staff20190618.pdf 49
 
How to Ensure Message Capture and Retention?
External Flooding Fire
* Intrinsic value of content              High Winds
  - Risk level (absolute, relative)      Seismic
  - Risk importance (absolute, relative)  Internal Flooding          Internal Events
  - Surprise
* Communication process
  - Message formulation
  - Delivery method
  - Tools 50
 
Current Mechanisms Documents and          Interactive Presentations        Discussion (Flatland)      (Storytelling) 51
 
Can We Do Better? Different Documents?                  Graphic Elements Small Font Questions                                                    Sidebars Embedded Graphics Conventional  TwoColumn, Conversational                  Graphical 52
 
Can We Escape Flatland?
* Tufte model: use rich displays and reports, encourage user to explore
    - Promotes active involvement of decision maker
    - Increases general trust?
* A graduated technical approach to assist?
Interface                        Interaction Mode Hyperlinked dashboards, reports  Manual Time    Video (with sound?)              AI assist Visual immersion Multisensory immersion 53
 
From Static to Interactive Dashboard. Then to SciFi?
M. Korsnick, Risk Informing the Commercial Nuclear Enterprise, Promise of a Discipline: Reliability and Risk in Theory and in Practice, University of Maryland, April 2, 2014.                            Graphic adapted from https://www.flickr.com/photos/83823904@N00/64156219/
(permission CCBY2.0) 54
 
Internal Risk Communication - Concluding Remarks Risk Communication    Technical Communication Communication
* General communication good practices are helpful but not sufficient: special characteristics of risk information pose additional challenges
* Intuitively better approaches are being developed; scientific testing could be helpful
* Communication involves people: one risk communication solution may not work for all actors 55
 
Internal Risk Communication - Extra Slides 56
 
Risk Information: Qualitative + Quantitative*
Risk  { i ,                          i,        i}
* What can go wrong?
* What are the consequences?
* How likely is it?
  *Kaplan/Garrick triplet definition has been adopted by NRC. See:
White Paper on RiskInformed and PerformanceBased Regulation (Revised), SRM to SECY98144, March 1, 1999 57  Glossary of RiskRelated Terms in Support of RiskInformed Decisionmaking, NUREG2122, May 2013 Probabilistic Risk Assessment and Regulatory Decisionmaking: Some Frequently Asked Questions, NUREG2201, September 2016
 
Sources of Breakdowns: Risk Communication Between Risk Managers and Public*
* Differences in perception of information
    - Relevance
    - Consistency with prior beliefs
* Lack of understanding of underlying science
* Conflicting agendas
* Failure to listen
* Trust
    *J.L. Marble, N. Siu, and K. Coyne, Risk communication within a riskinformed regulatory decisionmaking environment, International 58  Conference on Probabilistic Safety and Assessment (PSAM 11/ESREL 2012), Helsinki, Finland, June 2529, 2012. (ADAMS ML120480139)
 
Differences in Perspective (Example)
Our tendency is to focus on things that are interesting and                                      Decision make them important. The thing that we have to do is focus                                        Makers on what really is important Ron Rivera, 2020 is (developer)
Whats interesting            might be (practitioner)            important            Practitioners          Developers isnt* (decision maker)
The PRA/RIDM Community
*Or, at least, isnt necessarily - interesting and important are independence.
59
 
External Flooding Preference: Avoid Chart Junk Fire High Winds Seismic
* Visual effects (e.g., noninformative 3D with perspective) can add          Internal pop but distract from or even distort messages.                          Flooding          Internal Events
* Advanced animation tools can be even stronger attention grabbers with even greater distraction potential
    - Focus attention on effects rather than message
    - Saturate audiences, leading to the need for even stronger effects in future presentations to grab attention
* Use effects with moderation (if at all), recognizing that your audience External
    - has preferences that vary from person to person and over time (maybe      High Flooding Fire Winds they prefer 3D charts!)
Seismic
    - is likely subject to many presentations besides yours (imagine the clamor of highly animated presentations seeking attention to their    Internal specific messages)                                                    Flooding                Internal Events 60
 
Spatial Information - Underused Resource?
* Common practice in everyday risk communication
* Going beyond - add changes over time?
61
 
An OftIgnored External Risk Communication Lesson:
Comparisons Dont Work for Everybody U.S. Annual Deaths, Various Causes (20102019) 700,000 600,000 500,000 400,000 Deaths                                                        (2020) 300,000 200,000 100,000 0
Flu    Auto    Guns  Cancer  COVID19 62
 
One Size Doesnt Fit All, Part II:
1,000 words (a story) > a picture?
On the evening of June 25, a freshly graduated high school      Drunk Driving Accident Fatalities (2018) star QB was going over 100 mph on a neighborhood road,        1,800 trying to go fast enough to avoid speed camera detection
("whipping"). Out of control on a sweeping curve, the car hit a fence and two trees, and flipped. Two unbelted passengers No Alcohol were ejected and died at the scene. The QB and the front          10,600 seat passenger were seriously injured. All four were                                                    BAC > 0.08 g/DL teenagers. All had just left an underage drinking party and                                            0.01 < BAC < 0.07 g/DL were drunk. The QB was indicted on counts of vehicular                            24,100 manslaughter, alcohol related vehicular homicide and causing a lifethreatening injury while driving under the influence of alcohol. The parent of the girl hosting the party, who was present and knowledgeable, pled guilty to two criminal        Data from "Traffic Safety Facts 2018 Data: State AlcoholImpaired citations for allowing underage drinking at his home and was  Driving Estimates," DOT HS 812 917, June 2020. (Available from:
ordered to pay $5,000 in fines.                                https://crashstats.nhtsa.dot.gov/#!/DocumentTypeList/11) 63
 
A BRIEF HISTORY: PRA AND THE CHARACTERIZATION OF UNCERTAINTIES
* What drove us to where we are now?
* What are some of the major milestones?
64
 
PRA History: Challenges and Responses RIDM issues (e.g., realism, heterogeneity, aggregation)
PostFukushima issues (e.g., external hazards)
New/advanced reactors (e.g., conduct of operations)
Modern Applications Characterizing the fleet (variability)  Expansion Across Developing confidence for mainstreaming RIDM Industry Filling known gaps (completeness uncertainty)          Early Clarifying meaning: models and results        PRAs Quantifying accident probability Means to communicate risk                Hanford to WASH1400 1940      1950              1960          1970              1980            1990          2000    2010      2020 65
 
From Hanford to WASH1400 Technical Challenges: 1) Quantifying accident probability
: 2) Means to communicate risk WASH740                                                                                            Hanford AEC/NRC Credible Accident UKAEA Estimates:
not in the generation OpE (pessimistic) of the ACRS members                    Decomposition present                          (optimistic)
Recommend:                                                                  Farmer Curve                      WASH1400 accident                                              System chain              System reliability            reliability                    SGHWR analysis                    studies                    studies                      analysis 1950                        Windscale        1960                                                1970                              TMI2 1980 For more information: T.R. Wellock, A Figure of Merit: Quantifying the Probability of a Nuclear Reactor Accident, 66      Technology and Culture, 58, No. 3, July 2017, pp. 678721.
 
WASH1400 Uncertainties (Level 1)
WASH1400: it is reasonable to believe that the                WASH1400 Uncertainties (Estimated*)
core melt probability of about 5x105 per reactoryear predicted by this study should not be significantly larger and would almost certainly not exceed the value                    5th              50th            95th Surry of 3x104 which has been estimated as the upper                                                mean bound for core melt probability.
Peach Bottom Risk Assessment Review Group (NUREG/CR0400):
1.E05                            1.E04                      1.E03 We are unable to define whether the overall CDF (/ry) probability of a core melt given in WASH1400 is high or low, but we are certain that the error bands are    *Based on data from Tables V 314 (PWR) and 316 (BWR) of Appendix V, assuming distributions are lognormal; median values are somewhat higher than reported in Section 7.3.1 of the Main Report.
understated. We cannot say by how much.
67
 
Some Early Developments and PRAs Challenges: 1) Filling known gaps (completeness uncertainty)
: 2) Clarifying meaning: models and results Biblis Sizewell
(+aircraft)
(+DI&C)                                                    USDOE Clinch River        Oyster Creek                                                                                    NRC Indian Point (LMFBR)              (+seismic)
(full scope)
US Industry AIPA            Forsmark                                                                                                International Limerick (HTGR)            Koeberg                          Zion Millstone                                        Other Notable
(~WASH1400)                    (full scope)
Seabrook Super                                                                    (full scope)
Ph&#xe9;nix                                  RSSMAP/IREP (FBR DHR)                                                                                                            TMI1 Oconee (full scope)
Apostolakis                      Kaplan/                            (full scope)
Fleming                      (subjective                    Garrick                                                  EC/JRC Benchmarks (factor)                  probability)                    (risk)        NUREG/CR2300                          (systems, CCF, HRA) 1975                                        TMI2    1980                                                  1985 Chernobyl 68
 
Sample Level 1 Results Display 69
 
Sample Results - SubModel Uncertainty Effect Effects of fire model (COMPBRN) uncertainty on fire growth time N. Siu, "Modeling Issues in Nuclear Plant Fire Risk Analysis," in EPRI Workshop on Fire Protection in Nuclear Power Plants, EPRI NP 70 6476, J.P. Sursock, ed., August 1989, pp. 141 through 1416.
 
Sample Results - Model Uncertainty (User Effect)
Early core melt, containment cooling Early core melt, no containment cooling Damage State Frequency (/yr), Review 104                                                                        Late core melt, containment cooling Late core melt, no containment cooling Containment bypass Steam generator tube rupture Direct containment failure 106 Internal Events                                                                                External Events 108                                                                                                1.E03                                                                                        1.E03 1.E04                                                                                        1.E04 1.E05                                                                                        1.E05 Review 1.E06                                                                                        1.E06 Review 1.E07                                                                                        1.E07 1010                                                                                              1.E08                                                                                        1.E08 1.E09                                                                                        1.E09 1.E10                                                                                        1.E10 1.E11                                                                                        1.E11 1.E11  1.E10  1.E09  1.E08  1.E07  1.E06  1.E05  1.E04  1.E03                1.E11  1.E10  1.E09  1.E08  1.E07  1.E06  1.E05  1.E04  1.E03 1010          108            106            104                                                                      Original                                                                                        Original Damage State Frequency (/yr), Original Data source: G.J. Kolb, et al., Review and Evaluation of the Indian Point Probabilistic Safety Study, NUREG/CR2934, December 1982.
71                                        (ML091540534)
 
Severe                  Expansion Across Industry (US)
Accident Policy                  Technical challenges: 1) Characterizing the fleet (variability)
Statement                                              2) Developing confidence for mainstreaming RIDM Safety Goal                                          PRA Policy                        NRC Policy                                            Statement Statement                                                                              US Industry GL 8820 GL 8820            Supplement 4                        NUREG1560                NUREG1742 NUREG1150      NUREG1150 (draft)      (final)            1982      ASP Plant Class Models    SPAR Models IPEEEs IPEs 1985 Chernobyl                1990                            1995                      2000 9/11 72
 
NUREG1150 Estimated* Uncertainties (Level 1)
Model Uncertainty Model Uncertainty
  *Notes: totals shown in this
: 1) NUREG1150 does not aggregate the hazardspecific results. The totals shown are rough estimates assuming that the NUREG1150 distributions are lognormal.
73 2) The WASH1400 distributions are based on data from Tables V 314 (PWR) and 316 (BWR) of Appendix V, assuming that the distributions are lognormal. The median values are somewhat higher than reported in Section 7.3.1 of the Main Report
 
IPE/IPEEE - Variability Across Fleet Internal Events + Internal Floods                                                                    Total 40                                                                                40 BWR                                                                          BWR PWR                                                                          PWR 30                                                                                30 Number                                                                        Number 20                                                                                20 10                                                                                10 0                                                                            0 1x106  3x106  1x105  3x105  1x104  3x104  1x103                1x106  3x106  1x105  3x105  1x104  3x104  1x103 CDF (/ry)                                                                  CDF (/ry) 74
 
The Modern Era (US)
Technical challenges: 1) RIDM issues (e.g., realism, heterogeneity, aggregation)
SECY98144                                            2) PostFukushima issues (e.g., external hazards)
: 3) New/advanced reactors (e.g., conduct of operations)
RG 1.174 NUREG2150 ASME PRA                                      NRC Risk Standard        NTTF Request                  US Industry Informed for Information ROP                                    NUREG1855        (Reevaluations) 10 CFR 50.48(c)
NFPA 805 (Fire Protection)                            NFPA 805 LARs (Fire Protection)
SAMAs (Life Extension)
RiskInformed License Amendment Requests (LARs)
SPAR Models 2000  9/11              2005                      2010    Fukushima        2015                  2020 75
 
Variability in Recent Results (Level 1) 0.35 0.30 Population Mean:
4.7x105 0.25 Fraction of Plants 0.20 0.15 0.10 Lowest                                    Highest Reported:                                Reported:
0.05        3.5x106                                  1.3x104 0.00 6.0        5.5      5.0    4.5    4.0        3.5    3.0 1E6                    1E5              1E4                1E3 CDF (per reactor year) 76
 
Variability in Results - Comparison with IPE/IPEEE 1E3 0.001 0.50 Total CDF (IPE + IPEEE)
NFPA 805 Fraction of PRAs 0.40 IPE/IPEEE 0.30 1E4 0.0001 0.20 0.10 0.00 1  2        3  4      5  6        7  8        9  10 0.01      0.1          1          10          100        1000 1E5 0.00001 1E5 1.00E05            1E4 1.00E04          1E3 1.00E03                                    Fire CDF/Internal Events CDF Total CDF (Recent LARs) 77
 
Parameter, Model, and Completeness Uncertainty:
A Practical Categorization mod*el, n. a M (Model of the World):        representation of reality created with a specific Scope, structure              objective in mind.
i: Parameters                  A. Mosleh, N. Siu, C. Smidts, and C. Lui, Model
: Universe Uncertainty: Its Characterization and Quantification, Center for Reliability Engineering, University of Maryland, College Park, MD, 1995. (Also NUREG/CP0138, 1994)
PRA models for NPPs
* Typically an assemblage of sub models with parameters
* Implicitly include issues considered but not explicitly Known Unknowns                                      quantified Unknown Unknowns 78      For more discussion, see snippet on Treatment of Uncertainties
 
PRA History - Concluding Remarks NPP PRA:
* Has decades of experience with analyses and decision support applications
* Is strongly advocated and widely used internationally
* Has evolved in response to theoretical and practical challenges and will likely continue to do so with new challenges 79
 
PRA LESSONS FROM OPERATIONAL EXPERIENCE
* How can information from operational experience help PRA?
* How has this been explored and what has been learned?
* What might we do next?
See N. Siu, Nuclear Power Accidents and Incidents: Lessons for PRA, Research Seminar, University of Illinois UrbanaChampaign 80  (virtual), February 2, 2021 (ML20339A570) for a full seminar slide set.
 
OpE Input to Risk Assessment Operational Experience
(> statistics)
Adapted from NUREG2150 Other Considerations
* Current regulations
* Safety margins
* Defenseindepth
* Monitoring Quantitative Qualitative 81
 
Some Reactor Fuel Damage Accidents and Incidents*
Windscale 1                              TMI 2                                  Fukushima Daiichi 13 Graphite Pile, UK                          PWR, US                                      BWRs, Japan UMetal Fire                      Loss of Feedwater                          EQ + Tsunami, Loss of Power Fermi 1                          Chernobyl 4 LMR, US                        RBMK, Ukraine Flow Blockage                    Reactivity Accident Leningrad 1 RBMK, Russia Reactivity Accident St. Laurent 1              Bohunice A1                Paks 2 GCR, France          HWGCR, Slovak Republic        VVER, Hungary Fuel Misload          Fuel Loading Accidents  Spent Fuel Pool Accident 1950          1960              1970              1980            1990      2000              2010          2020 82  *Events involving fuel damage at power and/or production reactors
 
And Some Other                                              Rancho Seco PWR, US Madras 2 PHWR, India Maintenance Error                                            Tsunami Serious Incidents*                                    LOFW, TMI precursor                                            LOUHS Gundremmingen A                Turkey Point 3 & 4                      H.B. Robinson B/F  Bleed and feed cooling                        VVER, East Germany                  PWR, US                                PWR, US LOCA  Loss of coolant accident                          Training Error              Storm (Hurricane)                      Bus Fire (Arc)
LOFW  Loss of feedwater                            Partial LOOP, RV LOCA                  LOOP                          RCP Seal Challenge LOMCR Loss of main control room LOOP  Loss of offsite power Browns Ferry 1 & 2                        Narora          Maanshan        Fukushima Daiichi 5 LOUHS Loss of ultimate heat sink RV    Relief valve                                    BWR, US            DavisBesse    PHWR, India      PWR, Taiwan            BWR, Japan SBO  Station blackout (loss of AC power)            Cable Fire            PWR, US      Turbine Fire    Storm (Spray)        EQ + Tsunami Complicated Trip      LOFW, no B/F    SBO, LOMCR            SBO            Loss of all power LaCrosse                Armenia                        Blayais 1 & 2          Cruas 24          Duane Arnold BWR, US              VVER, Armenia                      PWR, France          PWR, France            BWR, US Switchyard Fire            Cable Fire                  Storm (Wind + Flood)    Flood (Debris)        Storm (Wind)
Partial Uncovery              SBO                      LOOP, Degraded UHS          LOUHS                LOOP 1950                1960                  1970                  1980                1990                2000                  2010              2020 83    *Selected nonfuel damage events with challenges to core cooling
 
NPP OpE Narratives
* Incident databases                                                    ETH = Eidgenssische Technische Hochschule
    - Many public (e.g., LERs, ETH) and nonpublic (e.g., IAEA IRS,    IRS = Incident Reporting System ICES = INPO Consolidated Event System INPO ICES) sources
    - Varying purposes (affecting fields, entry criteria), degrees of              Selected Reports on Fukushima:
coverage                                                                            Cumulative Pages
    - All contain narratives (unstructured text)                                12000 10000
* OpE narratives                                                                  8000
    - Content: subjective but potentially rich; can stimulate AND        Pages  6000 4000 temper imagination (possible mechanisms and scenarios)                    2000
    - Volume: ranges from terse (passing mentions) to overwhelming                  0
    - Perspectives and usefulness for PRA: varied                                        03/11/2011  03/10/2012  03/10/2013  03/11/2014  03/11/2015  03/11/2016  03/11/2017  03/11/2018 Date 84
 
Text Mining Cautions                        The big issue was the hydrogen bubble...
* Be aware of 2020 hindsight, a.k.a.
    - MMQB (Monday Morning Quarterbacking)
    - I knew it all along syndrome as a barrier to learning                                        Wasnt there
* Factual information is often uncertain, limitations can            a major persist in later records                                        human error?
    - Simplifications
    - Inconsistencies
    - Factual errors
* Postevent judgments are subject to normal human biases
    - Confirmation bias
    - Underestimation/undervaluation of uncertainty
* Reviews
    - Often reflect technical discipline perspectives
    - Often used to assess blame rather than identify opportunities for improvement 85
 
Some OpE Mining Case Studies
* PRAoriented reviews of
  - 30 fire events*
  - Great East Japan Earthquake and Tsunami (2013, 2016)
  - Selected storm and flood events (2018)
  - Selected seismic events (20192020
* General Objectives
  - Develop insights (observed mechanisms, scenarios) to support PRA technology development
  - Support staff learning (familiarization with events, PRA    Last two case studies approaches)
  - Support future activities (e.g., smart tool development) 86
 
Insights Relevant to PRA Technology Case studies :
* Strengthened basis for many previously recognized messages (e.g., potential importance of external hazards, errors of commission)
* Identified instances where (depending on the decision problem) PRA scope might need to be extended (e.g., multisite events, longduration events)
* Identified mechanisms/scenarios needing multidisciplinary attention (e.g., multiple shocks, induced hazards, scenario dynamics)
* Identified phenomena potentially warranting PRA community attention (seismicallyinduced reactivity excursions, seismicallyinduced HEAFs*)
* Identified previously unrecognized/underpublicized precursors to Fukushima (Hinkley Point, Turkey Point, Blayais)
* Identified potential need for supplementary measures/means to highlight incidents (boost the signal) for PRA community attention
    *The possibility of a seismicallyinduced HEAF has been recognized due to the 2007 KashiwazakiKariwa (station transformer) and the 2011 Onagawa (nonsafety switchgear) events. The insights are: a) generating mechanisms for observed nonseismically induced HEAFS 87  might be activated by a seismic event, and consequentially b) seismicallyinduced HEAFs might be risk significant (based on the impact of the Maanshan 2001 nonseismic HEAF).
 
Knowledge Management and Knowledge Engineering Tool Insights Connect the dots
* Knowledge Management
    - Useful learning experience for all participants
    - Demonstrated value of multidisciplinary perspectives
    - Would have benefitted from increased team interactions
* Knowledge Engineering Tools                                                                        Where does it say ?
12000
    - Still need deep subject matter expert (SME) expertise to                  10000 connect the dots, develop insights (not yet just analytics)            8000 Pages    6000
    - Tools need to deal with enormous, heterogeneous database                  4000
    - With humanintheloop implementation, could use improved                  2000 0
tools for screening documents, prioritizing remainder for further examination                                                              03/11/2011    03/10/2012  03/10/2013  03/11/2014  03/11/2015  03/11/2016  03/11/2017  03/11/2018 Date 88
 
PRA Lessons from OpE - Concluding Remarks
* Not many NPP accidents and serious incidents, but perhaps more than realized
* Events illustrate how things can fail, sometimes by unexpected pathways and mechanisms
* Review of events
    - Can inform PRA modeling (identification of possible scenarios)
    - Can broaden knowledge base of reviewer
    - Can support development of smart tools for data mining 89
 
Lessons From OpE - Extra Slides 90
 
Closing Remarks Reminder: Accidents are a real possibility Windscale 1                                                          TMI 2                                                                          Fukushima Daiichi 13 Graphite Pile, UK                                                      PWR, US                                                                                  BWRs, Japan UMetal Fire                                              Loss of Feedwater                                                                    EQ + Tsunami, Loss of Power Fermi 1                                              Chernobyl 4 LMR, US                                            RBMK, Ukraine Flow Blockage                                        Reactivity Accident
[Before TMI] core                                          damage was never never land Leningrad 1 RBMK, Russia Reactivity Accident R. Bari*
St. Laurent 1                              Bohunice A1                                                Paks 2 GCR, France                      HWGCR, Slovak Republic                                        VVER, Hungary Fuel Misload                        Fuel Loading Accidents                                Spent Fuel Pool Accident 1950                    1960                        1970                          1980                        1990                        2000                        2010        2020
      *Plenary Panel: Perspectives on Nuclear Safety Since the Three Mile Island Event, ANS Intl Mtg Probabilistic Safety Assessment (PSA 2019), Charleston, SC, 2019.
91
 
Reminder: Accidents [often] have precursors Hinkley Point                            Blayais              Fukushima Unpublicized                      a French problem Madras Unpublicized Leningrad            Chernobyl Unconfirmed until 1990 TMI Rancho Seco TMI similarity recognized 1980*
1950            1960                  1970                1980                1990                2000                2010      2020
    *a twoyear old incident that could easily have resulted in an outcome as serious as that of the accident at Three 92 Mile Island. [R.M. Bernero and F.H. Rowsome, Single Failure Potentially Leading to Core Damage, memorandum to H.R. Denton and C. Michelson, U.S. Nuclear Regulatory Commission, March 14, 1980. (ML19323J370)]
 
Closing Remarks Reminder: Increasing Realism / Reducing Conserva sm
* Known gaps* in broad scenario categories Rationale                                                                      Common Example(s)
Out of scope                                                                  security/sabotage, operation outside approved limits Low significance (preanalysis judgment)                                      external floods (many plants preFukushima)
Appropriate PRA technology unavailable                                        management and organizational factors PRA not appropriate                                                          software, security
* Known gaps in treatment of contributors within categories Category                                                                      Example(s)
External hazards                                                              multiple coincident or sequential hazards Human reliability                                                            errors of commission, nonproceduralized recovery Passive systems                                                              thermalhydraulic reliability
      *Terminology of Guidance on the Treatment of Uncertainties Associated with PRAs in RiskInformed Decision Making, NUREG1855 Rev. 1, March 2017; 93  a.k.a. known unknowns
 
References
* N. Siu, Nuclear Power Accidents and Incidents: Lessons for PRA, Research Seminar, University of Illinois UrbanaChampaign (virtual),
February 2, 2021. (ML20339A570)
* S.P. Nowlen, M. Kazarians, and F. Wyant, Risk Methods Insights Gained from Fire Incidents, NUREG/CR6738, September 2001.
* N. Siu, D. Marksberry, S. Cooper, K. Coyne, and M. Stutzke, PSA technology challenges revealed by the Great East Japan Earthquake, Proceedings of PSAM Topical Conference in Light of the Fukushima DaiIchi Accident, Tokyo, Japan, April 1517, 2013. (Paper:
ML13038A203, Presentation: ML13099A347)
* N. Siu, K. Compton, S. Cooper, K. Coyne, F. Ferrante, D. Helton, D. Marksberry, and J. Xing, PSA technology reminders and challenges revealed by the Great East Japan Earthquake: 2016 update, Proceedings of 13th International Conference on Probabilistic Safety Assessment and Management (PSAM 13), Seoul, Korea, October 27, 2016. (Paper: ML16245A871, Presentation: ML16270A522)
* N. Siu, I. Gifford, Z. Wang, M. Carr, and J. Kanney, Qualitative PRA insights from operational events, Proceedings of 14th International Conference on Probabilistic Safety Assessment and Management (PSAM 14), Los Angeles, CA, September 1621, 2018. (Paper:
ML18135A109, Presentation: ML18249A340), NonPublic Report: ML18248A117)
* N. Siu, J. Xing, N. Melly, F. Sock, and J. Pires, Qualitative PRA Insights from Seismic Events, Proceedings 25th Conference on Structural Mechanics in Reactor Technology (SMiRT25), Charlotte, NC, August 49, 2019. (Paper: ML19162A422, Presentation: ML19210D835),
NonPublic Report: ML20309A718)
Note: Expanded versions of the PSAM 14 paper (storms and floods) and SMiRT25 paper (seismic events) can be found in nonpublic staff reports and public versions of these reports (ML21081A038 and ML21081A040, respectively) 94
 
RISKRELATED REGULATORY R&D (R4&D)
* What is the purpose of R4&D?
* How has R4&D supported NRCs riskinformed activities?
* Why can it be difficult to assess the potential benefits of R4&D?
95
 
NRC Uses of Risk Information PRA Policy Statement (1995)
Regulations
* Increase use of PRA technology in all and Guidance regulatory matters
                                              - Consistent with PRA stateoftheart
                                              - Complement deterministic approach, R&D support defenseindepth philosophy Operational  Decision Licensing
* Benefits:
and        (1) Considers broader set of potential challenges Experience    Support Certification (2) Helps prioritize challenges (3) Considers broader set of defenses U.S. Nuclear Regulatory Commission, Use of Probabilistic Risk Assessment Methods in Nuclear Activities; Final Policy Statement, Federal Register, 60, p. 42622 (60 FR Oversight                      42622), August 16, 1995.
96
 
Regulatory R&D in Decision Support Decision Typical products (regulatory research)
* Detailed
* Ways to look at and/or approach
* Problemdriven                                problems (e.g., frameworks, Specific
* Need it now        Analyses methodologies)
* Points of comparison (e.g., reference Methods, Models,            calculations, experimental results)
Tools, Databases,    R&D
* Job aids (e.g., computational tools, Standards,              databases, standards, guidance: best Guidance,                practices, procedures)
* Broad
* Busy people => limited
* Problemspecific information (e.g.,
time for nonurgent                                                    results, insights, uncertainties) communication                        Foundational Knowledge        Side benefits
* Potential future uses =>
* Education/training of workforce needs to persist
* Networking with technical community Regulatory Decision Support 97
 
R4&D Product Examples - Frameworks/Methodologies Dynamic PRA1                                                                          Model uncertainty -
* Inspired by accident                                                                  quantification2 experience (TMI2,
* Focused on Chernobyl)                                                                                between model
* NRCsponsored                                                                              output and reality exploratory R&D
* Bayesian estimation (universities, labs)
* Includes user
* International                                                                              effect as well as interest (WGRISK,                                                                          fundamental IAEA)                                    Time (s)  Experiment (K)  DRM (K)              model/tool errors
* Futurefocused                              180 360 400 465 450 510 research                                    720          530          560 840          550          565 1 N. Siu, Dynamic PRA for Nuclear Power Plants: Not If But When? U.S. Nuclear Regulatory Commission, March 2019. (ML19066A390; see also slides in this presentation) 98  2 E. Droguett and Ali Mosleh, Bayesian methodology for model uncertainty using model performance data, Risk Analysis, 28, No. 5, 1457 1476, 2008.
 
R4&D Product Examples - Reference Points Fire PRA maturity                                                                        PRA lessons from and realism1                                                                              accidents/Incidents2
* Opinion papers
* Reviews of sparked by ongoing                                                                      accidents and debate                                                                                  incidents for real Maturity vs.                                                                        world scenarios and Realism                                                                            mechanisms Available evidence                                                                      3/11/2011
* Quantitative and                                                                            Fire events qualitative analyses                                                                      Storms and floods
* Basis for later                                                                              Earthquakes WGRISK Technical
* Also KM and KE Opinion Paper                                                                            benefits 1 N. Siu, K. Coyne, and N. Melly, Fire PRA Maturity and Realism: A Technical Evaluation, U.S. Nuclear Regulatory Commission, March 2017. (ML17089A537; see also ML15035A678 and NEA 7417) 2 N. Siu, D. Marksberry, S. Cooper, K. Coyne, and M. Stutzke, PSA technology challenges revealed by the Great East Japan Earthquake, 99  Proceedings of PSAM Topical Conference in Light of the Fukushima DaiIchi Accident, Tokyo, Japan, April 1517, 2013. (ML13038A203; see also this presentations slides on OpE lessons)
 
R4&D Product Examples - Job Aids COMBPRN1                                                                          Content Analytics2
* Developed with
* Exploratory study of NRC support for                                                                  Watson tech NPP fire PRA
* Unstructured (NUREG/CR2258)                                                                  database (corpus)
* Zone model                                                                          Identify and Time to target                                                                    characterize multi (cable) damage                                                                  unit events Uncertainty                                                                    Current CDFs analysis
* Basis for input to
* Used in multiple                                                                  current NRC AI/ML industry PRAs                                                                    initiatives 1 N. Siu, "Probabilistic Models for the Behavior of Compartment Fires," NUREG/CR2269, 1981.
2 N. Siu, K. Coyne, and F. Gonzalez, Knowledge Management and Knowledge Engineering at a RiskInformed Regulatory Agency:
100  Challenges and Suggestions, U.S. Nuclear Regulatory Commission, March 2017. (ML17089A538; see also ML16355A373)
 
R4&D Product Examples - Decision Support Post9/11 studies                                                                            Pressurized Thermal
* Shortterm analyses                                                                        Shock1 to support orders
* Technical basis for
* Longerterm                                                                                    revision PTS confirmatory                                                                                screening limit (10 analyses                                                                                    CFR 50.61)
* Advanced
* Quantitative communication                                                                                Event sequence Video                                                                                        analysis Nonlinear                                                                                  T/H analysis (PT) papers                                                                                  PFM analysis (TWCF)
* Qualitative (Level 2) 1 M. EricksonKirk, et al, Technical Basis for Revision of the Pressurized Thermal Shock (PTS) Screening Limit in the PTS Rule (10CFR50.61):
Summary Report, NUREG1806, 2006.
101
 
From R4&D to RIDM                          SECY98144 RG 1.174 Safety Goal        PRA    Revised      NUREG NUREG Policy          Policy    ROP        1860    2150 NUREG/CR                                          ASME/ANS 2300                                          PRA Standard Indian Point PRA                          IPE/IPEEE                NEI 1804 WASH                    NUREG                                            Level 3 1400                      1150                                              PRA 1970      1980                1990            2000              2010          2020 102
 
R4&D - From NearTerm to Blue Sky Activity Short Title                            DoB    Notes Now                            Blue Sky                Description
: 1. Already in use by some organizations; unknown effectiveness.
Automatic PRA                                2. Topic previously suggested by FSR, accepted conceptually by DRA/PRAB Characterize current Model                                  Light  3. Not requested by user offices, resisted by PRA old guard technologies                  4. Challenges: developing understanding of technologies, obtaining information from Construction users (international, private)
Identify and                  1. Widely recognized but vaguely characterized issue in move towards riskinformed Treatment of  prioritize gaps and              regulation; activity goes beyond current practices
: 2. Concept likely to have broad support Uncertainty in potential              Light  3. Challenges: prioritizing gaps considering ability to do something (quantifying model PRA            improvement                      uncertainty, reducing completeness uncertainty, improving communication of activities                        uncertainties, )
Prepare for future            1. Old research concept enabled by improved computational capabilities, external (U.S.
                                                                                              & international) R&D investments.
applications of more  Light Dynamic PRA                                  2. Likely a feature of some advanced reactor applications.
simulationoriented  Moderate 3. Pushed by R&D community, interesting to nonPRA types, resisted by PRA old guard PRA                            4. Challenge: demonstrating sufficient value
: 1. Not a new concept, but tools are better; could involve application of NRCs Level 3 Characterize value                PRA model.
: 2. Could examine some fundamental concepts (e.g., single failure criterion, Risk Impact of of regulatory Moderate    containment)
Regulation    requirements by risk          3. Likely to be resisted by some staff.
impact                        4. Challenges: defining risk metrics, treatment of uncertainties, extending lessons DoB = f{technological readiness,                                                            beyond single plant.
clarity of application, Piloting      Explore advanced              1. Current approaches involve flatland displays (possibly animated) and storytelling.
user skepticism}                                                                    Potential for advanced technologies (AR, VR, multisensory inputs) not yet discussed.
Through        technologies for risk  Strong  2. Possible resistance from decision makers Hyperspace    communication                  3. Challenge: completely unknown potential benefits 103
 
Concluding Remarks
* R4&D is an essential element of NRCs continuing efforts to increase its use of risk information in regulatory decision making
* R4&D has many purposes, longterm as well as shortterm
    -  Ways to look at and/or approach problems
    -  Points of comparison
    -  Job aids
    -  Problemspecific information
* The benefits of R4&D can be disruptive, but also unforeseen and delayed
* Blue Sky proposals are welcome: submit to the NRCs FutureFocused Research Program 104
 
R4&D - EXTRA SLIDES 105
 
Degrees of Blue - More R4&D Examples Natural Hazards                Decision Making SimulationBased                                        AIassisted RIDM Extreme Hazards                                Advanced techniques for Climate Change                          risk communication Correlated Hazards                  Advanced metrics for RIDM DoB = f{technological readiness, Blue  User                                              clarity of application, Sky Needs                                              user skepticism}
Automatic model construction AIbased data mining Dynamic PRA Errors of Commission Autonomous Reactors Full simulationbased PRA Org Factors in PRA Human/Org Factors            Computational Methods 106
 
More Product Examples - Frameworks/Methodologies NRCsponsored Fire PRA    Technology Neutral R&D (universities)        Framework
* Started after Browns
* Explored use of risk Ferry fire (1975)          metrics to identify
* Developed fire PRA          licensing basis events approach first used in
* Inspiration and part Zion and Indian Point      basis for current PRAs (early 80s), same      Licensing general framework          Modernization used today                  Program
* Started path leading to riskinformed fire protection (NFPA 805) 107
 
More Product Examples - Reference Points NUREG1150                          SOARCA
* Continuing point of
* Detailed analysis of comparison for                      potential severe Level 1, 2, 3 results                accidents and offsite
* Expectations                        consequences (ballpark)
* Updated insights on
* Basis for regulatory                margins to QHOs Peach Bottom analysis (backfitting, generic issue resolution)
NUREG1150 (Surry)
Surry            Sequoyah 108
 
More Product Examples - Methods/Models/Tools SPAR                      IDHEASG
* Independent plant
* Improved support for specific models          qualitative analysis (generic data)
* Explicit ties with cognitive
* Allhazards (many)        science (models, data)
* Support SDP, MD 8.3,
* General framework for ASP, GSI, SSC studies    developing focused
* Adaptable for specific    applications (e.g., IDHEAS circumstances            ECA)
SAPHIRE
* Benefits from NPP simulator studies
* General purpose
* Consistent with current modelbuilding tool HRA good practices
* Multiple user guidance (NUREG1792) interfaces From https://en.wikipedia.org/wiki/SAPHIRE 109
 
From R4&D to RiskInformed Fire Protection RG 1.205 NUREG/CR    NUREG 10 CFR 50.48(c) 2258, 2269  1150 ASME/ANS 10 CFR 50.48                            PRA Standard Appendix R NUREG/CR6850 BTP 9.51                              EPRI 1011989 RG 1.75                            NFPA 805 Indian        IPEEE                        NFPA 805 LARs Point PRA Browns 1970  Ferry 1980        1990        2000              2010            2020 110
 
TREATMENT OF UNCERTAINTIES
* What does treatment mean?
* What are the current approaches? Challenges?
* Can we do better? How?
111
 
Decision Making Under Uncertainty                                                                                                                  Safety Security p11                        Environment
* Uncertainties                                                                                                                              C11  Cost p12                        Reputation
      - About outcome of alternative                                                                                                          C12
      - Conditioned on situation,                                                    A1 p1M                                      11 state of knowledge                                                                                                                    C1M
* Aim: treat uncertainties                                                                                            p21                  C21 to ensure1                                                                                                            p12 A2                                                      C22
      - Effectiveness (best alternative)                                                                                                    p2N           
      - Efficiency                                                                                                                            C2N
      - Stakeholder confidence pij = P{outcome Cijlsituation, knowledge}
1Alternatively,  can bounce approaches against the Principles of Good Regulation: independence, openness, efficiency, clarity, reliability 112
 
Treatment > Characterization Use in Decision Characterization Communication    Making 113
 
Characterizing Uncertainties - A Pragmatic Framework M (Model of the World):
Scope, structure i: Parameters
: Universe Known Unknowns Unknown Unknowns 114
 
Characterization Challenges Early core melt, containment cooling Early core melt, no containment cooling
* Parameter Uncertainties                                                          Late core melt, containment cooling Late core melt, no containment cooling
    - Raw data preprocessing (selection and                                      Containment bypass Steam generator tube rupture interpretation)                                                              Direct containment failure
    - Potentially nonintuitive Bayesian updating                              Data source: G.J. Kolb, et al., Review and Evaluation of the Indian Point results                                                                  Probabilistic Safety Study, NUREG/CR 2934, December 1982. (ML091540534)
    - Stateofknowledge dependencies
    - Appropriate simplification: expert elicitation Opinions
* Model Uncertainties                                1) Failure to use uncertainty characterization
    - Serious consideration of alternative models        best or even good practices provides
    - Mainstreaming of quantitative approaches          an easy target for critics, can affect
* Completeness Uncertainties                            stakeholder confidence.
: 2) Pro forma, cookbook analyses can miss
    - Systematic identification of gaps                  potentially useful insights.
    - Serious efforts to reduce (transition to model) 115
 
Communicating Uncertainties                                                                                          Will somebody find me a onehanded scientist?!
Senator Edmund Muskie
* Content/format depend on audience and expected use                                                                          (Concorde hearings, 1976)
    - Different (and changing?) levels of comfort with
* Uncertainty
* Formal frameworks (parameter/model/completeness; aleatory/epistemic; probabilistic/nonprobabilistic) and displays
    - Different decisions => different information
* Fundamental questions
    - How confident am I (the analysis team) in the key results, insights, and implications?
    - Why should you (the decision maker) be confident in my characterization?
Quote from I. Flatow, Truth, Deception, and the Myth of the OneHanded Scientist, October 18, 2012. Available from:
116  https://thehumanist.com/magazine/novemberdecember2012/features/truthdeceptionandthemythoftheonehandedscientist
 
Communication Challenges
* Simpler than risk communication (fewer dimensions, perhaps less visceral reaction)                                                                  Hurricane Model Warning:
* Meeting the Goldilocks Principle                                                Useful Advice or Just Venting?
    - Are different types (per characterization frameworks) important to the decision?                                                              If anything on these products causes confusion, ignore the
    - Does too much uncertainty information cause a loss of attention?            entire product.
Reduce salience of key results, insights, implications?
    - Does too little uncertainty information breed overconfidence or even a suboptimal decision?
* Designing to increase cognitive engagement
    - Likely important for major, nonroutine decisions
    - Active learning => reduce package polish to encourage audience internal processing and dialog with team?1 1 Of course this possible (and untested) approach requires a willing audience.
117
 
Using Uncertainty Information in Decision Making (An Outsiders View)
* Deliberative vs. Naturalistic Decision Making
* Structured approaches
    - MultiAttribute Utility Theory (late 1960s,1 used Adapted from NUREG2150 by ASCE?)
    - Simplifications (e.g., Analytic Hierarchy Process,2 early 1980s)
    - Nowadays?
* Technical and social influences
    - Needs of problem
    - Views on uncertainty information (e.g., useful or confusing or even obfuscating?)                                                                    From NUREG2114
    - Heuristics and biases 1 See for example H. Raiffa, Decision Analysis: Introductory Lectures on Choices under Uncertainty, AddisonWesley, New York, 1968. (NRC Technical Library HD69.D4 R13 c.1) 118  2 See for example T.L. Saaty, Decision Making for Leaders: The Analytical Hierarchy Approach for Decisions in a Complex World, Lifetime Learning, Belmont, CA, 1982. (HD30.23 .S24 c.1)
 
Challenges in Using Uncertainty Information (An Outsiders View)
* Demonstrating value of/creating demand for beyond pro forma treatment
* Balancing
    - Rulebased (repeatable, transparent)
    - Knowledgebased (optimal use of evidence)
* Effective communication
    - With providers (what is the question)
    - With stakeholders (basis for decision) 119
 
Treatment of Uncertainties - Concluding Remarks
* Treatment covers characterization, communication, and use
* A longstanding concern for RIDM with
    - Accepted practices
    - Remaining challenges
* Improved methods and tools for treatment
    - Are feasible
    - Will provide better support for agency transformation
    - May need culture change for investment and use 120
 
Treatment of Uncertainties - Extra Slides 121
 
Uncertainties and Decision Making (Two Days Before Landfall)
Andrew (1992)                                            Irma (2017)
Hurricane Warning Hurricane Watch Evacuated Hurricane tracks adapted from University of WisconsinMilwaukee (https://web.uwm.edu/hurricanemodels/models/archive/)
122    Emergency response based on data from National Hurricane Center:
(https://www.nhc.noaa.gov/1992andrew.html)
 
Parameter Uncertainty: Current Practice
* Treatment involves Estimation (including expert elicitation, Bayesian updating)
Propagation
* Straightforward mathematics and mechanics
* Some practical challenges 123
 
Parameter Uncertainty Challenges
* Data preprocessing                                                                    Runtime Failures (MotorDriven Pumps)
      - Selection                                                          1.00
      - Interpretation                      Probability Density Function 0.80 0.60
* Effect of analysis shortcuts                                              0.40 (Normalized)
      - Standard (e.g., noninformative)                                  0.20 prior distributions                                                0.00 1.00E09  1.00E08    1.00E07      1.00E06        1.00E05    1.00E04  1.00E03
      - Simplified expert elicitation                                                                            Failure Rate (/hr)
      - Independence assumption                                                            Service Water        Normally Running          Standby
* Ensuring correspondence with
* 2015 Industrywide estimates from: https://nrcoe.inl.gov/resultsdb/AvgPerf/
stateofknowledge
* Service Water Pumps: 2 failures in 16,292,670 hours Normally Running Pumps: 225 failures in 59,582,350 hours
      - Basic events (micro view)
* Standby Pumps (1st hour operation): 48 failures in 437,647 hours
      - Overall results (macro view) 124
 
Model Uncertainty:                                                                                Hurricane Andrew 8/22/1992, 1200 UTC Current Practice                                                                                  Adapted from University of Wisconsin Milwaukee (https://web.uwm.edu/hurricane models/models/archive/)
* Important to acknowledge and treat (in context of decision)
* Multiple approaches
    - Consensus model
    - Sensitivity analysis
    - Weighted alternatives (e.g., SSHAC)
    - Output uncertainties                Adapted from V.M. Andersen, Seismic Probabilistic Risk Assessment Implementation Guide, EPRI 3002000709, Electric Power Research Institute, M.H. Salley and A. Lindeman, Verification and Palo Alto, CA, December 2013 Validation of Selected Fire Models for Nuclear Power Plant Applications, NUREG1824 Supplement 1/EPRI 3002002182, November 2016.
125
 
Quantification of Model Output Uncertainty Time (s)  Experiment (K)    DRM (K)
* Bayesian methods                                                                                              180          400            450
      - Framework consistent with overall PRA                                            Data                    360          465            510 720          530            560
      - Early approaches used in past PRAs                                                                      840          550            565
      - Can address practical issues (e.g., non                                                                                Temperature (K) homogeneous data)*                                                                                                Assume          Assume Non
* Challenges include                                                                                          Percentile Homogeneous      Homogeneous Output Uncertainty Data              Data
      -  Uncertainties in unmeasured parameters                                                                1st          415.2          372.8
      -  Submodel limits of applicability                                                                      5th          437.5          400.7
      -  Representativeness of computed results                                                                50th          457.1          470.5 95th          479.7          559.4
      -  Use in actual decision making 99th          509.1          608.7
          *See E. Droguett and Ali Mosleh, Bayesian methodology for model uncertainty using model performance data, 126        Risk Analysis, 28, No. 5, 14571476, 2008.
 
Model Uncertainty  Commentary
* Model uncertainties can be large; importance depends on decision
* Some practical NPP RIDM approaches (e.g.,                                            Hurricane Irma: 9/8/2017, 0000 UTC (about 2 days before FL landfall) consensus models, deterministic screening) can understate uncertainties
* Ensemble approaches (with SMEdetermined                  Outer best estimate) used by other disciplines                  prediction is closest
* Subjective probability framework =>                        to actual course
    - Need to consider user effect
                          /    Plot adapted from University of WisconsinMilwaukee (https://web.uwm.edu/hurricanemodels/models/archive/)
    - Raises question regarding fundamental meaning of weighted average approaches 127
 
Completeness Uncertainty:                NUREG1855 Rev. 1 (2017)
Current Practice                          Options:
* Progressive analysis (screening, bounding,
* Recognized concerns                    conservative, detailed)
      - Known gaps (missing scope)
* Change scope of risk
* Scenario categories              informed application
* Contributors within categories RG 1.174 Rev. 3 (2018)
      - Unknown gaps
* Treatment (Mind the Gap)
      - Analysis guidance
      - Additional analysis/R&D
      - Riskinformed decision making 128
 
NPP PRA Known Gaps1
* Broad scenario categories Rationale                                Common Example(s)
Out of scope                              security/sabotage, operation outside approved limits Low significance (preanalysis judgment)  external floods (many plants preFukushima)
Appropriate PRA technology* unavailable  management and organizational factors PRA not appropriate                    software, security
* Contributors within categories Category                                Example(s)
External hazards                        multiple coincident or sequential hazards Human reliability                        errors of commission, nonproceduralized recovery Passive systems                          thermalhydraulic reliability 1aka Known Unknowns 129
 
Completeness Uncertainty: Possible R&D
* Continue to develop technology to address        Event (NUREG/CR4839), 1992 known gaps                                      Aircraft impact Avalanche
    - Riskinformed prioritization                  Coastal erosion
    - Fully engage appropriate disciplines          Drought External flooding
    - Take advantage of general computational and  Extreme winds and tornadoes methodological developments                  Fire
* Facilitate reemphasis on searching              Fog Forest fire
    - Demonstrate efficiency and effectiveness with Frost Hail current tools (e.g., MLD, HBFT) vs.          High tide, high lake level, or high checklist/screening                          river stage
    - Develop improved tools (including OpE mining) 130
 
Different Perspectives: Logarithmic vs Linear Displays 131
 
ADDITIONAL RESOURCES 132
 
Selected Lectures, Seminars, and Talks1,2
* Nuclear Power Accidents and Incidents: Lessons for PRA, Research Seminar, University of Illinois UrbanaChampaign (virtual),
February 2, 2021. (ML20339A570)
* PRA and RiskInformed Decision Making at the NRC: Some Trends and Challenges, Nuclear Engineering Research Seminar (Virtual),
North Carolina State University, Raleigh, NC, October 22, 2020. (ML20293A370)
* PRA and RiskInformed Decision Making at the NRC: Some Trends and Challenges, Modeling, Experimentation, and Validation Summer School (Virtual), Idaho National Laboratory, July 27, 2020. (ML20195B157)
* Technology for the Treatment of Uncertainties: History, Status, Commentary and Challenges, prepared for CRIEPI/NRRC and OECD/NEA Workshop on the Proper Treatment of Uncertainties in Safety Analyses, Tokyo, Japan, May 2627, 2020 (postponed, new date TBD). (ML20080N774)
* PRA and RiskInformed Decisionmaking at the NRC: Some Trends and Challenges, B.J. Garrick Institute for Risk Sciences, University of California, Los Angeles, February 21, 2020. (ML20035G249)
* Research and Integrated Decision Making (IDM): A Personal Perspective, Workshop: Integrated and RiskInformed Decisionmaking Forum for Managers, U.S. Nuclear Regulatory Commission, November 13, 2019. (ML19310F243)
* Dynamic PRA: Not If But When? Invited Talk, IAEA Workshop on Advanced PSA Approaches and Applications, Alkmaar, The Netherlands. September 913, 2019. (ML19248C656)
* Nuclear Power Plant Probabilistic Risk Assessment (PRA) and RiskInformed Decision Making (RIDM), Independent Activities Period, Massachusetts Institute of Technology, January 1623, 2019.3 (ML19011A416)
* Advanced Knowledge Engineering Tools to Support Probabilistic Risk Assessment (PRA) Activities  A Whole New World? NRC Knowledge Management (KNOWvember) Webinar, November 21, 2017. (Webinar Video)
* PRA R&D - Changing the Way We Do Business? Invited Plenary Lecture, ANS International Topical Meeting on Probabilistic Safety Assessment (PSA 2017), Pittsburgh, PA, September 2428, 2017. (ML17292A552) 1The ML numbers refer to pdf versions. PowerPoint versions (with fullresolution graphics) will be provided in a separate ADAMS package.
133  2Although  some of the talk titles are duplicative, the material has been tailored to the different audiences and venues.
3Lectures, workshops, and reference material for a 1week intensive course (meant to cover material normally provided in a semester).}}

Revision as of 12:32, 19 January 2022

RES Seminar Part 2 - Short Takes
ML21138A792
Person / Time
Issue date: 05/13/2021
From: Nathan Siu
NRC/RES/DRA
To:
Siu, Nathan - 301 415 0744
Shared Package
ML21138A647 List:
References
Download: ML21138A792 (133)
v • d • e


Text

Short Takes:

Snippets on Some PRA Topics*

Nathan Siu Senior Technical Adviser for PRA Analysis Office of Nuclear Regulatory Research Division of Risk Analysis RES Staff Technical Seminar (Virtual) - Part 2 May 13, 2021 (3:004:00)

  • The views expressed in this presentation are not necessarily those of the U.S. Nuclear Regulatory Commission.

The Menu

  • Dynamic PRA Notes
1) Each snippet provides a 1520 minute talk on the
  • Identifying Scenarios - Weird Stuff and the subject.

Importance of Active Searching 2) Most snippets are accompanied by extra slides providing additional details.

  • Internal Risk Communication 3) Links to additional presentations (pdf versions) are
  • A Brief History: PRA and the Characterization provided in the Additional Resources portion of this slide set.

of Uncertainties 4) PowerPoint presentations (with fullresolution

  • PRA Lessons from NPP Accidents and graphics and fullfunctioning navigation links) will be uploaded into an ADAMS package.

Incidents a. Slides from this seminar

  • RiskRelated Regulatory R&D (R4&D) b. Presentations identified in Additional Resources
  • Treatment of Uncertainties c. Miscellaneous additional presentations, snippets, and notes1
  • Additional Resources 1Extended and illustrated notes on a subject not intended as an actual presentation but provided in PowerPoint form for convenience.

2

DYNAMIC PRA

  • What is it?
  • Why do we care?
  • Where are things now?

3

Fukushima Daiichi 1, 3/11/2011: Static Description

{Loss of Power} CORE DAMAGE AND {Loss of Isolation Condenser} CD AND {Failure of Alternate Cooling} LOSS OF DECAY HEAT REMOVAL LODHR

= {Core Damage}

LOSS OF ALL LOSS OF ISOLATION FAILURE OF AC AND DC POWER CONDENSER ALT COOLING LOP Ext LOIC Ext FALTC Ext 4

Adding Time, Motion (Kinematics)

Emergency Isolation Actions to Actions to Offsite LOOP EDG LongTerm Power Condenser Extend Shed Power (Seismic) Recovery Cooling (EDGs) (IC) IC Ops DC Loads Recovery Earthquake and LOOP (T = 0:00)

LOOPEQ EPS ISO EXT DCL OPR DGR LTC

  • Tsunami (T+0:40) 1 2 CD
  • Loss of all power (T+0:50) 3 4 CD 5
  • IC outboard valve closed (T+3:40)* 12 hr 12 hr 6

7 CD CD

  • Core damage (T+4:00, estimated 8 9 CD postaccident) 8 hr 10 11 CD 8 hr 12 CD 13 14 CD
  • What but not Why 4 hr 15 16 CD

- Closure of isolation condenser 4 hr 17 CD 18

- Delay in implementing alternate 19 CD 20 cooling (fire pumps) 1 hr 1 hr 21 CD 22 CD 5 *Manual action that had little actual effect; inboard valves already closed

Dynamic Interactions => Context => Why Time Hazard Systems Indications Operators/Workers ERC/ER team EP Time 14:46 0:00 Earthquake Scram MSIVs close, turbine trips, EDGs 14:47 0:01 Rx level drops start and load RV pressure decreases; RV level 14:52 0:06 ICs start automatically in normal range 40 minutes between Cooldown earthquake rate exceeding techand tsunami; 15:03 0:17 ICs removed from service Manually remove IC from service transition fromspec confident limits control to disbelief Disaster HQ established in TEPCO 15:06 0:20 Tokyo Determine only 1 train IC 15:10 0:24 needed; cycle A train First tsunami 15:27 0:41 arrives Second tsunami 15:35 0:49 arrives 15:37 0:51 Loss of AC Degradation and failure over time, 15371550: Gradual loss of instrumentation, indications gradually affecting operator 15:37 0:51 Determine HPCI unavailable (including IC valve status, RV information and ability to control level), alarms, MCR main lighting TEPCO enters emergency plan 15:42 0:56 (loss of AC power); ERC established D/DFP indicator lamp indicates 16:35 1:49 "halted" Review accident management Cannot determine RV level or Review accident management procedures, start developing injection status; work to restore procedures, start developing Declared emergency (inability to 16:36 1:50 procedure to open containment level indication; do not put IC in procedure to open containment determine level or injection) vent valves without power service vent valves without power 6

Dynamic PRA - What Is It?

  • Dynamic PRA PRA that explicitly treats interac ons among system elements and resulting motions (including rates of change), e.g.,

- Hardware component transitions (e.g., available to unavailable, or even intermediate states)

- Changes in operating crew situation awareness Plant I&C Crew

- Changes in plant thermalhydraulic state

  • Degree of treatment of dynamics => continuum of analyses, e.g.,

Environment

- Current PRA (phenomenological submodel; some direct Complexity dependencies, e.g., support systems)

- Taskoriented network models and simulations

- Largely mechanistic simulations with stochastic elements Frequent conceptualization of dynamic PRA 7

Dynamic PRA - Potential Benefits

  • Additional insights (suggesting alternative risk management strategies), e.g.,

- Untreated mechanisms (e.g., feedback loops)

- Timing of key events

- Causes of key events

  • Fewer intermediate and often conservativelyoriented simplifications (e.g., discretization, success criteria)

- More realistic results

- Improved use of available evidence (what we know) => improved DM confidence

  • Directly supportive of phenomenological whatif and optimization analyses, e.g.,

- Assessing effect of different parameter values (e.g., ATF properties, arrival times for offsite aid)

- Identifying potentially troublesome ranges of parameter values (cliffedge effects)1

  • Modeling in disciplinespecific terms (native language)

- Reduced chance of translation errors

- Increased stakeholder involvement and buyin

  • Engineering trends (integrated simulation) 1Analysis requires coupling of dynamic PRA model with appropriate mathematical searching and optimization tools.

8

Dynamic PRA - Where Are We?

  • Strong interest: academia, international
  • NPPs: tools, demonstrations
  • Nonnuclear: decision support applications (e.g., aerospace, hydropower)

Methodologies for Current PSA (Phased Mission, Dynamic PSA HighFidelity, Competing Risks, Tools and SimulationBased Level 3 PSA) Toolboxes Dynamic PSA Late Intermediate Early (Mature, Stable) (Adolescent, Developing) (Infancy, Emerging)

Developmental Stage 9

Dynamic PRA - Concluding Remarks

  • All NPP accidents have involved significant dynamic interactions among system elements
  • Explicit treatment of these interactions can benefit PRA studies and the PRA enterprise
  • Work (particularly decision support applications) is needed to achieve these benefits 10

Dynamic PRA - Extra Slides 11

Indicators of Technology Maturity1 Early Intermediate Late (Infancy, Emerging) (Adolescent, Developing) (Mature, Stable)

Many welltrained and experienced practitioners Small research community Larger number of practitioners Recognize limits of applicability of Small number of practitioners Larger number of experienced Practitioners Strong personality influences, researchers methods Can adapt methods to new situations competing schools of thought Can work with researchers to identify important issues New practicedriven research problems Most research driven by needs of Driven by perceived needs Some consensus positions for some practice Research Problem selection affected by personal broadly defined problem areas More abstract research addresses Agenda choice (e.g., due to ease of formulation Some unproductive research lines needs clearly identifiable by all or solution) abandoned concerned Incomplete coverage of topics Fast growth Local applications (addressing small Vocabulary has evolved Developing vocabulary Applications parts of larger problems) General framework exists Optimistic views on new methods; No broader framework Little selling of area limitations not well understood 1 Adapted from: Cornell, C.A., Structural safety: some historical evidence that it is a healthy adolescent, Proceedings of Third 12 International Conference on Structural Safety and Reliability (ICOSSAR 81), Trondheim, Norway, June 2325, 1981.

U2: start depressurization (stuck RV, then continue) Browns Ferry 1 & 2 U2: D DG tripped, multiple boards lost 19750322 U2: control panel malfunctions, scram, turbine trip, FW trip, MSIVs close U2: conditions U2: shutdown stabilized cooling established U1: start depressurization U1: FW tripped, HPCI and RCIC stopped, use CRD pump U1: enter RB U1: scram, 2/3 FW pumps to assess SSC U1: RV control tripped, multiple boards lost conditions restored U1: spurious alarms, actuations U1: loss of operating relief valves U1: shutdown cooling established Fire reported to U1/U2 MCR U1: prepare for RHR cooling (15 hr, 50 min)

OFD notified TVA notified Fire out CSR CO2 discharge CSR fire out, resume Start using water RB firefighting Smoke, CO2 enter MCR Fire start 0 2 4 6 8 10 12 13 Time from Start (hr)

Loss of EFW (burned cable Start laying temporary Greifswald 1 to 2nd EFP) cable to power EFPs 19751207 EFW restored Trial and error fault diagnosis (power to EFPs) actions => more failures (including instrumentation) Close primary hot leg main gate valves, start DGs start, power to forced circulation 1/2 emergency buses Natural circulation: MCR power restored; Turbine use SVs and cold FW pressurizer SVs open, 2/6 fail to reclose; Stable trip to control primary emergency cooling pump started cooling Start firefighting Corridor Fire Fire ventilation alarm out restored Heavy smoke, need respirators Fire start, spread 0 2 4 6 8 10 12 14 Time from Start (hr)

Start laying temporary power cable from Armenia 1&2 U2 DG to U1 emergency makeup pump 19821015 Station U1: only instrumentation is Feedwater makeup to Blackout primary pressure (local station) SGs (temporary cable)

Loss of main coolant pumps, MCR Power to U1 emergency lights, readouts, alarms, phones, makeup pump from DG power, normal and emergency makeup Manual SG SRV Operators manually open SG dump valves MCR power trip U1&2 opened in upper TB (wearing breathing masks) restored Offsite Break cable spreading Fire out FBs arrive room wall to access fire FB arrives, open MCR TB, transformer fires Fire hatch to spray vault under control controlled H2, transformer explosions Fire start, Smoke MCR smoke spread in MCR unbearable 0 2 4 6 8 10 12 15 Time from Start (hr)

Blayais 14 U1: shutdown 19991227 U4: 400 kV restored Level 2 Emergency Plan activated for U1; utility U1: Train A ESWS and regulator national emergency teams pumps submerged activated; agree to SG cooldown strategy U2: 400 kV restored U1 & U2: LHSI and CSS Regulator Walkdown discovers U1 Train A pump rooms flooded informed of U1 & ESWS pumps submerged U2 status and SG U2 & U4: Loss of 400 kV cooldown Use fire engines to assist 225 kV power (grid instability), scram strategy in pump floodwaters restored (U1U U1U4: Loss of 225 kV Level 1 Emergency Plan activated: onsite pumps power (fallen trees) for floodwaters, recover submerged equipment U4: High Site access regained; needed tide alarm offsite workers can arrive Floodwater pumping (continues to ~50 hr)

Flood overtops dyke, site access lost 0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 16 Time from Start (hr) 19:00 0:00 0:00 12/27 12/28 12/29

Dynamic PRA Fukushima Daiichi 1 20110311 Relative Time Hazard Systems Indications Operators/Workers ERC/ER team EP Time 14:46 0:00 Earthquake Scram MSIVs close, turbine trips, Rx level drops (1 of 3) 14:47 14:52 0:01 0:06 EDGs start and load ICs start automatically RV pressure decreases; RV level in normal range 15:03 0:17 40 minutes between earthquake and tsunami; ICs removed from service Cooldown rate exceeding Manually remove IC from tech spec limits service transition from confident control to disbelief Disaster HQ established in 15:06 0:20 TEPCO Tokyo Determine only 1 train IC 15:10 0:24 needed; cycle A train First tsunami 15:27 0:41 arrives Second tsunami 15:35 0:49 arrives 15:37 0:51 Loss of AC 15371550: Gradual loss of Determine HPCI instrumentation, Degradation unavailable and failure over time, 15:37 0:51 indications (including IC gradually affecting operator valve status, RV level),

alarms, MCR main lighting information and ability to control TEPCO enters emergency 15:42 0:56 plan (loss of AC power);

ERC established D/DFP indicator lamp 16:35 1:49 indicates "halted" Review accident Cannot determine RV level Review accident Declared emergency management procedures, or injection status; work to management procedures, (inability to determine start developing restore level indication; do start developing level or injection) 16:36 1:50 procedure to open not put IC in service procedure to open containment vent valves containment vent valves without power without power 17 17

Dynamic PRA Fukushima Daiichi 1 20110311 Time Relative Hazard Systems Indications Operators/Workers ERC/ER team EP Time (2 of 3) 16:45 16:55 1:59 2:09 Tsunami alert Determine RV level Workers on way to check Emergency cancelled D/DFP had to turn back Lose ability to determine Reentered emergency plan 17:07 2:21 External influence RV level or injection status Site superintendent directs 17:12 2:26 triggering work investigation of using fire protection to inject water 17:15 2:29 stoppage, temporary Estimated core uncovery in 1 hr Tsunami alert evacuation, 17:19 2:33 cleared accountability Dieseldriven fire pump Pressure above 100 psi Manually open valves (in started and left to idle dark) from fire protection system to core spray 17:30 2:44 system; take turns holding D/DFP switch to keep in standby Error3:32 18:18 of commission (disabling DC power partially returned MO3A and MO2A indicate closed passive safety system) possibly MO3A and MO2A Open IC valves MO3A and 18:18 3:32 opened 2A. Steam from condenser based on assumed low inventory observed MO3A closed Remove IC from service (usage) (concerned about failing lines). Entered R/B and T/B to manually open MOV for 18:25 3:39 FP lineup. Hard time finding valve, had wrong key, hard to operate hand wheel. Long time.

18

Fukushima Daiichi 1 20110311 Time Relative Time Hazard Systems Core damage (45 hr Indications Operators/Workers ERC/ER team EP 18:50 4:00 (3 of 3) 19:00 4:14 after trip)

Close valves for broken outdoor FP pipes. Broke Ask Tokyo for more fire engines lock to allow passage between Units 2 and 3.

Govt. declares nuclear 19:03 4:17 emergency InNohindsight, core damage pressure indication in MCR; Reactor pressure =

20:07 5:21 Game 6.89 MPa (1000Over psi) local for 1F1; indication Small portable generatorcontinuing 1F1 MCR recovery has temporary lighting 20:49 6:03 installed 20:50 6:04 activities and events impact Local authorities order evacuation within 2 km other units Level indication (1F2 and 1F3 core restored; 21:19 6:33 level = 0.20 m (8) above uncovery TAF on 3/14)

Prime minister orders 21:23 6:37 evacuation within 3 km; sheltering out to 10 km MO3A opened Place IC in service; steam 21:30 6:44 observed Access to RB restricted due 21:51 7:05 to dose rates - indirect indication of core uncovery Level = 0.55 m (21.7) 22:00 7:14 above TAF Drywell pressure = 0.50 Restoration team from 23:50 9:04 MPa (87 psi) above design ERC enables reading Offsite power supply 23:59 9:13 trucks arrive by midnight 19

Earthquake Fukushima Daiichi 16 2nd Tsunami 20110311 Request: Suspend Order: Vent Seawater Injection U1 and U2 Order: Confirmed:

Local Evac. Local Evac.

U5 Rx U2 Cont. U5 Level =

Depressurizing Venting TAF + 0.95m SBO (U1U5) U1 Cont. U3 Cont. U2 Core Loss of DC (U1U4) Venting Venting Uncovery U1 Core U1 RB U3 Core U3 RB U4 RB Damage (est.) Explosion Uncovery Explosion Explosion 3/11 3/12 3/13 3/14 3/15 20

Fukushima Daiichi 16 20110311 U5 SFP Cooling Restored U6 SFP SDF Truck Spray Cooling Restored U4 SFP Earthquake U2 Core U4 SFP Lev Uncovery <0.5m above 2nd Tsunami U5 Level = U4 RB 4/

SBO (U1U5) TAF + 0.95m Explosion U1 Core U1 RB U3 Core U3 RB Damage (est.) Explosion Uncovery Explosion 3/11 3/12 3/13 3/14 3/15 3/16 3/17 3/18 3/19 3/20 3/21 21

Local evacuation confirmed, 1st team dispatched Govt Start prep orders 2nd team dispatched, turned back (radiation) for venting venting Unsuccessful attempts to open AO90 Open AO72 1.0 manual venting of Pressure (MPa) wetwell Containment Venting:

  • Prevents catastrophic 0.5 lower head failure failure pressurization from core steam dome
  • Causes release to relocation to lower head drywell wetwell environment RPVTEPCO steam line rupture DWTEPCO WWTEPCO 0.0 0 5 10 15 20 25 30 3/11/2011 Time (hr) 14:46 Adapted from: R. Gauntt, Fukushima Daiichi Accident Study: MELCOR Analyses and Results, OECD/NEA Fukushima Accident Analysis Workshop, IssylesMoulineaux, France, June 1820, 2012.

See also R. Gauntt, et al., MELCOR Simulations of the Severe Accident at the Fukushima 1F1 Reactor, ANS Winter Meeting and Nuclear Technology Expo, San Diego, CA, November 1115, 2012.

22

INTERESTING Clickbait IDENTIFYING SCENARIOS - WEIRD STUFF AND THE IMPORTANCE OF ACTIVE SEARCHING

  • What is the concern?
  • What tools are available?
  • How might we do better?

23

Reminders

1) Risk = {si,Ci,pi}

Scenarios: what can go wrong? (qualitative)

2) All models are wrong, but some are useful.1
  • What isnt in the PRA model wont be quantified
  • What isnt conceived of might not be addressed in a riskinformed decision 1 G.E.P. Box and N.R. Draper, Empirical ModelBuilding and Response Surfaces, John Wiley and Sons, 1987. See the Wikipedia article All 24 models are wrong for background.

Youre analyzing a floating NPP. Have you thought of this one?

Chazhma Bay (August 10, 1985)1

  • Echo II class submarine K431 is nearly done refueling. Fresh fuel has been loaded, workers are preparing to reattach 12ton reactor vessel head which has control rods attached.
  • Workers see seal is not tight (there are leaks), decide to lift head using refueling ship crane. (Decision is against regulations and made without consulting supervisor. Did not drain primary loop to ensure no moderation, did not detach lattice used to keep control rods in place.)
  • Passing torpedo boat creates large wake, rocks refueling ship; crane pulls control rods out of the core.
  • Reactivity excursion causes steam explosion which blows head and fuel assemblies out of the reactor compartment, destroys the submarine pressure hull.

1See M. Takano, V. Romanova, H. Yamazawa, Y. Sivintsev, K. Compton, V. Novikov, and F. Parker, Reactivity Accident of Nuclear Submarine 25 near Vladivostok, Journal of Nuclear Science and Technology, Vol. 38, No. 2, pp. 143157 (February 2001).

What Can Go Wrong?

  • PRA scenarios need a starting point (initiating hazard or event)
  • Complementary methods to identify starting point:

- Inductive (e.g., FMEA, HAZOP)

- Deductive (e.g., Master Logic Diagram, Heat Balance Fault Tree)

- Lists (e.g., possible hazards, actual events, other PRAs)

  • Notes:

- Conventional focus on postinitiator scenario can blur or even miss important factors in preinitiator buildup

- Real events can involve unanticipated mechanisms and sequences of events that appear perfectly reasonable in hindsight. Click here for more examples.

26

Checklists can be useful but Aircraft impact Local intense precipitation

  • Might not actually be exhaustive Avalanche Biological events Low lake or river water level Low winter temperature
  • Can be confusing (e.g., overlapping Coastal erosion Drought Meteor or satellite strike Onsite chemical release categories) External fire External flooding Pipeline accident River diversion
  • Can promote oneatatime consideration Extreme winds and tornadoes Fog Sandstorm Seiche (actual events can involve multiple Forest fire Frost Seismic activity Severe temperatures categories) Hail Snow High summer temperature Soil shrinkswell
  • Can be inefficient (e.g., excessive attention High tide Hurricane Space weather Storm surge on ultimately unimportant categories) Ice cover Transportation accident Industrial/military facility accident Tsunami
  • Lengthy lists might trigger impulse to screen Internal flooding Landslide Turbinegenerated missiles Volcanic activity rather than explore Lightning 27

Active Searching

  • Searching emphasized in the early days it is incumbent upon the new industry and of nuclear power the Government to make every effort to recognize every possible event or series of
  • A fundamental first principles attitude: events which could result in the release of using understanding of system, look for unsafe amounts of radioactive material to potential problems (rather than expect the surroundings and to take all steps them to be revealed by some analytical necessary to reduce to a reasonable minimum the probability that such events process) will occur in a manner causing serious
  • Potentially valuable for new/novel overexposure to the public.

situations where operational experience W.F. Libby (1956)1 is weak or entirely lacking 1W. F. Libby (Acting Chairman, AEC) - March 14, 1956 response to Senator Hickenlooper [See D. Okrent, Reactor Safety, University of 28 Wisconsin Press, 1981. (NRC Technical Library TK9152 .O35, multiple copies)]

Hazard Identification Example: Checklist vs Active Search Checklist Active Search (aka Red Teaming)

General Process Stepping through list Looking at undesired state (e.g., failure of

1) Ask what each hazard might do key components), ask
2) Screen or retain for further analysis using 1) What conditions might cause this established criteria undesired state
2) What hazards or hazard combinations might create these conditions
3) If there are protective barriers preventing the undesired conditions, what might fail these barriers Advantages More complete More direct Methodical, easy to document Less restricted by categorization Engages imagination Challenges Not wasting time on unimportant categories Tempering imagination with plausibility Avoiding urge to screen (to finish the job) Ensuring reasonable completeness 29

Active Search >> Drawing Fault Tree

  • Need to identify plausible mechanisms

- Possible failures can always be added to a fault tree

- Reasonable causality needed for retention and quantification

  • Examples

- Operator disabling of safety systems (errors of commission)

- Seismicallyinduced reactivity transients 30

Example: Disabling a BWR Isolation Condenser OPERATOR TERMINATES Possible ISOLATION CONDENSER OPERATION but what ISOXHEEOCTERM reason?

31

Example: SeismicallyInduced Reactivity Excursion

  • Observations

- Global operational experience: at least 4 (perhaps 5)

North Anna Nuclear Generating Station earthquakes causing fluxinduced trips at 7 (perhaps 9) reactors1

- Some reactor designs have unstable operating regimes

- Systems with timedelayed feedback (e.g., restorative forces) can oscillate, even resonate

  • Q: Can a seismic event induce a resonance leading to a runaway reaction? Under what conditions?

Adapted from: https://earthquake.usgs.gov/earthquakes/

1Ground motion trips either not available (e.g., power loss) or not triggered (e.g., accelerations are too low) 32

Example: SeismicallyInduced Reactivity Excursion Seismic Hazard Controls Neutronics Plausible?

Structures Operational Experience Thermal Hydraulics Systems Integration

  • Movement? Bowing?
  • Reactivity effects?
  • Feedback?
  • Resonance? Perhaps not, but
  • Fluid flow & density effects?
  • Time scales?
  • Excursion?
  • Heat transfer effects?

ask the question 33

Looking Forward: OpE + Advanced Technology

  • Empirical evidence: strong argument for plausibility
  • Challenges

- Enormous and growing database (not just nuclear)

- Unstructured, natural language and heterogeneous (content, form, quality) data

- Inferencing

  • Exploratory study: advanced technology Adapted from:

1) 2)

https://str.llnl.gov/str/March02/March50th.html https://en.wikipedia.org/wiki/History_of_supercomputing#/media/File:Supercomputershistory.svg (AI/ML, Big Data) can help1 3) https://www.top500.org/news/japancapturestop500crownarmpoweredsupercomputer/

1See, for example

  • N. Siu, K. Coyne, and F. Gonzalez, Knowledge Management and Engineering at a Riskinformed Regulatory Agency: Challenges And Suggestions, white paper, U.S. Nuclear Regulatory Commission, 2017. (ML17089A538) 34
  • F. Gonzalez and N. Siu, Accident Sequence Precursors: Current Analyses, Challenges, and Future Research, WGRISK Annual Meeting, NEA HQ, BoulogneBillancourt, France, March 2022, 2019. (ML19071A160)

Identifying Scenarios - Concluding Remarks

  • A longstanding and continuing PRA goal: ensuring completeness
  • An important mindset: active searching (especially when dealing with new/novel situations)
  • Currently a variety of tools and resources to support searching; advanced technology (e.g., AI/ML) can lead to further improvements 35

Extra Slides - Examples of RealWorld Events and Mechanisms 36

External flooding: obvious now but back then?

  • Fukushima Daiichi (1990s)

- Added EDGs to supplement existing units (SAM modification)

- Aircooled EDGs1 installed at Units 2, 4, 6; crossties provided with Units 1, 3, 5

- All watercooled EDGs in building basements

- Aircooled EDGs installed on ground floor, metalclad switchgear in basement New 2011 EDG

- Earthquake => LOOP New M/C 2009 DB 10m

- Tsunami => SBO for Units 14 (W/C EDGs, M/C switchgear) Switchgear 1972 DB

- Unit 6 EDG supplies Units 5 and 6; air louver ~1m above tsunami height 1Not affected by loss of service water, e.g., due to tsunami. (Pumps are at elevation O.P.+4m.) Per IAEA Director Generals report, 37 choice of aircooled is due to current service water loads; unclear if diversity was a major factor.

Some Other Accidents Accident Notable Mechanisms/Events Sodium Reactor Reactor coolant pump organic coolant leaks into the primary circuit, causes flow blockages, Experiment (1959) higher fuel temperatures, interaction with cladding and formation of a lowmelting temperature alloy, coolant channel blockage, fuel damage, and release of radioactive gases and some volatiles into the sodium coolant and eventually the environment.

Fermi 1 (1966) Segments of zirconium sheets (installed late in construction as a safety barrier) tear loose during power ascension, blocking coolant flow. Two fuel assemblies melt. Following radiation alarms, reactor is manually scrammed.

Chernobyl 4 (1986) Interruption of a planned test due to offsite grid needs leads to Xenon poisoning, inability to achieve planned test conditions. Crew decides to proceed with the test despite the plant being in an unstable operating regime and disables an automatic scram to facilitate testing. A plant computer signal dictating immediate shutdown is ignored. The test initiates a positive reactivity excursion with a catastrophic steam explosion and core destruction some 44 seconds later.

38

Some Interesting Incidents Incident Notable Mechanisms/Events Vogtle 1 (1988) Smoke detector actuation (burnoff in a duct heater) led to pressurization of a preaction deluge system in cable spreading room, water discharge through leakoff valves (as designed), water seepage through a floor penetration into the main control room, and spurious opening of a PORV at power. Floor penetration design was faulty - assumed sealwelding (of embedded seal angles and upper angle iron assembly) would be watertight. See LER 424/88016.

Indian Point 3 Activation of an outdoor deluge system (in response to a transformer explosion and fire) led to (2015) bleed off water in a valve room adjacent to a vital 480V switchgear room. Due to insufficient drain system capacity, water backed up into the switchgear room. [Note: Although the water was not high enough to affect the switchgear, it constituted a potential industrial hazard that could have inhibited operator access to that room.] See Special Inspection Report ML15204A499.

39

Some RealWorld Mechanisms (1 of 4)

Mechanism Plant (Year) Description Unexpected U.S. plant

  • EDG oil fire due to fatigue cracking of undocumented instrumentation line.

/Unusual

  • Failure occurred during followup examination of a reported small oil leak; line was Loadings moved slightly [cause?]

Nogent

  • Unit 2 condenser circulating water system leak causes p between Turbine (2006) Building foundation and floor, lifts floor, fails manhole.
  • Water floods Unit 1 Turbine Building, enters ESW system gallery through penetrations, CCW pump room through drains.

Inadequate Forsmark 1

  • Offsite switchyard twophase short circuit during maintenance causes LOOP Protective (2006)
  • Inverters failed on overvoltage, causing loss of 2/4 trains of AC and DC power Systems 40

Some RealWorld Mechanisms (2 of 4)

Mechanism Plant (Year) Description Secondary Maanshan 1

  • Salt spray caused LOOP; electrical fault caused highenergy arcing fault (HEAF), loss Hazards (2001) of faulted safety bus
  • Heavy smoke from HEAF delayed access to switchgear room to restore power to undamaged safety bus => 2 hour2.314815e-5 days <br />5.555556e-4 hours <br />3.306878e-6 weeks <br />7.61e-7 months <br /> station blackout Cruas 24
  • Flood management actions lead to vegetation debris downstream, clogging of (2009) service water intake
  • Total loss of service water for Unit 4, partial loss for Units 2 and 3 Declared Blayais 12
  • During site flooding, rooms containing Unit 1 and Unit 2 lowhead safety injection Inoperability (1999) and containment spray pumps partially flooded
  • Foreign material (injectable sealant foam) found on floor of service water tunnel (1996)

Some RealWorld Mechanisms (3 of 4)

Mechanism Plant (Year) Description Worker Point Beach

  • Communications lost with diver working in Unit 2 (shutdown) circulating water Safety (2000) pump house Concerns
  • Manual shutdown of Unit 1 U.S. plant
  • Spurious evacuation alarm (smoke clogged radiation monitor)
  • Reactor building evacuated Operator Greifswald 1 During a severe power cable fire triggered by an electrician (performing a Choices (1975) demonstration for a trainee), operators manipulated switchgear to find intact cables for power (trial and error problem solving) but these actions caused additional failures TMI2 During a loss of feedwater event, operators throttled high pressure makeup in the (1979) mistaken belief that the reactor coolant system was going solid 42

Some RealWorld Mechanisms (4 of 4)

Mechanism Plant (Year) Description Operator DavisBesse

  • During a loss of feedwater transient, the shift supervisor did not implement Choices (1985) operating procedures for feed and bleed cooling (which would contaminate (cont.) containment), counting (correctly) on timely restoration of auxiliary feedwater

an equipment operator tossed keys to another ten feet ahead Vandellos During a Turbine Building fire (hydrogen deflagration, cascading burning oil),

(1989) operators (using breathing apparatus) entered dark, smoke filled areas to perform recovery actions Fukushima Operators isolate the isolation condenser in the mistaken belief that it was close to Daiichi 1 drying out and failing (which would provide a direct release path to the environment)

(2011)

Maintenance Rancho A maintenance worker dropped a lightbulb into a cabinet, shorting out nonnuclear Error Seco (1978) instrumentation. Propagating faults led to a scenario that could easily have resulted in an outcome as serious as that of the accident at Three Mile Island a year later 43

Some Resources

1. Fukushima Daiichi (2011): International Atomic Energy Agency, The Fukushima Daiichi Accident, Director General Report, Vienna, Austria, 2015.
2. Sodium Reactor Experiment (1959): P. Pickard, Sodium Reactor Experiment Accident July 1959, Sandia National Laboratories, August 29, 2009. (Available from:

https://www.etec.energy.gov/Library/Main/Pickard%20SRE%20presentation.pdf)

3. Fermi 1 (1966): Fermi Fuel Melt Accident, Nuclepedia.
4. Chernobyl 4 (1989): U.S. Department of Energy, Electric Power Research Institute, Environmental Protection Agency, Federal Emergency Management Agency, Institute of Nuclear Power Operations, and the U.S. Nuclear Regulatory Commission, Report on the Accident at the Chernobyl Nuclear Power Station, NUREG1250, January 1987.
5. Vogtle 1 (1988): Water Leakage into Control Room/Potential Exists for a Safety System Failure, Licensee Event Report 424/88016, November 22, 1988.
6. Indian Point 3 (2015): U.S. Nuclear Regulatory Commission, Indian Point Nuclear Generating - Special Inspection Report 05000286/2015010, July 23, 2015.
7. Nogent (2006): U.S. Nuclear Regulatory Commission, ConstructionRelated Experience with Flood Protection Features, IN 200906, July 21, 2009. (ML090300546)
8. Forsmark 1 (2006: U.S. Nuclear Regulatory Commission, Significant Loss of SafetyRelated Electrical Power at Forsmark, Unit 1, in Sweden, IN 200618, August 17, 2006.
9. Maanshan 1 (2001): Atomic Energy Council, Taiwan, The Station Blackout Incident of the Maanshan NPP Unit 1, April 18, 2001. (Available from:

https://www.aec.gov.tw/webpage/control/report/safety/safety_04_002.pdf)

10. Cruas 24 (2009): P. Dupuy, G. Georgescu, and F. Corenwinder, Treatment of the loss of ultimate heat sink initiating events in the IRSN Level 1 PSA, NEA/CSNI/R(2014)9, Nuclear Energy Agency, BoulogneBillancourt, France, 2014.
11. Blayais 12 (1999): Blayais Flood, Nuclepedia.
12. LaSalle 12 (1996): Foreign Material Injected Into Service Water Tunnel Causes Dual Unit Shutdown Due to Inadequate Work Control, Licensee Event Report 373/96008R01, November 25, 1996.
13. Point Beach (2000): Manual Reactor Trip Due to Concerns for Diver Safety, Point Beach Nuclear Plant Unit 1, Licensee Event Report 266/00010R00, November 22, 2000.
14. Greifswald 1 (1975): M. Rwekamp and E. Gelfort, Sicherheitsrelevanter Kabeltrassenbrand im Kernkraftwerk Greifswald Beschreibung und Einschtzung, GRSVSR 24491, Gesellschaft für Anlagen und Reaktorsicherheit (GRS) mbH, Kln, Germany, June 2004.
15. TMI2 (1979): D. Marksberry, F. Gonzalez, and K. Hamburger, Three Mile Island Accident of 1979 Knowledge Management Digest, Overview, NUREG/KM0001, rev. 1, U.S. Nuclear Regulatory Commission, June 2016.
16. DavisBesse (1985): U.S. Nuclear Regulatory Commission, Loss of Main and Auxiliary Feedwater Event at the DavisBesse Plant on June 9, 1985, NUREG1154, July 1985
17. Vandellos (1989): S.P. Nowlen, M. Kazarians, and F. Wyant, Risk Methods Insights Gained from Fire Incidents, NUREG/CR6738, U.S. Nuclear Regulatory Commission, September 2001.
18. Rancho Seco (1978): R.M. Bernero and F.H. Rowsome, Single Failure Potentially Leading to Core Damage, memorandum to H.R. Denton and C. Michelson, U.S. Nuclear Regulatory Commission, March 14, 1980. (ML19323J370) 44

INTERNAL RISK COMMUNICATION

  • What is it?
  • Why is it hard?
  • How might we improve?

45

Internal Risk Communication: Support RIDM Adapted from NUREG2150 With To Other Considerations

  • Current regulations
  • Safety margins
  • Defenseindepth
  • Monitoring Quantitative

+

Quantitative 46

Risk Information: Not Just for Current Decisions Prior (foundational) information affects DM processing of new information Specific Analyses

  • Recognition
  • Interpretation Methods, Models,
  • Judging/Weighting Tools, Databases, Standards, Guidance, Foundational Knowledge 47

Risk Information: Inherently Complex

  • Low likelihood => beyond personal Other Complications experience, intuition
  • Heterogeneous
  • Hyperdimensional o Qualitative and quantitative o Multiple views

- Scenarios (organizations, disciplines)

- Likelihood

  • Dynamic

- Multiple consequence measures o System changes (e.g.,

different operational modes,

  • Uncertain effects of decisions)

- Sparse or nonexistent data o New applications (and

- Multiple models contexts)

- Partial coverage 48

Other Challenges

  • Individual user differences, e.g.,

- Knowledge

- Preferences/heuristics

  • Social factors, e.g.,

- Trust

- Decision and group dynamics

  • Situational context, e.g.,

- Available time to consider

- Decision support vs. informational Source: https://www.nrc.gov/readingrm/doccollections/commission/slides/2019/20190618/staff20190618.pdf 49

How to Ensure Message Capture and Retention?

External Flooding Fire

- Risk level (absolute, relative) Seismic

- Risk importance (absolute, relative) Internal Flooding Internal Events

- Surprise

  • Communication process

- Message formulation

- Delivery method

- Tools 50

Current Mechanisms Documents and Interactive Presentations Discussion (Flatland) (Storytelling) 51

Can We Do Better? Different Documents? Graphic Elements Small Font Questions Sidebars Embedded Graphics Conventional TwoColumn, Conversational Graphical 52

Can We Escape Flatland?

  • Tufte model: use rich displays and reports, encourage user to explore

- Promotes active involvement of decision maker

- Increases general trust?

  • A graduated technical approach to assist?

Interface Interaction Mode Hyperlinked dashboards, reports Manual Time Video (with sound?) AI assist Visual immersion Multisensory immersion 53

From Static to Interactive Dashboard. Then to SciFi?

M. Korsnick, Risk Informing the Commercial Nuclear Enterprise, Promise of a Discipline: Reliability and Risk in Theory and in Practice, University of Maryland, April 2, 2014. Graphic adapted from https://www.flickr.com/photos/83823904@N00/64156219/

(permission CCBY2.0) 54

Internal Risk Communication - Concluding Remarks Risk Communication Technical Communication Communication

  • General communication good practices are helpful but not sufficient: special characteristics of risk information pose additional challenges
  • Intuitively better approaches are being developed; scientific testing could be helpful
  • Communication involves people: one risk communication solution may not work for all actors 55

Internal Risk Communication - Extra Slides 56

Risk Information: Qualitative + Quantitative*

Risk { i , i, i}

  • What can go wrong?
  • What are the consequences?
  • How likely is it?
  • Kaplan/Garrick triplet definition has been adopted by NRC. See:

White Paper on RiskInformed and PerformanceBased Regulation (Revised), SRM to SECY98144, March 1, 1999 57 Glossary of RiskRelated Terms in Support of RiskInformed Decisionmaking, NUREG2122, May 2013 Probabilistic Risk Assessment and Regulatory Decisionmaking: Some Frequently Asked Questions, NUREG2201, September 2016

Sources of Breakdowns: Risk Communication Between Risk Managers and Public*

  • Differences in perception of information

- Relevance

- Consistency with prior beliefs

  • Lack of understanding of underlying science
  • Conflicting agendas
  • Failure to listen
  • Trust
  • J.L. Marble, N. Siu, and K. Coyne, Risk communication within a riskinformed regulatory decisionmaking environment, International 58 Conference on Probabilistic Safety and Assessment (PSAM 11/ESREL 2012), Helsinki, Finland, June 2529, 2012. (ADAMS ML120480139)

Differences in Perspective (Example)

Our tendency is to focus on things that are interesting and Decision make them important. The thing that we have to do is focus Makers on what really is important Ron Rivera, 2020 is (developer)

Whats interesting might be (practitioner) important Practitioners Developers isnt* (decision maker)

The PRA/RIDM Community

  • Or, at least, isnt necessarily - interesting and important are independence.

59

External Flooding Preference: Avoid Chart Junk Fire High Winds Seismic

  • Visual effects (e.g., noninformative 3D with perspective) can add Internal pop but distract from or even distort messages. Flooding Internal Events
  • Advanced animation tools can be even stronger attention grabbers with even greater distraction potential

- Focus attention on effects rather than message

- Saturate audiences, leading to the need for even stronger effects in future presentations to grab attention

  • Use effects with moderation (if at all), recognizing that your audience External

- has preferences that vary from person to person and over time (maybe High Flooding Fire Winds they prefer 3D charts!)

Seismic

- is likely subject to many presentations besides yours (imagine the clamor of highly animated presentations seeking attention to their Internal specific messages) Flooding Internal Events 60

Spatial Information - Underused Resource?

  • Common practice in everyday risk communication
  • Going beyond - add changes over time?

61

An OftIgnored External Risk Communication Lesson:

Comparisons Dont Work for Everybody U.S. Annual Deaths, Various Causes (20102019) 700,000 600,000 500,000 400,000 Deaths (2020) 300,000 200,000 100,000 0

Flu Auto Guns Cancer COVID19 62

One Size Doesnt Fit All, Part II:

1,000 words (a story) > a picture?

On the evening of June 25, a freshly graduated high school Drunk Driving Accident Fatalities (2018) star QB was going over 100 mph on a neighborhood road, 1,800 trying to go fast enough to avoid speed camera detection

("whipping"). Out of control on a sweeping curve, the car hit a fence and two trees, and flipped. Two unbelted passengers No Alcohol were ejected and died at the scene. The QB and the front 10,600 seat passenger were seriously injured. All four were BAC > 0.08 g/DL teenagers. All had just left an underage drinking party and 0.01 < BAC < 0.07 g/DL were drunk. The QB was indicted on counts of vehicular 24,100 manslaughter, alcohol related vehicular homicide and causing a lifethreatening injury while driving under the influence of alcohol. The parent of the girl hosting the party, who was present and knowledgeable, pled guilty to two criminal Data from "Traffic Safety Facts 2018 Data: State AlcoholImpaired citations for allowing underage drinking at his home and was Driving Estimates," DOT HS 812 917, June 2020. (Available from:

ordered to pay $5,000 in fines. https://crashstats.nhtsa.dot.gov/#!/DocumentTypeList/11) 63

A BRIEF HISTORY: PRA AND THE CHARACTERIZATION OF UNCERTAINTIES

  • What drove us to where we are now?
  • What are some of the major milestones?

64

PRA History: Challenges and Responses RIDM issues (e.g., realism, heterogeneity, aggregation)

PostFukushima issues (e.g., external hazards)

New/advanced reactors (e.g., conduct of operations)

Modern Applications Characterizing the fleet (variability) Expansion Across Developing confidence for mainstreaming RIDM Industry Filling known gaps (completeness uncertainty) Early Clarifying meaning: models and results PRAs Quantifying accident probability Means to communicate risk Hanford to WASH1400 1940 1950 1960 1970 1980 1990 2000 2010 2020 65

From Hanford to WASH1400 Technical Challenges: 1) Quantifying accident probability

2) Means to communicate risk WASH740 Hanford AEC/NRC Credible Accident UKAEA Estimates:

not in the generation OpE (pessimistic) of the ACRS members Decomposition present (optimistic)

Recommend: Farmer Curve WASH1400 accident System chain System reliability reliability SGHWR analysis studies studies analysis 1950 Windscale 1960 1970 TMI2 1980 For more information: T.R. Wellock, A Figure of Merit: Quantifying the Probability of a Nuclear Reactor Accident, 66 Technology and Culture, 58, No. 3, July 2017, pp. 678721.

WASH1400 Uncertainties (Level 1)

WASH1400: it is reasonable to believe that the WASH1400 Uncertainties (Estimated*)

core melt probability of about 5x105 per reactoryear predicted by this study should not be significantly larger and would almost certainly not exceed the value 5th 50th 95th Surry of 3x104 which has been estimated as the upper mean bound for core melt probability.

Peach Bottom Risk Assessment Review Group (NUREG/CR0400):

1.E05 1.E04 1.E03 We are unable to define whether the overall CDF (/ry) probability of a core melt given in WASH1400 is high or low, but we are certain that the error bands are *Based on data from Tables V 314 (PWR) and 316 (BWR) of Appendix V, assuming distributions are lognormal; median values are somewhat higher than reported in Section 7.3.1 of the Main Report.

understated. We cannot say by how much.

67

Some Early Developments and PRAs Challenges: 1) Filling known gaps (completeness uncertainty)

2) Clarifying meaning: models and results Biblis Sizewell

(+aircraft)

(+DI&C) USDOE Clinch River Oyster Creek NRC Indian Point (LMFBR) (+seismic)

(full scope)

US Industry AIPA Forsmark International Limerick (HTGR) Koeberg Zion Millstone Other Notable

(~WASH1400) (full scope)

Seabrook Super (full scope)

Phénix RSSMAP/IREP (FBR DHR) TMI1 Oconee (full scope)

Apostolakis Kaplan/ (full scope)

Fleming (subjective Garrick EC/JRC Benchmarks (factor) probability) (risk) NUREG/CR2300 (systems, CCF, HRA) 1975 TMI2 1980 1985 Chernobyl 68

Sample Level 1 Results Display 69

Sample Results - SubModel Uncertainty Effect Effects of fire model (COMPBRN) uncertainty on fire growth time N. Siu, "Modeling Issues in Nuclear Plant Fire Risk Analysis," in EPRI Workshop on Fire Protection in Nuclear Power Plants, EPRI NP 70 6476, J.P. Sursock, ed., August 1989, pp. 141 through 1416.

Sample Results - Model Uncertainty (User Effect)

Early core melt, containment cooling Early core melt, no containment cooling Damage State Frequency (/yr), Review 104 Late core melt, containment cooling Late core melt, no containment cooling Containment bypass Steam generator tube rupture Direct containment failure 106 Internal Events External Events 108 1.E03 1.E03 1.E04 1.E04 1.E05 1.E05 Review 1.E06 1.E06 Review 1.E07 1.E07 1010 1.E08 1.E08 1.E09 1.E09 1.E10 1.E10 1.E11 1.E11 1.E11 1.E10 1.E09 1.E08 1.E07 1.E06 1.E05 1.E04 1.E03 1.E11 1.E10 1.E09 1.E08 1.E07 1.E06 1.E05 1.E04 1.E03 1010 108 106 104 Original Original Damage State Frequency (/yr), Original Data source: G.J. Kolb, et al., Review and Evaluation of the Indian Point Probabilistic Safety Study, NUREG/CR2934, December 1982.

71 (ML091540534)

Severe Expansion Across Industry (US)

Accident Policy Technical challenges: 1) Characterizing the fleet (variability)

Statement 2) Developing confidence for mainstreaming RIDM Safety Goal PRA Policy NRC Policy Statement Statement US Industry GL 8820 GL 8820 Supplement 4 NUREG1560 NUREG1742 NUREG1150 NUREG1150 (draft) (final) 1982 ASP Plant Class Models SPAR Models IPEEEs IPEs 1985 Chernobyl 1990 1995 2000 9/11 72

NUREG1150 Estimated* Uncertainties (Level 1)

Model Uncertainty Model Uncertainty

  • Notes: totals shown in this
1) NUREG1150 does not aggregate the hazardspecific results. The totals shown are rough estimates assuming that the NUREG1150 distributions are lognormal.

73 2) The WASH1400 distributions are based on data from Tables V 314 (PWR) and 316 (BWR) of Appendix V, assuming that the distributions are lognormal. The median values are somewhat higher than reported in Section 7.3.1 of the Main Report

IPE/IPEEE - Variability Across Fleet Internal Events + Internal Floods Total 40 40 BWR BWR PWR PWR 30 30 Number Number 20 20 10 10 0 0 1x106 3x106 1x105 3x105 1x104 3x104 1x103 1x106 3x106 1x105 3x105 1x104 3x104 1x103 CDF (/ry) CDF (/ry) 74

The Modern Era (US)

Technical challenges: 1) RIDM issues (e.g., realism, heterogeneity, aggregation)

SECY98144 2) PostFukushima issues (e.g., external hazards)

3) New/advanced reactors (e.g., conduct of operations)

RG 1.174 NUREG2150 ASME PRA NRC Risk Standard NTTF Request US Industry Informed for Information ROP NUREG1855 (Reevaluations) 10 CFR 50.48(c)

NFPA 805 (Fire Protection) NFPA 805 LARs (Fire Protection)

SAMAs (Life Extension)

RiskInformed License Amendment Requests (LARs)

SPAR Models 2000 9/11 2005 2010 Fukushima 2015 2020 75

Variability in Recent Results (Level 1) 0.35 0.30 Population Mean:

4.7x105 0.25 Fraction of Plants 0.20 0.15 0.10 Lowest Highest Reported: Reported:

0.05 3.5x106 1.3x104 0.00 6.0 5.5 5.0 4.5 4.0 3.5 3.0 1E6 1E5 1E4 1E3 CDF (per reactor year) 76

Variability in Results - Comparison with IPE/IPEEE 1E3 0.001 0.50 Total CDF (IPE + IPEEE)

NFPA 805 Fraction of PRAs 0.40 IPE/IPEEE 0.30 1E4 0.0001 0.20 0.10 0.00 1 2 3 4 5 6 7 8 9 10 0.01 0.1 1 10 100 1000 1E5 0.00001 1E5 1.00E05 1E4 1.00E04 1E3 1.00E03 Fire CDF/Internal Events CDF Total CDF (Recent LARs) 77

Parameter, Model, and Completeness Uncertainty:

A Practical Categorization mod*el, n. a M (Model of the World): representation of reality created with a specific Scope, structure objective in mind.

i: Parameters A. Mosleh, N. Siu, C. Smidts, and C. Lui, Model

Universe Uncertainty: Its Characterization and Quantification, Center for Reliability Engineering, University of Maryland, College Park, MD, 1995. (Also NUREG/CP0138, 1994)

PRA models for NPPs

  • Typically an assemblage of sub models with parameters
  • Implicitly include issues considered but not explicitly Known Unknowns quantified Unknown Unknowns 78 For more discussion, see snippet on Treatment of Uncertainties

PRA History - Concluding Remarks NPP PRA:

  • Has decades of experience with analyses and decision support applications
  • Is strongly advocated and widely used internationally
  • Has evolved in response to theoretical and practical challenges and will likely continue to do so with new challenges 79

PRA LESSONS FROM OPERATIONAL EXPERIENCE

  • How can information from operational experience help PRA?
  • How has this been explored and what has been learned?
  • What might we do next?

See N. Siu, Nuclear Power Accidents and Incidents: Lessons for PRA, Research Seminar, University of Illinois UrbanaChampaign 80 (virtual), February 2, 2021 (ML20339A570) for a full seminar slide set.

OpE Input to Risk Assessment Operational Experience

(> statistics)

Adapted from NUREG2150 Other Considerations

  • Current regulations
  • Safety margins
  • Defenseindepth
  • Monitoring Quantitative Qualitative 81

Some Reactor Fuel Damage Accidents and Incidents*

Windscale 1 TMI 2 Fukushima Daiichi 13 Graphite Pile, UK PWR, US BWRs, Japan UMetal Fire Loss of Feedwater EQ + Tsunami, Loss of Power Fermi 1 Chernobyl 4 LMR, US RBMK, Ukraine Flow Blockage Reactivity Accident Leningrad 1 RBMK, Russia Reactivity Accident St. Laurent 1 Bohunice A1 Paks 2 GCR, France HWGCR, Slovak Republic VVER, Hungary Fuel Misload Fuel Loading Accidents Spent Fuel Pool Accident 1950 1960 1970 1980 1990 2000 2010 2020 82 *Events involving fuel damage at power and/or production reactors

And Some Other Rancho Seco PWR, US Madras 2 PHWR, India Maintenance Error Tsunami Serious Incidents* LOFW, TMI precursor LOUHS Gundremmingen A Turkey Point 3 & 4 H.B. Robinson B/F Bleed and feed cooling VVER, East Germany PWR, US PWR, US LOCA Loss of coolant accident Training Error Storm (Hurricane) Bus Fire (Arc)

LOFW Loss of feedwater Partial LOOP, RV LOCA LOOP RCP Seal Challenge LOMCR Loss of main control room LOOP Loss of offsite power Browns Ferry 1 & 2 Narora Maanshan Fukushima Daiichi 5 LOUHS Loss of ultimate heat sink RV Relief valve BWR, US DavisBesse PHWR, India PWR, Taiwan BWR, Japan SBO Station blackout (loss of AC power) Cable Fire PWR, US Turbine Fire Storm (Spray) EQ + Tsunami Complicated Trip LOFW, no B/F SBO, LOMCR SBO Loss of all power LaCrosse Armenia Blayais 1 & 2 Cruas 24 Duane Arnold BWR, US VVER, Armenia PWR, France PWR, France BWR, US Switchyard Fire Cable Fire Storm (Wind + Flood) Flood (Debris) Storm (Wind)

Partial Uncovery SBO LOOP, Degraded UHS LOUHS LOOP 1950 1960 1970 1980 1990 2000 2010 2020 83 *Selected nonfuel damage events with challenges to core cooling

NPP OpE Narratives

  • Incident databases ETH = Eidgenssische Technische Hochschule

- Many public (e.g., LERs, ETH) and nonpublic (e.g., IAEA IRS, IRS = Incident Reporting System ICES = INPO Consolidated Event System INPO ICES) sources

- Varying purposes (affecting fields, entry criteria), degrees of Selected Reports on Fukushima:

coverage Cumulative Pages

- All contain narratives (unstructured text) 12000 10000

  • OpE narratives 8000

- Content: subjective but potentially rich; can stimulate AND Pages 6000 4000 temper imagination (possible mechanisms and scenarios) 2000

- Volume: ranges from terse (passing mentions) to overwhelming 0

- Perspectives and usefulness for PRA: varied 03/11/2011 03/10/2012 03/10/2013 03/11/2014 03/11/2015 03/11/2016 03/11/2017 03/11/2018 Date 84

Text Mining Cautions The big issue was the hydrogen bubble...

  • Be aware of 2020 hindsight, a.k.a.

- MMQB (Monday Morning Quarterbacking)

- I knew it all along syndrome as a barrier to learning Wasnt there

  • Factual information is often uncertain, limitations can a major persist in later records human error?

- Simplifications

- Inconsistencies

- Factual errors

  • Postevent judgments are subject to normal human biases

- Confirmation bias

- Underestimation/undervaluation of uncertainty

  • Reviews

- Often reflect technical discipline perspectives

- Often used to assess blame rather than identify opportunities for improvement 85

Some OpE Mining Case Studies

  • PRAoriented reviews of

- 30 fire events*

- Great East Japan Earthquake and Tsunami (2013, 2016)

- Selected storm and flood events (2018)

- Selected seismic events (20192020

  • General Objectives

- Develop insights (observed mechanisms, scenarios) to support PRA technology development

- Support staff learning (familiarization with events, PRA Last two case studies approaches)

- Support future activities (e.g., smart tool development) 86

Insights Relevant to PRA Technology Case studies :

  • Strengthened basis for many previously recognized messages (e.g., potential importance of external hazards, errors of commission)
  • Identified instances where (depending on the decision problem) PRA scope might need to be extended (e.g., multisite events, longduration events)
  • Identified mechanisms/scenarios needing multidisciplinary attention (e.g., multiple shocks, induced hazards, scenario dynamics)
  • Identified phenomena potentially warranting PRA community attention (seismicallyinduced reactivity excursions, seismicallyinduced HEAFs*)
  • Identified previously unrecognized/underpublicized precursors to Fukushima (Hinkley Point, Turkey Point, Blayais)
  • Identified potential need for supplementary measures/means to highlight incidents (boost the signal) for PRA community attention
  • The possibility of a seismicallyinduced HEAF has been recognized due to the 2007 KashiwazakiKariwa (station transformer) and the 2011 Onagawa (nonsafety switchgear) events. The insights are: a) generating mechanisms for observed nonseismically induced HEAFS 87 might be activated by a seismic event, and consequentially b) seismicallyinduced HEAFs might be risk significant (based on the impact of the Maanshan 2001 nonseismic HEAF).

Knowledge Management and Knowledge Engineering Tool Insights Connect the dots

  • Knowledge Management

- Useful learning experience for all participants

- Demonstrated value of multidisciplinary perspectives

- Would have benefitted from increased team interactions

  • Knowledge Engineering Tools Where does it say ?

12000

- Still need deep subject matter expert (SME) expertise to 10000 connect the dots, develop insights (not yet just analytics) 8000 Pages 6000

- Tools need to deal with enormous, heterogeneous database 4000

- With humanintheloop implementation, could use improved 2000 0

tools for screening documents, prioritizing remainder for further examination 03/11/2011 03/10/2012 03/10/2013 03/11/2014 03/11/2015 03/11/2016 03/11/2017 03/11/2018 Date 88

PRA Lessons from OpE - Concluding Remarks

  • Not many NPP accidents and serious incidents, but perhaps more than realized
  • Events illustrate how things can fail, sometimes by unexpected pathways and mechanisms
  • Review of events

- Can inform PRA modeling (identification of possible scenarios)

- Can broaden knowledge base of reviewer

- Can support development of smart tools for data mining 89

Lessons From OpE - Extra Slides 90

Closing Remarks Reminder: Accidents are a real possibility Windscale 1 TMI 2 Fukushima Daiichi 13 Graphite Pile, UK PWR, US BWRs, Japan UMetal Fire Loss of Feedwater EQ + Tsunami, Loss of Power Fermi 1 Chernobyl 4 LMR, US RBMK, Ukraine Flow Blockage Reactivity Accident

[Before TMI] core damage was never never land Leningrad 1 RBMK, Russia Reactivity Accident R. Bari*

St. Laurent 1 Bohunice A1 Paks 2 GCR, France HWGCR, Slovak Republic VVER, Hungary Fuel Misload Fuel Loading Accidents Spent Fuel Pool Accident 1950 1960 1970 1980 1990 2000 2010 2020

  • Plenary Panel: Perspectives on Nuclear Safety Since the Three Mile Island Event, ANS Intl Mtg Probabilistic Safety Assessment (PSA 2019), Charleston, SC, 2019.

91

Reminder: Accidents [often] have precursors Hinkley Point Blayais Fukushima Unpublicized a French problem Madras Unpublicized Leningrad Chernobyl Unconfirmed until 1990 TMI Rancho Seco TMI similarity recognized 1980*

1950 1960 1970 1980 1990 2000 2010 2020

  • a twoyear old incident that could easily have resulted in an outcome as serious as that of the accident at Three 92 Mile Island. [R.M. Bernero and F.H. Rowsome, Single Failure Potentially Leading to Core Damage, memorandum to H.R. Denton and C. Michelson, U.S. Nuclear Regulatory Commission, March 14, 1980. (ML19323J370)]

Closing Remarks Reminder: Increasing Realism / Reducing Conserva sm

  • Known gaps* in broad scenario categories Rationale Common Example(s)

Out of scope security/sabotage, operation outside approved limits Low significance (preanalysis judgment) external floods (many plants preFukushima)

Appropriate PRA technology unavailable management and organizational factors PRA not appropriate software, security

  • Known gaps in treatment of contributors within categories Category Example(s)

External hazards multiple coincident or sequential hazards Human reliability errors of commission, nonproceduralized recovery Passive systems thermalhydraulic reliability

  • Terminology of Guidance on the Treatment of Uncertainties Associated with PRAs in RiskInformed Decision Making, NUREG1855 Rev. 1, March 2017; 93 a.k.a. known unknowns

References

  • N. Siu, Nuclear Power Accidents and Incidents: Lessons for PRA, Research Seminar, University of Illinois UrbanaChampaign (virtual),

February 2, 2021. (ML20339A570)

  • S.P. Nowlen, M. Kazarians, and F. Wyant, Risk Methods Insights Gained from Fire Incidents, NUREG/CR6738, September 2001.
  • N. Siu, D. Marksberry, S. Cooper, K. Coyne, and M. Stutzke, PSA technology challenges revealed by the Great East Japan Earthquake, Proceedings of PSAM Topical Conference in Light of the Fukushima DaiIchi Accident, Tokyo, Japan, April 1517, 2013. (Paper:

ML13038A203, Presentation: ML13099A347)

  • N. Siu, K. Compton, S. Cooper, K. Coyne, F. Ferrante, D. Helton, D. Marksberry, and J. Xing, PSA technology reminders and challenges revealed by the Great East Japan Earthquake: 2016 update, Proceedings of 13th International Conference on Probabilistic Safety Assessment and Management (PSAM 13), Seoul, Korea, October 27, 2016. (Paper: ML16245A871, Presentation: ML16270A522)
  • N. Siu, I. Gifford, Z. Wang, M. Carr, and J. Kanney, Qualitative PRA insights from operational events, Proceedings of 14th International Conference on Probabilistic Safety Assessment and Management (PSAM 14), Los Angeles, CA, September 1621, 2018. (Paper:

ML18135A109, Presentation: ML18249A340), NonPublic Report: ML18248A117)

  • N. Siu, J. Xing, N. Melly, F. Sock, and J. Pires, Qualitative PRA Insights from Seismic Events, Proceedings 25th Conference on Structural Mechanics in Reactor Technology (SMiRT25), Charlotte, NC, August 49, 2019. (Paper: ML19162A422, Presentation: ML19210D835),

NonPublic Report: ML20309A718)

Note: Expanded versions of the PSAM 14 paper (storms and floods) and SMiRT25 paper (seismic events) can be found in nonpublic staff reports and public versions of these reports (ML21081A038 and ML21081A040, respectively) 94

RISKRELATED REGULATORY R&D (R4&D)

  • What is the purpose of R4&D?
  • How has R4&D supported NRCs riskinformed activities?
  • Why can it be difficult to assess the potential benefits of R4&D?

95

NRC Uses of Risk Information PRA Policy Statement (1995)

Regulations

  • Increase use of PRA technology in all and Guidance regulatory matters

- Consistent with PRA stateoftheart

- Complement deterministic approach, R&D support defenseindepth philosophy Operational Decision Licensing

  • Benefits:

and (1) Considers broader set of potential challenges Experience Support Certification (2) Helps prioritize challenges (3) Considers broader set of defenses U.S. Nuclear Regulatory Commission, Use of Probabilistic Risk Assessment Methods in Nuclear Activities; Final Policy Statement, Federal Register, 60, p. 42622 (60 FR Oversight 42622), August 16, 1995.

96

Regulatory R&D in Decision Support Decision Typical products (regulatory research)

  • Detailed
  • Ways to look at and/or approach
  • Problemdriven problems (e.g., frameworks, Specific
  • Need it now Analyses methodologies)
  • Points of comparison (e.g., reference Methods, Models, calculations, experimental results)

Tools, Databases, R&D

  • Job aids (e.g., computational tools, Standards, databases, standards, guidance: best Guidance, practices, procedures)
  • Broad
  • Busy people => limited
  • Problemspecific information (e.g.,

time for nonurgent results, insights, uncertainties) communication Foundational Knowledge Side benefits

  • Potential future uses =>
  • Education/training of workforce needs to persist
  • Networking with technical community Regulatory Decision Support 97

R4&D Product Examples - Frameworks/Methodologies Dynamic PRA1 Model uncertainty -

  • Inspired by accident quantification2 experience (TMI2,
  • NRCsponsored output and reality exploratory R&D
  • Bayesian estimation (universities, labs)
  • Includes user
  • International effect as well as interest (WGRISK, fundamental IAEA) Time (s) Experiment (K) DRM (K) model/tool errors
  • Futurefocused 180 360 400 465 450 510 research 720 530 560 840 550 565 1 N. Siu, Dynamic PRA for Nuclear Power Plants: Not If But When? U.S. Nuclear Regulatory Commission, March 2019. (ML19066A390; see also slides in this presentation) 98 2 E. Droguett and Ali Mosleh, Bayesian methodology for model uncertainty using model performance data, Risk Analysis, 28, No. 5, 1457 1476, 2008.

R4&D Product Examples - Reference Points Fire PRA maturity PRA lessons from and realism1 accidents/Incidents2

  • Opinion papers
  • Reviews of sparked by ongoing accidents and debate incidents for real Maturity vs. world scenarios and Realism mechanisms Available evidence 3/11/2011
  • Quantitative and Fire events qualitative analyses Storms and floods
  • Also KM and KE Opinion Paper benefits 1 N. Siu, K. Coyne, and N. Melly, Fire PRA Maturity and Realism: A Technical Evaluation, U.S. Nuclear Regulatory Commission, March 2017. (ML17089A537; see also ML15035A678 and NEA 7417) 2 N. Siu, D. Marksberry, S. Cooper, K. Coyne, and M. Stutzke, PSA technology challenges revealed by the Great East Japan Earthquake, 99 Proceedings of PSAM Topical Conference in Light of the Fukushima DaiIchi Accident, Tokyo, Japan, April 1517, 2013. (ML13038A203; see also this presentations slides on OpE lessons)

R4&D Product Examples - Job Aids COMBPRN1 Content Analytics2

  • Developed with
  • Exploratory study of NRC support for Watson tech NPP fire PRA
  • Zone model Identify and Time to target characterize multi (cable) damage unit events Uncertainty Current CDFs analysis
  • Basis for input to
  • Used in multiple current NRC AI/ML industry PRAs initiatives 1 N. Siu, "Probabilistic Models for the Behavior of Compartment Fires," NUREG/CR2269, 1981.

2 N. Siu, K. Coyne, and F. Gonzalez, Knowledge Management and Knowledge Engineering at a RiskInformed Regulatory Agency:

100 Challenges and Suggestions, U.S. Nuclear Regulatory Commission, March 2017. (ML17089A538; see also ML16355A373)

R4&D Product Examples - Decision Support Post9/11 studies Pressurized Thermal

  • Shortterm analyses Shock1 to support orders
  • Technical basis for
  • Longerterm revision PTS confirmatory screening limit (10 analyses CFR 50.61)
  • Advanced
  • Quantitative communication Event sequence Video analysis Nonlinear T/H analysis (PT) papers PFM analysis (TWCF)
  • Qualitative (Level 2) 1 M. EricksonKirk, et al, Technical Basis for Revision of the Pressurized Thermal Shock (PTS) Screening Limit in the PTS Rule (10CFR50.61):

Summary Report, NUREG1806, 2006.

101

From R4&D to RIDM SECY98144 RG 1.174 Safety Goal PRA Revised NUREG NUREG Policy Policy ROP 1860 2150 NUREG/CR ASME/ANS 2300 PRA Standard Indian Point PRA IPE/IPEEE NEI 1804 WASH NUREG Level 3 1400 1150 PRA 1970 1980 1990 2000 2010 2020 102

R4&D - From NearTerm to Blue Sky Activity Short Title DoB Notes Now Blue Sky Description

1. Already in use by some organizations; unknown effectiveness.

Automatic PRA 2. Topic previously suggested by FSR, accepted conceptually by DRA/PRAB Characterize current Model Light 3. Not requested by user offices, resisted by PRA old guard technologies 4. Challenges: developing understanding of technologies, obtaining information from Construction users (international, private)

Identify and 1. Widely recognized but vaguely characterized issue in move towards riskinformed Treatment of prioritize gaps and regulation; activity goes beyond current practices

2. Concept likely to have broad support Uncertainty in potential Light 3. Challenges: prioritizing gaps considering ability to do something (quantifying model PRA improvement uncertainty, reducing completeness uncertainty, improving communication of activities uncertainties, )

Prepare for future 1. Old research concept enabled by improved computational capabilities, external (U.S.

& international) R&D investments.

applications of more Light Dynamic PRA 2. Likely a feature of some advanced reactor applications.

simulationoriented Moderate 3. Pushed by R&D community, interesting to nonPRA types, resisted by PRA old guard PRA 4. Challenge: demonstrating sufficient value

1. Not a new concept, but tools are better; could involve application of NRCs Level 3 Characterize value PRA model.
2. Could examine some fundamental concepts (e.g., single failure criterion, Risk Impact of of regulatory Moderate containment)

Regulation requirements by risk 3. Likely to be resisted by some staff.

impact 4. Challenges: defining risk metrics, treatment of uncertainties, extending lessons DoB = f{technological readiness, beyond single plant.

clarity of application, Piloting Explore advanced 1. Current approaches involve flatland displays (possibly animated) and storytelling.

user skepticism} Potential for advanced technologies (AR, VR, multisensory inputs) not yet discussed.

Through technologies for risk Strong 2. Possible resistance from decision makers Hyperspace communication 3. Challenge: completely unknown potential benefits 103

Concluding Remarks

  • R4&D is an essential element of NRCs continuing efforts to increase its use of risk information in regulatory decision making
  • R4&D has many purposes, longterm as well as shortterm

- Ways to look at and/or approach problems

- Points of comparison

- Job aids

- Problemspecific information

  • The benefits of R4&D can be disruptive, but also unforeseen and delayed
  • Blue Sky proposals are welcome: submit to the NRCs FutureFocused Research Program 104

R4&D - EXTRA SLIDES 105

Degrees of Blue - More R4&D Examples Natural Hazards Decision Making SimulationBased AIassisted RIDM Extreme Hazards Advanced techniques for Climate Change risk communication Correlated Hazards Advanced metrics for RIDM DoB = f{technological readiness, Blue User clarity of application, Sky Needs user skepticism}

Automatic model construction AIbased data mining Dynamic PRA Errors of Commission Autonomous Reactors Full simulationbased PRA Org Factors in PRA Human/Org Factors Computational Methods 106

More Product Examples - Frameworks/Methodologies NRCsponsored Fire PRA Technology Neutral R&D (universities) Framework

  • Started after Browns
  • Explored use of risk Ferry fire (1975) metrics to identify
  • Developed fire PRA licensing basis events approach first used in
  • Inspiration and part Zion and Indian Point basis for current PRAs (early 80s), same Licensing general framework Modernization used today Program
  • Started path leading to riskinformed fire protection (NFPA 805) 107

More Product Examples - Reference Points NUREG1150 SOARCA

  • Continuing point of
  • Detailed analysis of comparison for potential severe Level 1, 2, 3 results accidents and offsite
  • Expectations consequences (ballpark)
  • Updated insights on
  • Basis for regulatory margins to QHOs Peach Bottom analysis (backfitting, generic issue resolution)

NUREG1150 (Surry)

Surry Sequoyah 108

More Product Examples - Methods/Models/Tools SPAR IDHEASG

  • Independent plant
  • Improved support for specific models qualitative analysis (generic data)
  • Explicit ties with cognitive
  • Allhazards (many) science (models, data)
  • General framework for ASP, GSI, SSC studies developing focused
  • Adaptable for specific applications (e.g., IDHEAS circumstances ECA)

SAPHIRE

  • Benefits from NPP simulator studies
  • General purpose
  • Consistent with current modelbuilding tool HRA good practices

From R4&D to RiskInformed Fire Protection RG 1.205 NUREG/CR NUREG 10 CFR 50.48(c) 2258, 2269 1150 ASME/ANS 10 CFR 50.48 PRA Standard Appendix R NUREG/CR6850 BTP 9.51 EPRI 1011989 RG 1.75 NFPA 805 Indian IPEEE NFPA 805 LARs Point PRA Browns 1970 Ferry 1980 1990 2000 2010 2020 110

TREATMENT OF UNCERTAINTIES

  • What does treatment mean?
  • What are the current approaches? Challenges?
  • Can we do better? How?

111

Decision Making Under Uncertainty Safety Security p11 Environment

  • Uncertainties C11 Cost p12 Reputation

- About outcome of alternative C12

- Conditioned on situation, A1 p1M 11 state of knowledge C1M

  • Aim: treat uncertainties p21 C21 to ensure1 p12 A2 C22

- Effectiveness (best alternative) p2N

- Efficiency C2N

- Stakeholder confidence pij = P{outcome Cijlsituation, knowledge}

1Alternatively, can bounce approaches against the Principles of Good Regulation: independence, openness, efficiency, clarity, reliability 112

Treatment > Characterization Use in Decision Characterization Communication Making 113

Characterizing Uncertainties - A Pragmatic Framework M (Model of the World):

Scope, structure i: Parameters

Universe Known Unknowns Unknown Unknowns 114

Characterization Challenges Early core melt, containment cooling Early core melt, no containment cooling

  • Parameter Uncertainties Late core melt, containment cooling Late core melt, no containment cooling

- Raw data preprocessing (selection and Containment bypass Steam generator tube rupture interpretation) Direct containment failure

- Potentially nonintuitive Bayesian updating Data source: G.J. Kolb, et al., Review and Evaluation of the Indian Point results Probabilistic Safety Study, NUREG/CR 2934, December 1982. (ML091540534)

- Stateofknowledge dependencies

- Appropriate simplification: expert elicitation Opinions

  • Model Uncertainties 1) Failure to use uncertainty characterization

- Serious consideration of alternative models best or even good practices provides

- Mainstreaming of quantitative approaches an easy target for critics, can affect

  • Completeness Uncertainties stakeholder confidence.
2) Pro forma, cookbook analyses can miss

- Systematic identification of gaps potentially useful insights.

- Serious efforts to reduce (transition to model) 115

Communicating Uncertainties Will somebody find me a onehanded scientist?!

Senator Edmund Muskie

  • Content/format depend on audience and expected use (Concorde hearings, 1976)

- Different (and changing?) levels of comfort with

  • Uncertainty
  • Formal frameworks (parameter/model/completeness; aleatory/epistemic; probabilistic/nonprobabilistic) and displays

- Different decisions => different information

  • Fundamental questions

- How confident am I (the analysis team) in the key results, insights, and implications?

- Why should you (the decision maker) be confident in my characterization?

Quote from I. Flatow, Truth, Deception, and the Myth of the OneHanded Scientist, October 18, 2012. Available from:

116 https://thehumanist.com/magazine/novemberdecember2012/features/truthdeceptionandthemythoftheonehandedscientist

Communication Challenges

  • Simpler than risk communication (fewer dimensions, perhaps less visceral reaction) Hurricane Model Warning:
  • Meeting the Goldilocks Principle Useful Advice or Just Venting?

- Are different types (per characterization frameworks) important to the decision? If anything on these products causes confusion, ignore the

- Does too much uncertainty information cause a loss of attention? entire product.

Reduce salience of key results, insights, implications?

- Does too little uncertainty information breed overconfidence or even a suboptimal decision?

  • Designing to increase cognitive engagement

- Likely important for major, nonroutine decisions

- Active learning => reduce package polish to encourage audience internal processing and dialog with team?1 1 Of course this possible (and untested) approach requires a willing audience.

117

Using Uncertainty Information in Decision Making (An Outsiders View)

  • Deliberative vs. Naturalistic Decision Making
  • Structured approaches

- MultiAttribute Utility Theory (late 1960s,1 used Adapted from NUREG2150 by ASCE?)

- Simplifications (e.g., Analytic Hierarchy Process,2 early 1980s)

- Nowadays?

  • Technical and social influences

- Needs of problem

- Views on uncertainty information (e.g., useful or confusing or even obfuscating?) From NUREG2114

- Heuristics and biases 1 See for example H. Raiffa, Decision Analysis: Introductory Lectures on Choices under Uncertainty, AddisonWesley, New York, 1968. (NRC Technical Library HD69.D4 R13 c.1) 118 2 See for example T.L. Saaty, Decision Making for Leaders: The Analytical Hierarchy Approach for Decisions in a Complex World, Lifetime Learning, Belmont, CA, 1982. (HD30.23 .S24 c.1)

Challenges in Using Uncertainty Information (An Outsiders View)

  • Demonstrating value of/creating demand for beyond pro forma treatment
  • Balancing

- Rulebased (repeatable, transparent)

- Knowledgebased (optimal use of evidence)

  • Effective communication

- With providers (what is the question)

- With stakeholders (basis for decision) 119

Treatment of Uncertainties - Concluding Remarks

  • Treatment covers characterization, communication, and use
  • A longstanding concern for RIDM with

- Accepted practices

- Remaining challenges

  • Improved methods and tools for treatment

- Are feasible

- Will provide better support for agency transformation

- May need culture change for investment and use 120

Treatment of Uncertainties - Extra Slides 121

Uncertainties and Decision Making (Two Days Before Landfall)

Andrew (1992) Irma (2017)

Hurricane Warning Hurricane Watch Evacuated Hurricane tracks adapted from University of WisconsinMilwaukee (https://web.uwm.edu/hurricanemodels/models/archive/)

122 Emergency response based on data from National Hurricane Center:

(https://www.nhc.noaa.gov/1992andrew.html)

Parameter Uncertainty: Current Practice

  • Treatment involves Estimation (including expert elicitation, Bayesian updating)

Propagation

  • Straightforward mathematics and mechanics
  • Some practical challenges 123

Parameter Uncertainty Challenges

  • Data preprocessing Runtime Failures (MotorDriven Pumps)

- Selection 1.00

- Interpretation Probability Density Function 0.80 0.60

  • Effect of analysis shortcuts 0.40 (Normalized)

- Standard (e.g., noninformative) 0.20 prior distributions 0.00 1.00E09 1.00E08 1.00E07 1.00E06 1.00E05 1.00E04 1.00E03

- Simplified expert elicitation Failure Rate (/hr)

- Independence assumption Service Water Normally Running Standby

  • Ensuring correspondence with

stateofknowledge

  • Service Water Pumps: 2 failures in 16,292,670 hours0.00775 days <br />0.186 hours <br />0.00111 weeks <br />2.54935e-4 months <br /> Normally Running Pumps: 225 failures in 59,582,350 hours0.00405 days <br />0.0972 hours <br />5.787037e-4 weeks <br />1.33175e-4 months <br />

- Basic events (micro view)

  • Standby Pumps (1st hour operation): 48 failures in 437,647 hours0.00749 days <br />0.18 hours <br />0.00107 weeks <br />2.461835e-4 months <br />

- Overall results (macro view) 124

Model Uncertainty: Hurricane Andrew 8/22/1992, 1200 UTC Current Practice Adapted from University of Wisconsin Milwaukee (https://web.uwm.edu/hurricane models/models/archive/)

  • Important to acknowledge and treat (in context of decision)
  • Multiple approaches

- Consensus model

- Sensitivity analysis

- Weighted alternatives (e.g., SSHAC)

- Output uncertainties Adapted from V.M. Andersen, Seismic Probabilistic Risk Assessment Implementation Guide, EPRI 3002000709, Electric Power Research Institute, M.H. Salley and A. Lindeman, Verification and Palo Alto, CA, December 2013 Validation of Selected Fire Models for Nuclear Power Plant Applications, NUREG1824 Supplement 1/EPRI 3002002182, November 2016.

125

Quantification of Model Output Uncertainty Time (s) Experiment (K) DRM (K)

  • Bayesian methods 180 400 450

- Framework consistent with overall PRA Data 360 465 510 720 530 560

- Early approaches used in past PRAs 840 550 565

- Can address practical issues (e.g., non Temperature (K) homogeneous data)* Assume Assume Non

  • Challenges include Percentile Homogeneous Homogeneous Output Uncertainty Data Data

- Uncertainties in unmeasured parameters 1st 415.2 372.8

- Submodel limits of applicability 5th 437.5 400.7

- Representativeness of computed results 50th 457.1 470.5 95th 479.7 559.4

- Use in actual decision making 99th 509.1 608.7

  • See E. Droguett and Ali Mosleh, Bayesian methodology for model uncertainty using model performance data, 126 Risk Analysis, 28, No. 5, 14571476, 2008.

Model Uncertainty Commentary

  • Model uncertainties can be large; importance depends on decision
  • Some practical NPP RIDM approaches (e.g., Hurricane Irma: 9/8/2017, 0000 UTC (about 2 days before FL landfall) consensus models, deterministic screening) can understate uncertainties
  • Ensemble approaches (with SMEdetermined Outer best estimate) used by other disciplines prediction is closest
  • Subjective probability framework => to actual course

- Need to consider user effect

/ Plot adapted from University of WisconsinMilwaukee (https://web.uwm.edu/hurricanemodels/models/archive/)

- Raises question regarding fundamental meaning of weighted average approaches 127

Completeness Uncertainty: NUREG1855 Rev. 1 (2017)

Current Practice Options:

  • Progressive analysis (screening, bounding,
  • Recognized concerns conservative, detailed)

- Known gaps (missing scope)

  • Change scope of risk
  • Scenario categories informed application
  • Contributors within categories RG 1.174 Rev. 3 (2018)

- Unknown gaps

  • Treatment (Mind the Gap)

- Analysis guidance

- Additional analysis/R&D

- Riskinformed decision making 128

NPP PRA Known Gaps1

  • Broad scenario categories Rationale Common Example(s)

Out of scope security/sabotage, operation outside approved limits Low significance (preanalysis judgment) external floods (many plants preFukushima)

Appropriate PRA technology* unavailable management and organizational factors PRA not appropriate software, security

  • Contributors within categories Category Example(s)

External hazards multiple coincident or sequential hazards Human reliability errors of commission, nonproceduralized recovery Passive systems thermalhydraulic reliability 1aka Known Unknowns 129

Completeness Uncertainty: Possible R&D

  • Continue to develop technology to address Event (NUREG/CR4839), 1992 known gaps Aircraft impact Avalanche

- Riskinformed prioritization Coastal erosion

- Fully engage appropriate disciplines Drought External flooding

- Take advantage of general computational and Extreme winds and tornadoes methodological developments Fire

  • Facilitate reemphasis on searching Fog Forest fire

- Demonstrate efficiency and effectiveness with Frost Hail current tools (e.g., MLD, HBFT) vs. High tide, high lake level, or high checklist/screening river stage

- Develop improved tools (including OpE mining) 130

Different Perspectives: Logarithmic vs Linear Displays 131

ADDITIONAL RESOURCES 132

Selected Lectures, Seminars, and Talks1,2

  • Nuclear Power Accidents and Incidents: Lessons for PRA, Research Seminar, University of Illinois UrbanaChampaign (virtual),

February 2, 2021. (ML20339A570)

  • PRA and RiskInformed Decision Making at the NRC: Some Trends and Challenges, Nuclear Engineering Research Seminar (Virtual),

North Carolina State University, Raleigh, NC, October 22, 2020. (ML20293A370)

  • PRA and RiskInformed Decision Making at the NRC: Some Trends and Challenges, Modeling, Experimentation, and Validation Summer School (Virtual), Idaho National Laboratory, July 27, 2020. (ML20195B157)
  • Technology for the Treatment of Uncertainties: History, Status, Commentary and Challenges, prepared for CRIEPI/NRRC and OECD/NEA Workshop on the Proper Treatment of Uncertainties in Safety Analyses, Tokyo, Japan, May 2627, 2020 (postponed, new date TBD). (ML20080N774)
  • PRA and RiskInformed Decisionmaking at the NRC: Some Trends and Challenges, B.J. Garrick Institute for Risk Sciences, University of California, Los Angeles, February 21, 2020. (ML20035G249)
  • Research and Integrated Decision Making (IDM): A Personal Perspective, Workshop: Integrated and RiskInformed Decisionmaking Forum for Managers, U.S. Nuclear Regulatory Commission, November 13, 2019. (ML19310F243)
  • Dynamic PRA: Not If But When? Invited Talk, IAEA Workshop on Advanced PSA Approaches and Applications, Alkmaar, The Netherlands. September 913, 2019. (ML19248C656)
  • Advanced Knowledge Engineering Tools to Support Probabilistic Risk Assessment (PRA) Activities A Whole New World? NRC Knowledge Management (KNOWvember) Webinar, November 21, 2017. (Webinar Video)
  • PRA R&D - Changing the Way We Do Business? Invited Plenary Lecture, ANS International Topical Meeting on Probabilistic Safety Assessment (PSA 2017), Pittsburgh, PA, September 2428, 2017. (ML17292A552) 1The ML numbers refer to pdf versions. PowerPoint versions (with fullresolution graphics) will be provided in a separate ADAMS package.

133 2Although some of the talk titles are duplicative, the material has been tailored to the different audiences and venues.

3Lectures, workshops, and reference material for a 1week intensive course (meant to cover material normally provided in a semester).

Retrieved from "https://kanterella.com/w/index.php?title=ML21138A792&oldid=3410304"