ML21138A793

RES Seminar Part 1 - Nightmares

ML21138A793
Person / Time
Issue date:	05/13/2021
From:	Nathan Siu NRC/RES/DRA
To:
Siu, Nathan - 301 415 0744
Shared Package
ML21138A647	List: ML21138A778 ML21138A779 ML21138A780 ML21138A781 ML21138A792 ML21138A793 ML21138A805 ML21138A810
References
Download: ML21138A793 (49)

Text

I Still Have Nightmares About That Class*

PRA: why its complicated and why it doesnt have to be Nathan Siu Senior Technical Adviser for PRA Analysis Special Guests:

Office of Nuclear Regulatory Research Prof. George Apostolakis Dr. Harold S. Blackman Division of Risk Analysis Dr. Dennis C. Bley Dr. Robert J. Budnitz RES Staff Technical Seminar (Virtual) - Part 1 Prof. Ali Mosleh John W. Stetkar May 13, 2021 (2:00-3:00)

Dr. Thomas R. Wellock

• The views expressed in this presentation are not necessarily those of the U.S. Nuclear Regulatory Commission.

After 40+ years, PRA seems intuitive to me Typewriters, punch cards => laptops It cant be done => modern risk-informed regulator Indian Point PRA Quad COMPBRN Summer Cities (NRC-support) at NRC IPEEE Browns Ferry Fire, Join Join Join Join WASH-1400 PLG MIT INL NRC 9/11 Fukushima COVID-19 1975 1980 1985 1990 1995 2000 2005 2010 2015 2020 Punch card graphic adapted from: https://en.wikipedia.org/wiki/Punched_card#/media/File:FortranCardPROJ039.agr.jpg. Publicly available under Creative Commons Attribution-Share Alike 2.5 Generic conditions, 2

but it might not be to others An old survey More recently Carolyn Kenny Christopher (12) (9) (4) You no longer need to Who does The Nuclear be a mathematical genius Wha? The Daddy Regulatory government Me to run a reliability or risk work for? Commission analysis.

Makes sure nuclear He reads a lot of What does plants dont go he do? overboard or stuff and goes Write - Ola Bckstrm (2021)1 to meetings something like that 1Ola Bckstrm, The role of digital insight in a safer nuclear industry, Power, January 28, 2021. (Available from:

3 https://www.powermag.com/the-role-of-digital-insight-in-a-safer-nuclear-industry/)

Talk Outline

• PRA: what is it and why do it? Alphabet Soup PRA = Probabilistic Risk Assessment

• Challenges and complications RIDM = Risk-Informed Decision Making

• Strategies for reducing complexity

• Closing remarks 4

PRA: WHAT AND WHY 5

Risk Assessment

• Risk (per Kaplan and Garrick,1 adopted by NRC2) Whats in a word?

- What can go wrong? analysis, n., process of

- What are the consequences? separating an entity into its constituent elements; process as

- How likely is it? a method for studying the nature of something or determining its

• Qualitative as well as quantitative essential features and their relationships

• Non-prescriptive, flexible

- Does not define wrong or prescribe metrics for assessment, n., an estimation or judgment of value [emphasis consequences or likelihood added] or character

- Does not define how risk is to be assessed 1S. Kaplan and B.J. Garrick, On the quantitative definition of risk, Risk Analysis, 1, 1981.

2See, for example:

6 - White Paper on Risk-Informed and Performance-Based Regulation (Revised), SRM to SECY-98-144, March 1, 1999.

- Glossary of Risk-Related Terms in Support of Risk-Informed Decisionmaking, NUREG-2122, May 2013.

PRA Risk assessment where likelihood is quantified in terms of probability

• Still flexible - definition does not mandate Subjective Interpretation of Probability1 specific methods (e.g., event tree/fault tree analysis)

• Probability quantifies degree of belief

• Appropriate for decision support

• Typically: engineering analysis process

• Inherent in current PRAs (e.g., Bayesian

- Models facility/process as an integrated system updating)

• Not universally accepted

- Attempts to address all important scenarios Subjectivity uncomfortable for many (within study scope) Technical objections (appropriateness of a lottery model for characterizing

- Attempts to use all practically available, subjective uncertainty) relevant information (not just statistics) 1See:

- G. Apostolakis, Probability and risk assessment: the subjectivistic viewpoint and some suggestions, Nuclear Safety, 9, 305-315(1978).

- G. Apostolakis, The concept of probability in safety assessments of technological systems, Science, 250, 1359-1364(1990).

7 - M. Granger Morgan, Use (and abuse) of expert elicitation in support of decision making for public policy, National Academy of Sciences Proceedings (NASP), 111, No. 20, 7176-7184, May 20, 2014.

Why PRA? Risk assessment is a set of tools, not an end in itself. The limited resources available should be spent to generate information that helps risk managers PRA Policy Statement (1995)1 to choose the best possible course of action among the available options.

• Increase use of PRA technology in all regulatory - National Research Council, 1994 matters It [fire PRA] aint perfect but its the

- Consistent with PRA state-of-the-art best thing weve got.

- Complement deterministic approach, support defense- - G. Holahan in-depth philosophy Our tendency is to focus on things

• Benefits: that are interesting and make them (1) Considers broader set of potential challenges important. The thing that we have to do is focus on what really is (2) Helps prioritize challenges important (3) Considers broader set of defenses - R. Rivera, 2020 1U.S. Nuclear Regulatory Commission, Use of Probabilistic Risk Assessment Methods in Nuclear Activities; Final Policy Statement, 8 Federal Register, 60, p. 42622 (60 FR 42622), August 16, 1995

Risk information has uses beyond immediate decision Adapted from NUREG-2150 support 9

9

Moving Forward

• Past successes1 => expectation of future successes

• Past results => anticipation of Average Plant CDF 1.00 Probability {one or more accidents before t}

0.90 International Fleet ~ 440 rx 10-4/ry future challenges 0.80 0.70 5*10-5/ry

• Continued investment => readiness 0.60 0.50 to meet challenges, maintain NRC 0.40 0.30 0.20 10-5/ry international leadership 0.10 0.00 0 10 20 30 40 50 Years from Now 1For examples, see Probabilistic Risk Assessment and Regulatory Decisionmaking: Some Frequently Asked Questions, NUREG-2201, September 2016.

10

NPP PRA: ITS CHALLENGING 11

Fatality Rate by Vehicle Type (2018)

Fatalities/105 Vehicles Lots of Data => Statistical Analysis 15.0 10.0 Cars SUVs 5.0 Pickups 0.0 Vans 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 Alcohol-Impaired Driving From Traffic Safety Facts: Research Note, U.S. Dept. of Transportation, 2016.

Fatality Rates per 106 VMT (2018)

Motor Vehicle Fatalities U.S. Average: 0.32 Fatality Rate (per 100M VMT) 60,000 6.00 Maryland: 0.20 50,000 5.00 40,000 4.00 Fatalities 30,000 3.00 Accident Causes 2005-2007 Driver Errors 20,000 2.00 10,000 1.00 Recognition Driver Decision 0 0.00 Vehicle Performance 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 Environment Non-Performance Unknown Other Data from https://crashstats.nhtsa.dot.gov 12

Fundamental NPP PRA Accident In a nutshell Note TMI 2 Anticipated transient + Unlikely confluence Challenge: Little/No Plant- (1979) additional failures and errors of likely events Level Data Chernobyl 4 (1986)

Systems test in unstable regime, violating procedures Single-minded aim to perform test Fukushima

• Sparse data Daiichi 1-3 Beyond design basis tsunami Extremely unlikely catastrophic event

- Few accidents/serious incidents (2011)

- Statistical relevance challenged by design and 2021: ~18700 reactor-years operational changes

- Interest in specific plant => further reduced significant data set precursor

• Coping strategies

- Decomposition-based systems modeling (e.g., event trees, fault trees) precursor

- Specialized estimation procedures (e.g.,

Bayesian statistics, expert elicitation) for model elements

=> Complexity (no free lunch) Licensee Event Reports 1969-2019 (~4360 ry)

(No significant precursors since 2002; one under review) 13

PRA Complications

• Inherent in problem, e.g., com*pli*cat*ed, adj. consisting of many parts not easily

- Complex phenomenology (often beyond separable; difficult to analyze, understand, explain, etc.

experience)

- Multiple technical disciplines, roles, and For many years, risk assessment required perspectives a high level of abstraction and an elite team of analysts fully immersed in the

• Highlighted (or even introduced) by ways of every single component and their failure profiles. A heady task for any risk analyst, but one made doubly hard by the coping strategies for sparse data exacting requirements of nuclear.

- Ola Bckstrm (2021)1 1Ola Bckstrm, The role of digital insight in a safer nuclear industry, Power, January 28, 2021. (Available from:

14 https://www.powermag.com/the-role-of-digital-insight-in-a-safer-nuclear-industry/)

Complex Phenomenology: Scenario Dynamics (1)

Time Hazard Systems Indications Operators/Workers ERC/ER team EP Time 14:46 0:00 Earthquake Scram MSIVs close, turbine trips, EDGs 14:47 0:01 Rx level drops start and load RV pressure decreases; RV level 14:52 0:06 ICs start automatically in normal range Cooldown rate exceeding tech 15:03 0:17 ICs removed from service Manually remove IC from service spec limits Disaster HQ established in TEPCO 15:06 0:20 Tokyo Determine only 1 train IC 15:10 0:24 needed; cycle A train First tsunami 15:27 0:41 arrives Second tsunami 15:35 0:49 arrives 15:37 0:51 Loss of AC 1537-1550: Gradual loss of instrumentation, indications 15:37 0:51 Determine HPCI unavailable (including IC valve status, RV level), alarms, MCR main lighting TEPCO enters emergency plan 15:42 0:56 (loss of AC power); ERC established D/DFP indicator lamp indicates 16:35 1:49 "halted" Review accident management Cannot determine RV level or Review accident management procedures, start developing injection status; work to restore procedures, start developing Declared emergency (inability to 16:36 1:50 procedure to open containment level indication; do not put IC in procedure to open containment determine level or injection) vent valves without power service vent valves without power 15

Complex Phenomenology: Scenario Dynamics (2) 16

Coping with Dynamics

• Aggregation (bundling)

• Simplified timing + success criteria For an early discussion of transitions between sequences, see G. Apostolakis and T.L. Chu, Time-dependent accident sequences 17 including human actions, Nuclear Technology, 64, 115-26 (1984).

Complication: Multiple Disciplines, Multiple Roles Different points of view:

• Whats important to the analysis?

• Whats an acceptable solution approach?

Analysts/

Users Reviewers Plant Systems Electrical Human Factors Mechanical Civil NPP Fire Protection Materials PRA Earth Sciences Probability Nuclear & Statistics Operational Systems Developers Experience Science 18

External Flooding at Plant X: Model Scope?

U.S. watershed image from https://www.nps.gov/miss/riverfacts.htm 19

Diverse Views: From Coping to Benefitting?

From You PRA Guys/Gals to Us PRA Guys/Gals?

• Clear definition of analysis needs, interfaces

• Stakeholders 101: early, open engagement

• Future: integrated native language analysis (e.g., dynamic PRA)?

20

Complication: Numerous Possibilities

• Many paths to core damage

• Many ways to fail each barrier in path 21

Coping with Multiple Scenarios

• Model simplifications, e.g.,

- Screening

- Grouping (often with bounding quantification)

• Boolean algebra, reliability theory,1 e.g.,

for independent basic events, where CAFTA RISKMAN 1 1 1 1 Risk Spectrum

• Software tools to implement theory 1 See, for example, R.E. Barlow and F. Proschan, Statistical Theory of Reliability and Life Testing Probability Models, To Begin 22 With, Silver Spring, MD, 1975. (Available in the NRC Technical Library: TS173.B37 c.1)

Complication: Sparse Data Potomac River Flooding (Little Falls, VA) 30 28 26 Flood Height (ft) 24 22 20 18 16 Major Flood 14 Moderate Flood 12 10 1930 1940 1950 1960 1970 1980 1990 2000 2010 2020 Data from: https://water.weather.gov/ahps2/crests.php?wfo=lwx&gage=brkm2&crest_type=historic 23

Coping with Sparse Data: Modeling + Bayesian Estimation Potomac River (Little Falls, VA)1

• First cut bounding analysis: major flood1 => catastrophic flood Date Flood Height (ft)

• Frequency of major flooding ()

5/14/1932 15.25

- Prior state-of-knowledge: minimal 2/27/1936 14.69

- Evidence: 12 major floods over 1932-2019 (87 years) 3/19/1936 28.10

- Bayes Theorem: , 4/28/1937 23.30

, 10/30/1937 15.62

- Posterior state-of-knowledge: Poisson Non-informative 10/17/1942 26.88 4/29/1952 14.17 05 = 0.079/yr probability density 8/20/1955 17.60 prior 50 = 0.13/yr 6/24/1972 22.03 posterior 95 = 0.21/yr mean = 0.14/yr 11/7/1985 17.99 1/21/1996 19.29 0.00 0.05 0.10 0.15 0.20 0.25 0.30 9/8/1996 17.84 Major Flood Frequency (/yr)

• More sophisticated analysis if needed (e.g., frequency-magnitude analysis (perhaps with expert elicitation) 1 Data from: https://water.weather.gov/ahps2/crests.php?wfo=lwx&gage=brkm2&crest_type=historic 2Major Flood: height > 14 ft 24

More Complications: Expert Elicitation >> BOGGSAT1 what we know

• Mechanism to support decision making what we believe P{XlC,H}

- Diverse, authoritative views

- Broad range of evidence proposition/event conditions of of concern probability

• Social process => social biases; need statement

- Formal elicitation processes (e.g., SSHAC2) Level Characteristics

- Sufficient time and resources 1 TI only (literature review, personal experience)

• Need to remember purpose and context; 2 TI interacts with proponents and resource experts follow-on experimentation, analysis, etc. 3 TI brings together proponents and resource experts may be needed 4 TFI organizes expert panel to develop estimates TI = Technical Integrator TFI = Technical Facilitator/Integrator 1BOGGSAT: Bunch of guys and gals sitting around a table 2SSHAC: Senior Seismic Hazard Analysis Committee. See R. J. Budnitz, et al., Recommendations for Probabilistic Seismic Hazard 25 Analysis: Guidance on Uncertainty and Use of Experts, NUREG/CR-6372, 1997.

You no longer need to be a mathematical genius to run a reliability or risk analysis.

- Ola Bckstrm (2021)1 SO PRA CAN BE COMPLICATED.

DOES IT HAVE TO BE?

1Ola Bckstrm, The role of digital insight in a safer nuclear industry, Power, January 28, 2021. (Available from:

26 https://www.powermag.com/the-role-of-digital-insight-in-a-safer-nuclear-industry/)

It depends. (Tough problems => increased complexity)

• Technically challenging

- Complex phenomenology

- Multiple disciplines, roles, perspectives

• Tough decisions (higher-fidelity solutions)

- high stakes

- multiple stakeholders

- multiple risk attributes

- uneven distribution of risks and benefits

- large uncertainties From Indian Point Emergency Plan (ML15357A005) 27

Reducing PRA Complexity Source Simplification Strategy BUT Complex

• Simplify regulated systems/processes

• Beware of simplistic characterizations (e.g.,

phenomenology

• Increase certainty in rarity of off- gravity never fails => natural circulation normal conditions (facilitates cooling will always work) screening)

• Remember real-world testing and

• Obtain more empirical data (reducing maintenance needs => extra bits and pieces, need for sub-modeling) off normal configurations and procedures

• Improve PRA technology1 to improve

• Remember even simple systems can have focus on whats important complex behaviors (e.g., dynamic resonances)

Multiple disciplines, Improved communication Beware of unintended side effects (e.g., reducing roles, perspectives diversity through forcing a view)

Tough decision Reduce stakes (e.g., by reducing

• Recognize some risk metrics (e.g., for problem (driving potential consequences), enabling enterprise risk) might be less sensitive to need for high-fidelity lower-fidelity model design/operational changes PRA model)

• Recognize technical arguments for reduced concern might not be accepted 1PRA 28 Technology = PRA methods, models, tools, data

Internal Risk Communication Challenge

• Principle: the decision maker should be an informed consumer of risk information

• What do the DMs need to know? Is perceived complexity a barrier to effective Adapted from NUREG-2150 communication?

Barriers?

Other Considerations

• Current regulations

• Safety margins

• Defense-in-depth PRA is for my PhDs

• Monitoring Quantitative Qualitative 29

Reducing Perceived Complexity Strategy BUT Improve training and communication: ensure focus is

• Beware of turning PRA into a black box oracle; DMs on what DMs need to know need to appreciate (without overemphasizing) limitations and uncertainties

• Ensure NRC has (or has access to) experts who understand and can communicate limitations and uncertainties, especially when addressing novel applications (designs, processes, decision problems)

Improve PRA technology1 to increase focus on whats Same as above but ever so much more so important (e.g., analytics-informed automated PRA)

Wait: take advantage of growing societal experience Dont wait too long (technology rejection is the result of with and acceptance of analytics (e.g., sports), social processes, established attitudes can be difficult to modeling (e.g., weather), real-world risk scenarios2 overcome) and trade-offs (e.g., climate change, pandemics) 1PRA Technology = PRA methods, models, tools, data 2According to https://www.etymonline.com, the current, common use of scenario (Italian, sketch of the plot of a play) as an imagined 30 situation first occurred in 1960 as a reference to hypothetical nuclear wars.

Were Not Alone

• Other industries and other countries perform risk 1978 assessments for a wide range of applications (simple to complex). Examples:

- Chemical process industry

- NASA 1985

- Netherlands (all industries, all hazards)

• Potentially instructive: review of requirements and practices for lower-risk applications 2020 1Oosterscheldedam photo from 31 https://commons.wikimedia.org/wiki/File:Oosterscheldedam_storm_Rens_Jacobs.jpg

Example: Layers of Protection Analysis (LOPA)1

• Intention: reduce inconsistency in qualitative assessments without requiring full PRA

Purpose:

estimate risk (order-of-magnitude frequencies, qualitative consequences), assess adequacy of protection layers

• Adequacy assessed via risk matrix 1See M. Kazarians and K. Busby, Use of simplified risk assessment methodology in the process industry, Proceedings International 32 Conference Probabilistic Safety Assessment and Management (PSAM 14), Los Angeles, CA, September 16-21, 2018.

Change Emphasis to Improve Communication?

(And Banish Nightmares?)

The Engineering Story System Familiarization: Scenario Analysis Risk-Informed Decision Making

- How do things work?

- How can they fail?

33

PRA Simplification: Some Cautionary Notes

• Past NPP PRA simplifications have gravitated to more detailed models

- RSSMAP/IREP1 => NUREG-1150

- ASP plant class models => SPAR

• Simplified model results and insights can be harder to interpret and use

- Reduced scope => unknown importance of out-of-scope contributors

- Game over conservatism => masking of important contributors

• Better, cheaper, and faster - realistic Risk Reduction Alternatives (notional) result of learning or wishful thinking?

1RSSMAP = Reactor Safety Study Methodology Applications Program (4 plants, 1978-1982) 34 IREP = Interim Reliability Evaluation Program (4 plants, 1980-1982)

CONCLUDING REMARKS 35

The Bottom Line PRA can be complicated You know about conservation of mass,

• Inherent problem complexities energy, etc. Today were going to talk about

- Systems and phenomenology the Conservation of Difficulty.

- High-stakes issues

• Coping strategies for problem complexity can introduce technical complexity Hoo boy.

- Modeling simplifications and math Gotta get out

- Estimation procedures to address sparse data of this class!

• Multiple disciplines/communities => added complexity but complexity can [sometimes] be reduced

• Simplify problem (e.g., simplify analyzed system, reduce stakes of decision)

• Improve PRA technology (methods, models, tools, data)

• Improve training 36

Acknowledgments My views on PRA have, of course, been strongly influenced by my interactions with others. I can truthfully say that Ive learned from all of my colleagues and that Im still digesting some of these lessons. Special acknowledgments go to Professor George Apostolakis (my adviser and mentor in grad school and beyond, who gave me a framework and tools for thinking about PRA and its use); Dr. B. John Garrick (the importance of aiming for the truth, even if unpopular); Professor Norman Rasmussen (the importance of pragmatic engineering approaches even in R&D, theres no such thing as a worst case),

John Stetkar (the basics of practical NPP PRA in the field); Dr. Harold Blackman (the importance and rigor of human factors engineering); Professor Ali Mosleh, Dr. Dennis Bley, and Dr. Robert Budnitz (gracious sounding boards for ideas, wild or otherwise); and Dr. Thomas Wellock (the early history of PRA and what skeptics think about the enterprise). My particular thanks go to Dr. Dana Kelly, gone too soon, for fruitful discussions. I regret that we never got to write the Details Matter paper we were toying with.

37

ADDITIONAL SLIDES 38

Everyday Risk-Informed Decisions

• Should I

- Go for a run in the woods? Teach me to

- Cross the street against the light? ignore that High

- Eat that last doughnut? Wind warning

- Click on that emailed link?

- Go to the office when Im coughing?

- Get vaccinated?

- Visit NYC?

• What do I know?1 What are the current conditions?

• What are the risks? The benefits?1

• N.B. Risk is input to decision problem (choice among alternatives), not just FYI 1 And of course: What are the rules? What are the margins? Is there any defense in depth? Can I monitor the outcome(s) to influence future choices?

39

Risk information - not always for decision support.

(Sometimes people just want to know.)

MoCo Covid-19 Cases (%)

0.06 0.05 Daily Cases (%)

0.04 0.03 MoCo Dailies %

MoCo 7-Day (%)

0.02 0.01 0

COVID-19 data from: https://coronavirus.maryland.gov/datasets/mdcovid19-casesbycounty 40 Estimated population for Montgomery County (2020): 1M

RIDM: A Changing Environment

• Internal

- Overall direction (transformation)

- Initiatives (e.g., Be riskSMART)

• External

- Risk communication: risk maps, e.g.,

• Tsunami inundation zones (explicit), e.g., https://www.conservation.ca.gov/cgs/tsunami/maps

• Industrial risks (explicit), e.g., https://www.risicokaart.nl/

• Wildfire extent (implicit), e.g., https://inciweb.nwcg.gov/

• COVID-19 extent (implicit), e.g., https://coronavirus.maryland.gov/

- Explicit representation of uncertainties (e.g., hurricane tracks)

- Explicit acknowledgment of expert judgment informed by models (e.g., weather forecasting)

- Tough, widely discussed risk problems (e.g., climate change, COVID-19) 41

On Using the Right Tool: Some Cautions

• If all you have is a hammer Event tree/fault tree analysis for a fundamentally continuous process?

• Using the wrong tool might not only be ineffective or inefficient, it might damage the tool Using PRA to prove a facility/process is safe?

42

Complexity: In the Eye of the Beholder Analysts/

Users Reviewers Developers

, 0 , 1, ,

43

Challenges and Whats Important:

In the Eye of the Beholder

• Near-term solutions: heavy

• Fundamental nature of risk problem time/budget pressure (complexity, uncertainty, multiple consequence

• Huge problem size and types and potentially large magnitude, complexity Analysts/ multiple stakeholders, )

Multiple technical Users communities/cultures Reviewers

• Competing problems with attentional and

• State of technology: Too resource demands much/little diversity, Holes

• Academic contribution Developers

• Nexus between personal/professional and external interests

• Support (especially with declining budgets) 44

Increasing Model Completeness (and Confidence)

Information Sources Attitude it is incumbent upon the

• Hazard analysis tools, e.g.,

• Be open to possibilities new industry and the

- Failure Modes and Effects

• Use checklists but also search Government to make every Analysis (FMEA) for ways to get in trouble, e.g., effort to recognize every

- Hazard and Operability Studies possible event or series of

- What might prompt operators (HAZOPS) to operate in an unstable events which could result

- Master Logic Diagrams (MLD) regime? Disable safety in the release of unsafe

- Heat Balance Fault Trees systems? amounts of radioactive

- What could cause a complete material to the

- System-Theoretic Accident Model and Processes/Systems- loss of AC and DC power? surroundings Theoretic Process Analysis - What could cause coolant (STAMP/STPA) channel blockage? - W.F. Libby (1956)1

• Past events - What could cause removal of

• Other studies all control rods?

1W. F. Libby (Acting Chairman, AEC) - March 14, 1956 response to Senator Hickenlooper. [See D. Okrent, Reactor Safety, University of 45 Wisconsin Press, 1981. (NRC Technical Library TK9152 .O35, multiple copies)]

Harnessing Imagination:

Credible Possibilities Need Support (Causality)

OPERATOR TERMINATES Possible ISOLATION CONDENSER OPERATION but ISO-XHE-EOC-TERM plausible?

46

Integrator Expert Elicitation Easy Button General Process Process

1) Preparation Design 2) Piloting/Training Group Workshop 3) Interactions (Workshops) a) Evaluate evidence b) Develop, defend, and revise judgments Interaction Model Data c) Integrate judgments With Structure 4) Participatory Peer Review Interaction Individual Experts Interaction Group Workshop Interaction Model Ground Motion Uncertainty With Parameter Forecast Assessment Individual Experts Interaction Interaction Interaction Integrator Interaction Adapted from: R. J. Budnitz, et al., Recommendations for With Integration Probabilistic Seismic Hazard Analysis: Guidance on Uncertainty and Individual Experts Use of Experts, NUREG/CR-6372, 1997.

47

Sources of Risk Communication Breakdowns1

• Differences in perception of information

- Relevance

- Consistency with prior beliefs

• Lack of understanding of underlying science

• Conflicting agendas

• Failure to listen

• Trust 1J.L. Marble, N. Siu, and K. Coyne, Risk communication within a risk-informed regulatory decision-making environment, International 48 Conference on Probabilistic Safety and Assessment (PSAM 11/ESREL 2012), Helsinki, Finland, June 25-29, 2012 (ADAMS ML120480139).

Listed causes are for breakdowns between risk managers and the public, but appear to be relevant to internal risk communication as well.

Bowtie Diagrams:

Different Visualization => Different Insights? Decisions?

From W. Nelson, How Things Fail - e.g. Deepwater Horizon and Fukushima - and Occasionally Succeed, presentation to U.S.

49 Nuclear Regulatory Commission, Det Norske Veritas AS, November 2, 2011.