ML17265A846

Risk-Informed Thinking Workshop (Public Slides)

ML17265A846
Person / Time
Issue date:	08/02/2017
From:	Samson Lee Special Projects and Process Branch
To:
Lee S/DORL-LSPB/301-415-3168
References
Download: ML17265A846 (85)

Text

Risk-Informed Thinking Workshop Welcome August 2, 2017 (Public Slides)

NRC iLearn ID_280148

Agenda 8:00 to 9:00 EST Risk-Informed Decision Making Refresher See-Meng Wong, NRR 9:00 to 9:30 Senior Management Perspective Marc Dapas, Office Director NMSS 9:30 to 10:35 Risk-Informed Thinking Donnie Harrison, NMSS, and Laura Kozak, Region III 10:35 to 10:45 Break 10:45 to 12:15 Scenario #1, Joe Rivers, NSIR 12:15 to 1:00 Lunch 1:00 to 2:30 Scenario #2, Steve Arndt, NRR 2:30 to 2:40 Break 2:40 to 4:00 Scenario #3, Kathy Lyons-Burke, OCIO 2

Sponsors/Leads/Mentors NMSS: Donnie Harrison NRO: Martin Stutzke NRR: Steven Arndt, Paul Clifford, James Hickey, Allen Hiser, Kamal Manoly, Jeff Mitman, See-Meng Wong NSIR: Joseph Rivers OCIO: Kathy Lyons-Burke RES: Thomas Nicholson, Nathan Siu, Sushil Birla Regions: Laura Kozak, Bill Cook, Dave Loveless, George MacDonald, Scott Freeman Agency Senior Level Advisors, Senior Reactor Analysts (SRAs) or workshop instructor 3

Objectives At the end of this workshop, you will be able to:

• Discuss how risk information can be used to:

- Complement the traditional evaluation approaches

- Focus the evaluation process for effectiveness and efficiency

• Communicate and apply the risk-informed decision-making process to agency work 4

Workshop Outline

• Background material

- What is risk-informed?

- Why does NRC strive to be risk-informed?

• Risk-informed decision making (RIDM)

- The Risk Triplet

- Integrated decision process

• Hands-on use of RIDM on several scenarios

- Instructor examples

- Student Teams

• Conclusion - Risk-Informed Thinking Certificate 5

RISK-INFORMED DECISION MAKING REFRESHER 6

What is Risk-Informed?

Why does NRC use it?

What is risk-informed regulation?

• A philosophy whereby risk insights are considered together with other factors to establish requirements that better focus licensee and regulatory attention on design and operational issues commensurate with their importance to health and safety.*

• SRM on SECY-98-144, 3/1/99 8

What is risk?

• In everyday usage, "risk" is often used synonymously with the probability of a loss.

• In the context of evaluating risk from a nuclear power plant, risk is commonly expressed as the risk triplet:

1. What can go wrong (accident scenario)?

2. How likely is it (frequency on a reactor-year basis)?

3. What are the consequences (impact on the plant or on people)?

9

The 5 Key Principles of Risk-Informed Note - while this focuses on changes, Regulation the principles may be applied to other types 2. Consistent with of regulatory defense-in-depth decisions philosophy

1. Meets current regulations (unless it 3. Maintain sufficient is explicitly related safety margins INTEGRATED to an exemption or DECISIONMAKING rule change)

5. Use performance- 4. Increases in risk (CDF) are measurement small (consistent with the strategies to monitor Commissions Safety Goal the change Policy Statement)

• RG 1.174, 11/2002 10

Compliance and Defense-in-Depth

• Compliance with regulations (unless exemption is granted)

• Consistency with defense-in-depth philosophy is maintained if:

- A reasonable balance is preserved among prevention of core damage, prevention of containment failure, and consequence mitigation.

- Over-reliance on programmatic activities to compensate for weaknesses in plant design is avoided.

- System redundancy, independence, and diversity are preserved commensurate with the expected frequency, consequences of challenges to the system, and uncertainties 11

Defense-in-Depth (Continued)

• Consistency with defense-in-depth philosophy is maintained if:

- Defenses against potential common cause failures are preserved, and the potential for the introduction of new common cause failure mechanisms is assessed.

- Independence of barriers is not degraded.

- Defenses against human errors are preserved.

- The intent of the plants design criteria is maintained.

Source: RG 1.174, 11/2002 12

Safety Margins

• With sufficient safety margins:

- Codes and standards or their alternatives approved for use by the NRC are met.

- Safety analysis acceptance criteria in the LB (e.g.,

FSAR, supporting analyses) are met, or proposed revisions provide sufficient margin to account for analysis and data uncertainty.

Source: RG 1.174, 11/2002 13

Risk

• Qualitative approaches may be considered for ensuring risk is acceptably low if appropriate

• NRR also has quantitative risk acceptance guidelines for reactors - refer to P-101 or similar training

• This course: Uses the risk triplet to qualitatively assess risk

- Recognize that more detailed risk assessment may be necessary in some cases Source: RG 1.174, 11/2002 14

Performance Measurement

• In risk-informed licensing actions, the primary goal for performance monitoring strategies is to ensure that no adverse safety degradation occurs because of the changes to the licensing basis

- Possibility that the aggregate impact of changes that affect a large class of systems, structures, and components (SSCs) could lead to an unacceptable increase in the number of failures from unanticipated degradation, including possible increases in common cause mechanisms

- Implementation and monitoring plan to ensure that the conclusions that have been drawn from the engineering evaluation remain valid (continues to reflect the actual reliability and availability of SSCs that were evaluated.

Source: RG 1.174, 11/2002 15

What is Risk-Informed Thinking?

• Consider risk information when thinking about an issue

• A culture that thinks and discusses risk

• An integrated decision-making approach to incorporate risk insights in evaluating issues

• You do not have to be a probabilistic risk analysis (PRA) specialist to think risk-informed 16

Why is risk information used?

• Commissions policy statement on the use of PRA* included four main statements:

1. Increase use of PRA to the extent supported by the state-of-the-art and in a way that complements traditional engineering approaches

2. Use PRA both to reduce unnecessary conservatism in current requirements and to support proposals for additional regulatory requirements

3. Be as realistic as practicable

4. Consider uncertainties appropriately when using the Commission's safety goals and subsidiary numerical objectives

• 8/16/95 17

What is the Commissions direction?

(Detailed Background)

Sample of Staff Requirements Memoranda (SRM):

• Staff Requirements-SECY-98-144-White Paper on Risk-Informed and Performance-Based Regulation, March 1, 1999.

• Staff Requirements-SECY-99-100-Framework for Risk-Informed Regulation in the Office of Nuclear Material Safety and Safeguards, June 28, 1999.

• Staff Requirements-SECY-11-0024-Use of Risk Insights to Enhance the Safety Focus of Small Modular Reactor Reviews, May 11, 2011.

18

Why is it important to consider risk insights?

It can make us a more effective and efficient regulator

• Risk information can complement deterministic approach in integrated decision-making to make better decisions

• Better inform the scope of staff review and help focus resources on significant issues

• Better communicate the significance of issues 19

Why is it important to consider risk insights? (Continued)

• Resources expended by staff and industry on issues/reviews should be commensurate with their risk significance

• Focus efforts on risk significant issues/reviews

- Consider risk insights to enhance effectiveness and efficiency

• NRC provides reasonable assurance, not absolute assurance

- Leverage risk insights in reaching a reasonable assurance determination

• Collaboration of staff to make better decisions 20

How can I consider risk insights in licensing review?

• We follow guidance in Standard Review Plan, which is traditionally deterministic

• However, use of risk insights is addressed in the Standard Review Plan:

Risk insights can also be used in determining the depth of review. Standard Review Plan, Introduction, Scope of Review of License Applications, NUREG-0800, Revision 2, March 2007.

21

How can I consider risk insights in licensing review? (Continued)

• A review should be effective and efficient

• Standard Review Plan allows for judgment, for example, it does not specify the number of Requests for Additional Information (RAIs)

• We can use risk insights to help:

- Determine how deep to dig

- Focus the scope of the review

- Establish review priority 22

Why arent our decisions risk-based?

• Risk-based would mean we decide using only the numerical results and insights of a risk assessment - if risk assessments are so helpful, why not?

- We cant measure risk - we have to evaluate it using models

• The models should address all contributors but do so with varying degrees of rigor and realism

• Data on many failures or initiating events is sparse

• Uncertainties may be large, but in principle we know how to deal with them

- However, we cannot know everything, and therefore our models are incomplete, e.g., there could be previously unknown failure mechanisms.

• Therefore, we still consider traditional deterministic concepts such as defense-in-depth and safety margins, as well as performance monitoring, to accommodate our incomplete knowledge 23

RISK-INFORMED THINKING Risk Triplet Integrated Decision Making Process

Review (from a few slides back)

The Risk Triplet helps us to get a ball-park idea of the relative risk of an issue or activity This is done qualitatively by asking:

• What can go wrong?

• How likely is it?

• What are the consequences?

25

What is Risk? (Continued)

Example: Medical radiotherapy What is risk?

• What can go wrong? Misadministration of radiation treatment resulting in patient receiving a dose much greater than prescribed

• How likely is it? Several times a year nationwide

• What are the consequences? Depending on the dose and radiated site, there could be a significant undesirable health effect on the patient 26

Sample Decision-Making Process Source - NRR Office Instruction LIC-504, Integrated Risk-Informed Decision-Making Process for Emergent Issues 27

Sample Decision-Making Process (2)

Source - NUREG-2150, A Proposed Risk Management Regulatory Framework 28

5 Step Process for this Workshop

1. Characterize the issue (including initial risk estimate)

2. Define decision options

3. Assess each option (5 key principles +)

4. Integrate assessment of options and make recommendation

5. Communicate assessment and recommendation 29

Step 1 - Characterize the Issue

• Characterize the issue in terms of the physical impact on the plant and the potential impact on safe operation, well enough to:

- Make initial assessment of those organizations that must be involved

- Determine what information will be needed to perform subsequent steps

- Consider any sources of information that may provide useful information on the issue:

• UFSAR and Safety Evaluation Report

• Inspection reports

• Results of team inspections

• Incident reports

• Use the risk triplet to the extent practicable to help determine the urgency and importance of the issue 30 30

Step 2 - Define Decision Options

• Determine decision environment: Urgency, available tools, resources

• Define the decision options to resolve the issue; e.g.:

- Immediate plant shut down

- Shut the plant down within a specified time period

- Continued operation with the implementation of compensatory actions

- Delay the decision until more information is available.

• Describe the guidelines or criteria for acceptability or rejection of each decision option

• Describe the factors that determine the approach to the analysis of the issue and the selection of options

• Identify the potential primary decision makers

• NOTE: Choice of options may change as more information becomes available. Process is iterative to allowing adjustments to the original scope or approach.

31 31

Step 3 - Assess Each Option

• Analyze and document the assessment of each option

- Determine factors that differentiate among options

• e.g., 5 key principles of risk-informed regulation

• Other factors that will drive the decision

- Assess each option in terms of how each factor is affected

- Assess the technical adequacy of each analytical method used

• Document the analysis of each input to the decision in a similar manner:

- What is affected by the issue

- How the option addresses the issue

- The uncertainties associated with the analysis

- The assumptions made to deal with those uncertainties

- The degree of confidence in the conclusion of the analysis 32 32

Option Assessment

• The next slide shows how one team summarized its evaluation of options in an actual LIC-504 report (step 3 of our process)

• Purpose is not to read the slide, but to see one convenient way of organizing the assessment of options

• This approach might be useful for todays scenarios!

Source; LIC-504 report, DEGRADATION OF BAFFLE-FORMER BOLTS IN PRESSURIZED-WATER REACTORS DOCUMENTATION OF INTEGRATED RISK-INFORMED DECISIONMAKING PROCESS IN ACCORDANCE WITH NRR OFFICE INSTRUCTION LIC-504, ML16225A341 33

34 Step 4 - Integrate Assessment Results

• Summarize the results for each decision option

- The driving factors for the assessment of the option. Driving factors are those that play the most significant role in the decision (e.g.,

defense in depth, safety margin, risk, etc.).

- Key technical inputs. A key technical input is an essential input to the analysis that enables the conclusion of acceptability or non-acceptability

- Assessment of the validity and applicability of each technical input

- Assessment of the confidence in the assessment, recognizing the uncertainties in the technical inputs

• Determine the preferred option - the decision that the technical team is recommending to the decision authority 35 35

Option Integration

• The next 2 slides show how one team integrated its evaluation of options to arrive at a recommendation (step 4 of our process)

• Again, the purpose is not to read the slide, but to see one convenient way of comparing and contrasting the options

• This also might be useful for todays scenarios!

Source; LIC-504 report, DEGRADATION OF BAFFLE-FORMER BOLTS IN PRESSURIZED-WATER REACTORS DOCUMENTATION OF INTEGRATED RISK-INFORMED DECISIONMAKING PROCESS IN ACCORDANCE WITH NRR OFFICE INSTRUCTION LIC-504, ML16225A341 36

37 38 Step 5 - Communicate Assessment and Recommendation

• Purpose - provide decision maker the information needed to make properly informed decision

• Suggested content (may be formal or informal):

Background:

Sufficient information on the issue and the decision to be made

- Decision: State the decision that is required clearly and concisely

- Options: Present individually and concisely

• Present driving factors for accepting or rejecting each option, including uncertainties

• Document the logical basis for accepting/rejecting each option

- Recommendation: Summarize the logic for the recommended option

- Supporting Details: Provide qualitative insight into causes, uncertainties, assumptions, sensitivities and affected outcomes for a given situation

- Other relevant information: (e.g., generic implications, stakeholder 39 concerns) 39

HANDS-ON SCENARIOS Instructor Examples Student Teams

INSTRUCTOR EXAMPLES 41

NMSS Scenario Donnie Harrison Senior Level Advisor for Risk Assessment NMSS

NMSS Dry Cask Welding Scenario

1. Characterize the issue and risk Weld inspections/tests are supposed to be performed on each of 3 successive welds to ensure no cracks in weld as part of sealing canister

- NRC determined that some weld tests were performed improperly on one canister per National Codes and Standards. Subsequently, the licensee identified five additional canisters with similar improper weld tests

- All canisters have passed leak tests (i.e., though welds not tested properly, currently no leaks detected from canister).

The licensee submits a license amendment requesting exemptions from Codes and Standards.

43

NMSS Dry Cask Welding Scenario (Continued)

- What can go wrong?

The welds can crack and the canister could fail under design basis events, such as a seismic event.

- How likely is it?

Weld tests were not performed properly, but design basis events (e.g., seismic) are very unlikely.

- What are the consequences?

Confinement breach may occur, releasing radiation to the environment 44

NMSS Dry Cask Welding Scenario (Continued)

2. Define decision options Option 1:Deny the exemption request and require the licensee to repair the welds per Codes and Standards Option 2:Reexamine the top weld layer and perform additional leak testing Option 3:Approve exemption request with no additional weld testing requirement 45

NMSS Dry Cask Welding Scenario (Continued)

3. Assess each option - OPTION 1 Deny the exemption request and require the licensee to repair the welds per Codes and Standards

- Applicable regulations: 10 CFR 72.212(b)(3) and (11). Exemption request is per 10 CFR 72.7.

- Defense in depth: There are two barriers to prevent release: fuel cladding and canister.

- Safety margins: Passed Code-required leak test. Complies with Codes and Standards

- Risk assessment: Risk is well below the quantitative health limit based on staffs estimation. ALARA concerns because of potential for significant worker exposure

- Performance monitoring: Radiation level is monitored at the independent storage facility site boundary.

46

NMSS Dry Cask Welding Scenario (Continued)

3. Assess each option - OPTION 2 Reexamine the top weld layer and perform additional leak testing

- Applicable regulations: 10 CFR 72.212(b)(3) and (11).

- Defense in depth: There are two barriers to prevent release: fuel cladding and canister.

- Safety margins: Passed Code-required leak test. Complies with Codes and Standards

- Risk assessment: Risk is well below the quantitative health limit based on staffs estimation. ALARA concerns because of potential for significant worker exposures

- Performance monitoring: Radiation level is monitored at the independent storage facility site boundary.

47

NMSS Dry Cask Welding Scenario (Continued)

3. Assess each option - OPTION 3 Approve exemption request with no additional weld testing requirement

- Applicable regulations: 10 CFR 72.212(b)(3) and (11). Exemption request is per 10 CFR 72.7.

- Defense in depth: There are two barriers to prevent release: fuel cladding, and canister.

- Safety margins: Passed Code-required leak test. Does not comply with Codes and Standards

- Risk assessment: Risk is well below the quantitative health limit based on staffs estimation. No worker ALARA concerns.

- Performance monitoring: Radiation level is monitored at the independent storage facility site boundary.

48

NMSS Dry Cask Welding Scenario (Continued)

4. Integrate assessment of options and make recommendation

- Previous evaluations of seismic hazards indicate very low probability of seismic events that can topple the canister.

- Additional weld inspection or repair is an ALARA concern.

- Efficient use of resources to address low risk issue

- Compliance issue, but low risk significance 49

NMSS Dry Cask Welding Scenario (Continued)

5. Communicate assessment and recommendation

- Describe assessment of options

- Integrate risk insights with seismic hazards Recommendation: Risk is very small and there are two barriers. Exemption may be granted.

50

Region III Scenario Laura Kozak Senior Reactor Analyst Region III

Region III Inspection Scenario Scenario: You are an inspector at a nuclear power plant. Plant operators identified a failed indicator in the main control room for a pump and determined that it did not impact the function of the pump. Use a risk informed decision process to prioritize inspection of the issue.

Summary of the Issue

• Plant operators identify a failed speed indicator on main control board for a non-safety related turbine-driven pump.

• Plant operators conclude the pump itself is fully functional.

• Although non-safety related, the pump is considered to be risk-significant.

• The pump is operated about once a year and is tested every 90 days. The pump has not been tested in the last 30 days.

53

Background

• The pump is non-safety related but risk significant.

• The pump is normally in standby and operates during reactor transient events to provide core cooling and inventory control.

• The pump has a 14 day Technical Specification Allowed Outage Time (AOT).

• If the pump is non-functional for greater than the AOT due to a performance deficiency, it could be greater than green significance in the ROP.

54

Risk-informed decision-making Options:

• No follow-up inspection necessary.

• Perform an inspection in the near term to evaluate the operability/functionality of the pump

• Perform an inspection at a later date to follow-up on the corrective action taken for the speed indicator failure.

55

Risk-informed decision-making

1. Characterize the issue and risk

- What can go wrong?

- How likely is it?

- What are the consequences

2. Define Decision Options

3. Assess Each Option

4. Integrate Assessment of Options and make recommendation

5. Communicate assessment and recommendation 56

STUDENT TEAMS FOR HANDS-ON SCENARIOS 57

How can I apply Risk-Informed Thinking in NRC work?

Group Exercises - Scenarios:

1. NSIR, Joe Rivers

2. NRR, Steve Arndt

3. OCIO, Kathy Lyons-Burke 58

NSIR Scenario Joseph Rivers Senior Level Advisor for Security NSIR

Definitions

• Physical Protection System (PPS) an integrated system of equipment, personnel, and procedures intended to prevent the completion of a malicious act.

• Vulnerability Analysis (VA) A systematic, performance-based process that is used to evaluate the ability of a physical security system to meet performance requirements 60

Uses of Vulnerability Analysis (VA)

• Determine the relative effectiveness of alternative protective strategies and systems

• Ensure that systems provide balanced protection of assets

• Identify strengths and weaknesses in existing or planned protection systems

• Support cost-effectiveness studies for system design alternatives 61

Information Needs

• Site Data

- Site description

- Facility layouts and relationships

- Environment/climate information

- Current DBT

- Existing Vital Area Analysis

- Preliminary theft and/or sabotage target list

- Previous VAs, inspections, audit results

• Identify protection elements

- Detection

- Delay

- Response 62

Scenario: NSIR Scenario: The licensee has asserted that the security program at the nuclear power plant is more than adequate after constant upgrades over the last 15 years. They have submitted a security plan change that they claim will have marginal reduction in overall security effectiveness. How would you assess this security plan change?

- Team 1: Removal of protective force positions

- Team 2: Discontinuance of the use of side arms

- Team 3: Replacing owner controlled area (OCA) patrols with unmanned aerial vehicles (UAVs) 63

Scenario: NSIR (Continued)

Risk-informed decision-making:

1. Characterize the issue and risk

- What can go wrong?

- How likely is it?

- What are the consequences?

64

Scenario: NSIR (Continued)

2. Define decision options

3. Assess each option

- Applicable regulations

- Defense in depth

- Safety margins

- Risk assessment

- Performance monitoring

4. Integrate assessment of options and make recommendation

5. Communicate assessment and recommendation 65

NRR Scenario Steve Arndt Senior Level Advisor for Digital I&C NRR

Why is it important to consider risk insights?

• Resources expended by staff and industry on issues/reviews should be commensurate with their risk significance

• Focus efforts on risk significant issues/reviews

- Consider risk insights to enhance effectiveness and efficiency

• NRC provides reasonable assurance, not absolute assurance

- Leverage risk insights in reaching a reasonable assurance determination

• Collaboration of staff to make better decisions 67

NRR Scenario: NRC Regulatory Changes After Fukushima

• Commission directed a methodical and systematic review of the safety of U.S. facilities in light of events in Japan

• Created a task force to provide recommendations to enhance safety at US plants

• Near-Term Task Force report issued July 12, 2011 68

Near-Term Task Force Recommendations

• Re-evaluate/upgrade Seismic

• Regulatory framework for low and flooding protection probability, high consequence

• Strengthen mitigation against events prolonged loss of AC power

• Greater attention to defense-in-

• Improve containment venting depth for low probability events capabilities

• Seismically induced fires and

• Enhance spent fuel pool cooling floods and make-up capabilities

• Hydrogen control mitigation inside

• Integrate severe accident buildings procedures with EOPs, EDMGs,

• EP topics for multiunit events and

& EP activities prolonged SBO

• Strengthen emergency response

• EP topics on decision making, capabilities to address enhanced radiation monitoring, and public capabilities above education 69

Scenario: NRC Regulatory Changes After Fukushima

• Spent fuel pool cooling:

- At Fukushima the earthquake and tsunami resulted in a protracted SBO condition, no ac power was available and batteries were depleted

- No onsite capability to provide water inventory or cooling to the spent fuel pools

- Operators were significantly challenged to understand the condition of the spent fuel pools because of a lack of instrumentation or because of instrumentation that was not functioning properly

- Eventually, spent fuel cooling was provided by helicopter and pumper trucks to spray water from a distance in to the spent fuel pools 70

Scenario: Prioritization of Recommendations

• The Task Force recommends enhancing spent fuel pool makeup capability and instrumentation for the spent fuel pool.

- Order licensees to provide sufficient safety-related instrumentation, able to withstand design-basis natural phenomena

- Order licensees to provide safety-related ac electrical power for the spent fuel pool makeup system

- Order licensees to revise their technical specifications to address requirements to have one train of onsite emergency electrical power operable for spent fuel pool makeup and spent fuel pool instrumentation

- Order licensees to have an installed seismically qualified means to spray water into the spent fuel pools, including an easily accessible connection to supply the water at grade outside the building

- Initiate rulemaking or licensing activities or both to require the actions related to the spent fuel pool described in detailed recommendations 71

NRR Scenario: Prioritization of Recommendations

• Your team has been assigned to review the NTTF recommendations on spent fuel pool cooling and instrumentation and determine, how the recommendations should be implemented and what kind of requirements (if any) are appropriate (orders versus rulemaking or guidance, safety versus non-safety, etc.) (focus primarily on the question of does the solution need to be safety related?)

• Use the risk informed decision making process to brainstorm, develop and assess options and make a recommendation.

• Be prepared to support your recommendation.

72

Scenario: NRR (Continued)

Risk-informed decision-making:

1. Characterize the issue and risk

- What can go wrong?

- How likely is it?

- What are the consequences?

73

Scenario: NRR (Continued)

2. Define decision options

3. Assess each option

- Applicable regulations

- Defense in depth

- Safety margins

- Risk assessment

- Performance monitoring

4. Integrate assessment of options and make recommendation

5. Communicate assessment and recommendation 74

OCIO Scenario Kathy Lyons-Burke Senior Level Advisor for Information Security OCIO

Outsourced Project Scenario

• You are a project manager for an outsourced system/software contract.

• The contractor delivers systems and system changes that do not meet specified requirements, and there is pressure to place them into operations as quickly as possible.

• The issues identified are risk significant and would place the information and ability to perform the mission at risk.

• The contractor states that it will take at least 10 business days to correct the issues and another 5 business days to assess if the issues are resolved.

• The functionality provided by the system is needed in order to meet mission requirements.

• Mission offices indicate that the required functionality must be available in 5 business days due to an externally mandated requirement.

• Current software assessment processes reveal the same issues keep recurring.

• Use a risk informed decision process to identify and prioritize actions to be taken at this time.

76

Outsourced Project Scenario Summary of the Issue

• System assessors reveal significant cybersecurity and functionality issues during testing.

• There is a need to get the new/modified products into the operational environment as quickly as possible.

• Identified issues are considered to be risk-significant.

77

Outsourced Project Scenario Risk-informed decision-making Options:

1. Ignore the issues and place the system into operation.

2. Identify the most risk significant issues and work with the contractor to get those issues resolved in the 5 business day timeframe and then place the system into operation.

3. Place pressure on the contractor to correct the risk significant issues in a shorter timeframe and place the system into operation after the issues are corrected.

4. Place the system into operation after the 15 business days required to fully address all the risk significant issues.

78

Risk-informed decision-making

1. Characterize the issue and risk

- What can go wrong?

- How likely is it?

- What are the consequences

2. Define Decision Options

3. Assess Each Option

4. Integrate Assessment of Options and make recommendation

5. Communicate assessment and recommendation 79

RISK-INFORMED THINKING CERTIFICATE 80

How to complete Risk-Informed Thinking Certificate?

• Sign up for the certificate and follow instructions in iLearn

• Complete the following 3 learning activities:

- One PRA class (P-101, P-105, P-111, or P-400), or alternative (see next slide)

- This Risk-Informed Thinking Workshop

- Individual Study Activity -- must be the last learning activity (directions in iLearn)

• iLearn will record completed activities and issue certificate completion (no penalty if not continue) 81

How to complete Risk-Informed Thinking Certificate? (Continued)

Alternative to PRA class (your choice of one):

• Certified Contracting Officers Representative (COR)

• Federal Acquisition Certification for Program and Project Managers - Information Technology (FAC-P/PM-IT)

• Complete all the following online classes in iLearn:

- IT Project Management Essentials: Managing Risks in an IT Project

- ITIL 2011 Edition OSA: Introduction to Problem Management

- Nonstatistical Analysis Methods in Six Sigma

- Risk Response and Control (PMBOK Guide Fifth Edition)

- Leadership Advantage: Critical Thinking (Web-Based)

Note: Self-certify in iLearn if you are a COR or FAC-P/PM-IT 82

How to complete Risk-Informed Thinking Certificate? (Continued)

Individual Study Activity:

• Select one of your recent or current assignments or agree to one with your mentor (see Slide #3)

• Apply the 5-step risk-informed decision-making process

- Identify any potential effectiveness and efficiency improvement from thinking about risk

- Discuss study activity with your mentor

- Present assessment and recommendation to supervisor

- Complete summary sheet

• Self-certify in iLearn 83

PRA Certificate 3-Month PRA Rotation Risk-Informed Thinking Basic PRA Competency Certificate Independent Study Activity Introductory PRA Risk-Informed Class Thinking Workshop 84

85