ML21176A179
| ML21176A179 | |
| Person / Time | |
|---|---|
| Issue date: | 06/22/2021 |
| From: | NRC/SECY |
| To: | |
| Shared Package | |
| ML21050A177 | List: |
| References | |
| M210622B | |
| Download: ML21176A179 (120) | |
Text
1 UNITED STATES NUCLEAR REGULATORY COMMISSION
+ + + + +
MEETING ON TRANSFORMATION AT NRC:
MID-YEAR REVIEW
+ + + + +
- TUESDAY, JUNE 22, 2021
+ + + + +
The Commission met via Video Teleconference, at 9:00 a.m. EDT, Christopher T. Hanson, Chairman, presiding.
COMMISSION MEMBERS:
CHRISTOPHER T. HANSON, Chairman JEFF BARAN, Commissioner ANNIE CAPUTO, Commissioner DAVID A. WRIGHT, Commissioner ALSO PRESENT:
ANNETTE VIETTI-COOK, Secretary of the Commission MARIAN ZOBLER, General Counsel
2 NRC STAFF:
MARGARET DOANE, Executive Director for Operations NICHOLAS BUGGS, OCIO JUNE CAI, ADM ERIN CARFANG, Region I CANDACE de MESSIERES, NRR STEPHANIE MORROW, RES TAMMIE WILLIAMS, NRR
3 P R O C E E D I N G S 1
9:00 a.m.
2 CHAIRMAN HANSON: Good morning, everyone. I 3
convene the Commission's public meeting on NRC transformation. This is 4
the mid-year review. Our last meeting on the subject was in September of 5
2020.
6 We will hear today from two panels. First we'll hear from 7
the NRC staff, and following that, after a short break, we'll hear from 8
participants in our external panel. With each panel, we'll hold questions until 9
the end, and then we'll hear questions from the Commissioners.
10 Before I start, I'll ask if my colleagues have any remarks to 11 make. No? Okay, so with that, we'll begin with the NRC staff panel. We'll 12 begin with our Executive Director of Operations, Margie Doane.
13 MS.
DOANE:
Good
- morning, Chairman and 14 Commissioners. We look forward to providing you an update on the NRC's 15 transformation to become a modern, risk-informed regulator. As you will hear 16 today, we continue to make progress in all transformation focus areas.
17 Our mission to protect public health and safety, security, and 18 the environment has not changed. What is changing is how we go about 19 meeting this mission. Transformation is a whole Agency endeavor, and 20 though today we have success stories showing change across the Agency in 21 technical, corporate, policy, and legal programs, we know that there is more 22 to be done as this a journey, not a one-time event.
23 For background, I'll briefly discuss the highlights of our 24
4 transformation journey and how we got here. Although I'm sure that this is 1
sounding familiar, it will be helpful to set the stage for today's briefing.
2 As we discussed in our last meeting in September 2020, the 3
NRC built upon its 2019 futures assessment to develop the dynamic futures 4
for NRC mission areas report, which describes four hypothetical future 5
scenarios in which the NRC might operate in the year 2030 and beyond.
6 Using the futures report and information we gathered from 7
various efforts like the NRC Futures Jam, we identified four focus areas for 8
transformation: our people, use of technology, Be RiskSMART, and 9
innovation. These form the basis for a transformation journey roadmap.
10 This roadmap first identified and prioritized eight initiatives, 11 all of which are complete, except for our ongoing efforts to achieve the 12 Agency's desired culture. That culture initiative is focused on ensuring that 13 behavioral norms and sustainable processes are incorporated into how we do 14 business. You'll hear more about this later in the presentation.
15 The goal for 2020 was largely about developing the tools 16 and resources that would fuel our transformation. Each of the eight initiative 17 teams worked throughout the first year to familiarize the staff with their topics 18 and work collectively to identify ways to modernize.
19 Each team developed tools or products in pursuit of their 20 desired outcomes or goals and supported individual offices and regions in 21 developing and implementing their own transformation efforts. This work has 22 now put in place new infrastructure and tools for the staff to use to continue 23 the change journey, such as the Be RiskSMART framework.
24
5 We also have more than 200 transformation champions and 1
ambassadors, many of whom were previously on the initiative teams. They 2
regularly meet to discuss the specific initiatives they support, challenges they 3
face, and share the lessons learned that can help make progress across the 4
entire Agency.
5 Having built the infrastructure, our efforts in 2021 are 6
focused on sustaining the change that we have seen in the first year by 7
seeking ways to encourage widespread application of these new tools, 8
approaches, and resources across the Agency.
9 Our goal of transformation has always been on enhancing 10 our ability to solve the challenges of today and tomorrow, including those that 11 may not even be on our radar yet. And assessing risk is at the center of this 12 effort.
13 As a safety regulator, NRC has had a longstanding risk-14 averse culture in our approaches to meeting our mission. While this has 15 served us well in many areas, it has also led to long stretches with little or no 16 innovation and approaches that did not take advantage of modern technology 17 or other resources.
18 Through our transformation efforts, we've been working to 19 have a broader perspective of risk and have more consistent application and 20 management of risk across disciplines. For example, we're trying to be more 21 proactive in identifying how certain well-managed risk can lead to better 22 decisions that consider the most current information. These efforts will be 23 highlighted in our presentation today.
24
6 The focus of this meeting is on these newly-developed risk 1
tools and other resources that were developed in 2020, and how the Agency 2
is making progress to sustain these efforts at both the Agency and staff levels.
3 Next slide, please.
4 Okay, so today we'll hear from Stephanie Morrow from the 5
Office of Nuclear Regulatory Research, who will provide an update on the 6
Agency's desired culture efforts. Candace de Messieres from the Office of 7
Nuclear Reactor Regulation, who will discuss the Be riskSMART framework 8
in decisionmaking.
9 And we will also two mini perspectives regarding the Be 10 riskSMART framework. First, Erin Carfang from Region I will discuss the 11 reactor materials regional perspective. And then Tammie Williams from the 12 Office of Administration will present on the Be riskSMART framework from the 13 corporate perspective.
14 Nicholas, or Nick, Buggs from the Office of the Chief 15 Information Officer, OCIO, will provide an overview of the NRC's enterprise IT 16 roadmap and data strategy. And finally, June Cai from the Office of Nuclear 17 Materials Safety and Safeguards will discuss the transformation of the 18 rulemaking process, including the application of agile project management 19 concepts.
20 But before I turn it over to my fellow presenters, I wanted to 21 highlight a few significant achievements since our last meeting with you on 22 transformation. Notably, the COVID-19 pandemic has underscored the 23 Agency's transformation efforts and galvanized the already increasing use of 24
7 technology in our day-to-day work efforts.
1 We transitioned our communication platform from Skype to 2
Teams, that we adopted midstream during the pandemic. We have 3
dashboards proliferating throughout the Agency showing data in a more 4
understandable and useful way. Our volume of easily accessible data has 5
increased by millions of documents that have been digitized from microfiche, 6
and we are actively breaking down silos of information to enhance 7
retrievability.
8 We are currently developing a technology road map that will 9
ensure access and use of data continues to be enhanced. We have worked 10 almost entirely paperless during the pandemic. These and other 11 transformation efforts enable the NRC to continue to meet its mission in this 12 very dynamic environment. We gained significant experience that will carry 13 forward in our lessons learned.
14 I cannot say enough positive things about the NRC staff's 15 efforts during the pandemic to meet our mission and to embrace positive 16 change in the ways that we work. Many of these were sparked by the 17 transformation effort. This has not been easy, and it underscores our staff's 18 dedication and commitment to making the NRC a modern regulator.
19 It bears repeating that we have more work to do as we 20 cannot say that we are transformed. So with this framework established, 21 we're focusing on what's next and how to keep this momentum going.
22 This is especially challenging considering that our focus 23 areas for the Agency are making sure that we continue to work well during the 24
8 pandemic and that our eventual full reentry into our building is supportive of 1
our staff who are key to meeting our mission efforts. We are keeping all of 2
this in mind as we continue to make transformation changes.
3 One thing that we are doing to keep our momentum for 4
change going is transitioning our transformation efforts from the Office of the 5
EDO to the offices themselves. These next steps are designed to help 6
empower not only the offices collectively, but each and every one of the NRC 7
staff to make innovations on their own and adopt innovations by others.
8 One example we mentioned during our last meeting got its 9
fueling innovation at the staff level is IdeaScale that was launched by Innovate 10 NRC 2.0 Initiative Team. It enables the NRC staff to share successful 11 innovations and crowd source solutions.
12 Last week, we celebrated the first year of the platform, which 13 has been visited by over 80% of our staff. Using IdeaScale, we've achieved 14 the completion of 17 challenge campaigns and captured over 480 success 15 stories.
16 To hold ourselves accountable for sustaining change 17 throughout the Agency, our offices have also developed objectives and key 18 results, or OKRs, for the 2021 calendar year.
19 One aspect to highlight is that this year, one of our key 20 results is to achieve positive feedback through a first-time survey we will be 21 asking of our external stakeholders about whether they perceive any changes 22 in our mission execution as a result of our transformation efforts.
23 This concludes my introduction. I will now turn the 24
9 presentation over to Stephanie for her update on the Agency's desired culture 1
effort. Next slide, please.
2 MS. MORROW: Thank you, Margie. Good morning, 3
Chairman and Commissioners. It's a pleasure to be speaking with you and 4
providing this update today. I'm Stephanie Morrow, a Human Factors 5
Engineer in the Office of Nuclear Regulatory Research and a member of the 6
Agency culture team.
7 Today I'd like to give you an update on our efforts to 8
intentionally manage our culture as part of focusing on our people. I'll 9
highlight some recent accomplishments, how we're measuring progress in 10 closing our culture gaps, things we've learned over the past year, and how we 11 plan to maintain our momentum. Next slide, please.
12 First, culture is important because it sets the tone for how 13 we interact with each other, how we make decisions, and how we innovate 14 and collaborate with one another. Our culture drives how we accomplish our 15 public health, safety, and security mission.
16 And even as we've experienced significant changes in our 17 work environment over the past year, our culture helped to ground us and 18 keep us focused on our mission. If anything, our recent experiences have 19 given us even more of an opportunity to reflect on what behaviors and norms 20 help us be successful. Next slide, please.
21 Over the past year, we've accomplished a lot, and not just 22 the Culture Team, but our entire agency as we've continued forward with our 23 transformation focus areas and found new ways to work together. This slide 24
10 includes just a few accomplishments that reflect the shifts we're making in our 1
culture.
2 We've engaged staff in culture change. One example is 3
growing our change agent network to over 200 people from across our 4
different offices and regions. We've increased recognition. We've done this 5
in informal ways, like taking time to acknowledge people in the moment, but 6
also more formal ways. The Innovate NRC Team incorporated a new 7
innovation awards category into our formal awards program.
8 We've enhanced communications. Last year our EDO 9
hosted a first-even virtual town hall meeting with all staff, and we committed 10 to continuing this practice on a quarterly basis as part of our culture 11 improvement plans. Over 1600 staff attended each of the two quarterly 12 meetings we've held so far this year.
13 And we shared good practices. We've used a variety of 14 forums, including our change agent network, meetings with individual offices 15 and regions, and Agency leadership meetings to intentionally create 16 opportunities for shared learning, exchange ideas on what's working, and 17 address common challenges.
18 We've also been documenting successes in our innovation 19 platform, IdeaScale. There are currently over 60 culture success stories in 20 IdeaScale, and that number continues to grow. Next slide, please.
21 As we make changes and implement new things, we're 22 using surveys and other assessment tools to measure whether we see 23 corresponding changes in what our staff experience. Last year, the culture 24
11 team took a holistic approach to assessing our culture, which included 1
integrating data from different surveys, interviews, and focus groups. That 2
was our 2020 baseline culture assessment.
3 This baseline allowed us to measure where we were starting 4
from. At that time, we also defined what our ideal culture would look like so 5
we would know where we wanted to go. We know that culture change takes 6
time, so we plan to do a full assessment again in 2022, two years after our 7
baseline.
8 For this year, we conducted an abbreviated survey as a 9
pulse check on our culture. We also have the results from the 2020 Federal 10 Employee Viewpoint Survey. The FEVS is one of the data sources we used 11 as part of our culture assessment process.
12 And both the recent Culture Pulse Survey and the 2020 13 FEVS provide indications of whether we are moving in the right direction with 14 our culture and climate. Next slide, please.
15 So I'll start with the Federal Employee Viewpoint Survey.
16 NRC staff took the 2020 FEVS in September, about one year after the launch 17 of our transformation initiative, and six months after our baseline culture 18 assessment. Our FEVS results show a four percent increase in the 19 employee engagement index, which is a key result we track as part of the 20 Agency objective and key results, or OKRs.
21 This is the highest our engagement index has been since 22 2011. Moreover, the FEVS questions that increased the most touch on areas 23 that have been a focus for our transformation efforts. Staff have seen 24
12 improvement in recognition, a 14% increase in the item, differences in 1
performance are recognized in a meaningful way.
2 And the next highest item, with a 12% increase, employees 3
are recognized for providing high quality products and services. Leadership, 4
an 11% increase in senior leaders generate high levels of motivation and 5
commitment.
6 Innovation, an eight percent increase in; I feel encouraged 7
to come up with new and better ways of doing things. And communication, a 8
seven percent increase in satisfaction with the information received from 9
management. Next slide, please.
10 The Culture Pulse Survey was open to employees in April 11 of this year, just over one year after our baseline assessment. The results 12 show slight increases in constructive behaviors. People feel like there's more 13 of an expectation to encourage others, help others to grow and develop, and 14 give positive rewards to others.
15 We also saw significant decreases in defensive behaviors.
16 People feel like there's less of an expectation to look for mistakes, remain 17 aloof, or do things for the approval of others, behaviors that can be detrimental 18 when practices in excess.
19 Finally, we saw improvements in measures of employee 20 involvement, communication, and perceptions of the agencys ability to adapt 21 to change, consistent with what we saw in the FEVS. Collectively, these 22 results tell us we are heading in the right direction toward closing our culture 23 gaps, but we still have room to grow and improve. Next slide, please.
24
13 Part of measuring our progress on a regular basis is learning 1
and adapting as we go. Weve learned that we need leadership at all levels.
2 The constructive behaviors we want to see in our ideal culture have direct ties 3
to our NRC Leadership Model, and they apply to everyone, not just those in 4
formal leadership roles - we can all encourage others and help others grow 5
and develop.
6 Our Culture Pulse Survey results show that managers and 7
senior executives experience our culture as much more constructive than 8
those at other levels. So s we move forward, we want to place even more 9
emphasis on role-modeling constructive behaviors so that our first-line 10 supervisors and non-supervisory staff also experience these behaviors as 11 valued and expected in our culture.
12 We've learned that we need to be very intentional with 13 connecting the dots. More communication does not have to be information 14 overload. And we need to be clear about how our activities fit together and 15 work to support accomplishing our mission so they don't feel like flavors of the 16 month or add to change fatigue.
17 We've also learned that it's important to actively create safe 18 spaces to bring our whole selves to work, fostering psychological safety.
19 Whether it's taking a moment in a team meeting to talk about what's going on 20 in our lives, or a larger forums to share perspectives on what's going on in the 21 world around us, like issues of racism and social justice.
22 Creating the psychological safety to speak up helps us to 23 make new connections with each other, more freely share information, and 24
14 ultimately makes us more effective at working together to accomplish our 1
mission. Next slide, please.
2 As we continue this effort, some of the ways we plan to 3
sustain our momentum and embed our desired culture into our processes and 4
practices include aligning our employee development programs and plans 5
with the NRC leadership model to address our culture gaps and develop 6
constructive behaviors, skills, and habits. Role model constructive behavior 7
with leaders and in teams to reinforce the behaviors we want to see in our 8
desired culture.
9 We're starting by facilitating conversations with senior 10 leadership teams in each office and region, and we plan to expand this 11 approach to first-line supervisors and non-supervisory staff.
12 We'll also include our NRC leadership model behaviors in 13 performance plans, starting with our Senior Executive Service and Senior 14 Level Service members, to make behavior expectations clear and ensure 15 accountability. Next slide, please.
16 Finally, I'd like to thank and acknowledge the rest of our 17 culture team. Patrice Reid, our team lead. Our full-time rotational staff, who 18 have taken on the heavy load of implementing our Agency culture plans. My 19 fellow part-time support who I've been working with on transformation since 20 the Futures Jam in 2019. And our senior executive sponsors, who continue 21 to provide their time and enthusiasm in championing this effort.
22 I also include our 200-plus member change agent network 23 as part of this culture team. Their grassroots involvement is critical to 24
15 reaching all parts of our agency.
1 Ultimately, our goal as the culture team is to facilitate 2
change, start conversations, and help apply what we learn so we can 3
consistently reinforce norms and expectations that will help us be a modern, 4
risk-informed regulator.
5 Thank you for your time. I will now turn it over to Candace 6
de Messieres to talk about Be RiskSMART.
7 MS. de MESSIERES: Thank you, Stephanie, and good 8
morning, Chairman and Commissioners. My name is Candace de 9
Messieres, a Technical Assistant for the Office of Nuclear Reactor Regulation.
10 It's an honor to be here today to share how the NRC staff can use the Be 11 RiskSMART framework in all decisionmaking.
12 As you know, the NRC has a long history and firm 13 foundation in risk-informed decisionmaking that includes many successful 14 applications of the risk triplet. That is, the set of three questions that NRC 15 uses to define risk: What can go wrong, how likely is it, and what are the 16 consequences.
17 While we have continuously improved how we use risk 18 insights to make better-informed decisions to meet our important safety and 19 security mission, risk-informed decisionmaking was not consistently applied 20 across the Agency or used for all types of decisions. And that's why I'm 21 thrilled to be here to discuss Be riskSMART. But before I do, I want to 22 highlight that while being risk-informed can provide for efficiencies and help 23 identify unnecessary conservatisms, it more importantly leads to a focusing of 24
16 attention on those issues of greatest importance.
1 This includes increased regulatory attention and oversight 2
on those items that matter for safety and security. We are making great 3
strides in fostering an Agency culture that recognizes the benefits that risk-4 informed decisionmaking can afford, as evidenced in the recent survey results 5
that Stephanie mentioned that indicate nearly 70% of staff thinks risk 6
information is important to accomplish our mission. Next slide, please.
7 The Be riskSMART risk-informed decisionmaking 8
framework was developed to provide staff confidence to consistently apply 9
and communicate risk insights for all NRC decisions without compromising 10 our mission. It is applicable in all NRC programmatic areas, including 11 corporate, technical, and legal support, and is written in plain language.
12 That is, the framework provides a common vocabulary that 13 can help users break down a problem and make sound decisions based on 14 the best available information, regardless of the application. The framework 15 helps remove the stigma from risk, the term risk, and replaces it with an 16 understanding of the value added by considering risk information, and how a 17 systematic approach can help ensure NRC is focused on items of greatest 18 importance.
19 Importantly, the framework does not revise or change 20 existing requirements or criteria, such as those supporting oversight or 21 licensing decisions. It also does not invent new concepts, but rather serves 22 as an umbrella to increase consistency, awareness, and usability of existing 23 risk-informed approaches.
24
17 Finally, the framework recognizes the multi-faceted nature 1
of risk and helps staff consider those multi-faceted risks systematically and in 2
a transparent manner, whether those risks are technical, programmatic, 3
reputational, information technology, human capital, or others. Next slide, 4
please.
5 This slide summarizes the key steps of the Be riskSMART 6
Framework. The first step is be clear about the problem. The problem could 7
range from a simple binary question that one faces on a daily basis, to a more 8
complex decision involving multiple individuals or entire organizations, such 9
as how to enhance a process to more fully realize the NRC's principles of good 10 regulation.
11 The next step is Spot. Spotting involves evaluation of the 12 risk triplet. But this step also explicitly includes consideration of both the 13 challenges and opportunities associated with a decision. That is, what can 14 go wrong or right. This step can involve both quantitative or qualitative 15 considerations.
16 Once spotting is completed, then you manage what you can.
17 That is, you consider strategies to reduce the likelihood or negative 18 consequences or enhance what can go right. These strategies could take 19 many forms, from increased communications or training, to additional 20 inspections or review resources.
21 The Act on the Decision step includes considering all 22 stakeholder perspectives and evaluating what you spotted and managed in 23 the context of the risk appetite for the decision.
24
18 In plain language, risk appetite or risk tolerance is simply the 1
amount of risk one is willing to accept. Depending on the decision, it can be 2
defined by an individual, a group of individuals, or an entire organization. And 3
it can be different for different aspects of the decision.
4 While simple decisions may not require extensive 5
documentation, the framework does provide a tool from enterprise risk 6
management called a heat map to help decisionmakers visualize the 7
challenges and opportunities, the effects of management techniques, and the 8
risk appetite associated with the decision. It is essential for all stakeholders 9
to be transparent about the risk appetite. A populated heat map can support 10 discussions about different perspectives and lead to an optimal decision.
11 The framework doesn't stop at the action. The Realize step 12 involves implementing the decision while managing what you can and 13 measuring your performance and progress. Realizing could involve 14 effectiveness reviews and other similar assessment tools to measure results 15 that can then be used as part of the continuous performance management 16 strategy to adjust and fine-tune.
17 The Teach step involves sharing knowledge to a broader 18 audience, enabling staff to apply best practices in new contexts to address 19 novel problems.
20 Finally, as a continual learning organization, the arrow 21 reflects the iterative nature of the framework, encouraging users to revisit any 22 of the steps at any time, or to even formulate a new problem. Next slide, 23 please.
24
19 Now I will shift to discuss how the Be riskSMART initiative 1
team is using a diagnostic tool to assess how the framework is inculcated in 2
our agency's culture. This slide summarizes the tool's features. The vertical 3
and horizontal axes of the graph depict the individual contributor and manager 4
roles in driving risk-informed decisionmaking.
5 In an ideal state, managers have established the 6
appropriate infrastructure and culture and play active and key roles.
7 However, risk informing starts earlier in the decision process and occurs 8
organically as part of everyday operations. The ideal state considers 9
everyone's opinion, individual contributors, supervisor, every layer of 10 management, and external stakeholders.
11 The importance of achieving the ideal state cannot be 12 overstated. This state fosters receptivity to new ideas and thinking, promotes 13 participative decisionmaking, transparency and trust, as well as encourages 14 diverse views.
15 Data collected by the Be riskSMART Initiative Team in 2020 16 reflected a majority of NRC staff see their organizational unit falling into the 17 center box, reflecting good progress. However, additional work is needed to 18 fully empower individuals.
19 Notably, the tool also includes specific strategies for an 20 organization to achieve the ideal state, such as enhanced communication, 21 training, and practices that engender trust.
22 I should also mention this tool is not the only way the Be 23 riskSMART team is tracking progress, as questions on the staff use and 24
20 attitudes towards risk-informed decisionmaking are included in several office 1
and Agency-wide level surveys, such as the surveys that Stephanie 2
mentioned.
3 While additional work is needed, the Be riskSMART initiative 4
is helping the Agency become a more modern risk-informed regulator, as 5
evidenced by the 2021 Culture Pulse Survey results I mentioned earlier, 6
showing that nearly 70% of staff thinks risk information is important to 7
accomplish our mission. Next slide, please.
8 I would now like to briefly reiterate that the framework 9
envelops the entire spectrum of NRC activities. Specifically, the team 10 deliberately evaluated the mapping of specific guidance or processes in all 11 areas, such as material safety, as highlighted on the top left of the slide, as 12 well as in corporate support, research, reactor safety, legal support, materials 13 safety, and security, emergency preparedness, and response.
14 As you can see, the framework is compatible with and can 15 enhance the many effective existing approaches and has broad applicability 16 across business lines. A few early examples are highlighted here for 17 illustration, and you will hear about a few more shortly from Erin and Tammie.
18 Next slide, please.
19 Being a risk-informed regulator is a journey. While it takes 20 a great deal of effort from all staff and unwavering support from senior 21 leadership, it is worth it. And Be riskSMART is there to meet staff wherever 22 they are.
23 A few items that demonstrate the team's progress are 24
21 highlighted here, including the issuance of a new knowledge management 1
NUREG, the launch of new NRC staff training, and the advancement of the 2
framework across the Agency through objectives and key results, or OKRs.
3 The NUREG provides a one-stop resource for using the 4
framework, including the case studies, mappings of discipline-specific 5
processes and guidance, and initial diagnostic tool and survey data. The new 6
training enables staff to explore how the framework can be used both 7
generally and to make discipline-specific decisions.
8 The course consists of three modules, including a general 9
introductory training video, a choose-your-own-adventure scenario section, 10 and a knowledge check.
11 Finally, aggressive goals have been established through the 12 OKRs, including that 85% of staff complete the training and that the framework 13 is then used in at least 50% of key decisions by the end of calendar year 2021.
14 In addition, OKRs encourage the featuring of the Be riskSMART application in 15 the staff IdeaScale success gallery platform.
16 I'm pleased to report that while we're still in the early stages, 17 offices are making great progress toward meeting OKRs, including that nearly 18 20% of staff have now completed the Be riskSMART training. Next slide, 19 please.
20 Lastly, I would like to thank the entire Be riskSMART 21 Initiative Team, ambassadors, and community of practice. It has been a 22 pleasure working over the past two years with such an amazing and diverse 23 team of talented and dedicated individuals empowered by our team lead, 24
22 Mirela Gavrilas, and our several executive sponsors.
1 Thank you for your attention, and with that, I'll turn it over to 2
Erin.
3 MS. CARFANG: Thank you, Candace. Good morning, 4
Chairman and Commissioners. My name is Erin Carfang, and I'm a Branch 5
Chief in Region I, with recent experience at both the reactors and materials 6
programs.
7 I have two examples where Region I had an opportunity to 8
demonstrate the concepts captured in the Be riskSMART framework Candace 9
presented. Next slide, please.
10 In the reactor program, there are many priorities inspectors 11 need to balance. One site in Region I recently experienced several safety-12 related breakers failing to actuate as expected over a four-month time period, 13 and licensee actions were slow to address the issue.
14 This resulted in the question of should additional resources 15 be assigned to reviewing the breaker failures, given the actions that licensees 16 had already taken to address the failures. Using this model, we spotted that 17 the licensee had followed the established maintenance practices and took 18 these issues into the corrective action program and identified the cause of the 19 failures.
20 The Resident Inspectors were concerned that the site's 21 planned actions were not timely and did not identify the correct extent of 22 condition. Additional electrical expertise was requested by the Resident, and 23 this request coincided with high demand for inspection resources.
24
23 The regional staff and management worked together to 1
align on the best path forward, reviewing inspection resources, adequacy of 2
licensee action, and concerns with equipment performance. We agreed that 3
the breaker inspection deserved priority.
4 And as a result of the extra inspection efforts, the licensee 5
agreed to pull forward the scheduled maintenance, schedule these tasks for 6
the high priority susceptible breakers this summer.
7 Additionally, a review of the other preventive maintenance 8
strategies and the procedures is being performed. Teaching is one of the 9
most important steps in this process. The Residents have presented to the 10 technical staff and management on the issue, ensuring that other inspectors 11 are aware of the potential failure when conducting inspections. Next slide, 12 please.
13 A gamma sterilization irradiator licensee requested 14 authorization to replaces sources in its panoramic irradiator. This activity is 15 typically performed by the source manufacturer and involves bringing in casks, 16 lowering the casks into the pool, and loading source racks with the source 17 pencil under water.
18 The irradiator serves a critical function because it used to 19 sterilize medical supplies. The licensee was initially prompted to submit a 20 license amendment request due to COVID-19 travel restrictions. However, 21 they now feel they need this authorization because they anticipate reduced 22 source replacement support from the source manufacturer in the future.
23 This brings us to the clear question should the region 24
24 approve the licensing action. So we evaluated the risk triplet and considered 1
the benefit of taking the action. The benefit of the action is improved reliability 2
for resourcing in the future for the licensee. While the challenge is potentially 3
improperly installed sources that could be brought to the surface.
4 The potential consequences include the benefit of ensuring 5
critical medical supplies remain available, while the challenge could result in 6
an extremely high dose rate to site personnel if the source is exposed on the 7
surface of the pool.
8 In managing the risk, the licensing staff considered the 9
likelihood of consequences is highly dependent on the training provided to the 10 licensee staff to conduct the activity safely, which is firmly within the licensee's 11 control.
12 The regional staff worked collaboratively with Headquarters 13 to ensure other views outside of the region were considered in the 14 decisionmaking. In consultation with NMSS and OGC, Region I acted by 15 issuing the license amendment in May of 2021, allowing the licensee to 16 perform the action.
17 The results of this licensing action include the inspection of 18 the source racks, given recent operating experience, as well as the inspection 19 being a high priority observation item for the regional staff while providing 20 operational stability for the licensee.
21 Knowledge transfer sessions are planned on this topic 22 within the region and the activity slated for inclusion in the Region I monthly 23 newsletter put out by our regional risk-informed decisionmaking working 24
25 group.
1 Thank you for your time. I will now turn it over to Tammie 2
Williams to talk about a corporate perspective using the Be riskSMART 3
framework.
4 MS. WILLIAMS: Thank you, Erin. Good morning, 5
Chairman and Commissioners, my name is Tammie Williams. I'm a Senior 6
Space Management Specialist in the Office of Administration, and I'm here 7
today to talk about how the Be riskSMART framework can be used broadly in 8
different ways beyond the Program Office to include a corporate support 9
perspective.
10 To help with that, I'll walk you through an overview of a 11 corporate example we have in our training materials on how Be riskSMART 12 can be used. We took the implementation of the Two White Flint restack 13 project, which is an example of how the decisionmaking process can benefit 14 from accepting well-managed risk, and assessed it using the new risk model.
15 Next slide, please.
16 NRC committed to releasing one floor of the Two White Flint 17 building to the General Services Administration, or GSA, by the end of the 18 fiscal year 2020. This was in support of federal-wide initiatives to reduce 19 office space across the country.
20 The restack process impacted several offices, Research, 21 OCFO, NSIR, Admin, OCIO, and multiple floors across the building. Early in 22 the restack process, the COVID-19 public health emergency occurred, and we 23 were challenged with finishing the project in a compressed timeframe and to 24
26 adapt many aspects of the project to a virtual environment for the first time.
1 The team needed to determine if there were any parts of the 2
process that could be expedited, delayed, or avoided without imposing 3
unacceptable risk. The team spotted where things could go right or wrong, 4
the consequences, and the likelihood of occurrence. For example, if 5
elements of the planning and office selection phases were missed, then 6
collective bargaining agreement, CBA, commitments with the union may be 7
impacted.
8 Experience showed a high likelihood for non-compliance if 9
these earlier phases were expedited, affecting the subsequent phases and 10 ultimately resulting in an overall schedule delay. Unbudgeted rents would 11 incur, and NRC would experience a reputational impact with the GSA.
12 The risks outweighed the potential benefit of saving time 13 during these early phases. Instead, the risks inherent in the physical aspects 14 of the move were managed by utilizing experienced movers and technicians 15 and relying on staff. The risk of conducting some activity virtually for the first 16 time was managed by increasing communication and coordination with the 17 office point of contact.
18 One of the actions taken to expedite the process was 19 conducting the move incrementally as the office selections were still being 20 completed. Additionally, the turnover process including a tour of the space 21 was conducted virtually with GSA.
22 The result what was realized was the majority of the moves 23 were completed by the end of the fiscal year. Decommissioning activities 24
27 took place in the October timeframe, and turnover of the space to GSA 1
occurred in November.
2 The timely completion of the project allowed the Agency to 3
release one floor to GSA, enhanced NRC's reputation for fulfilling 4
requirements on time, and reduced NRC's rent and real property-related 5
costs.
6 To teach what we've learned, this example is included as a 7
case study in the NUREG that Candace had mentioned earlier. We're also 8
implementing lessons learned in other projects. Next slide, please.
9 Looking back at the assessment exercise, it helped us to 10 realize that the Be riskSMART framework can be applied anywhere, and that 11 many of the existing processes fit within the framework or enhanced by it.
12 It showed us that staff at a working level in corporate support 13 are already organically assessing and managing risks in their daily activities, 14 with the most successful aspects of the project being those that were staff-15 driven, which is consistent with the diagnostic ideal state that Candace had 16 described earlier.
17 Additionally, we realized the importance and the key 18 interdependence on other NRC initiatives for the success of Be riskSMART, 19 including the Agency desired culture initiative and the associated tools, such 20 as speed of trust and the NRC leadership behaviors, that really helped move 21 us to the ideal state.
22 Thank you very much for your time, and I will now turn it over 23 to Nicholas Buggs to talk about the NRC enterprise IT roadmap and data 24
28 strategy.
1 MR. BUGGS: Why, thank you, Tammie. Good morning, 2
Chairman and Commissioners, my name is Nicholas Buggs and I'm a Senior 3
Enterprise Architect in the Office of the Chief Information Officer, or OCIO.
4 I would like to thank you for the opportunity to speak with 5
you today about the IT roadmap and the data strategy. Next slide, please.
6 As exemplified in the previous topics, the Agency continues 7
its transformation journey towards becoming a more modern risk-informed 8
regulator. Throughout this journey, the role of strategic investments in IT 9
modernization as mission enablers has become more apparent than ever.
10 Process changes, culture shifts, workforce enhancements, 11 and a greater reliance on collaborative tools in a mixed working environment 12 all require changes to the IT architecture to enable this transformative change.
13 Next slide, please.
14 The desire for change inspired by innovative thinking, a 15 culture of transformation, must be balanced by strategic thinking and 16 collaborative planning to ensure that the Agency is able to meet its goals in 17 the most efficient and effective way possible.
18 This balance can only be achieved when Agency leadership 19 is aligned on its priorities and promotes the implementation of new initiatives 20 that provide the greatest value to the Agency. Next slide, please.
21 To this end, the Office of the Chief Information Officer, in 22 consult with subject matter experts and leadership across the Agency, have 23 been working collaboratively to develop an IT roadmap. An IT roadmap is a 24
29 strategic planning tool that aids in the identification and selection of key 1
strategies and initiatives to support the continual operation and improvement 2
of the Agency's IT portfolio.
3 This approach enables the Agency to more effectively select 4
and implement the appropriate set of IT investments to maintain operational 5
readiness and to meet evolving Agency mission needs.
6 Initial development of the IT roadmap took a three-phased 7
approach. The first phase was designed to articulate the current state 8
architecture of the Agency. This involved outreach to subject matter experts 9
in every office and region to identify the key processes, stakeholders, and 10 systems involved in meeting functional and organizational mission objectives.
11 Through exploration of the current state, we developed a set 12 of observations that suggested opportunities for enterprise-level improvement.
13 The second phase was designed to articulate the future 14 state architecture of the Agency by validating the opportunities for 15 improvement and envisioning the desired capabilities to close operational 16 gaps and enhance operational effectiveness. With the current state 17 articulated where we are, the future state defines where we want to be.
18 The final phase includes the development of key strategies 19 and initiatives that are needed to achieve the desired future state over the next 20 three to five years. This phase of the effort is still ongoing as we seek to 21 consolidate strategic input and the set of projects needed to meet our strategic 22 objectives. Next slide, please.
23 Some of the key business drivers that have been identified 24
30 throughout this effort include optimized Agency business processes and 1
modernized workflow and workload management solutions to support greater 2
process efficiency, improved visibility into staff workloads, and more effective 3
resource allocation. Improved staff ability to access and analyze system 4
data, as well as documents within Agency repositories, as a means of 5
promoting the staff's ability to leverage the Agency's collective body of 6
knowledge.
7 Expanded use of modern IT solutions to collect and 8
communicate information from and to external stakeholders to enhance the 9
automated use of data for decisionmaking and downstream analysis.
10 And lastly, optimized workspace design to support a hybrid 11 working environment to accommodate full and part-time on premise and 12 remote work. Next slide, please.
13 Once complete, the IT roadmap will serve as a strategic 14 planning tool in the selection of strategic investments year after year. It will 15 help to ensure that fact-of-life changes, emergent requests, and risks and 16 issues are considered with respect to existing strategies, while providing 17 opportunities to consider strategic course changes when needed. As such, 18 the IT roadmap will enable agility while providing a common strategic thread 19 to support decisionmaking. Next slide, please.
20 One significant input into the continued evolution of the IT 21 roadmap among many is the Agency data strategy. The data strategy will 22 describe the Agency's overarching goals for the enterprise management and 23 use of data and the key strategies needed to achieve them.
24
31 These strategies are used as input to the ever-evolving IT 1
roadmap by articulating the strategic initiatives needed to maximize the NRC's 2
ability to use data as a strategic asset. The data strategy will consider our 3
workforce, our processes, our use of technology, and of course our 4
stakeholders.
5 It will articulate the Agency's approach to meeting its priority 6
objectives through initiatives designed to enhance our data culture, which 7
represents our collective desire to promote the use of data, inform the 8
evolution of our business processes, leverage modern capabilities, and 9
improve our ability to collect actionable information from external 10 stakeholders. Next slide, please.
11 The first step in the development of the data strategy was 12 the definition of the set of strategic objectives or goals that the Agency is 13 seeking to achieve. These goals include promote the secure management 14 and control of Agency data, promote the collection, transformation, and use of 15 data. Promote the continual data management and maturity of Agency staff 16 and processes, and promote the use of data-driven insights.
17 Pursuit of these goals will enable the Agency to continue to 18 advance in the areas of data architecture and governance, data management, 19 data maturity, and data analytics. Next slide, please.
20 With these goals in hand, the staff will work with 21 stakeholders across the Agency to gather information and to perform 22 assessments across a number of key areas. These areas included Agency 23 systems and tools, data assets, and data management maturity. These 24
32 inputs were used to identify key findings, observations, and recommendations 1
for improvement which served as the foundation for the development of 2
specific data strategies needed to achieve the data goals.
3 Examples of these recommendations include expanding the 4
integration of system data sources and the data warehouse, establishing 5
enterprise data standards for key cross-functional data sets, and filling data 6
literacy gaps through selective hiring, cross-training, and professional 7
development. Next slide, please.
8 These strategies not only identify new opportunities for 9
improvement, but they also encourage the continued exploitation of existing 10 data initiatives, such as expansion of the Agency's enterprise data warehouse 11 to the identification and ingestion of new data.
12 Expanded use of the Mission Analytics Program, or MAP, 13 through Agency-wide exploration of business analytics use cases. The 14 enhanced data stewardship of functional organizations through the localized 15 identification and curation of data for analytics. And promoting the expanded 16 use of data in the generation of evidence to support programmatic evaluations 17 to answer priority Agency questions. Next slide, please.
18 As you can see, the IT roadmap and the data strategy 19 provide the Agency with strategic planning tools to promote the identification, 20 selection, and implementation of key IT investments that will support the 21 NRC's transformation journey. These tools are meant to evolve in concert 22 with the business needs of the Agency's workforce, business processes, 23 technologies, and the stakeholders.
24
33 Again, thank you all for your time. I will now turn it over to 1
June Cai to talk about the transformation of the rulemaking process.
2 MS. CAI: Thank you, Nick. Good morning, Chairman and 3
Commissioners. I am June Cai, a Branch Chief from the Rulemaking Center 4
of Expertise in the Office of Nuclear Materials Safety and Safeguards. Today 5
I will be providing an overview of our recent rulemaking innovation effort and 6
showing you some of our completed products.
7 Before I begin, I would like to take the opportunity to 8
acknowledge a number of individuals who have contributed to the 9
presentation contents you will see and hear about today, including Caylee 10 Kenny, Dan Doyle, Mary Anderson, Alexa Sieracki, Cindy Bladey, Ilka Berrios, 11 and Tania Martinez Navedo. This has been truly a full team effort. Next 12 slide, please.
13 In 2019, the NRC's Rulemaking Center of Expertise self-14 initiated an effort to strengthen the Agency's rulemaking process to align with 15 our vision of being a modern risk-informed regulator. For this initiative, we 16 focused on identifying opportunities to enhance the quality and timeliness of 17 our rulemaking products and further strengthen staff and stakeholder 18 engagement.
19 We took a holistic, integrated review of all aspects of the 20 rulemaking process that were within the staff's control to look for ways to 21 innovate and improve. In carrying out our efforts, we leveraged the diversity 22 in the backgrounds and skillsets of our staff. We energized and empowered 23 our staff to take ownership so that innovation and continuous improvement 24
34 can become more fully integrated into our culture and routine operations.
1 Next slide, please.
2 We reviewed a variety of internal and external inputs and 3
resources. Based on our analysis, we identify opportunities for 4
enhancements at both strategic and tactical levels and we organize them to 5
five key enhancement areas. This simple graphic provides a summary of the 6
results. Next slide, please.
7 The first area, decision tools for developing a graded 8
approach to rulemaking, establishes a framework for facilitating a strategic 9
risk-informed decision path for rulemaking and expected to provide the most 10 impact. Next slide, please.
11 The other four areas help develop and implement 12 rulemaking activities in more efficient, effective, innovative ways. These 13 include strategic thinking and planning and prioritization; process 14 enhancements and efficiencies, such as application of agile project 15 management concepts; product development and concurrence, such as 16 modernizing review and concurrence practices; and organizational 17 effectiveness, such as serving as early adopters of Agency transformation 18 tools and processes.
19 The full set of initiatives are designed to operate together to 20 support the entire rulemaking process. We issued a report about our effort in 21 July 2020 and have been implementing and refining the associated actions 22 since that time. This is a living and continuous effort. We have completed 23 some products, others are in progress, and some new efforts have been 24
35 recently started.
1 Over the next several slides, I will be highlighting some 2
examples of our recently completed key products. Next slide, please.
3 We call this our super infographic. It serves as our 4
navigation tool and contains references to all the products we have developed 5
to support the full process, from initiating rulemaking to developing and 6
reviewing rulemaking products, to highlighting successful outcomes. We 7
plan to keep this graphic updated as we develop additional items. Next slide, 8
please.
9 To begin the rulemaking process, here are the two tools we 10 have developed to help guide the decisionmaking to identify the best path 11 forward. The first tool helps us determine if rulemaking is the most 12 appropriate path to meet a regulatory need.
13 And the second tool helps us to develop a tailored approach 14 to meet that need in a risk-informed manner based on consideration of factors, 15 such as the complexity of the issues involved and effective stakeholder 16 engagement. Next slide, please.
17 An important consideration on many rulemaking products is 18 appropriate engagement with Agreement States as our regulatory partners in 19 the materials safety area. This infographic serves as a helpful tool for both 20 internal Agency use and for communicating with our stakeholders to explain 21 why and how we engage Agreement States through the different stages of the 22 rulemaking process and the expected benefits.
23 Once we delve into the developmental process, the tools 24
36 described in the next three slides are helpful for supporting different project 1
stages. Next slide, please.
2 First, to facilitate overall management of rulemaking 3
products, these two infographics provide helpful insights on the potential 4
application of concepts from agile project management methods, such as the 5
general Scrum framework, which is traditionally used in the IT arena to 6
achieve improved project outcomes, such as increasing product quality and 7
minimizing project redirection. Next slide, please.
8 Once we move into development of our rulemaking 9
packages, our staff can use these two infographics, which provide best 10 practices and helpful tips for writing quality to develop well-written materials 11 and reduce the need for significant revisions. Next slide, please.
12 After we complete development of the rulemaking package, 13 we use many of the practices on this infographic to facilitate the review and 14 concurrence process in a more streamlined manner, leverage new technology 15 tools, and tailor approaches based on specific project needs.
16 In identifying and developing these practices, we 17 coordinated closely with the Agency's Process Simplification futures team, 18 and we designed these to complement their efforts. Next slide, please.
19 Finally, once our rulemaking projects are completed, we 20 have recently started publishing publicly available summaries called 21 rulemaking highlights to help illustrate how specific rulemaking activities have 22 contributed to the Agency's mission and how staff have applied specific 23 innovations and efficiency measures during the development process. Next 24
37 slide, please.
1 Next, I'd like to share some examples of the impacts of our 2
progress. To begin with, we have been able to develop tailored approaches 3
to meet different regulatory needs and schedule needs. I have three specific 4
examples to share here.
5 First, for the rulemaking on the development of a risk-6 informed, technology-inclusive regulatory framework for advanced reactors for 7
Part 53, we're performing significant public outreach, including releasing 8
preliminary proposed rule language for public discussion and holding monthly 9
public meetings and workshops.
10 Second, by applying process efficiencies, we have been 11 tightening timeframes needed for conducting rulemaking to address spent fuel 12 casks' certificate of compliance amendments, while at the same time being 13 able to accommodate an increase in volume of requests during Fiscal Year 14 2021.
15 Third, we have taken focused steps to provide additional 16 opportunities for Agreement State members to participate directly on working 17 groups on a number of different rulemaking projects in the materials area.
18 The next example of an area of impact is we have been able 19 to demonstrate the ability to quickly adapt to using new technologies, such as 20 new IT tools for conducting collaboration and parallel reviews. Also, we 21 recently used IdeaScale platform to seek Agency-wide input for the 22 categorical exclusions from environmental reviews rulemaking.
23 We have also been able to quickly incorporate new process 24
38 improvements, such as the new commission paper alignment process into 1
ongoing projects. Furthermore, we have been able to enhance how we 2
communicate with the stakeholders using more innovative and modern tools, 3
such as the case studies I just described, as well as through multiple social 4
media channels.
5 Finally, we have been able to strengthen our organizational 6
capacity. Our staff have been able to learn to use new IT tools. They've 7
been able to broaden their project management practices. And they've been 8
able to work together on cross-discipline topics.
9 In addition, all these efforts will help us facilitate knowledge 10 management and the growth of the next generation of rulemaking staff. Next 11 slide, please.
12 In closing, I'd like to share some reflections on what worked 13 well in this initiative from a change management perspective. First, we 14 recognize that there's always room for improvement. Even in an area like 15 rulemaking where there are many requirements we have to meet and 16 processes and procedures we have to follow, there are opportunities to 17 innovate and make improvements.
18 Second, this has truly been a grassroots effort, and we were 19 fortunate to also have strong management support, which helped amplify our 20 progress.
21 Third, it truly takes a village, or in this case, a center of 22 expertise. We have staff across our organization working and pulling 23 together to achieve these results.
24
39 Next, I want to highlight the importance of focusing on 1
progress, not perfection. Incremental improvements can truly add up over 2
time to make more significant and lasting impacts.
3 And finally, I want to emphasize the importance of creating 4
a safe environment for taking informed risk and applying creativity. In order 5
to innovate, our staff has to feel comfortable taking some calculated risks and 6
try out new things, even if they don't work out perfectly. The only way we can 7
grow and adapt is if we keep trying and learning from our experiences. Next 8
slide, please.
9 Our rulemaking innovation efforts are expected to mature, 10 grow, and adapt over time. NRC's Rulemaking Center of Expertise is 11 committed to continuously strengthening the Agency's rulemaking process 12 and serving as a model of federal rulemaking excellence.
13 I am honored to have been able to share about innovation 14 efforts with you today and for this chance to represent the committed and 15 talented staff from the Agency's Rulemaking Center of Expertise. I look 16 forward to answering any questions you may have. I will now turn it back to 17 our EDO for closing remarks.
18 Thank you.
19 MS. DOANE: Okay, thanks, June.
20 As you've heard today, we have realized so many 21 accomplishments on our journey, while also responding to the dynamic 22 environment in which we are currently operating. Our operating experience 23 has taught us that transformation is absolutely necessary and should be 24
40 maintained. We will continue to closely monitor our external environment in 1
order to appropriately and timely respond to outside factors.
2 We must do this so as to not find ourselves in a position 3
where we are at risk of not being able to keep pace with what is occurring 4
around us.
5 We will continue to encourage and support innovative 6
approaches to carrying out our important safety and security mission. And to 7
do this, we'll work on realizing our ideal culture, one that supports creativity, 8
diversity in all its forms, and appreciates the need for change.
9 We will continue to carry forth the valuable lessons we have 10 learned throughout our transformation journey to best position ourselves for a 11 successful future. As you've heard today, while there is always room for 12 innovation, there's always room to learn as well, especially in a safe 13 environment for taking informed risks while creating these innovative 14 approaches.
15 These past few years have been about developing a strong, 16 clear plan for transformation. Now that the framework and goals have been 17 established, we want to empower the staff to continue to evolve these efforts 18 and truly make them their own.
19 I wanted to say thanks to these individuals at the table with 20 me today and to the core team, especially Maria Arribas and Mary Frances 21 Woods, who is on a rotation to the OEDO's office from OGC, for their hard 22 work in putting this material together. I also wanted to acknowledge all those 23 that have supported in one way or another the initiative teams, and to the staff 24
41 at large for embracing and helping to achieve all of these successes.
1 As I have said many times and I feel like we should say it 2
again, our staff is what makes the NRC a great place to work.
3 This concludes our remarks, Chairman and Commissioners, 4
and we are happy to take your questions.
5 CHAIRMAN HANSON: Thank you very much, Margie, and 6
thanks to the staff for that presentation. I appreciate it very much. We're 7
going to start the questions this morning with Commissioner Caputo.
8 COMMISSIONER CAPUTO: Good morning, everyone.
9 Thank you very much for all of these presentations. It's clear that there is a 10 significant amount of staff engagement and creativity being applied here, and 11 I thank everyone for their presentations.
12 Margie, I'm going to start with you because as Executive 13 Director, the success of the Agency's transformation rests upon your 14 leadership.
15 This meeting reflects the Commission's recognition of the 16 need for transformation, and our focus on it over the past several years.
17 Transformation, innovation and agility have entered our lexicon and have 18 become frequent buzzwords.
19 I see innovation as an integral part of our value of 20 excellence, high
- quality, continuously improving and self-aware.
21 Transformation, however, is different and implies a much larger, wide ranging 22 change.
23 We spoke early on about the goal of our transformation 24
42 efforts and the result was to become a modern risk-informed regulator.
1 Margie, you stated that we are making progress in all 2
transformation focused areas. Other than culture surveys, what objective, 3
measurable metrics are you tracking to support that conclusion?
4 MS. DOANE: Okay. Thank you, Commissioner. So let 5
me focus on what the areas are, so people, technology, and Be riskSMART, 6
and I'll come up with the fourth. I don't know, for some reason I'm blanking.
7 But any focus areas, what we have done is a number of different things.
8 So let's focus on our people. From the perspective of our 9
people, the objective tracking measures that we've been using are two-fold.
10 First, we've been monitoring what's been going on with the 11 creativity in our journey, our staff journey, and also the other activities that the 12 initiative team focused on our people from the perspective of mid-career staff 13 and have been identified as what would be helpful to improving their journey 14 and also their experiences. And so, as you know, we have created a number 15 of different tools for them to use.
16 And so the way that we've objectively monitored whether 17 those tools have been helpful is by monitoring how many people touched the 18 Web site, how many people use it and also we get feedback regularly on that 19 Web site. So that's one thing that we're doing and focusing on our people.
20 Another area that we're looking at focusing on our people is 21 the culture. And as you heard we're doing Pulse Surveys, and both our FEVS 22 and our Pulse Surveys have improved in meeting our desired culture. So that 23 means --
24
43 (Simultaneous speaking.)
1 COMMISSIONER CAPUTO: Right. We've already heard 2
about those this morning. Can you give me a different --
3 MS. DOANE: Okay.
4 COMMISSIONER CAPUTO: -- set of metrics?
5 MS. DOANE: Okay. So just heading into these, okay?
6 And then IdeaScale, I went over a few of those metrics, who has touched the 7
site and where our success stories are and things like that. So those are the 8
metrics that we're using for innovation.
9 For technology, what we're using are various tools like how 10 to measure whether we're retrieving data better, whether we're using -- we 11 keep all kinds of metrics on whether we're using the technology.
12 So for example, I don't have the figures with me, but we 13 tracked how many people were using Teams. And our daily metrics showed, 14 like for example when we went into the pandemic that we went from almost 15 little to no usage to immediate usage, very effective usage. And then we also 16 tracked our transition to Teams to see that we had the same uses and effective 17 uses.
18 So we had various metrics, all kinds of metrics in that area.
19 COMMISSIONER CAPUTO:
So you're tracking 20 implementation of a lot of tools and whether or not the tools are being used.
21 Do you have any results to suggest it's actually improving staff productivity or 22 increasing efficiency?
23 MS. DOANE: Yes. So let me start with -- and Annette's at 24
44 the table so she can help me, but let's talk about like productivity, for example, 1
on process simplification.
2 For process simplification, we focused on SECY papers, as 3
you know, and depending on how -- I've seen two different data measures for 4
this, but depending on how you are measuring whether you've seen efficiency 5
in this area, we've seen a 30 percent, at least a 30 percent reduction in the 6
time it took to put papers forward.
7 So that's an example of a very specific outcome and a 8
measurement from what we did before and what we did now and how that's 9
an improvement. Like I said --
10 (Simultaneous speaking.)
11 COMMISSIONER CAPUTO: So do you have any way of 12 comparing the time the staff has spent training on these tools, learning these 13 tools and making advances, transformation advances? Do you have any way 14 of comparing that to those productivity improvements to see if we're, you 15 know, seeing benefits from this activity?
16 MS. DOANE: I mean, I think we have a way of measuring 17 the benefits. The time it took to get there is not -- I don't think -- we do 18 measure it because they have to keep track of their time. But we haven't 19 done a comparison of, like, for example, the first -- so let's say we have on 20 these routine papers we've seen a 30 percent increase in the time it took to 21 get the papers to the Commission, right?
22 And we haven't done a comparison to well, did it take 30 23 percent of staff, more than 30 percent of staff time to get trained and to learn 24
45 these things? No.
1 And because that's like a one-time investment, you would 2
have to do that over a series of time. In other words, you would have to watch 3
these things over a series of time to know whether your initial investment.
4 But, no, we haven't done that function, that comparison.
5 COMMISSIONER CAPUTO: Okay. I think that would be 6
a worthwhile review just to understand how much benefit we're getting out of 7
the amount of staff time being invested.
8 I'm going to move on to a question about IT. So Nicholas 9
discussed a lot of important concepts and aspirational goals with regard to the 10 IT roadmap. But Margie, I'm going to address this question to you.
11 It would be more fairly addressed to the Chief Information 12 Officer and certainly unfair to ask Nicholas. But let me start by saying our 13 people are our most important asset. And we have strategic workforce 14 planning to ensure we have the right people for the right work at the right time.
15 They're our most important tool.
16 And our second biggest line item is IT. And while I 17 appreciate Nicholas' presentation and consider it a step forward, I'm not clear 18 how detailed the roadmap will be.
19 So my question, Margie, and my understanding is that it is 20 not ready yet and thus has not been available to inform budget development.
21 So my question to you, Margie, is how much longer are we 22 going to continue to spend in excess of $150 million on IT without having a 23 comprehensive plan to ensure we have the right tools at the right time to meet 24
46 the Agency's evolving needs and prevent last minute scrambling to address 1
easily foreseen issues such as software obsolescence and Wi-Fi installation?
2 MS. DOANE: Okay. Let me see if I can -- thank you, 3
Commissioner, for these questions. Let me see if I can answer them in the 4
way that you put them.
5 Okay. So how much longer until we have a comprehensive 6
plan? We are anticipating this for later during the summer that we'll have the 7
plan. So I appreciate that the budget formulation process that's going on now 8
hasn't been informed by that process.
9 But we certainly have had planning in place for many, many 10 years. It's how we were ready for the pandemic in going all virtual. What we 11 didn't have, and I will tell you this honestly, so we had, for example, we knew 12 we needed to move to laptops. And we knew needed to move to Wi-Fi. We 13 knew we needed to do a lot of things.
14 So we were bringing the Agency down. We were reducing 15 the number of staff and corporate investment. We were going down, not up.
16 So we were trying to do a number of different things in a sequenced event so 17 that we -- sequencing them so that we could meet this plan, like, for example, 18 get laptops, Wi-Fi and different things, but we were doing it over a number of 19 years because we never had the funds to do it in any one year. So we did 20 have planning.
21 We had a number of things that were in the queue. We had 22 laptops. We had Office 365 that was coming down. We had Skype at the 23 time, as you know, Commissioners. We've had this conversation before.
24
47 So we had a lot of things that were good things that were in 1
the planning and were being carried out as funding became available.
2 But there were pressures on this same corporate support 3
funding. And so we couldn't fund all of them. So it looks frenetic as you tried 4
to then fund them later or pick them up from resources that become available 5
but in fact it wasn't. We had these things listed, and we always knew that 6
these were where we wanted to go.
7 What the transformation did for us is it helped us focus.
8 And I give the OCIO credit for this, Dave Nelson. It helped us focus on why 9
we weren't putting these technology advances into our working environment.
10 And it was because we were not focused on technology adoption for our staff.
11 We didn't look at what was keeping them from doing it. And that's really been 12 the transition and that's made all the difference.
13 So I think we have had good planning. I think we've got a 14 lot of great ideas. We just haven't been able to implement them.
15 Why is this different? So then you might say well, then, 16 why do you need this IT roadmap and data strategy? Because information 17 and data are flowing in at an increasing rate, and technology is continuing to 18 evolve. And so the benefit that we have now that we're getting this -- the staff 19 is now accepting using this technology is now we have a transformation that 20 will enable us to use technology at a faster rate and to make better decisions.
21 We also have --
22 COMMISSIONER CAPUTO: All right. My time has 23 expired. So I'm going to cut you off right there. I think it's usually important 24
48 when managing expenditures of that size that there be a significantly detailed 1
plan to ensure we're spending the money prudently and meeting needs.
2 And I'm going to beg my colleague's indulgence. This is 3
my last meeting. There's one last comment I would like to make. I'm sorry 4
that we haven't had a speaker here today from the CFO's office. And I want 5
to take a moment to commend them for their work on budget execution 6
dashboards.
7 This has been a strong step forward toward the better use 8
of data in budget formulation and execution. And I think it clearly reflects a 9
fair amount of hard work, and I appreciate their efforts.
10 However, I would characterize this as an innovation, not a 11 transformation. Our oversight and licensing workload is down 45 percent 12 since 2016 and likely to decrease further given the potential for as many as 13 five reactors to close within the next year.
14 In 2021, this workload was the equivalent of 439 FTE. That 15 means for every FTE doing oversight and licensing work, there are 5-1/2 16 others doing other things.
17 In 2016, that ratio was 1 to 3-1/2. This is not a sustainable 18 trend. I personally believe this is an enterprise risk, and there is a strong 19 need for transformation in budget formulation execution to manage it. Thank 20 you for the time, Chairman.
21 CHAIRMAN HANSON: Commissioner Wright.
22 COMMISSIONER WRIGHT: Thank you, Mr. Chairman.
23 And thank you so much, everyone, for your presentations.
24
49 I continue to be impressed with your enthusiasm for 1
transformation and the impact that it's having on the way we do business and 2
even at last week's innovation celebration, staff called IdeaScale, I think, an 3
ecosystem of innovation, if I remember correctly. And then that connects and 4
inspires us. And I like that term, you know. And I think that it also applies to 5
the network of interconnected transformation projects.
6 We could also say that we have an ecosystem of 7
transformation that drives us toward our goal of becoming a modern risk-8 informed regulator. So with that in mind, I've got several questions I'd like to 9
get to. And Margie, I'm going to stay with you.
10 There have been several change initiatives undertaken in 11 the past decade and longer. But it seems transformation seems to be one of 12 the most or the more successful in terms of sustained engagement and 13 somewhat measurable results.
14 You know transformation, it's a broad term. I've really not 15 personally been a big fan of that word as a standalone for the Agency because 16 I kind of agree here with Commissioner Caputo here. I think it's more what 17 we're trying to do and what we're going through at the Agency, it's really big 18 on innovation. It's really big on adaptation of how to use data in better ways.
19 You can call it transformation if you want. That's fine. But 20 I do think it's probably more than just one word. I mean, we're morphing.
21 We're changing. We're constantly moving. And I guess that's what you 22 mean when you're using the term transformation.
23 But it doesn't work unless we have a proper vision of where 24
50 we want to go, right? What's our end game? Otherwise I think it's 1
problematic.
2 What do you think is different about this effort that is 3
contributing to the successes that you have seen so far that you point to?
4 MS. DOANE: So I think -- first I will tell you that past 5
transformations had their own successes like, I know, the Reactor Oversight 6
Program, the ROP. So there have been successful transformations. It's just 7
that we didn't sustain a culture of transformation and innovation.
8 So what I think is really different about what we're doing 9
today is we're really marrying these transformation efforts. We can call them 10 innovations or I'm open to any of it. But they're just good ways of enabling 11 staff to do their work better.
12 And so we're marrying that with really paying very close 13 attention to the health of the organization. And having a real effort on culture 14 and what a culture that embraces change looks like, why that's good for the 15 staff, making this like a really, making every day a great experience for the 16 staff as they go through doing different things and looking for ways to improve 17 how they do their work, all with an ending towards making better decisions.
18 That's in the end. That's our product, right? That's the 19 NRC's product. It's our decision-making. We're a regulator. We want to 20 make better decisions, more efficient, more timely, better, taking into 21 consideration more data.
22 So the vision is that we'll get to a point where our surveys 23 are showing that this culture has really moved. We've seen a real shift. And 24
51 so now we have the culture and how the organization is marrying up with these 1
different things that you're hearing about. You know, you can't help but be 2
excited about all the different things that staff are working on. But there 3
needs to be this marrying of the two, the health of the organization with all the 4
goals that we're setting.
5 COMMISSIONER WRIGHT: I --
6 MS. DOANE: And I think -- I'm sorry.
7 COMMISSIONER WRIGHT: No, go ahead, finish. Go 8
ahead.
9 MS. DOANE: So that's really the vision. The vision is, you 10 know, it morphs into -- what it morphs into is no longer this like push down 11 from the top saying, you know, we have to do things better. We have all these 12 areas of stagnation, different things, to the staff, telling us. And they're 13 already doing it. You're not going fast enough. You're not doing this. You 14 know, we could do more. And that's really the vision of a transformed agency 15 for the NRC.
16 COMMISSIONER WRIGHT: All right. So, I mean, I'm a 17 huge fan of the use of people like Embark and Innovate and IdeaScale and 18 those. I'm a big fan.
19 So this brings me to two questions. And it's two different 20 questions, but I'm asking them at the same time.
21 So how are we capturing those lessons learned for future 22 initiatives? That's the first one.
23 And my second question, how often are you recalibrating or 24
52 refocusing on the vision as a whole for the Agency? Because you mentioned 1
just a second ago that it's constantly morphing. So how often are you taking 2
a hard look and recalibrating?
3 MS. DOANE: Okay. So first how are we capturing the 4
lessons learned? So we are doing that in a number of different ways. So 5
mostly we use our Web site to capture where we have gone in these various 6
initiative areas and then we get feedback from the staff and then we're 7
continually trying to answer that feedback.
8 And so depending on how significant the issue is, we'll take 9
it on all the way up to the EDO's level because it's something that really needs 10 that kind of attention or it will be done at the grassroots level.
11 But we're trying to identify a number of different areas where 12 staff can give us feedback. And so that's how we're really learning the lesson 13 of are we moving in the direction that we want? And maybe that goes to the 14 second really about, you know, are we looking at the vision?
15 The other thing we're doing is we know that once we built 16 the infrastructure and thank you for mentioning Embark. It's very important.
17 They're our innovation spark plug if you ever -- you know, I know you've 18 worked with them and you've gotten briefings from them many times.
19 But what we really want to see is we want to see a place in 20 the Agency for these things long-term. And so what we're looking at now is 21 whether we have the right organizational structure. We want to make sure 22 that we can sustain these efforts.
23 We're trying to, like I said, survey and do other things. Look 24
53 at the products that are being produced and see whether we really do see 1
innovations or changes in small groups and also in large projects. There are 2
really important things that are going along to make sure we are seeing 3
progress in these areas.
4 So we're constantly looking at these issues. We have a 5
directorate that pays attention to the survey results, pays attention to the 6
issues that are going on. The assistant for operations, as you know in the 7
EDO's office, is really focused with a good portion of her time on 8
transformation looking at these various things.
9 And the way that we make sure that we are capturing the 10 lessons learned is every other Wednesday we bring a transformation example 11 to our executives, every Wednesday, every other Wednesday we do this, so 12 that they're hearing what's going on around the Agency and other parts of the 13 Agency. And they bring them into their organization.
14 And then finally IdeaScale is really good at capturing 15 successes. And what it's linked with is something called Nuclepedia, which 16 just like Wikipedia, it has pages of how we've learned different lessons from 17 going forward during transformation so that staff can either mimic these or 18 learn from these experiences or use in a training tool.
19 So there's about, you know, too many different things for me 20 to capture. But I hope that that's given kind of a glimpse of how we're 21 capturing the lessons learned and then relooking at the vision by we still have 22 a steering committee for transformation, and we have the AO looking at this.
23 We also have survey results, and these different issues that are brought to us.
24
54 And then we bring them to the executive core.
1 COMMISSIONER WRIGHT: Okay. Thank you. I've got 2
about a minute and a half left. I'm going to switch over to Candace for a 3
second. So, Candace, it's good to see you. And I'd like to again take this 4
opportunity to thank you for all you did as my session coordinator for my RIC 5
grid panel that I'm on already. So thank you. I look forward to working with 6
you in the future, too.
7 You mentioned the success of the Be riskSMART 8
framework for risk-informed decision-making. We've heard many examples 9
of its use today from Erin and from Tammie as well during the business line 10 briefings.
11 So Candace, but Erin and Tammie can jump in if they want 12 and they're welcome to add their thoughts, now that you've had some 13 experience using the framework in multiple parts of the Agency and you've 14 completed your first Pulse check, do you think any revisions are needed?
15 And if so, what might they look like?
16 MS. de MESSIERES: Thank you, Commissioner. And 17 we are still in the early phases of deployment. But I think some of the 18 foundational work that the initiative team completed has really paid dividends 19 in this regard.
20 For example, we built off of the multiple decades worth of 21 experience that pockets of the organization has in risk-informed decision-22 making. And we also exercised extensive pilots and case studies.
23 We sought staff feedback throughout the process of 24
55 development at multiple office and divisional level meetings and individual 1
targeted outreach across disciplines.
2 And one of our guiding principles was that we first do no 3
harm. We deliberately ensured that the framework could accommodate, be 4
consistent with, or enhance the many successful current NRC processes and 5
guidance. And therefore, it's broad, it's flexible, it's scalable.
6 And, you know, given our initial progress and successes, I 7
don't think that, you know, revisions are needed at this time. But as you've 8
heard throughout this whole meeting, you know, we are a continual learning 9
organization.
10 And I think that we are always open to feedback. I mean, 11 throughout the whole process, we've been open to feedback, and we've 12 adjusted as needed. The little arrow on the Be riskSMART logo there talks 13 about the iterative nature and that continual learning.
14 And so while I don't think we need revisions at this time, I 15 would say that I think a lot of additional work will be needed in the future to 16 continue the positive progress to increasing the use of risk-informed decision-17 making and also maintaining a supportive agency culture.
18 I think this is really continual work that needs to be done in 19 this area every single day at all levels of the organization.
20 COMMISSIONER WRIGHT: Thank you so much for that.
21 I just want to, again, focus back on the vision. I really appreciate the fact that 22 we are constantly looking to improve and stuff. And that does not happen if 23 you do it just to do it, you're not accomplishing anything. So the fact that you 24
56 are taking a good hard look at how you are doing things and looking how you 1
can improve on that I think is very important and just not checking a box. So 2
thank you so much. Mr. Chairman, I yield.
3 MS. de MESSIERES: Thank you.
4 CHAIRMAN HANSON: Thank you. Thanks again to the 5
staff for the updates this morning. You know, as I've said before, I think the 6
challenge for transformation going forward is kind of fourfold.
7 The first is to keep the safety mission front and center in all 8
of our efforts. The second is to sustain a culture of innovation throughout the 9
Agency.
10 The third is to realize the return on investment from past 11 initiatives to make sure that the learnings, as Commissioner Wright pointed 12 out, are getting captured and that we're institutionalizing some of the business 13 processes that we've been driven to change. And the fourth is then to expand 14 and apply transformation thinking to some new areas.
15 I was really encouraged at the level of participation 16 throughout the Agency in transformation efforts, right? There were, you 17 know, something on the order of 1,600 people that were engaged in IdeaScale 18 events. But I have also heard some concerns from staff that, you know, 19 transformation is really just, you know, a means to cutting people or 20 regulations or that it's change just for the sake of change.
21 And Margie, I guess I'd like to hear from you just a little bit 22 about how Agency leadership are responding to some of those concerns.
23 MS. DOANE: So, thank you, Chairman. And I think that 24
57 one of the main things that we try to do in all of our communications is to make 1
sure that this is, and you heard it with the speakers, but make sure that it's an 2
iterative process so that we can get our messages out.
3 And we always start with, you know, what's our mission?
4 It's what unites us. We explain to staff that our mission is something that 5
unifies the staff and the staff believes, every one of them believes in that 6
commitment. And we make sure that we focus ourselves in those meetings 7
of thinking about how did it enable us to meet our mission, to do better, not to 8
somehow impact it.
9 So I understand that concern. In fact, a lot of what we did 10 at the very beginning when we were setting up these initiatives is to ask 11 ourselves, we kind of kept this as a mantra, like, you know, how do we make 12 sure that whatever we do doesn't impact our ability to meet our mission but 13 instead enhances it?
14 Okay? So first of all, one thing I would tell you is the 15 concerns about staff, that this is just a means of cutting or doing less. We 16 counter that every day with both our communication from our executives but 17 also we have to be role models.
18 And so one of the things we've said as being a role model 19 to show that we are improving in this area is the way that we're looking at the 20 culture, to empower staff, to let them bring their perspectives forward, their 21 whole self forward, that is a culture. If we get to our desired result, we will 22 have many more perspectives to bring to the issues. And when staff feels 23 comfortable speaking up, right, then we have more perspectives so we can 24
58 make better decisions.
1 So we're countering this in a number of different ways.
2 Those are two examples.
3 And to make sure that it isn't change for change, we have 4
had to refocus this actually. Because we have had situations where we got 5
a little excited about, like, you know, doing things.
6 And we didn't put in any guardrails because we just wanted 7
to spark innovation. Then we just had innovation everywhere. It was very 8
frenetic. And some of the staff said, you know, it's too much.
9 So we're putting in guardrails to make sure that we are 10 identifying the problems, and we're doing this in all different areas. You heard 11 Candace talk about it, an IdeaScale. You have to identify this problem.
12 We were also looking at Embark, Innovation Accelerator 13 and making sure that we're really focusing on the important problems and that 14 we're prioritizing in a way that really moves the Agency forward with what's 15 most important.
16 So the first year was all about just running as fast as we 17 could to show innovation successes and things like that. But now we're really 18 trying to hone that in a bit, put guardrails in to make sure that we're focusing 19 on those that we're prioritizing and getting results.
20 CHAIRMAN HANSON: Thanks very much for that, Margie.
21 I appreciate your response on that.
22 June, I've got a question for you. I kind of want to follow 23 through on this theme just a little bit. I was really interested in your 24
59 presentation about kind of improving the rulemaking process, particularly 1
some of the things that we're doing internally.
2 And I think it's important for us to look at those ways in 3
making the rulemaking process more efficient. But I want to make sure that 4
efficient doesn't mean reducing opportunities for public involvement in the 5
process. And so can you talk a little bit about what staff is doing to strike the 6
right balance there?
7 MS. CAI: Thank you, Chairman, for the question. So 8
stakeholder engagement is definitely very important to us in rulemaking. And 9
it's one of the centerpieces of our decision-making tools that we built in.
10 I mentioned one of the tools talks about developing an 11 approach, address whatever the issue and the needs are. And within that 12 consideration is definitely how do we effectively involve stakeholders?
13 So we have different examples listed, different guidance, 14 you know, tools that staff can use, public meetings, putting documents out, 15 Federal Register notices, you know, social media communications.
16 And I also mentioned Agreement States involvement. And 17 we have our separate infographic on how to engage effectively with the 18 Agreement States.
19 So in addition to the focus on the process improvement 20 efficiencies, the effective stakeholder engagement involvement was also a 21 strong focus area for this whole innovation effort and rulemaking.
22 CHAIRMAN HANSON: Okay. Thank you very much.
23 Stephanie, I thought your discussion on culture was really important. And, 24
60 you know, I think as Commissioner Caputo has said on occasion you get what 1
you measure in a lot of these things.
2 And one of the ways we measure progress in culture and 3
encourage people to take calculated risks and other things is in part through 4
kind of the appraisal system. And are there changes kind of in the appraisal 5
system that allow for risk acceptance to innovate and change to send the right 6
signals to staff that they should be able to come up with new ideas without 7
fear of failing or being penalized in some way?
8 MS. MORROW: Well, I can't speak directly to the appraisal 9
system, but one thing that we've been working on with the culture team is 10 including more constructive behaviors as part of the appraisal system. That's 11 something we've been working with OCHCO on, focusing first on our senior 12 executive service.
13 There's also, I can't say that it is uniform, but there is 14 organizational effectiveness as an element and standard in our appraisal 15 system that I feel like has evolved over the years in terms of how that's 16 evaluated.
17 And there seems to be much more of a focus on improving 18 the organization and really contributing to the organization. And that is where 19 I would see, you know, the opportunity to embrace risk and make that -- you 20 know, accept that as part of our culture and incorporate that into our appraisal 21 system.
22 CHAIRMAN HANSON: Great. Thank you. So we know 23 that one of the ways that kind of culture changes and is transmitted is through 24
61 stories. So I was really interested in Erin and Tammie's kind of case studies 1
of the application of the Be riskSMART framework because I think sometimes 2
these things can be a little abstract. I admit I don't always understand what 3
we're talking about when we talk about these things.
4 So I was really kind of pleased to see this. And I know 5
Candace, you and others have briefed me on this before, but could you share 6
another example of a success of application of the Be riskSMART framework?
7 And particularly could you share an example which maybe the riskSMART 8
framework resulted in putting more resources towards a particular activity due 9
to its risk significance?
10 MS. de MESSIERES: Sure. Thank you, Chairman. Yes.
11 There's many, many examples. And maybe I'll briefly touch on a couple.
12 You know, first of all the framework can be used for not just 13 technical but process enhancement. So we do have examples where, you 14 know, in preparing for important decisions, like the 10 CFR 52.103(g) finding 15 for Vogtle Unit 3, the framework was used not for the actual decision but to 16 ensure the process enhancements were available there.
17 The early implementation of the principles was used in 18 FLEX decisions regarding the finding level. So, you know, should all FLEX 19 findings be green? And in that case, you know, no, that wasn't appropriate 20 and that used the principles from the framework.
21 We have examples, you know, from operator reactor 22 licensing where, you know, it may be that it's more important to use, you know, 23 rulemaking processes to address an issue or, you know, do formal exemptions 24
62 versus other, you know, processes.
1 You heard from Erin about her example from the region.
2 But, you know, there's also many examples of using the framework that 3
resulted in increased inspection resources, for example, evaluating safety-4 related breakers at Region I. So the list, you know, goes on.
5 I also just want to mention from a personal standpoint in my 6
use of it on a daily basis, and I recognized this right away during the pilot, one 7
of the pilots that's highlighted in the NUREG, that there is a -- some of the 8
benefit of the enhancement area is that the decision is based on a full suite of 9
possibilities, scenarios.
10 It really enables you to hear all of the perspectives, to talk 11 frankly about the risk appetite, get, you know, enhanced documentation.
12 There are things that I've come and just again daily use of the framework that 13 I just wouldn't have thought about managing if I hadn't really stepped through 14 that systematic approach.
15 And so I think it enhances personally my decision-making 16 every day. And I've witnessed with many examples. I think I'll --
17 CHAIRMAN HANSON: Okay.
18 MS. de MESSIERS: -- take any other questions.
19 CHAIRMAN HANSON: Thank you very much, Candace.
20 I appreciate it. Commissioner Baran.
21 COMMISSIONER BARAN: Well, thank you all for your 22 presentations and your work to encourage innovation at NRC.
23 I'd like to ask about the effort to improve the concurrence 24
63 process for sign off on rulemaking packages and policy papers to the 1
Commission.
2 I think it's fair to say that the Agency has struggled with a 3
concurrence process that often takes a long time to complete and leaves 4
people on the concurrence chain uncertain about what they're responsible for 5
reviewing.
6 June, you talked about this a bit for the rulemaking process.
7 Are the practices you discussed on Slide 46 the same ones used for policy 8
papers to the Commission or are those two different process improvement 9
initiatives?
10 MS. CAI: So I'll speak for rulemaking products and then 11 perhaps turn it to Margie on the overall paper process.
12 So what you see on Slide 46, this is really a very organic 13 collection of best practices that rulemaking staff developed over the last 14 couple of years to help us improve the concurrence process because we have 15 a lot of documents that we're working on, you know, at any given time and 16 they all have to go through concurrence.
17 And so we had our staff coming up with these great ideas.
18 And I want to give a shout out to Alexa Sieracki and Caylee Kenny because 19 Alexa worked to collect all these good practices you see on the slide from the 20 various staff. And then Caylee worked with her to put it all together.
21 Around the same time, the Agency process simplification 22 team was doing their work. And so Alexa actually served on that group and 23 shared many of these practices and some of the ideas that we've been trying 24
64 out in our experiences. So she sent our experiences directly to that group.
1 And so you'll see in the end many of these practices complement what the 2
Agency is doing under the process simplification team.
3 There is a lot of focus on making sure that we get early 4
alignment. There is an alignment meeting that's held under this new 5
Commission paper development process to make sure we have all the key 6
players at the table and we agree who should be on concurrence and what 7
their roles are. And so we're definitely implementing that. And we have 8
earlier versions of that practice as reflected in the infographic.
9 We've also been using parallel concurrence extensively.
10 We made that shift about a year, a year and a half ago when we started using 11 OneDrive and SharePoint more readily in the Agency. And we were really 12 able to leverage those IT tools to do parallel reviews. And now we'll be able 13 to quickly to the e-concurrence system in there and it's parallel review feature.
14 And so that's part of the, I think, process simplification team's focus as well.
15 So I will say, to summarize, it's been very complementary.
16 We were trying a lot of these things out and at the same time feeding the 17 inputs to that team. And in the end, I think, where we came out are very 18 closely aligned.
19 Margie, did you want to kind of talk about the overall?
20 MS. DOANE: Yes, if that's all right, Commissioner?
21 COMMISSIONER BARAN: Sure. Go ahead.
22 MS. DOANE: Okay. So, yes. So the answer to your 23 question is it's not just for rulemaking. It is being used for policy papers. And 24
65 is there significant enough -- the good thing that came out of an SRM, and 1
they're affecting a large program area. So there is a significant enough policy 2
question.
3 There's alignment all the way up to the EEO's office, but it 4
could be just within an office. So the alignment is helping because it clarifies 5
issues upfront. And so we're seeing that this is leading to more expedited 6
decision-making because we understand the problems earlier in the process 7
and also we can get alignment more quickly so that less issues come up at 8
the end. That's the first thing.
9 Also by just sort of documenting the concurrence process in 10 this matrix, everybody can understand their role and understand how much 11 time they're going to have.
12 Even they put a time limit on me. Sometimes there's, like, 13 you know, the EDO gets two days. And so I know this thing is coming, and 14 I'll try to get it early on in some other form because I try to keep to the staff 15 schedule.
16 And we also try to make sure that we expedite more quickly 17 things that already got alignment in the beginning. So it's a work in progress, 18 but we're seeing this used throughout -- for significant so definitely policy 19 papers and rulemaking. And then I would say probably the best example, 20 Commissioner --
21 COMMISSIONER BARAN: Mm-hmm.
22 MS. DOANE: -- the recent example is the gas pipeline 23 issue where they used -- that wasn't a rulemaking but where they used all the 24
66 different tools like parallel concurrence and e-concurrence. And they used 1
the SharePoint site to really move that report along quickly but to get all of the 2
right people involved in the decision-making.
3 COMMISSIONER BARAN: Great. Well, let me follow-up 4
on a few of those things. There are a lot of good ideas there.
5 In terms of the parallel concurrence, I can see how that 6
would save quite a bit of time if people were reviewing things simultaneously.
7 How is that practice working? Is it saving time? Is it creating any challenges 8
to take that approach?
9 And I guess I could, maybe I'll ask June if she wants to 10 weigh in on kind of the rulemaking piece, hone a little bit on the rulemaking 11 concurrence process just to keep it kind of clear. And, obviously, Margie, if 12 you prefer to weigh in but, June, otherwise.
13 MS. DOANE: No, no, I wanted to say Annette. I wanted 14 to turn you to Annette, who has been the head of this concurrence process if 15 you --
16 COMMISSIONER BARAN: Annette is sitting behind you, 17 which is really hard, but that's fine.
18 MS. DOANE: I'm sorry. I didn't understand the room.
19 COMMISSIONER BARAN: That's okay.
20 MS. DOANE: But June is fine, too. They both have been 21 putting a lot of effort into this.
22 COMMISSIONER BARAN: I turn it over to you guys.
23 Who wants to talk about this?
24
67 MS. CAI: I can jump in a little bit and then maybe Annette 1
could add on because I guess that we've been working closely together.
2 So for us, the parallel concurrence has, I would say, been 3
transformative. It's really made a huge impact.
4 And one thing that's really helped us is availability of these 5
IT tools. You know, I look back to even a year and a half ago, we were still 6
handwriting comments, scanning them, emailing them to each other, calling 7
each other. I can't read the handwriting. I mean, that's really, you know, 8
some of the true examples of what we were doing.
9 And then we started using these tools and we would put it 10 out there. And now we have all the branch chiefs reviewing together, all the 11 division directors reviewing together.
12 And it's also helping not just the time it takes, but also that 13 they can kind of see each other's comments, and we don't have to, we being 14 the rulemaking staff, have to work so hard to reconcile the comments.
15 Because I can, for example as a branch chief, I can see what the other branch 16 chiefs are commenting on. And I can respond or I agree with their comments.
17 And now that saves me time from having to write.
18 So those are some of the tangible improvements we've seen 19 with the parallel concurrence. And truly we were only able to do it because 20 of the availability of these IT tools.
21 And so, again, we were doing this, you know, pretty early 22 on, like a year, a year and a half ago. And then when the Agency transitioned 23 to e-concurrence and it has the parallel concurrence feature, we were able to 24
68 really make that shift quickly because we were already operating in that way.
1 COMMISSIONER BARAN: Great. And it looks like based 2
just on the slide, another practice is under the heading of need to concur.
3 I assume the idea here is to try to keep the concurrence 4
chain as short as practical and reduce the number of individuals who need to 5
review and sign off on a document.
6 How is the staff deciding who should be on the concurrence 7
chain? Is that tough to figure out? And how is this approach working? Is 8
the staff doing a good job identifying the right individuals? Are we seeing 9
cases where people don't end up weighing in, and it raises issues later?
10 MS. CAI: So I think this practice has been working pretty 11 well. And I think there's two main factors that are contributing to it.
12 First, we have been really making a concerted effort to make 13 sure we have early alignment on rulemaking projects before we, you know, go 14 down too far. Like I said, we have the decision tool so we're making sure the 15 key decision-makers, the key players, are coming in early and having open 16 discussions, making sure we identify the path forward.
17 So during that early alignment, we make sure, okay, who 18 are all the impacted parties who all have, you know, some input and make 19 sure we're engaging with them and then, of course, we would include them on 20 concurrence.
21 The other piece is with this new Commission paper 22 process enhancement, we do have to do the alignment, early alignment 23 meetings with Margie. And so as part of that process, she mentioned the 24
69 concurrence matrix.
1 We have to document clearly here are all the people who 2
are going to concur and here's what their role is. So it's not just that they're 3
concurring perhaps to the whole package, but they're looking at it, for 4
example, from a legal perspective or editorial, you know, style perspective or 5
only on this technical issue or only this other piece. So it's very clear what 6
their role is.
7 And so at that alignment that we have with Margie at the 8
outset of the project, we make sure we clearly document and include all the 9
various parties.
10 So I think it's been working very well. And so we're able to 11 hone in on exactly who needs to be on concurrence but in addition also specify 12 what their roles are. And I think between those two pieces, it really helps us 13 be more efficient.
14 I wanted to add one other piece. The alignment agreement 15 does also have a section about awareness.
16 So a lot of times we do have a wider distribution network for 17 our packages where we share documents with people for awareness or early 18 engagement but not actually putting them on formal concurrence but that they 19 do have, you know, an opportunity to take a look at it. And, of course, if they 20 have any concerns that they have a chance to weigh in and give us their input.
21 COMMISSIONER BARAN: Great.
22 MS. VIETTI-COOK: I just want to mention, too.
23 COMMISSIONER BARAN: Sure.
24
70 MS. VIETTI-COOK: What's interesting, what we're 1
observing -- even though we have limited data, what we're observing is that 2
we're actually, even though we were encouraging collaboration with as many 3
people as you need to and only put those people that you really feel need to 4
be on concurrence and be thoughtful about it.
5 And, of course, like she said, we also encouraged, you 6
know, courtesy reviews, provided for awareness to all those people that you've 7
collaborated with. And if they did have an issue, wanted to review the paper, 8
they could always go to the author. They could go to their branch chief. And 9
they can always go to the differing views programs if they feel like they're not 10 getting satisfaction.
11 But what we're observing is that we are seeing a big 12 reduction in the concurrence time, but that we're not really seeing a huge drop 13 in the number of people concurring.
14 So what we think is happening is number one, the alignment 15 is helping move, you know, early alignment of expectations of what the 16 stakeholders want to see in the product. And I think, you know, 17 documentation of that and then, you know, so that -- and then realigning as 18 necessary as new information becomes available but also the technology 19 improvements, like all of the collaboration tools, you know, for sharing 20 documents for, you know, collaborating over Teams.
21 We did templates so that they're all -- you know, these are 22 all the questions you should be considering on data, on, you know, 23 concurrence and all these various issues.
24
71 And, of course, we have templates for SECY papers and 1
then there's the e-concurrence system. So it's really tracking and people can 2
see how much time people are spending on concurrence. It's very visible to 3
everybody.
4 So we actually have seen a 45 percent drop in the 5
concurrence of the annual papers that we've been tracking. So we are 6
seeing progress on the time. And it was an interesting observation that we 7
really haven't seen the number of people getting reduced on the concurrence 8
process. So we think it's these other things that are moving it along.
9 COMMISSIONER BARAN: Great. Well, I think it's great 10 that the staff is trying some new tools and approaches in this area. It sounds 11 like it's going well.
12 I think it's the kind of process improvement that has the 13 potential to increase both the timeliness and the quality of the concurrence 14 reviews. And that, of course, can increase the timeliness and quality of the 15 rulemaking or the SECY papers which is what in the end has the real world 16 benefit. So that's great. Thanks so much for walking me through that.
17 CHAIRMAN HANSON: Thank you. And now we'll take a 18 five-minute break. We'll reconvene at 10:52 and, yes, thank you.
19 (Whereupon, the above-entitled matter went off the record 20 at 10:47 a.m. and resumed at 10:52 a.m.)
21 CHAIRMAN HANSON: All right. Thank you, everyone.
22 The meeting will recommence now with the external panel. Each panelist will 23 have about 8 minutes to present.
24
72 And we'll start with Reginald Mitchell, the Chief Financial 1
Officer and Enterprise Risk Management Secretariat for the United States 2
Agency for International Development.
3 Mr. Mitchell.
4 MR. MITCHELL: Yes. Good morning, Chairman and 5
Commissioners. Thank you for the invitation to participate in today's briefing.
6 It is indeed a pleasure to be back before the Commission with an opportunity 7
to share the U.S. Agency for International Development's journey in 8
developing our risk appetite statement.
9 As the CFO, I serve as agency Enterprise Risk Management 10 Secretariat, also referred to as the ERM Secretariat. And in this capacity, I 11 lead the agency's ERM governance program.
12 As some of you know, USAID is the federal government's 13 largest development agency working in partnership with international and 14 humanitarian organizations, and we operate in over 100 countries worldwide.
15 Our work requires our foreign service officers to work in remote and 16 sometimes dangerous environments, delivering assistance: everything from 17 food security programs, to fighting pandemics such as COVID-19.
18 As we develop our programming within a variety of country 19 contexts, it is imperative for us to actively identify and manage our enterprise 20 risk. Today I was asked to focus on the development, best practices, and 21 use of USAID's risk appetite statement in our agencys day-to-day decision-22 making.
23 But before I begin, I'd like to, I think it would be helpful for 24
73 me to share a few moments to go over our ERM overarching governance 1
framework which uses our risk appetite statement as a governing principle for 2
managing risk.
3 Next slide, please.
4 Key to the success of the ERM program is an effective risk 5
governance structure. With support and engagement from top leadership 6
that reaches stakeholders and every employee within the agency.
7 As you can see on the chart, our framework is comprised of 8
three levels, with the head of the agency at the top. What is important to note, 9
is the framework is designed so that information flows in both directions, up 10 and down the organization. I will start with the assessable unit, which we 11 define as the roles of an organizational unit of the agency, which includes our 12 missions overseas and independent offices, each with their own management 13 controls on risk and internal controls.
14 Next is the bureau level, with each bureau having its own 15 risk council, like the accessible units, and it's tasked with reviewing and 16 evaluating aggregated data collected by missions and offices. Each bureau 17 then determines which high-risk items will be forwarded to the next level in our 18 ERM governance framework.
19 Our three agency councils are:
20 First, we have the Risk Management Council which covers 21 programmatic risks identified by the bureau.
22 Second, we have the Senior Assessment Team, referred to 23 as SAT, where SAT is responsible for financial risk, and financial-related 24
74 issues identified through internal controls, financial compliance assessments, 1
and internal and external audits.
2 And, finally, we have the Executive Management Council for 3
Risk and Internal Controls. We call it the EMCRIC, which receives input and 4
recommendations from the Risk Management Council and the Senior 5
Assessment Team on risk identification and mitigation for leadership 6
consideration. The EMCRIC is chaired by our deputy administrator and the 7
council makes final recommendation on risk adoption and mitigation strategies 8
to the administrator.
9 Now let me go over our risk appetite statement.
10 Next slide, please.
11 As previously mentioned, USAID works in very challenging 12 environments and has vast experience managing risk. We are a highly 13 decentralized organization due to the geographical footprint of the agency and 14 the remote locations we operate within. We believe it was critically important 15 to document the agency leadership's expectations related to identifying and 16 managing enterprise risk.
17 The agency leadership provided broad-based guidance on 18 the amount and type of risk AID is willing to accept based on evaluations of 19 opportunities and threats at a high level and in key categories.
20 In addition, OMB Circular A-123 encourages the agency to 21 develop a risk appetite statement as a high-level document that employees 22 can make use of as they conduct their day-to-day activities. So, we began 23 with a need-based analysis. We thought it was essential to develop formal 24
75 agency guidance to help staff identify where it made sense to encourage risk 1
forward thinking, and to seize opportunities and where we needed a more risk-2 cautious approach.
3 We used a collaborative approach by establishing a core 4
cross-functional working group with expertise in program development, 5
operations, performance, and management. The group was overseen by 6
agency leadership. The core group was primarily responsible for developing 7
the seven categories we used to identify each type of risk.
8 We have in the agency our risk categories are:
9 programmatic, fiduciary, reputational, legal, security, human capital, and 10 information technology. Each category of risk was assigned an owner 11 responsible for evaluating their assigned risk category. The owners made 12 recommendations on the risk tolerance to the agency leadership for approval.
13 For example, I own the fiduciary risk category, and I 14 provided a recommendation that all financial activity has a low tolerance level.
15 However, I recommended a medium risk level be adopted when implementing 16 our funding through our overseas partners.
17 Once we had our statement developed, we began our road 18 show, and we briefed OMB and our Oversight Committee. Our statement 19 was well received by both OMB and the Oversight Committee. As a matter 20 of fact, as USAID was one of the early adopters of a risk appetite statement 21 in the federal government, several agencies have reached out to us and asked 22 that we share our development process.
23 As every organization is different and has a unique culture, 24
76 I encourage agencies to determine the culture that best fits their needs and 1
here are some best practices for your consideration.
2 In addition to the strategic revision of policy and procedures, 3
ERM requires significant change in agency culture and the fostering of internal 4
and external partnerships.
5 As an institution, avoid being too risk averse. At USAID the 6
risk appetite statement encourages us to be bolder in those areas where our 7
tolerance for risk is high, for example, when seeking out new implementing 8
development partnerships. But also remain vigilant in areas where risk 9
tolerance is low, such as cyber security, financial, sexual abuse, and 10 exploitation.
11 Next slide, please.
12 Our risk appetite statement is an integral component in our 13 risk posture. As mentioned earlier, the risk appetite statement includes those 14 categories, it also includes gradation of risk categories, that is high, medium, 15 and low, to explain how each risk may be used in specific situations.
16 I would now like to share what we believe our best practices 17 for socializing a risk appetite statement.
18 We operationalize the risk appetite statement by conducting 19 agency-wide training on conceptual components of the risk appetite and our 20 Enterprise Risk Management program. The administrator issues 21 communications to all employees reiterating their responsibility for using the 22 risk appetite statement when determining and managing risks in their daily 23 work, and to encourage staff at all levels to get comfortable with taking risks.
24
77 When developing a risk appetite statement, agencies may 1
want to assess what others have done when deciding on structure. For 2
example, we surveyed other federal government agencies, state and local 3
governments, and other international organizations. And we built on our 4
expert knowledge and consulted the General Accountability Office ERM 5
Playbook for applicability.
6 I would like to leave you with three thoughts on socializing a 7
risk appetite statement we found specific to our success of its adoption, and 8
that is:
9 Educate the staff on identifying managing risk; 10 Communicate to the staff a leadership commitment; 11 And encourage the staff to be smart risk-takers.
12 Next slide.
13 In closing, I believe our risk appetite statement equips our 14 staff with the capability of making better-informed decisions.
15 And, again, thank you for the opportunity to share USAID's 16 risk appetite journey. This concludes my statement to the Commission.
17 CHAIRMAN HANSON: Thank you, Mr. Mitchell. It's great 18 to have you back here at the NRC, even just for a little while this morning.
19 With that we'll move on to Ms. Teresa Gerton, President and 20 CEO of the National Academy of Public Administration.
21 Ms. Gerton.
22 MS. GERTON: Good morning, Commissioners, I'm Terry 23 Gerton, President of the National Academy of Public Administration. And I 24
78 am honored to represent the academy before you today.
1 Next slide, please.
2 Before I begin, I want to commend the breadth and progress 3
of the NRC's transformation efforts that I've heard about this morning. You 4
all are already adopting critical aspects of the Agile government principles and 5
demonstrated profound results.
6 In the interests of time, I will assume that you all are familiar 7
with the National Academy of Public Administration. It's one of only two 8
congressionally chartered national academies. And I'll jump right into the 9
discussion of Agile government that you all have requested.
10 Next slide, please.
11 In November of 2019, when the Academy announced our 12 12 grant challenges in public administration, we also established the Agile 13 Government Center as an enabler for the seminal concepts that would 14 underlie any success against the grand challenges agenda. We knew that 15 every major issue facing government would require an intergovernmental, 16 intersectoral, and interagency approach to development, and that would 17 require a new way of doing government business, would require more agility.
18 The Center itself is a partnership between the Academy and 19 the IBM Center for the Business of Government. Over the past 18 months it 20 has become a hub with the establishment of our Agile government network of 21 organizations and individuals who are exploring the implementation of Agile 22 principles in government programs around the world.
23 Next slide, please.
24
79 As you've already heard this morning, Agile is a new 1
management paradigm. Compared to the sequential waterfall project, Agile 2
is a very different way for projects to be accomplished. And if it's adapted for 3
broader use, it's a different way for governments to operate.
4 Building on and understanding of Agile principles for 5
software development and IT initiatives, and their incorporation by some 6
federal agencies into their IT projects 7
(connection difficulties) 8
.organizations, and federal agencies themselves are now 9
adopting and adapting the principles for their systems and applications.
10 This is driving changes in numerous federal IT software 11 areas, and it's now increasingly common for federal agencies to use Agile for 12 projects, especially for IT projects. However, it is much rarer for federal 13 agencies to use Agile as a way of managing their organizations.
14 Next slide, please.
15 Through the research in cases promulgated by our Agile 16 Government Center, the Academy has developed and shared 10 fundamental 17 principles of Agile government. I won't read these to you, but you will 18 recognize several, such as the empowering of staff members, or small teams 19 doing work in multiple short periods of time, a critical focus on iteration and 20 learning, and an early identification of risk is already operational within the 21 NRC.
22 Far more important is the mental picture these principles 23 create. Imagine a government organization at any level that practiced these 24
80 principles. It would be more focused on program outcomes and less on strict 1
compliance, with a deliberate approach to managing risk.
2 It would be open to engagement with partners across a 3
variety of boundaries, including with private and nonprofit sector partners, 4
without concern about who gets the credit.
5 It would be intent on learning quickly what works, measured 6
by both what its customers think and what the data says, and it would be open 7
to trying new solutions while accepting that failure often accompanies early 8
trials, and it would be an engaging place for its employees who are both sold 9
on the mission and empowered to seek creative solutions that encourage 10 mission accomplishment.
11 And with that compelling vision, we might ask ourselves why 12 don't we see more of this in the government?
13 Next slide, please.
14 You can already imagine some of the institutional stumbling 15 blocks to implement Agile approaches in government. For example, there 16 exists a general lack of knowledge and understanding about Agile. Many 17 people simply don't know what it is.
18 There's cultural and behavior barriers, like risk aversion, and 19 a perceived absence of permission to implement Agile principles.
20 And, third, there are real regulatory and procedural barriers 21 in contracting, hiring, oversight, and budgeting.
22 The Academy is currently developing a training module at 23 the Federal Executive Institute that will introduce federal leaders to the 24
81 concept of Agile government as a first step to addressing the knowledge and 1
culture challenges in implementing Agile. And we'll conduct our first class 2
next month.
3 Next slide, please.
4 The Academy's recent report on implementing Agile 5
principles in the federal government, which is linked on this slide, include 6
several recommendations to address these challenges more broadly.
7 First, we recommend that Agile should be a cornerstone of 8
the President's management agenda.
9 Second, it should be incorporated into existing cross-10 agency performance goals, especially the goal to improve customer 11 experience with federal services. And ensure that Agile management is used 12 whenever appropriate for organizations, programs, and projects.
13
- Third, we recommend that the General Services 14 Administration's Office of Government-wide Policy should assign or establish 15 an organizational unit designed to assist other departments and agencies with 16 their Agile management journey.
17
- And, fourth, we recommend that the President's 18 Management Council coordinate across its member agencies to develop and 19 facilitate the implementation of strategies for accomplishing Agile-related 20 goals.
21 Next slide.
22 Making these proposals for implementation of Agile across 23 the federal government does not preclude any agency from taking up the Agile 24
82 challenge itself. The Academy's report offers readiness factors, and 1
assessment tools for both organizations and individuals to determine how 2
ready they are to embark on an Agile government transformation. They also 3
document case studies of activities that have embraced agility in their projects, 4
programs and organizations.
5 The concept of Agile government is catching on around the 6
world, and we are excited to see the first embers of interest within the Biden 7
administration. And we look forward to continuing to support this effort.
8 I'll be happy to answer any questions that you have on this 9
topic. And I thank you for the opportunity to present it to you this morning.
10 CHAIRMAN HANSON: Thank you very much, Ms. Gerton.
11 Next we'll hear from Dr. Steven Simon, President and 12 Founder, Culture Change Consultants, Incorporated.
13 Dr. Simon.
14 I think you're on mute, Dr. Simon.
15 MR. SIMON: Thank you.
16 CHAIRMAN HANSON: There we go.
17 MR. SIMON: Even after a year of virtual communication. I 18 appreciate it.
19 Thank you, Commissioner and to the Commission for the 20 invitation to speak today.
21 So, some brief background so you know who I am and how 22 I came to this topic of safety culture -- of culture transformation, and why I'm 23 focusing specifically on safety culture.
24
83 I guess first is that I am proud to be a pioneer and, in fact, 1
the originator of the concept of safety culture. In 1983 I coined the term 2
"safety culture," and for the last 38 years have been working to develop 3
approaches that show culture is the most powerful force to drive safe behavior 4
and achieve breakthrough safe performance.
5 A second part of my background is that I have been a roll-6 up-your-sleeves culture change consultant, and have led transformation 7
efforts at organizations, large organizations such as General Motors, GE, and 8
numerous electric and gas utilities.
9 And I guess a third point of specific relevance is I am trained 10 as a psychologist, so I approach culture and safety from the people side, not 11 the technical.
12 So, one of the I think contributions I might be able to make 13 today is lessons from, you know, my career in working with organizations who 14 have attempted to change and in any case to successfully transform their 15 safety cultures.
16 One of the real critical success factors for any approach to 17 safety culture transformation -- this may seem obvious, but often it's not -- is 18 actually to focus on the culture. There is in the safety field, as we all know, a 19 default to go to programs. And I use a model to differentiate safety programs 20 from safety culture to keep leadership focused in on the target.
21 Here's how it works.
22 I'd like you to imagine a pot of stew on the stove. Think of 23 it as a safety stew. And the solid ingredients, the meat and potatoes, the 24
84 vegetables, those are the programs that we use and, frankly, whether it's in 1
safety or other areas. But if you're making a good stew, you get the best 2
ingredients you can and then you have to drop it in something. The solid 3
ingredients go in liquid, which is known as the broth.
4 And in this analogy, the broth is the culture and the 5
programs, the meat and vegetable, are the solid ingredients.
6 I think the point is that you can have really excellent safety 7
program ingredients, but if it's a rancid broth, it doesn't matter how good your 8
sirloin is. And this goes whether it's for risk smart or whether it's for incident 9
investigation, safety, job hazard analyses, and so forth.
10 In a broth or culture where there's good caring, good dignity, 11 psychological safety, freedom to speak up, trust, obviously this is a two-factor 12 theory, you get the most out of the program. The program and the culture 13 work together.
14 In those cultures where you've got mistrust, lack of concern, 15 focus on numbers over people, blame fixing, no management visibility and so 16 forth, it doesn't matter how good the sirloin is, your program is, if it's in a rancid 17 broth you don't get the benefits that you want.
18 So, I have found this model very useful in working with 19 operational leaders who typically think that safety is the job of the Safety 20 Department.
21 And in the compliance part, the development of programs 22 certainly belongs to the experts, but the culture, the broth obviously is set by 23 the marks of the leadership.
24
85 So, I want to share -- and if we can go to the Slide 4, the 1
safety culture maturity path -- I would like to, I would like to share a maturity 2
path that I think talks to -- and it's a few slides previous, if you don't mind.
3 One more. Thank you very much.
4 And this is a maturity path that I developed that identifies 5
different stages in the development of the safety culture. And it gives you a 6
bit of a metric for self-assessment on where any gaps might lie. As I look 7
back on the history of how the concept of safety culture has evolved in the last 8
almost four decades now, I do see five phases.
9 And I want to emphasize that this maturity model is not a 10 values model. Some models state your organization should proceed from a 11 reactive to a proactive culture, or beginning to a leading culture, or dependent 12 or independent, or an interdependent culture. And that's great. Those are 13 important values-based maturity paths.
14 But this much more, instead of providing a direction for what 15 kind of organization your -- sorry -- culture your organization should strive 16 towards has to do with how to get there. And the five levels each talk to what 17 I see as some of what might be missing or present in an organization's 18 development.
19 So, number one, back in the '80s, safety culture 1.0 was 20 establishing safety as a value. Obviously, NRC has had it from the very 21 beginning. Not every company, not every organization, many companies 22 created safety as a value for their annual reports. For many it was just lip 23 service. The key question for safety culture 1.0 and the values is, obviously, 24
86 whether the value is real. But it's also the safety vs. production value, which 1
I know you deal with all the time.
2 Safety culture 2.0 is about leadership. The shadow of the 3
leader impacts, and impacting behavior of employees.
4 In the '90s, just a bit of a historical maturity path, many 5
consulting groups started to focus on the shadow that leaders cast, training of 6
upper management, assessing development and so on. Only more recently 7
has leadership been defined not only as upper management, but leaders at 8
all levels which, of course, for a culture is critical.
9 Which leads to safety culture 3.0, which is grass roots 10 engagement.
11 So, I would offer an effective safety culture cannot be driven 12 just top down, but also needs to be driven from the bottom up as well. This 13 has been true, of course, for the compliance aspects of safety, but even more 14 true when it has to do with employees focusing on the cultural foundations of 15 injury prevention.
16 To kind of paraphrase a commonly expressed meme, I think 17 all culture is local and, yes, upper levels have to be -- of leadership have to be 18 committed to safety with an urgent intensity.
19 But within that overall organizational concept -- context, 20 what happens off shift, what happens when people are alone, I guess my 21 favorite definition of safety culture is what happens when no one's watching.
22 That's when the grass roots matter because they are the only ones that are 23 there. And if they don't have control of their culture then you get safety culture 24
87 4.0, which is work group norms and the inevitable drift that comes in the way 1
we do things around here.
2 We all have our policies and procedures. Safety culture 4.0 3
provides tools for the work group not only to look at its behavior, but also the 4
invisible norms of the -- you know, that influence that, that behavior.
5 I would offer that very few organizations have gone past 6
safety culture 1.0, 2.0, and 3.0. They've done the value. Some did 7
leadership. Even some prepared engagement. Very few actually looked at 8
the work group level when it comes to culture.
9 And that leads to safety culture 5.0, which I would, you 10 know, even a layperson through the decades looking at the NRC would say 11 you've been clear leaders in the term which is now called psychological safety.
12 But, obviously, you know, going back a long way has to do with the speak-up 13 culture, speaking up retaliation -- without retaliation. And that's something 14 that is starting to gain more encouragement in industry in general.
15 So, I think that the point of this maturity model is for 16 organizations to look at whether there are some areas that they are missing.
17 I think the traditional focus, as I said, is on leadership. It's not only leadership 18 but all levels.
19 Next slide, please.
20 I would also offer that engagement is not enough. And 21 we're all in favor of employee engagement. You know, we all love apple pie, 22 motherhood, the flag, and employee engagement. And employee 23 engagement is critical but it is not enough. There has to be also a focus on 24
88 the norms that drive employee behavior, and a focus on culture and not just 1
programs.
2 So, I think, finally -- next slide, please -- I would like to say 3
from my standpoint as someone who's not been involved with the theory but 4
with implementing change, you know, throughout my career, that the couple 5
of lessons I've learned the biggest are that if you have the luxury of choice, 6
implementing village by village is always better when it comes to culture, not 7
programs, than enterprise-wide or systemwide or, you know, I'll just say 8
systemwide, and in a single, even in a single plant, because each of the 9
subcultures is so different. And I guess my mantras for change with culture 10 work is it's different from program work. You go slow to go fast. You go 11 small to go big because you want it to stick in the end.
12 And I think with that my time is up. And I appreciate the 13 opportunity to share some of my lessons accumulated on safety culture 14 transformation. And from having watched a lot of Senate hearings, I yield my 15 time.
16 Thank you very much.
17 CHAIRMAN HANSON: Thank you, Dr. Simon.
18 Next we'll hear from Jeff Place, Executive Vice President for 19 Strategy at the Institute for Nuclear Power Operations.
20 Mr. Place.
21 MR. PLACE: Good morning, Chairman and Commissioners.
22 Thank you for the privilege and the opportunity to discuss how new technology 23 is leading to improvements in operational performance of our nuclear fleet.
24
89 I'm going to focus my remarks on two main topical areas.
1 First I'm going to provide a high level overview of some of the new 2
technologies that nuclear stations are implementing.
3 And then, second, I will discuss how INPO is using 4
technology to improve our support and oversight of the nuclear industry.
5 Next slide.
6 So, in the past few years we've seen an acceleration in the 7
use of new technology, driven primarily by the efforts to improve safety and 8
reliability, increase efficiency, and reduce costs. The list of improvements is 9
extensive. And I'm only going to be able to highlight just a few today. And 10 as such, the discussion that follows is not fully representative of all the efforts 11 and activities across the nuclear fleet.
12 Use of robotics has a long history in our industry, but the 13 advancement in their capabilities has led to many more applications. In 14 general, robots greatly improve industrial and radiological safety, decrease the 15 time to conduct the work, and increase the frequency of monitoring.
16 Some examples would include submersibles that can now 17 support underwater intake system inspections without putting humans in 18 harm's way. And the use of drones have drastically increased, both inside 19 and outside the plant, including inside systems and also inside of containment 20 while at power.
21 The use of digital sensors of all types is now widespread.
22 Routing this data through software platforms has given engineers, operators, 23 and managers key information to identify and sometimes predict equipment 24
90 degradation before failure occurs.
1 Artificial intelligence and machine learning are now 2
beginning to be used to automate some knowledge work and tasks, 3
specifically focused on asset management and equipment reliability. For 4
example, some utilities are now using software applications to automate 5
corrective action program and work order screening, to automate parts 6
purchasing, intra-equipment anomaly detection. In these cases, the software 7
is making initial classifications or decisions and the plant staff review and 8
approve the classifications prior to taking action.
9 Training is also being enhanced by using virtual reality. We 10 are just really beginning to tap into the use of this technology to enhance 11 industrial safety, operations, and technical skills training.
12 To touch on just a few other uses of new technology, we're 13 seeing the use of digital platforms to build electronic procedures and work 14 packages. Some of these have embedded video and human error prevention 15 tools. Digital 3-D printers are being used to build training mock-ups and 16 produce replacement parts.
17 Overall, we're seeing a rapid adoption and expansion of new 18 technologies. The industry is just beginning to learn from each other and 19 understand all the applications in which technology can improve safety, 20 reliability, and cost competitiveness.
21 Next slide.
22 Like the industry we support, INPO is using new technology 23 to improve our work. Some of this work has been evolving over years, and 24
91 some of it has been accelerated because of the pandemic. And INPO leads 1
the National Academy for Nuclear Training. And through the Academy, we 2
conduct leadership courses and seminars for nuclear professionals from the 3
first line supervisor up to the board of directors.
4 When the pandemic hit, we were initially unable to carry out 5
this training, but our staff quickly began developing a virtual training platform 6
we refer to as VOLT, or Virtual Online Leadership Training. We began with 7
a few specific training modules, but we eventually fully converted some of our 8
one-week seminars into the virtual setting. The industry response has been 9
outstanding, and we've actually been able to reach individuals we would have 10 not otherwise had we done it just in person in the building in Atlanta.
11 While we are planning to return to face-to-face classrooms 12 later this year, we will continue to offer some of our course content virtually.
13 A longstanding role of INPOs has been to facilitate the 14 sharing of operating experience across the industry. Recently we began 15 producing podcasts to share some of this information. Again, industry 16 feedback's been very positive. People seem to like the more personal 17 approach of hearing people that were involved in the operating experience 18 giving their firsthand accounts.
19 We are currently working on other delivery methods to make 20 the operating experience more accessible in different formats and on different 21 platforms.
22 I'm going to now shift and discuss some fundamental 23 changes in ways we are monitoring and shaping industry performance.
24
92 Next slide, please.
1 The industry and INPO have long used indicators to 2
compare performance, set targets, and drive performance improvements.
3 Today, INPO collects between 700 and 800 datapoints and operating 4
experience records every month. From this data, approximately 2,200 5
separate indicators are created, and about 200 of these are displayed on our 6
member website.
7 Also, over the past four decades INPO has put teams of 8
nuclear professionals onto the nuclear sites at about a two-year periodicity to 9
conduct performance evaluations and peer reviews. An assessment of 10 performance is provided after each evaluation.
11 While we have other operations and activities to monitor 12 performance, we weren't satisfied with our ability to understand performance 13 in between this two-year evaluation period, so we began developing a plant 14 performance indicator that would give us a more current view on at least a 15 quarterly basis.
16 To help us accomplish this view, we used neural network 17 modeling techniques to find patterns between the indicators that correlate to 18 performance assessment scores over time. Derived models are designed to 19 think like our staff as they go about implementing the evaluation process. We 20 are on our third iteration of this model.
21 The original model was a simple linear model. And the 22 current one that utilizes neural modeling techniques is 96 percent accurate, 23 with precision of plus or minus 3.5 points.
24
93 We further use these techniques to create neural models for 1
all 15 individual functional and cross-functional areas that INPO assesses as 2
part of our operations. The accuracy of these area models range anywhere 3
from 76 to 95 percent, with a precision of plus or minus 5 points.
4 None of our indicators or models fully substitute for onsite 5
evaluations, however, we do leverage the information from the models to 6
performance base our operations. For example, the composition of our 7
teams are adjusted in part based on this information. Our base team for an 8
INPO evaluation today is about six professionals. And then we augment that 9
team based on our review of performance through, in part, these models.
10 Our teams now typically range between 6 and 12 team 11 members, compared to a previous team size of 25 to 30. The result of our 12 evaluations have not suffered based on self-assessments that we have 13 conducted and feedback from the industry.
14 We are also using machine learning to advance new 15 models, some of which we are anticipating will be more predictive. A recent 16 area of focus for us and for the industry is understanding and measuring 17 sustainability of performance improvement. Currently, our staff draw 18 conclusions on sustainability through manual review of data, interviews, and 19 field observations.
20 As the staff collaborates and assigns numerical 21 assessments on their work platform, artificial intelligence and machine 22 learning software is running in the background, trying to replicate the results 23 by developing correlations between our data and indicators that will give 24
94 similar results. Our goal is to be able to be better at automating our view of 1
sustainability, minimizing subjectivity, and reducing errors from new staff.
2 Next slide, please.
3 In wrapping up, I would say that the tools we've developed 4
to date give us a more current, accurate, and complete picture of performance, 5
and at a frequency that we did not have in the past. Using these tools, we've 6
been able to redirect our resources where the industry needs them most. It's 7
also been a catalyst for greater industry comparison and supports the 8
industry's continuous learning culture.
9 However, we have more work to do to make better use of 10 data science and advanced analytical techniques in support of our, INPO's, 11 transition to a new operating model over the next year. We've seen a nearly 12 constant slope of improvement in the industry over the past 8 years. By 13 nearly all metrics, the industry is performing at near its highest levels ever.
14 While there are many factors for this improvement, the use of new technology 15 has certainly played an important part in increased operational performance 16 and efficiency gains.
17 The graph in front of you today is actually a simple roll-up of 18 the average of the neural plant performance indicator for the entire U.S. fleet.
19 The industry as a whole passed into our exemplary range in early 2017 and 20 has continued to improve.
21 Again, thank you for the opportunity to speak with you all 22 this morning. And I will be happy to answer any questions you may have.
23 CHAIRMAN HANSON: Thank you, Mr. Place.
24
95 Commissioner Caputo.
1 COMMISSIONER CAPUTO: Good morning.
2 Mr. Place, I will stay with you for my first question.
3 So, for those not familiar with INPO, I'm just going to recap 4
their mission: to promote the highest levels of safety and reliability, to promote 5
excellence in the operation of commercial nuclear power plants.
6 And I'd like to ask you a little bit about INPO's indicator 7
index, which I think you've also expanded on that by talking this morning about 8
the Neural Performance Indicator Index, and stressed how the industry's 9
safety performance has improved over time. And so, you know, among those 10 many indicators that INPO tracks includes reactor performance, safety system 11 performance, occupational safety, chemistry performance, and radiation 12 exposure.
13 At our Commission meeting a couple weeks ago we 14 discussed the declining trend of green findings and findings or are low safety 15 significance. And while the NRC staff acknowledged the industry's improved 16 safety performance, they attributed the declining trend in findings to staff's 17 increased focus on risk significance.
18 So, my question is this: if we are a modern, risk-informed 19 agency, shouldn't we expect to see a decline in findings if industry safety 20 performance has improved? After all, if actual safety performance has 21 improved but it's not reflected in the number of findings, that would suggest 22 we're not risk-informed.
23 So, wouldn't the decline in findings be a logical outcome of 24
96 the combination of the agency's risk-informing and the industry's improved 1
safety performance?
2 MR. PLACE: Yes, Commissioner, thank you for the 3
question.
4 I don't know if I can speak directly to, you know, the reasons 5
why the number of green findings would be reduced, but I can share with you 6
what we are seeing through the way we look at performance and how that 7
may correlate to issues that would be of regulatory scrutiny.
8 As you know, we collect these operating experiences, and 9
we call them events. And not all of them are of great significance, but we do 10 characterize them. And we have different categories:
11 We have significant events, consequential events.
12 We've seen significant events in the last several years 13 almost entirely eliminated. We just don't see them.
14 Consequential events have been reduced by 50 percent 15 from 2010 to 2020. Fuel reliability is amongst the best -- well, it is the best 16 ever. SCRAMs are not complicated anymore.
17 And through our own evaluation process we're not 18 necessarily seeing a reduction in areas for improvement, which is what we like 19 to help drive, identify and drive closure of gaps we see in the industry. But 20 what we are seeing is that they're written at a different threshold. We've 21 lowered the threshold. So we're seeing many more of our gaps or our areas 22 for improvement written at the behavior level before real consequence has 23 occurred.
24
97 And so I think, you know, you roll those types of things up 1
and I think that it's possible that that would also mean that there's probably 2
more things -- less things that are happening in the power plant that would be 3
viewed under regulatory scrutiny that may result in a green finding or other 4
findings.
5 COMMISSIONER CAPUTO: Okay. Thank you very much.
6 Ms. Gerton, I have a question for you.
7 Getting staff to set and track outcome-based metrics is a 8
challenge. There is a constant temptation to set the bar low, to set the bar at 9
something that you know will be accomplished, thereby guaranteeing success 10 when, in reality it's ensuring business as usual under the aura of a shiny new 11 program.
12 So, whether it's our current evidence-based policymaking 13 effort currently underway, or the Agile method that you discussed, the ultimate 14 question is how do we get employees to embrace and use these strategies to 15 drive performance improvement rather than simply going through the 16 motions?
17 MS. GERTON: Thank you, Commissioner, that's a great 18 question.
19 One of the key aspects for setting good outcome measures 20 is good data collection. And, frankly, you don't always get the outcome 21 measures set exactly right on the first time. But as you begin to collect data 22 about your processes and your outcomes, you begin to see the trends and 23 you begin to see the gaps where performance can be improved.
24
98 So, it's a leadership challenge as well as a data collection 1
effort. But as you begin to monitor, even if you set those first metrics and 2
they give you 100 percent mission accomplishment, the next thing you need 3
to think about is, well, how do we, how do we extend ourselves and how do 4
we take a bit more risk to get better performance? And as you begin to do 5
that and you begin to see performance improvements over time, you begin to 6
tighten those metrics so that they continue to push you to improve outcomes.
7 And as the staff sees that and as they begin to perceive the 8
improved performance, and likely what goes along with that is improved 9
customer satisfaction or improved mission delivery, they begin to buy into the 10 process.
11 So, if you're just new to setting performance metrics, it's not 12 necessarily a bad thing to set them and understand how your data tracks and 13 informs your performance and how you measure it. But over time, with the 14 staff and the leadership support you begin to collect better data, you have a 15 better understanding of your processes and how they impact your outcomes, 16 and you could begin to fine tune those data so that they continue to drive 17 increased performance.
18 COMMISSIONER CAPUTO: All right. Thank you very 19 much.
20 One last question for Mr. Mitchell.
21 You stress the need to manage risk holistically. And you 22 listed fiduciary risk as a category for enterprise risk management. Can I just 23 get you to clarify, is that strictly limited to, you know, internal controls and 24
99 auditing? Or do you also look at risk in terms of budget formulation and 1
execution?
2 MR. MITCHELL: Thank you, Commissioner, for the 3
question.
4 Yes, it's the latter. We do look at it holistically, not just the 5
execution side of the house, but budgeting and execution. And when we look 6
at our partnerships that we developed in implementing our programs, that's a 7
fiduciary risk as well because of the actors that we come across and oversee 8
your arena.
9 COMMISSIONER CAPUTO: So, I mentioned this a little bit 10 earlier, but in 2016 we collected 320 million in fees for licensing and oversight 11 work. In 2021 this workload is down 45 percent. That means our work went 12 from being 31 percent of our total budget to only being 21 percent of our total 13 budget.
14 How important is data-driven decision making to anticipating 15 and managing risk in budget formulation?
16 MR. MITCHELL: From my perspective it's very important.
17 When you look at you're setting a budget, you're setting a budget to complete 18 a certain program with certain objectives. And those objectives will always 19 be impacted by the funding that you are able to obtain to meet the stated 20 goals.
21 So, again, if you don't factor in the risk that you are taking 22 on from a holistic perspective of the environment that you operate within, 23 understand the external risk, meaning that stakeholders, which is both of ours, 24
100 Congress and OMB, then you do, you risk, not being able to get the funding 1
you need to carry out your program because you didn't take adequate notice 2
of what are the risks that you have to be able to navigate in order to get the 3
funding that you need to do your work.
4 COMMISSIONER CAPUTO: Thank you very much for 5
joining us today. That's it for my questions.
6 CHAIRMAN HANSON: Commissioner Wright.
7 COMMISSIONER WRIGHT: Thank you very much, Mr.
8 Chairman.
9 Reggie, I'm going to stay with you for a minute. Thank you 10 for your discussion and your comments here recently here with Commissioner 11 Caputo on best practices and, you know, when you're looking at risk appetites.
12 So, you mentioned the need to foster partnerships, both 13 externally and internally. And I'm trying to understand that a little bit more.
14 I am familiar with USAID, and have worked with them when 15 I was a member of the state economic regulators through NARUC, the 16 National Association of Regulatory Utility Commissioners. So I'm familiar 17 with what you do.
18 But can you tell me, talk to me a little bit more about some 19 of the external, I guess, partnerships that you fostered and how those 20 partnerships have impacted your practices in this area?
21 MR. MITCHELL: Certainly. Thank you for the question, 22 Commissioner.
23 A couple of the external partnerships that I'm alluding to is 24
101 like the U.N. We do a number of partnerships with the U.N. on a variety of 1
different programs that we have like interests in.
2 We also partner with our NGOs and other implementing 3
partners on getting perspectives from them on risk and what they're involved 4
with. As you probably know, Commissioner, most of our work is performed 5
by implementing partners, whether that's governmental, whether private 6
sector, other public entities such as the U.N.
7 And when we went through our process of developing our 8
risk profile we solicited their perspective from where they sit and tried to 9
incorporate that into our thinking on, you know, the risks in these different 10 perspective areas so that we could pass that on down to the staff level.
11 But as relates to the external stakeholders, that's what I was 12 alluding: primarily Congress, OMB, the U.N., and others.
13 COMMISSIONER WRIGHT: All right, thank you.
14 So, the NRC's reputation as the gold standard for nuclear 15 safety, that's at the heart of who we are and how we do our work. And our 16 reputation as a safety regulator is sometimes questioned by members of the 17 public and other stakeholders when we change our processes and procedures 18 and, you know, because there's a fear that we may be reducing safety or 19 something.
20 So, can you talk to me about how, if at all, reputational risk 21 plays into your risk appetite model, and what advice you'd give to the NRC on 22 this point?
23 MR. MITCHELL: Certainly. Again, I guess I spent about 26 24
102 years at the Nuclear Regulatory Commission, so I'm somewhat present, 1
although it's dated now. But I am familiar with the reputational challenges 2
that we had early on as well, and then how we got to where you're at today.
3 When it comes to USAID, we deal with a lot of actors.
4 Okay? And some of these actors are not good actors. Okay? And but we 5
may not know that until after the fact. So, we have to deal with sexual 6
exploitation. We have to deal with fraudulent claims. We have to deal with 7
strategies where some of our workers are targeted and sought after by these 8
bad actors.
9 So, when you're operating in the international development 10 area, there's a lot of money, as you well know, coming through USAID and 11 being put, placed into implementing partners and governments, the host 12 governments that we work with. And we constantly have to, you know, put 13 up the barriers to make sure that we have the internal controls and other 14 mitigating, you know, strategies in place so that we are able to catch any bad 15 actors, you know, early on.
16 And when we don't, it hits the Washington Post. And 17 constantly that's one of the things I always tell my administrator: my job is to 18 keep you out of the Washington Post on the financial side of the house.
19 (Laughter.)
20 COMMISSIONER WRIGHT: Gotcha. Thank you.
21 So, Ms. Gerton, I'm going to come to you. So, thank you 22 for your presentation and for your work in this area.
23 You know, on the first panel we heard about the application 24
103 of Agile to our rulemaking process. And on one of your slides you listed some 1
implementation issues and challenges with applying Agile. As we, as the 2
NRC starts to use this process more broadly, what do you see as key actions 3
we could take to avoid those challenges?
4 MS. GERTON: Well, thank you, Commissioner. And I was 5
really excited to hear about the work in Agile regulation. And the Academy's 6
getting ready to start a government-wide project this fall that will look at better 7
ways to apply Agile to the regulatory process because we know that it is very 8
challenging to keep up with changes in, especially in the scientific 9
environment.
10 So, as you're thinking about changing your internal process, 11 especially in the rulemaking, a couple of things you want to keep in mind.
12 Advertising this effort to a broader staff than just those who are engaged in 13 the rulemaking transformation is important to building agency-wide 14 consensus.
15 And I think it gets to your earlier point about reputation.
16 One of the key principles of Agile is customer experience.
17 And so, engaging your stakeholders to communicate how you're working this, 18 being very transparent in the results of the rulemaking process changes 19 themselves, taking small bites at it so you're demonstrating progress in a very 20 positive way, and then continuing to demonstrate through the outcome 21 measures that we talked about earlier that you are in fact improving safety, 22 and improving operations, rather than introducing risk into the system. All of 23 those key pieces of the Agile framework I think fit exactly what you're trying to 24
104 do here in the Agile regulatory transformation:
1 Engaging you staff so that they broadly understand what 2
you're taking on; engaging your customers or stakeholders so that they 3
understand you're taking small bites at the process so that the changes can 4
be easily understood, and that the implications of those changes can be 5
understood; and then being very transparent in the impact so that you continue 6
to buy into the process, demonstrate change and over time you build a pretty 7
comprehensive transformation into the process. It sounds like you're already 8
well on your way.
9 COMMISSIONER WRIGHT: Thank you so much for your 10 answer.
11 So Jeff, I'm going to come to you in some of the time I've got 12 left. And I want to just, I want to thank Commissioner Caputo for her 13 questions to you and comments.
14 Nobody wants an unsafe plant. And as licensees out there 15 and nuclear operators, you know, in the industry it's important for them to be 16 safe, and reliable, and up and running. And so the things that you're doing 17 and the way that INPO is peer reviewing, it's been, it's been interesting to 18 watch over the last, you know, 8 or 10 years just how things have moved up 19 the scale, as you showed.
20 And you're trying to do a lot of things, right? I mean, you're 21 trying to do a lot of new technology and things that maybe some of those 22 things are going to require some type of regulatory hurdle to be jumped.
23 So, in those -- I'm going to go to just kind of combine two 24
105 questions, if I could.
1 Are there any other applications of advanced analytics that 2
the industry is considering? And do you see any potential technological or 3
regulatory hurdles to more broadly adopt some of these technologies?
4 CHAIRMAN HANSON: Sorry, Mr. Place, I think you're on 5
mute.
6 MR. PLACE: I beg your pardon.
7 Yes, Commissioner, thank you for the question. I think as 8
an industry we really are at the beginning of just being able to use advanced 9
analytics. I know there's a number of projects right now that our utilities are 10 working with national labs on other new ways to digest the data.
11 I mean, as we have digital sensors that are beginning to be 12 put in the plant, there's discussion about making decisions and making 13 decision trees based off of the vibrations and temperatures and flows that 14 these sensors now, you know, you used to be able to take an individual, an 15 operator to go out into the plant and take these readings. And now even 16 analog gauges are being able to be digitized and then almost on real-time 17 basis understand what's going on in the plant.
18 So, there's discussion about robotics, to be able to even 19 maybe do operator rounds.
20 So, I definitely would envision that there's going to be many 21 discussions to have to have with regulators.
22 Do I know if there's any specific hurdles right now? I don't.
23 I think the regulatory framework is there for those discussions to be had. I 24
106 think they're just going to require very open, transparent, and good 1
conversations with the regulators to get to an efficient decision point so things, 2
so decisions can be made and improvements being put into the plants.
3 COMMISSIONER WRIGHT: Okay. Thank you so much, 4
Mr. Chairman. My time is up.
5 CHAIRMAN HANSON: Commissioner Baran.
6 Oh, sorry, it's my turn next. I was jumping ahead.
7 Skipped myself. Sorry.
8 Ms. Gerton, let me start with you.
9 I was introduced to this idea of agility in government a 10 couple of years ago in my interactions with the National Nuclear Security 11 Administration and their efforts to develop what they were calling at the time 12 a more agile and responsive stockpile management and development system 13 for our nuclear weapons complex.
14 And the way they wanted to do that was to create a program 15 in which they were exercising certain capabilities within the weapons complex 16 on a regular basis so that, if needed, that is, in order to respond to 17 technological, or geopolitical, or policy contingencies, that they could be able 18 to respond and implement changes in a relatively rapid fashion. Right? This 19 is another industry that's focused on safety, has long time horizons, extremely 20 expensive, all things that we could also say potentially about the nuclear 21 industry.
22 But what was striking to me at the time was that this what 23 they called the Stockpile Responsiveness Program was in addition to the other 24
107 things that they were doing. That is, it was an added cost.
1 And I'm wondering, in your case studies around Agile 2
government are you seeing this as a -- are you seeing Agile efforts, or efforts 3
to become more agile as it were, as adds? And how are agencies kind of 4
financing or budgeting for these things?
5 Are they doing it within existing capabilities? Or are they 6
special initiatives that require additional budgets, or, or what?
7 MS. GERTON: Thank you, Commissioner. That's a great 8
question, and you may know that the Academy just recently completed a five-9 year congressionally-directed assessment of NNSA and its various programs.
10 And so we're very familiar with the program that you mentioned.
11 But your broader question is does it cost more to do Agile, I 12 think.
13 CHAIRMAN HANSON: Yes.
14 MS. GERTON: And, in truth, what we see in the early 15 efforts is Agile is often a response to crisis, and you can see that in some of 16 the programs that the U.S. Digital Service has been called in to fix. And so 17 the question about cost is relative to what? Is it more expensive relative to a 18 system failure? Is it more expensive relative to a crisis in service delivery?
19 And so I would say, in those cases, it's really not. It's taking 20 the assets that you currently have and thinking about a new way to solve a 21 problem. So you're using people, you're using the tools that you have, and 22 you may be developing new tools. A lot of times we see the need to develop 23 a new IT system as the camel's nose under the tent in Agile implementation 24
108 because those new IT systems, which are typically focused on better service 1
delivery and better customer satisfaction or more security, drive with them 2
changes in process. And that begins to get people oriented to a more 3
iterative form of project improvement.
4 And so, over time, certainly the Agile processes come to 5
replace and be more efficient and productive than your old processes, but 6
there may be a period of time initially where you are incurring cost because 7
you are bringing in a new capability, often through IT systems, to the challenge 8
at hand or you're repairing or fixing a crisis where new tools and investment 9
are necessary. But as you begin to build those Agile intuitions and Agile 10 processes within your organization, you should, in fact, over time actually find 11 that you're reducing cost because you're delivering better service, your 12 customers are happier, and your people are more in a mode of constantly 13 improving using the resources that they have to deliver services better.
14 And so I guess I would say there may be a period of time 15 where you may see some increased costs in implementation, over time it 16 should streamline your operations and actually improve your cost delivery 17 model.
18 CHAIRMAN HANSON: Thank you. That's super 19 interesting. In fact, you kind of anticipated my next question, and you may 20 have answered it already but I'll ask it anyway. And that was kind of, and you 21 touched on this, and what's the role of IT or IT investments in the overall ability 22 of agencies to adopt an Agile framework? And are there elements of IT that 23 you think are most important? So, for instance, knowledge management 24
109 and/or data warehousing or mobile technology or collaboration tools or 1
advanced computing, et cetera. Kind of what are you seeing out there in 2
other agencies?
3 MS. GERTON: Another great question. I think one of the 4
places where we see IT driving Agile practices immediately, and you can see 5
it over the past 18 months in response to the pandemic, is as organizations 6
focus on their customer, we might think of the IRS and how they have 7
delivered a massive increase in stimulus checks to individuals and 8
corporations. You might think of how DMVs are saying, well, we can't do 9
driver testing and we can't renew driver's licenses in person, so what are we 10 going to do? As they focus on better customer service, whether that is an 11 improved website, whether that's improved data collection so that there's a 12 single-user sign-on. There are a variety of ways. But that kind of IT 13 implementation where it's customer facing has been the most powerful in 14 driving Agile behavior because it changes how organizations think and it 15 changes how quickly they feel like they need to respond to changes in 16 customer demand.
17 I think the second most important piece of IT revolves 18 around data collection, data warehousing, and data analytics. You can't do 19 enterprise risk management without a really solid data system and data 20 management process. You can't do it without good data analytics so that 21 you're collecting the right data and using it to track your trend.
22 So as organizations are -- sometimes, it's very different to 23 implement an Agile culture change from the front-end, right? Inviting people 24
110 through the door to Agile doesn't necessarily motivate them. But going at it 1
from the back-end, which is putting in IT systems that are addressing 2
operational concerns and improving mission accomplishments can often 3
begin to inspire that kind of Agile behavior where a little ways down the road 4
a leader can't say, well, you weren't really interested in Agile, you know, on 5
the surface, but look at what we've done already in how Agile is improving how 6
we deliver our services and meet our customers' needs. Now let's take it to 7
the next level and be very intentional about our next and broader 8
implementation of Agile.
9 So those would be the two places in IT that I would say are 10 really driving change: customer focus and data warehousing and analytics.
11 CHAIRMAN HANSON: Thank you very much. And I 12 think, once again, you've kind of anticipated my next question. But my 13 question, actually, for Mr. Place. And I was very interested in your 14 presentation about technology adoption. We've gone through this, I think, 15 during the public health emergency, as well in the NRC, where we had, you 16 know, made some investments, and I think the public health emergency in a 17 lot of ways accelerated the leveraging of those adoptions in a serious way, 18 everything from mobile computing and collaboration tools to, actually, like the 19 podcast that Alysia Bone does. You were talking about podcasts as a great 20 way to transmit culture and knowledge throughout an organization. I think 21 we've found the same thing here.
22 But it does, for me, kind of raise the question of do you have 23 suggestions for how an agency like NRC can systematically evaluate and 24
111 potentially invest in and/or potentially adopt and continue down the path of 1
innovation in this area without kind of the pandemic prompt, if you will.
2 You're on mute again, Mr. Place.
3 MR. PLACE: I apologize. Chairman, we started an 4
innovation group, actually, several years ago, and that group, which is made 5
up of individuals, we've made purposeful decisions to run every one of our 6
operations through that innovation group. So all of our major operations, we 7
put a strategy in place, and we don't see that ending. So we've undertaken 8
that. We're in the process of developing a new operating model beginning at 9
the end of this year. We're putting a new ten-year strategy in place.
10 All of those things are driving us to continue to go back to 11 this innovation group. We don't have to use all pieces and parts of the 12 innovation. That's just one of the things we've had to do. You know, Dr.
13 Simon talked about culture, and we've had to adjust our culture because, you 14 know, we're very conservative, like the nuclear industry is in general. And at 15 first, everybody thought, well, everything had to go through all five parts of the 16 innovation process. Well, no, it doesn't based on what it is you're looking to 17 change, you know, how far you want to make those changes.
18 So that's very much part of how we do business now. And 19 then we engage the industry, as you know. We are the self-regulation arm, I 20 guess, or group for the industry, so we engage them along every one of these 21 changes to make sure that we are not taking on more risk than they're willing 22 for us to take. Also, in some cases, they push us to take risks, and we may 23 have to go back and forth and collaborate, and we make a final decision that, 24
112 no, we're not going to make that change.
1 But it's really constantly, through our strategy, pushing 2
things back through that innovation process.
3 CHAIRMAN HANSON: Great. Thank you. Now 4
Commission Baran.
5 COMMISSIONER BARAN: Well, thank you all for joining 6
us and for sharing your thoughts. I think it's been a good discussion.
7 I have a couple of big-picture questions that I'd like to pose 8
to the whole panel and invite anyone who has thoughts to weigh in. The first 9
is about innovation. Obviously, innovation is a good thing and we want to 10 foster a culture of innovation at the agency so that we're looking for and open 11 to better ways of doing our work. Of course, some balance with stability is 12 necessary. I think Margie alluded to this earlier in her comments. If the staff 13 is constantly in a state of churn or continuous change, it can be hard for them 14 to keep their footing and remain focused. And if our approaches and 15 expectations are constantly in flux, there could be a lack of predictability and 16 stability for regulated entities and other interested stakeholders.
17 Does anyone on the panel have thoughts about how to 18 strike this balance between encouraging innovation while maintaining a 19 reasonable level of stability for the NRC staff and external stakeholders?
20 MS. GERTON: I'll take a first crack, if I could.
21 COMMISSIONER BARAN: Please.
22 MS. GERTON: One of the things that we strongly suggest 23 that agencies who are embarking on any kind of Agile project do as a first step 24
113 is engage their oversight community. You all are an oversight and regulatory 1
agency yourself, but you have inspectors general, you have auditors, you have 2
a variety of folks in the community who are interested in oversight of your 3
operation. And we all know that oversight agencies are heavily invested in 4
stability and status quo.
5 So one of the ways to strike this balance is to engage your 6
oversight community with your transformation team or your Agile project team 7
to make sure that, as you are moving toward Agile, the oversight community 8
is there to go along with you and say we can push that far, but we're not 9
comfortable going farther than that, that there are some boundaries that the 10 oversight community may not be ready to breach in a first or second or even 11 third round of Agile transformation.
12 When you do that, that also helps address the concern on 13 your staff that you might be pushing too much transformation too quickly. If 14 the oversight community is along for the ride with you, that can oftentimes 15 allay some fears amongst the community that would prefer more stability 16 while, at the same time, you're doing the iterative learning and transformation 17 projects as you go.
18 So that's one method that we have identified about how to 19 manage that tension between innovation and stability, both within your 20 workforce and in your impacted community.
21 COMMISSIONER BARAN: Thank you very much. Any 22 other thoughts on that? I know it's a big --
23 MR. PLACE: Commissioner Baran, I would just add on to 24
114 what Ms. Gerton said. I think the advisory board or advisory groups, we put 1
those in place and it's a mixture of internal and external people. I think that 2
helps.
3 One of the things that we've begun or we're beginning to do 4
in the industry, the licensee does quite a bit, especially when they're in high 5
levels of change, they use pulse surveys. They don't wait until an annual 6
survey to understand what's going on in the organization. And so much more 7
frequent pulse surveys, sometimes even weekly, dependent on the rate of 8
change and what's going on can help give you a picture if it's too much.
9 COMMISSIONER BARAN: Great.
10 MR. SIMON: And, Commissioner, if I may, I would jump in 11 also just adding on. It's axiomatic but it's worth restating anyway that this sort 12 of challenge for whether it's balancing out innovation and people's resistance 13 to change or any other part, any other focus of culture change, to the degree 14 that you can establish grassroots teams all across the system that are 15 engaged in driving it, it's obviously going to go better and, often, that's the first 16 thing that goes. Just wanted to mention that.
17 COMMISSIONER BARAN: Great. Thanks. Another 18 high-level question I'd be interested in your thoughts about is about how talk 19 about risk and risk appetite. Because NRC is a public health and safety 20 regulator, we talk about risk in the context of radiological risk every day. Our 21 core mission is to keep those risks very low.
22 The Be SMART risk initiative, on the other hand, is more 23 focused on how we consider all kinds of risks in our day-to-day decision-24
115 making on corporate support projects or hiring or legal strategies or 1
rulemaking approaches. In that context, we want to be open to trying new 2
things and maybe taking a chance on a new process or approach that may be 3
an improvement over how we've traditionally done things.
4 Do any of you have any advice on how we can avoid 5
conflating these concepts so that we're clear that the agency is not interested 6
in accepting increased radiological risk on behalf of the American public.
7 Maybe Dr. Simon is the place to start on this question but, others, I'm 8
interested in your thoughts, too.
9 MR. SIMON: Let me go second on this, if I may.
10 COMMISSIONER BARAN: Fair enough.
11 (Laughter.)
12 COMMISSIONER BARAN: Of course, that requires 13 someone else to go first.
14 MR. MITCHELL: Let me just jump in, Commissioner.
15 Thank you for the question. Yes, I think it's one of, you know, laying down a 16 framework that clearly stipulates, you know, when you're looking at risk and 17 how you're looking at that risk. You're absolutely correct. I don't think 18 anyone on this panel or on this call wants the NRC to relax or take 19 unnecessary risks with nuclear power. That's a given. But when you look at 20 the whole aspect of the other part that makes up the entity NRC, then you 21 have to bring it in and conduct findings with respect to when you're looking at 22 the socialization and you've got four different generations working in the 23 workforce and you've got some generations that want to be out there and 24
116 doing new stuff and you got generations such as myself that, you know, kind 1
of slow on the uptake.
2 So my perspective is, you know, by clearly setting up a 3
framework to understand that, when you're talking about technical risk in and 4
of itself and the analysis and all the other aspects that you deal with in the risk 5
of a nuclear power plant, you know, that's one form of risk and that's a 6
technical risk that you're going to work and a health risk. But then when you 7
deal about the agency itself and is it open, does it allow collaboration, does it 8
include inclusion with staff up and down the organization.
9 COMMISSIONER BARAN: Thanks.
10 MR. SIMON: Yes. And the only thing I think I would add 11 to that is, look, I think most culture change efforts fail because they're too big, 12 they're too broad. And everybody wants to look at the culture and identify ten 13 things that need to change and, you know, go to work on it.
14 The focus, I often think putting something in the bullseye, 15 one part of the culture, and then working it up and down and across. And I 16 don't, frankly, think it makes too much difference if it's an agency change or if 17 it's a plant change. You know, the principles are the same. It just doesn't 18 have to do with the procedures and the tasks and the models and the 19 programs. It has to do only with the people, and it has to do with engaging 20 everybody, you know, from bottom-up to top-down.
21 But most important, it has to do with workgroup norms.
22 And this, to me, I work in a couple of nuclear plants, you know, safety in plants 23 is so much better than almost anywhere else in industry and the attention, the 24
117 focus, the understanding of culture, it's just terrific. But workgroups are 1
where it, you know, the battle is won or lost. You know, you go out on the 2
highway. It says 65 miles an hour. What do we drive? You know, we don't 3
follow the policy. You know, we drive 75 or 72. If you're driving 75 in a 65 4
mile-an-hour zone, are you following the law or are you breaking the law?
5 You are breaking the law, but I would offer your following the most important 6
laws that govern behavior which is the norms of the group. And don't take 7
this to driver's court. You know, I tried once, and the judge threw me out and 8
doubled my fine. But, you know, until you get the workgroup itself working on 9
that drift, you know, you can have all the procedures and training that you 10 want. They learn it in the training center, and then they go and they pick up 11 what's done in the workgroup.
12 So I think this is true of innovation, I think this is true with 13 risk, I think it's true with all programs. But I'm just, you know, kind of putting 14 that blanket around it, you know, hoping that gives a little bit of a framework 15 from the people's side of the transformation.
16 COMMISSIONER BARAN: Great. Thanks. Well, 17 maybe I'll just leave it there, Chairman. Thank you. Thanks so much, 18 everyone.
19 CHAIRMAN HANSON: Thanks very much to our external 20 panel. Mr. Mitchell and Ms. Gerton, Dr. Simon, Mr. Place, really appreciate 21 your contributions today. I thought it was a great way to kind of follow-up the 22 staff panel. Obviously, it touched on many of the themes. So thanks to 23 SECY for pulling all this together for us this morning.
24
118 As we wrap up, I want to take a moment to recognize 1
Commissioner Caputo. This is her final public meeting, as the end of her 2
current term is quickly approaching. Thank you, Commissioner, for your 3
years of dedicated service to our agency, not just over the last three years but 4
also working as congressional staff and in the industry. The NRC has greatly 5
benefitted from your extensive knowledge and experience in nuclear energy 6
policy, obviously even before you came here. And I have no doubt that you 7
will continue to be involved in our important work in one form or another going 8
forward.
9 And in that spirit, this isn't a goodbye but, rather, a 10 recognition of your work over the years and a sincere expression of thanks.
11 COMMISSIONER CAPUTO: Thank you very much, Mr.
12 Chairman.
13 COMMISSIONER BARAN: Well, let me just add all the 14 best of luck in all your future endeavors, whether at NRC or elsewhere. All 15 the best.
16 COMMISSIONER WRIGHT: Commissioner Caputo, or 17 Annie, as I've known you over the years, you know, I first met you sometime 18 back, I don't know, '05 - '06 time frame, I think, when I was an economic 19 regulator and you were working for your bosses on the Hill or the committees 20 at the time working the issues of the day. You've always been passionate 21 about everything that you've done, whether it be on the Hill or be it here at the 22 NRC.
23 And, you know, during the last three years, I've gotten the 24
119 opportunity to get to know you on a personal level, get to know your family, 1
and I've learned that you're more passionate about your family than you are 2
about anything that we're doing here. And the things that you have done that 3
you're also, you know, you love the outdoors, and I've discovered just how 4
talented you are at quilting. I never met someone who really knows that like 5
you do.
6 You know, I'm a very hopeful man, and so I'm hopeful that 7
our professional and personal relationship here will continue for many years 8
to come. So wish you the best.
9 COMMISSIONER CAPUTO: Thank you. Thanks to each 10 of you for your very kind remarks. During my time here, I have worked with 11 all of you and earned a huge measure of respect for each of my colleagues 12 and learned a lot from each of you in terms of how committed you are and 13 dedicated to your job and the talents and skills that you bring to it.
14 And I also, you know, I think we all get spoiled to a certain 15 extent. We have the most talented staff here at the NRC, and I really have 16 developed a huge respect for the staff in ways that I didn't have in my previous 17 capacities and have relished working with them during my time here.
18 So thanks again to each of my colleagues, and thanks again 19 to Chairman Hanson for his leadership over the last several months. I've 20 appreciated working with all of you, and I've definitely enjoyed my time here.
21 Thank you.
22 CHAIRMAN HANSON: Thank you. Thank you all.
23 We're adjourned.
24
120 (Whereupon, the above-entitled matter went off the record 1
at 12:15 p.m.)
2