ML19092A323

M190328: Transcript - Transformation at the NRC: Innovation (Public)

ML19092A323
Person / Time
Issue date:	03/28/2019
From:	NRC/SECY
To:
Shared Package
ML19009A241	List: ML19087A089 ML19092A323
References
M190328
Download: ML19092A323 (71)

Text

1 UNITED STATES NUCLEAR REGULATORY COMMISSION

+++++

COMMISSION MEETING ON TRANSFORMATION AT THE NRC: INNOVATION

+++++

THURSDAY, MARCH 28, 2019

+++++

ROCKVILLE, MARYLAND

+++++

The Commission met in the Commissioners= Hearing Room at the Nuclear Regulatory Commission, One White Flint North, 11555 Rockville Pike, at 9:00 a.m., Jeff Baran, Commissioner, presiding.

COMMISSION MEMBERS:

JEFF BARAN, Commissioner STEPHEN G. BURNS, Commissioner ANNIE CAPUTO, Commissioner DAVID A. WRIGHT, Commissioner ALSO PRESENT:

ANNETTE VIETTI-COOK, Secretary of the Commission MARIAN ZOBLER, General Counsel

2 NRC STAFF:

MARGARET DOANE, Executive Director for Operations ANGELA BUFORD, Chair, NRC Innovation Forum EXTERNAL PANEL:

MIKE WETKLOW, Deputy Chief Financial Officer & Director of the Division of Financial Management, National Science Foundation JOSHUA DI FRANCES, Executive Director, Presidential Innovation Fellows BETHANY BLAKEY, Lead, Change Management Center of Excellence, Technology Transformation Services, General Services Administration ROBERT D. ATKINSON, President, Information Technology and Innovation Foundation

3 1 PROCEEDINGS 2 (9:05 a.m.)

3 COMMISSIONER BARAN: Good morning, I call this meeting to order.

4 Unfortunately, Chairman Svinicki is unable to be here this morning so she asked me to chair 5 today's meeting for her.

6 The Commission meets today to continue the conversation about 7 transformation and innovation. For this meeting we reached outside of the nuclear sector to 8 hear from panelists with a wealth of knowledge and experience related to transformation and 9 innovation at other agencies in other subject matter areas.

10 We invited them to talk about best practices, success stories, lessons learned, 11 and any ideas they wish to share.

12 Before we get to our external panel, we will hear from Margie Doane, the 13 Executive Director for Operations, and Angela Buford, Chair of the NRC Innovation Forum.

14 Margie will briefly share her thoughts on transformation at the Agency and Angela will give us a 15 quick update on InnovateNRC.

16 After their presentations, we have a total of five minutes to do a quick, 17 informal, lightning round of clarifying questions. Before we turn it over to Margie, do any of my 18 colleagues have any opening comments or thoughts?

19 Okay, then let's get started. Margie?

20 MS. DOANE: Good morning, Commissioners. The Staff 21 greatly appreciates this opportunity to present to you today on the topics of innovation and 22 transformation. Transformation as an agency is about being and becoming a modern regulator 23 that promotes and embraces innovative approaches to achieve our mission.

4 1 In May of 2018, the senior executives got together, including me, and we held 2 a retreat on transformation and we set a vision for the Agency to be a modern, risk-informed 3 regulator.

4 And I'm often asked, a lot of times by Commissioner Wright, what will success 5 look like? And I thought I'd take just a few minutes here today to give you a glimpse. Here's a 6 glimpse.

7 Our most challenging milestones will be behind us, including development of a 8 proven regulatory framework for digital instrumentation and control, advanced reactor 9 applications, and advanced fuel, as well as a successful completion of our first design review for 10 a Small Modular Reactor, just to name a few of these milestones that we have going on right 11 now.

12 The Agency will routinely decide any safety, security, or environmental issue, 13 even the most complex or for the newest technology informed by the best available risk insights 14 in a collaborative, timely, and predictable way.

15 We'll be modern, using technology to advance our operations and 16 decision-making and we will have laid the groundwork to meet our important mission by 17 preparing our workforce of today and for the future.

18 So I know this sounds ambitious. I'm often asked how we can accomplish all 19 of this. I relayed to our Staff the old saying, how to eat an elephant, one bite at a time. I tell 20 them we'll make one decision at a time but I underscore the urgency of our situation.

21 The transformation needs to take place now. Given the dynamic, external 22 environment and the pace of change in the industry that we regulate, we have to be making 23 changes now.

5 1 Some decisions will be great and others will be more incremental, but with 2 time we will have built a new culture of a modern regulatory decision-making agency.

3 Next slide, please. As shown on this graphic, innovation is a key element to 4 transformation. Innovation is occurring at all levels and has been for many years at the NRC.

5 We are a learning organization, our Staff are continually making 6 improvements to how we achieve our mission. One of my priorities for Agency leadership is 7 ensuring we have ways for our Staff to actively bring innovative ideas to bear.

8 In a Commission Meeting last October, the Staff highlighted a significant effort 9 in this area, SECY-18-0060, Achieving Modern Risk-informed Regulation. That paper 10 recommends specific revisions to NRC's regulatory framework as it relates to new technologies.

11 12 But as you'll hear today, there's innovation going on all over the Agency in 13 many areas. Today, we are here to highlight another aspect of our transformation efforts: the 14 work of InnovateNRC across functional Working Groups facilitated by members of all 15 EDO-reporting offices including the regions.

16 Close to two years ago, this effort was created as a grassroots way for people 17 to raise, enhance, and share ideas across organizational boundaries. Angie Buford, the 18 Co-Chair of InnovateNRC, is here to provide highlights of their great work.

19 InnovateNRC's origins began with offices providing collection box approaches for 20 submitting ideas on a wide spectrum of topics. Everything was open.

21 As engagement with these programs picked up, we recognized that we 22 needed to create more consistency across the Agency for idea submission as well as be able to 23 fully address cross-cutting ideas that impact multiple areas.

6 1 So, we developed the Agency Innovation Forum to facilitate an agency-wide 2 approach. Together the Agency and office-level forums comprise what we are now calling 3 InnovateNRC.

4 The Staff who serve in these efforts do so as a collateral duty on a voluntary 5 basis in addition to their important work in the Agency. I'm pleased to report that we have had 6 a great response to InnovateNRC as you will hear from Angie.

7 As she'll describe it shortly, our Staff wants to innovate and have many great 8 ideas on how to do so. InnovateNRC has helped us grow and develop our innovative muscle 9 and it is strengthening our mindset towards innovation as an organization.

10 As our experience and efforts continue to evolve, we are focusing on several 11 areas going forward to take our innovation efforts to the next level. First, we are 12 taking steps to make sure we are appropriately evaluating, prioritizing, and following through 13 with the implementation of recommended ideas using an open and transparent process. In 14 addition, we plan to begin efforts to target innovation around specific topic areas.

15 Finally, we are honing in on a clear communication of progress and results in order to 16 sustain Staff engagement, because if they don't hear from us, they won't be encouraged to 17 continue to provide ideas for innovation. I'm excited about where our innovation efforts are 18 headed.

19 Next slide, please. Before I turn the presentation over to Angie, I want to 20 take this opportunity to recognize and express my appreciation for the Members of 21 InnovateNRC who have been so dedicated and passionate about helping improve our Agency 22 as well as our organizational psychology, who have been dedicated and passionate about 23 helping improve our Agency.

7 1 Our psychology experts from the Office of Research have been using their 2 expertise to help us as well in organizing and commenting on a number of different areas, and 3 also building platforms for how we are going to better engage our staff to harness their ideas for 4 innovation in the future.

5 We look forward to building on their momentum to continue advancing and 6 maturing our innovation focus. I will now turn the presentation over to Angie.

7 MS. BUFORD: Good morning, Commissioners. Thank you, for the 8 opportunity to speak to you today on behalf of InnovateNRC and the NRC's innovation 9 community. Next slide, please.

10 As Margie mentioned, InnovateNRC consists of the Agency Innovation Forum 11 and office-level innovation programs.

12 InnovateNRC was developed to break down barriers to innovation by 13 providing ideas and an efficient path to Agency decision-makers, and by sharing innovative 14 solutions across offices.

15 With a formalized structure, we provide an ongoing opportunity for NRC 16 employees to innovate. By having participant from every office, we avoid siloed communication 17 and by taking ideas directly to decision-makers, we cut through the red tape that can exist within 18 a large organization.

19 InnovateNRC provides a framework to solicit, evaluate, refine, and 20 recommend ideas so that we can bring forth a variety of innovations from small ideas that may 21 incrementally change our day-to-day work, to disruptive ideas with the potential to transform the 22 way we do business.

23 Our goals are for all Staff to have an innovative mindset, to promote and foster

8 1 creativity, and to make the NRC a more effective, efficient, and modern regulator.

2 Next slide, please. This slide addresses the structure of InnovateNRC. The 3 Agency Innovation Forum has at least one member from almost every office. In the past 16 4 months, more than a dozen new office-level innovation programs have been established in 5 addition to three that already existed.

6 This approach to innovation was modeled after the Region II idea, 7 Greenhouse, and research done by a senior executive service candidate development project 8 on enhancing innovation.

9 You see in this slide some of the logos for the office-level programs. In total, 10 approximately 130 NRC employees have been directly involved with innovation, with hundreds 11 more who have participated by submitting ideas.

12 Next slide, please. To date, InnovateNRC has processed approximately 900 13 ideas. Those ideas have come from two sources: one, Staff across the Agency via their office 14 or region innovation panel, and two, the March 2018 transformation team call for ideas which 15 solicited ideas from both internal and external sources.

16 Over half of all ideas received came from Staff submissions to Office/Region 17 Innovation Programs, and about 85 percent of those ideas were handled at the office level.

18 Many of those ideas have been implemented and I'll just give you a few examples.

19 Region II developed the individual development plan exchange, where 20 inspectors could work with their supervisors to exchange their positions and services for a 21 pre-defined period of time with another inspector who had similar qualifications.

22 The Office of Nuclear Material Safety and Safeguards, or NMSS, implemented 23 an idea to designate a single 2.206 petition coordinator for the Office, which has resulted in

9 1 efficiency and improved document quality for petitions.

2 Additionally, an idea that came out of Region III was to change the frequency 3 for dosimetry exchange from once a quarter to semiannually. This change was made 4 agency-wide and it resulted in a contract cost savings for dosimetry processing and savings in 5 Staff time for the exchange process.

6 The remainder of Staff submissions to the Office/Region Innovation Programs 7 were forwarded to the Agency innovation Forum due to their cross-agency applicability.

8 In addition to that, 362 ideas were directly transferred to the Agency 9 Innovation Forum from the March 2018 transformation team initiative. The ideas handled by 10 the Agency Innovation Forum are dispositioned in one of four ways.

11 First, some ideas are shared for informational purposes. These are ideas 12 that have been implemented at the office or region level but may benefit other offices and 13 regions as well.

14 The Agency Innovation Forum facilitates communication of these ideas to 15 encourage use of best practices across the Agency. The second way we disposition an idea is 16 to identify if the idea is within the scope of an ongoing Agency effort or would enhance an effort 17 that's currently underway.

18 When this is the case, the innovator is made aware of the current effort and 19 when appropriate, the team working on the current effort is made aware of the innovative idea.

20 This has helped to foster communication about an enhancement of our current efforts.

21 I'll give you a couple of examples there. We had many ideas come in 22 regarding the use of electronic concurrence. Through InnovateNRC's research, we found that 23 the Office of the Chief Information Officer, or OCIO, had already developed a prototype for

10 1 e-concurrence.

2 So InnovateNRC worked with OCIO to facilitate multiple pilot efforts to better 3 inform the design of that prototype and make other offices aware of this effort in order to avoid 4 duplication.

5 Another example was to obtain Transportation Security Administration, or 6 TSA, PreCheck for Agency employees to make airport travel more efficient. During the 7 evaluation phase, we actually found that TSA was already pursuing this government-wide.

8 While we are still waiting on TSA to formally roll out the program, our 9 knowledge of the effort will allow the NRC the opportunity to be an early adopter of the program.

10 Third are the ideas that are found to be truly novel and have no related 11 Agency efforts. These ideas must be researched, formally evaluated, and when appropriate, 12 further developed so that actionable solutions can be brought to Agency decision-makers for 13 consideration.

14 Fourth and finally, some ideas that are received are determined to be 15 infeasible or not actionable for a variety of reasons. When this is the case, the submitter is 16 provided feedback for transparency.

17 Overall, InnovateNRC has enabled Staff to submit their good ideas, to follow 18 their idea status, and to know the eventual outcome.

19 This careful consideration of ideas and transparency in the process, as well as 20 sharing successes on multiple levels, reinforces the desired culture of Staff empowerment and 21 embracing and promoting innovation.

22 Next slide, please. In the past 18 months, we've made strong progress.

23 What we know so far is that Staff want to innovate. This process is allowing Staff to utilize their

11 1 own creativity to improve the Agency to better support and accomplish our mission.

2 Nevertheless, moving forward, we designed InnovateNRC to be a learning 3 organization and as such, we are evolving as we apply lessons learned. We recognize that 4 sustained engagement in innovation requires positive gains in efficiency and effectiveness 5 through submitted ideas.

6 The Agency Innovation Forum is currently working through the evaluation 7 process for approximately 190 ideas. Moving forward, an area of focus is to build a robust 8 model for prioritizing and implementing the ideas both incremental and disruptive.

9 Our colleagues in the Office of Research are working on developing an 10 end-to-end innovative model and are looking to external counterparts for best practices. We 11 are also looking at ways to innovate around specific topics or challenges that the Agency is 12 facing.

13 Strengthening our innovation muscle will require investment and resources, 14 willingness to learn, and persistence.

15 We will continue to work creatively and collaboratively towards building a 16 culture where innovation is part of our daily life.

17 Thank you, and with that I'll turn it back over to Margie.

18 MS. DOANE: So, as you've heard, InnovateNRC has made great progress.

19 Our Staff has been very engaged. It is a great asset of the NRC that our people care so much 20 about our mission and see their work directly contributing to it. You've heard many examples 21 of ideas that have helped us work better and smarter already.

22 Investing in activities like InnovateNRC will help us more effectively achieve 23 our mission and create the conditions needed to enable the workforce and the workplace of

12 1 today and tomorrow to make us successful today and for the future.

2 So I look forward to hearing your questions in the lightning round. So we're 3 ready.

4 COMMISSIONER BARAN: The lightning round.

5 MS. DOANE: Angie and I are ready.

6 COMMISSIONER BARAN: Thank you. Do any of my colleagues have any 7 quick questions for Margie or Angie?

8 COMMISSIONER BURNS: The one question I would have is with respect to 9 this Innovation Forum. I recognize it was formed under the prior EDO but I look at the 10 membership and except for the Office of the Chief Financial Officer, these are all EDO offices 11 and none of the Commission-level offices.

12 So I'm trying to understand why that might be and what inhibition that that 13 might pose in terms of that. Because I'm looking at you and her and me as general counsels 14 and former general counsels, and we get a lot of complaints sometimes, it's about the lawyers.

15 And so anyway, I'm leaving that out there, I'd be interested to know the 16 answer to that question.

17 MS. DOANE: All right, Angie, I'm going to take it and you probably don't want 18 to even add anything to it but you can if you want, if you want to bail me out.

19 So, every office was invited to have its own forum and the EDO offices were 20 directed to have these forums and as often we'll do when we're in the Office of the General 21 Counsel, I can tell you, and other smaller offices, we'll watch to see what happens at the office 22 level and then see whether we can take advantage of it.

23 But we were doing many innovative actions in OGC when I was there, and I'm

13 1 sure Marian can add to this, and we had participation, I think Tison Campbell for example had 2 many innovative ideas on how to do things that were innovative for tracking of cases, for files.

3 We were getting great support but we were doing them very particular to the 4 activities that we were doing. These ideas, however, are giving us the opportunity across the 5 Agency Forum across the Agency to put in ideas and to make changes that would be 6 cross-cutting.

7 But as far as the individual offices that were concerned, OGC wasn't tasked to 8 do it but we did keep up with it.

9 COMMISSIONER BURNS: But my question then is so these ideas might be 10 innovative, they might be cross-agency, but how is that communication transmitted?

11 MS. DOANE: I see what you're saying. I'm sorry.

12 COMMISSIONER BURNS: And how is that transferred?

13 Because I agree with you, I can think of times over the years where one of her 14 deputies in effect, when there was a lot of complaining from the Office of Nuclear Reactor 15 Regulation about reviews of license amendments, she did a thorough search in terms of what 16 we had done or what was the nature of it and focused on, and helped our Staff focus on: these 17 are the types of problem areas you need to focus on. That's where we can be more effective in 18 our review. So I'm interested in the cross-agency communication on some of these ideas.

19 MS. DOANE: So I will tell you that starting with Vic, and I am continuing this, 20 we have all of the executives engaged.

21 And one of the things they do when they find an idea that could be 22 cross-cutting is, first of all, they work very closely with the lawyers.

23 Angie, I don't know if you want to give the example of where we worked on the

14 1 --

2 MS. BUFORD: The fitness?

3 MS. DOANE: Yes, or just any of them. But the lawyers get engaged early 4 on -- I'm sorry to put you on the spot like that, Angie -- the lawyers get engaged very early on in 5 helping us to make these decisions.

6 But in addition, the lawyers were invited in to talk about what things would 7 help the Agency, and then those issues were taken and handed down to the Staff. And we 8 continually have these conversations at that level.

9 So, for example, decision-making and a lot of the details on decision-making 10 that were put into my SOAPs came from --

11 COMMISSIONER BURNS: Tell people what --

12 MS. DOANE: I'm sorry. So we issued Strategies on a Page very early on 13 right after I became Executive Director for Operations, where we focused on three areas.

14 We focused on commitment to our mission, making better decisions, and then 15 communicating those decisions. Those strategies were put together and they set where we are 16 today and where we want to go, and those were set by executives at all offices including the 17 General Counsel's Office, OCA, OPA.

18 And they put a lot of effort into helping us think about where we need to go 19 and where we need to improve and where we need to innovate. So that's another area.

20 I don't know if you want to hear specific examples about how we work with the 21 offices?

22 COMMISSIONER BURNS: No, because I know the time is short. My point 23 is not that it's about whether it's legal or not or it meets that litigated risk.

15 1 It's whether if these ideas help this office work better, they potentially help her 2 office work better and Nader Mamish's office work better. And that's why I'm interested, to 3 make sure that there is this kind of broad communication --

4 MS. DOANE: There absolutely is.

5 COMMISSIONER BURNS: -- and ability for contribution across the Agency.

6 Thank you.

7 COMMISSIONER BARAN: Other questions? Commissioner Caputo?

8 COMMISSIONER CAPUTO: I agree we need to develop our innovation 9 muscle and that we're going to achieve transformation one good decision at a time.

10 With the Innovation Forum, we seem to have a healthy innovation effort at grassroots 11 level that's generating real benefits, and I want to thank all the Staff who have contributed their 12 time and thought to the Innovation Forum.

13 However, leadership development is critical to the success of transformation.

14 So how are we working to grow an innovation culture at the leadership level?

15 MS. DOANE: Okay, so I'm going to talk about -- we have the executives 16 working on innovation and transformation for the Agency, both setting the mission and being 17 coordinated on where our goals are. I've already talked about the Strategies on a Page. We all 18 worked as executives on this common theme of where what are now, where we're going to go, 19 how we're going to get there. This is number one.

20 Number two, as you know, we did a futures assessment so we could 21 understand scenarios for the future, knowing that it won't be any one of those scenarios, but to 22 make sure that we are thinking externally and improving our external awareness.

23 Because that was something that the Commission had noted would move us forward in

16 1 this regard. So we're working on that.

2 We have also put performance standards in our performance appraisals for 3 our executives that have to do with leading change, setting standards for them to lead change.

4 At our last senior leadership meeting, a topic of the discussion was to share 5 best practices and also to think about ways that we could improve our support for, I'll say 6 support for our first-line supervisors because they're at the crossroad of where these good ideas 7 come from, and then how they might get through the rest of the Agency.

8 And also, they're encouraging their Staff and giving them time to do corollary 9 duties that are outside their normal working station.

10 So, these are some of the things that we're doing but mainly what we're doing 11 is we know that we will not innovate or transform if the leaders don't have the same vision but 12 also support these efforts, be open to new ideas.

13 So these are some of the activities that we're working on.

14 COMMISSIONER CAPUTO: Thank you.

15 COMMISSIONER BARAN: Commissioner Wright?

16 COMMISSIONER WRIGHT: Good morning, how are you? So, to follow up 17 on Commissioner Caputo, the transforming one decision at a time thing, it kind of matches up 18 well with what we're trying to do with transformation and innovation too.

19 So, I really have a couple of questions about the one decision at a time 20 concept. Is it really a key part of the transformation process the way you're trying to implement 21 it?

22 MS. DOANE: I think it is because I think when you're trying to transform 23 across an Agency in an area of decision-making, which is very general, then you have to

17 1 understand that each decision is going to be made in a different environment and with different 2 facts and different circumstances.

3 And they're might not be a one-size-fits-all but we can all think about how to 4 take risk insights into consideration, how we can make a more timely decision. We're working 5 at our desks, we know where things are being held up.

6 So we can all make decisions to move us forward in that area, that's across 7 the Agency, and so by its very nature it has to be one decision at a time.

8 But some of these things I want to emphasize are going to be big decisions 9 like your decision for a design review for a Small Modular Reactor.

10 That's a big decision but, nevertheless, it will be an individual item. So that's 11 how we think about it so that we make it clear that this really isn't just something that's 12 addressed towards new reactors or something like that.

13 It's really across the Agency for decision-making.

14 COMMISSIONER WRIGHT: Okay, thank you. Could you indulge me one 15 more? So, I want to go to -- I guess both of you can answer but, Angela, maybe this one will 16 be for you.

17 So in your slide you talked about the Office/ Region Innovation Program, so 18 what I=d like to know is how are we rewarding Staff? The Staff Members, or whoever might 19 submit innovative ideas. Are they recognized in some tangible way or are we doing enough to 20 get folks excited about what's going on?

21 And I'll just share this real quickly, I did talk with someone who had submitted 22 an idea and they were excited about the idea they submitted but then they were told, okay, you 23 own it.

18 1 And I'm like, I'm not sure what that means, you know? Is that just getting 2 more work for them on top of what they're already doing? How are you trying to get them 3 excited about that?

4 MS. DOANE: Angie, why don't you start?

5 MS. BUFORD: Okay, so your first question on how are we rewarding Staff, 6 that's a really good question and it's something that is a key area of focus in the Agency 7 Innovation Forum. It's important because it allows for continued engagement.

8 It's something we're working on at the agency level. We enlisted a Tiger 9 Team of five or six Staff Members that actually were outside of the membership of the Agency 10 Innovation Forum to develop a white paper on rewards and recognition.

11 That was last fall and they actually presented us with a white paper, maybe a 12 ten-pager that looked at options for ways to reward both monetarily and non-monetarily. We 13 know that there are ways to reward from a monetary sense.

14 We met with the Chief Financial Officer and Maureen said that there are ways 15 to incorporate funds. And so that's a work in progress and something that we're focused on.

16 And we hope to develop a model at the agency level and then to share that with the offices.

17 And obviously offices are going to do it a little bit differently. I know that in 18 the Office of Research, I think they may be the furthest along. They have an official certificate 19 of appreciation and a symbolic award.

20 I know the other offices are trying to follow suit and also trying to integrate the 21 Office/Region innovation program into existing rewards and recognition processes.

22 I think the logistics of that -- especially when we're trying to do it at a 23 grassroots and Staff level, trying to come up with ideas for doing that as well as navigate the

19 1 rules for the ability to do that.

2 So that's a current work in progress. I'm not sure if you want to add 3 anything?

4 MS. DOANE: I think that was great. I think you can see the energy around 5 it. We know that rewarding innovation is a big part to both setting an example, informing Staff 6 about where what are innovating, and then also encouraging people to continue to give these 7 great ideas.

8 So we know this is an area that needs additional focus so thank you for that 9 feedback and that conversation.

10 As you said, we're going to continue to work on this because what we've been 11 doing is really we're just using our typical model, which isn't bad, which is giving individual 12 awards, giving group awards. But it's not enough attention.

13 We have a new website, I showed it on one of the slides, and that's one of the 14 things people are thinking about, that we could give recognition so the Agency can see real-time 15 what we're doing with innovation.

16 We need to do better on that and it's an important part of the process to 17 motivate change.

18 COMMISSIONER WRIGHT: Okay, thank you. Yes, ma'am?

19 MS. BUFORD: You had another question about someone who had 20 submitted an idea and said that they --

21 COMMISSIONER WRIGHT: No, I just wanted to -- this conversation --

22 MS. DOANE: We'll take that as feedback, I think that's helpful.

23 COMMISSIONER WRIGHT: Thank you.

20 1 COMMISSIONER BARAN: Okay, thank you both. I'll now invite our guest 2 panelists to join us at the table. Thank you all very much for being here.

3 We are pleased to have with us Mike Wetklow who is the Deputy Chief 4 Financial Officer and Director of the Division of Financial Management at the National Science 5 Foundation, Joshua Di Frances, who is the Executive Director of the Presidential Innovation 6 Fellowship Program, Bethany Blakey, who leads GSA's Change Management Center of 7 Excellence, and Dr. Robert Atkinson, who is President of the Information Technology and 8 Innovation Foundation. We'll have you present in order starting with Mr. Wetklow. After all 9 four presentations are completed we'll ask our questions. So Mr. Wetklow, welcome.

10 MR. WETKLOW: Thank you. Good morning, thank you to the NRC and the 11 Commissioners for the opportunity for NSF to be part of your Commission briefing.

12 Transformation and innovation is a passion of mine and I'm very grateful to tell 13 you about our developing practices, very much developing. You are not alone in what you're 14 going through. And I look forward to learning from the other panelists any questions you may 15 have.

16 But simply put, these are really exciting times for transformation efforts and if 17 you can see on the slides, from a government-wide governance perspective -- next slide, please 18 -- we have one of the best President's Management Agendas I've ever seen in my time.

19 And it's supported by a clear vision of the financial management for the future 20 by the Treasury Department and at NSF we have an initiative called Renewing NSF that we 21 could speak about throughout the presentation.

22 For me, all these transformations, strategic directions boil down to a common 23 theme, dealing with the proliferation of data and risk, leveraging technology, and reducing cost

21 1 of compliance efforts, and ultimately building a stronger government.

2 A University of Maryland professor put this to me in simple terms and I'm 3 going to go through this in the deck. But he calls what we're going through the ABCDs of 4 technology impacts where A is for artificial intelligence and automation, B is for blockchain, C is 5 for cybersecurity, and D is for data analytics.

6 I'll go through these ABCDs briefly and hopefully we can take a deeper dive 7 into them for questions and answers you may have.

8 But another way to think of these ABCDs is these could perhaps be building 9 blocks to kind of frame the next steps in maybe contemplating with the NRC paper on achieving 10 risk and modern risk-informed regulation.

11 So if we could go to the next slide, please, with artificial intelligence? Okay, 12 this is how we at NSF are learning about artificial intelligence and automation.

13 This is a helpful kind of spectrum from the Treasury Department where on one 14 end of the spectrum you have basic entry-level automation-type things and at the end of the 15 spectrum you have more advanced artificial intelligence.

16 But we started at the entry level focusing on robotic processing automation. I 17 don't know if you all have ever seen on TV with the commercials, they have this CFO and all 18 these robots come into join the CFO at the boardroom.

19 We don't have robots at NSF but what robotic processing automation is is 20 think of an employee or your own desktop where you have Word and Excel.

21 There's this software package that you can program and put on your computer 22 that is programmed to do work, manual work that employees typically do not like. It can go in 23 and out of systems of the Treasury Department, at NSF, and basically, relieve Staff of manual

22 1 work that they don't like to do.

2 And it just frees up a tremendous amount of effort so Staff can focus more on 3 decision-making and analytics. So we've started off with a very small pilot.

4 The Treasury Department has innovation programs to help fund, give seed 5 money and I think my fellow panelist is going to speak a little bit to that.

6 But it's really a great way to get started and we're moving into more advanced 7 things that we can talk about perhaps at the Q&A session.

8 The next area, blockchain, this is something that is very much an exploratory 9 initial stage but I believe that this is going to change the world as we know it, very similar to the 10 Internet when it came up in the '90s.

11 Sometimes in the news, blockchain gets a bad rap from Bitcoin or 12 cryptocurrency-type news stories but in my world as an accountant, the way to think of it is a 13 distributed ledger technology.

14 It's basically an infrastructure-type thing where you have these blocks of 15 information on a network that can connect information between us and universities throughout 16 the country.

17 And we're just exploring this, trying to wrap our heads around it, but it's very 18 secure. Basically, the blocks are secured by math, science, these formulas to kind of make the 19 security they call it immutable, hard to crack.

20 But I believe over the next five to ten years if not sooner, this is going to 21 change accounting as we know it and I think there's other use cases that it can be applied to.

22 Next slide, C for cybersecurity. I'm not going to spend a lot of time on this but 23 as a Federal employee, and personally, many of us have maybe gotten calls from credit card

23 1 companies that your card's been hacked, and as a Federal employee, the OPM data breach.

2 But my main message here is cybersecurity is not only a CIO initiative, it's an 3 enterprise risk to all government and all organizations.

4 And what we're trying to do at NSF is really have the CFO community and the 5 CIO community work together to deal with this and work with our program offices.

6 And then we're also exploring shared services as a way to outsource all the 7 systems that we have.

8 We're at very much the beginning stages of exploring this with Treasury, 9 because it's just hard to keep up with all the systems and if you have less systems, your risk is 10 lower to secure all that.

11 And then finally, D for data analytics, what we're doing here is, again, this is a 12 very entry-level beginning effort.

13 So you can start at one end of the spectrum and get to more complicated but 14 in my world we have an area called an internal control and quality assurance program.

15 Have you ever heard of Sarbanes-Oxley in the private sector with Enron and 16 there's all this compliance work that went along with that?

17 We've done that for years in the Federal Government through this A-123 18 regulation but now with the technology we're looking at, we're looking at all the manual 19 compliance reviews.

20 And with the flexibilities and the direction with the new President's 21 Management Agenda, we are seeking to stop a lot of the compliance things that we do.

22 And instead of doing process assessments of things to really get in the weeds 23 and the nitty-gritty of internal controls, trying to build data analytic use cases where the data just

24 1 kind of helps inform your oversight of your business processes.

2 So it's actually very refreshing because that compliance infrastructure that we 3 have maintained over the years is heavy over time and it's really just a breath of fresh air I feel 4 that we're actually stopping things and starting up new things that leads to better opportunities 5 for us.

6 A final thought, I wanted to share a white paper that may help you all as well, 7 but it's one of my favorite papers from the Harvard Business Review on leading change and 8 transformation.

9 The title of the paper -- you might not be able to see it in detail -- is Why 10 Transformation Efforts Fail. So in my simple words I say, okay, I'm going to try not to do that, 11 and these are the eight areas that we're kind of embedding into our transformational efforts.

12 Establishing a sense of urgency, I second the presentation earlier and maybe 13 we can talk more in the Q&A. This is something we've got to do now because the risk of not 14 acting is just going to grow with time.

15 We built coalitions throughout NSF working with our CFO, our CIO, our 16 program offices to kind of grapple with this transformation together. Our Agency leadership 17 has really empowered through our NSF renewal effort to just build this into the culture and 18 everything we do.

19 So hopefully we can talk about that more with the questions you have. But 20 thank you very much and I'll turn things over to my fellow panelists for their presentations.

21 MR. DI FRANCES: Great, thank you so much. Thank you so much for 22 having me today. I wanted to share a little bit about the Presidential Innovation Fellows 23 Program, our best practices, maybe some select success stories.

25 1 So to kick it off just with a little background for everyone in the room, the 2 Presidential Innovation Fellows Program was established by the former United States Chief 3 Technology Officer, Todd Park, in the White House Office of Science and Technology Policy in 4 2012 to attract into Government top innovators from the private sector capable of tackling issues 5 of convergence of technology, policy, and process.

6 The goal is to embed talented technology leaders into agencies to challenge 7 existing paradigms by rethinking problems and leveraging novel and agile approaches.

8 Fellows are mid-to-senior-career technologists so they're not early stage in 9 their career like some other great programs like PMF. They're mid to senior career that have 10 demonstrated a proven track record of success.

11 They're from multiple industries with expertise that includes data science, 12 engineering, artificial intelligence, and machine-learning, product management, design, 13 customer experience, and technology strategy.

14 Fellows focus on some of the nation's biggest and most pressing challenges 15 such as improving access to benefits, fueling job creation in the economy, and expanding the 16 public's ability to access its personal health data.

17 Since 2012, we've had 135 fellows partner with leaders and change-makers 18 across 35 Federal Agencies. Further, and this is really an interesting statistic, over 50 percent 19 of the fellows end up staying on in government and moving into senior leadership roles, often 20 C-suite positions.

21 We've had the current Chief Technology Officer of the VA, Charles 22 Worthington is a former fellow, former CTO of the VA, former CMO of Census. Kind of all 23 across Government there's quite a few fellows that have stayed on in senior roles to continue to

26 1 drive transformation.

2 The PIF Program was administered as a partnership between the White 3 House Office of Science and Technology Policy, the White House Office of Management and 4 Budget, and the General Services Administration.

5 And then in 2013, the PIF Program established a permanent home office 6 within GSA. Our Board of advisors includes the Federal Chief Information Officer, the Federal 7 Chief Technology Officer, the Head of OMB, the Head of the Office of the White House of 8 eGovernment, and the GSA Administrator.

9 And within GSA, PIF now sits within the technology transformation services 10 along with 18F, which is another team started by PIF, Office of Products and Platforms, which 11 includes data.gov, code.gov, login.gov, the Centers of Excellence, which Bethany is going to 12 share more about, and the Office of Acquisition.

13 And together these organizations are working along with GSA to modernize 14 technology across Government.

15 And then in 2017, the president signed the TALENT Act which received strong 16 bipartisan support in both the House and the Senate and codified the program into law to 17 ensure that future generations of innovators would continue to bring the principles, values, and 18 practices of America's innovation economy into the Federal Government.

19 A little on how the fellows work and how the program is structured, on a 20 biannual basis, the PIF Program recruits some of the nation's top technologists and innovators 21 from the private sector. The program selects 1.5, on average, to 3.5 percent of Applicants and 22 pays the fellows at a GS-15 level to ensure that we can be competitive.

23 We look for individuals that demonstrate technology expertise, entrepreneurial

27 1 spirit, and have a proven track record of success in the private sector. Previous fellows have 2 included the lead developer of Google Maps and the former CEO of Symantec.

3 Fellows are hired by us then detailed to agencies for one to two years, where 4 they are paired with Government executives. Fellows work on strategic challenges that they 5 have the opportunity to lead, shape, and define.

6 They workshop, deep dive, design, do live user testing and rapid prototyping, 7 ultimately uncovering insights and creating new ways to improve the Government.

8 We also carve out, and this is an interesting element of the program, we carve 9 out 20 percent of the fellows' time to collaborate with other fellows in the program so that they 10 are able to bring in others who have expertise to help with their projects when necessary.

11 And what it ends up allowing is for Agency partners to tap into the entire 12 cohort of expertise. So, if an engineer and a data scientist are on a team and they need to 13 speak to a product person, they can pull them into their project as necessary.

14 Some examples of current projects include working with Department of 15 Agriculture to re-imagine how it engages with its customers, America's farmers, ranchers, 16 conservationists and private foresters, helping the U.S. Department of Veteran's Affairs, better 17 support veterans transitioning from military to civilian life, adoption of medical innovations to 18 improve health outcomes, and reducing processing time for benefits claims.

19 And one of the fellows recently had a really great breakthrough there 20 regarding digitizing the benefits process.

21 Helping the Food and Drug Administration provide more streamlined and 22 efficient regulatory oversight of software-based medical devices through a program called 23 Pre-Cert at FDA, and aiding the Department of the State in leveraging its data as a strategic

28 1 asset to better gather insights on mission needs, risks, and results.

2 And a little about how we work and our best practices, we leverage 3 human-centered design, we take a citizen-centric approach to problem-solving, applying design 4 thinking.

5 We work to understand how each of us interacts with our government, 6 identifying at the source the problems that we should be solving. The fellows design 7 responsively, they design solutions that can be tested quickly and approved iteratively, saving 8 time, effort, and taxpayer money.

9 They build agile solutions, working to solve complex problems in a world of 10 changing requirements that requires them to be agile.

11 Fellows form teams quickly, deliver on short cycles, and have constant 12 communication with the people that they're here to serve and their Agency partners.

13 And they look to measure impact so we hold all our fellows accountable of the 14 impact on the projects as they turn ideas into products and services.

15 Fellows leverage lean methodologies that they bring in from the private sector 16 so our goal is to create a responsive government with solutions and services that work for 17 anyone and everyone.

18 We want success to mean saving lives, saving taxpayer money, and 19 improving how the government serves people. And we're fortunate to be part of this program 20 and we have had a tremendous amount of success over the past few years since 2012. I'm 21 going to share a few of these. Last year in 2018, a fellow helped pass the Foreign Investment 22 Risk Review Modernization Act of 2018. Michael Brown, who was the fellow, his research at 23 DoD on China's technology transfer strategy and testimony before the House of Financial

29 1 Services Committee and Subcommittee on Monitoring Policy and Trade was instrumental in 2 Congress passing FIRRMA and it being signed into law in August.

3 Then in September 2018, Michael Brown was selected to lead the Defense 4 Innovation Unit where he was working. Fellows have launched trials.cancer.gov a few years 5 back in partnership with the National Cancer Institute and the National Institutes of Health.

6 The streamlined, friendly site for searching NCI-supported clinical trials 7 includes a new API, which was really instrumental for that. A fellow created online training to 8 counter terrorist use of the Internet last year at the Department of Homeland Security.

9 This fellow spearheaded innovative technological approaches towards 10 countering terrorist content online and helping stand up data-driven approaches towards 11 countering foreign influence.

12 This includes the development of the 2018 Countering Terrorist Use of the 13 Internet, which is an online training that has been adopted or is starting to be adopted by the 14 tech industry to help train the content moderators.

15 A team of fellows developed the Veteran's Employment Center with the 16 Department of Veteran's Affairs in connection with the White House Joint Forces Initiative and 17 Department of Labor.

18 This was the first interagency website connecting veterans and transitioning 19 service members and their spouses to meaningful employment opportunities. The portal has 20 resulted in cost savings of more than $27 million to the VA.

21 Multiple round of fellows have worked with the Department of Health and 22 Human Services and the VA to expand the reach of the Blue Button initiative, which provides 23 more than 150 million Americans with access to their health records online.

30 1 And among their greatest accomplishments, fellows have sowed the seeds for 2 lasting change across government by establishing the United States Digital Service in the White 3 House, 18F, which is a team of over 100 technologists at GSA, and additional groups that have 4 been off-shot from those teams.

5 A little bit in closing, we are at a remarkable time in history. The fourth 6 industrial revolution brings with it certain responsibilities. We must now not only grow our 7 workforce but nurture and reskill our current Federal workforce to be prepared for the future.

8 For us at PIF that means working alongside senior Government officials and 9 front-line managers to help nurture our workforce with more nimble and agile management 10 approaches and keep our nation competitive in this future.

11 Given our expertise, we understand that the U.S. Government must leverage 12 innovation within the private sector to solve critical governance issues of the future.

13 The Government will function best as a 21st-century service provider when it 14 creates agile, design-driven programs that meet citizen needs.

15 To meet these needs, we are developing new methods of procurement, 16 accelerating speed to service, and encouraging risk-taking through public-private partnerships.

17 We are committed to making the Government effective, efficient, and 18 accountable by modernizing technology to better serve the American people.

19 COMMISSIONER BARAN: Thank you. Ms. Blakey?

20 MS. BLAKEY: Thank you. I am going to rely on the slides, so if we can 21 bring those up? Who's controlling that by the way? Okay, at least we see the person who's 22 driving the machine. Wonderful. So, Bethany Blakey, I'm with the Centers of Excellence in IT 23 modernization. But what you'll hear is that my job is about change management and my job is

31 1 actually not about IT or anything technological at all.

2 So if we go to the first slide, I'll tell you a little bit more about the CoEs. We 3 go into an agency as a team of people, a multidisciplinary team of people to work with that 4 Agency on their IT modernization.

5 But as you can see, it's not all about IT. Yes, there's cloud and in some 6 agencies there's infrastructure optimization, cybersecurity is in there as well. And we have 7 data analytics, there's lots of synergy there.

8 We also have customer experience and we have a contact center, and the 9 newest kid on the block is the change management piece because there is certainly a 10 realization that the technology stuff is the easy part.

11 It's the change adoption that's the difficult part and that involves people. We 12 have to address the change management aspects.

13 So if we flip to the next slide, I'm going to zoom in on pieces of the change 14 management piece. We have kind of a three-part approach, we're there to assess and address 15 change readiness.

16 How ready is that organization to accept and adopt change going forward?

17 The last piece there is promoting 21st century work and workforce. These are about those 21st 18 century skills.

19 You talked about the World Economic Forum and those 21st century skills.

20 We're about trying to future-proof the workforce because the change is going to continue to 21 happen, it's going to keep rolling through at a faster and faster pace.

22 You can't keep up with it unless you learn to use and adopt on a regular basis 23 those 21st century skills that will keep you fresh and current all the time.

32 1 The piece I want to talk with you about today is the part about identifying and 2 cultivating your change agents. So you made a reference, Commissioner Wright, about the 3 person who had this great idea and everyone else thinks it's a great idea. Who's going 4 to build it? That's what I want to talk about. The innovator is not necessarily the builder.

5 They could be the same person, in many cases they're not and it's really 6 important to draw that distinction between the person who has those skills to re-imagine and 7 test and prototype how things can be different.

8 And it may be a whole different person and team to take that idea, to move it 9 forward throughout the organization. It's not necessarily the same thing.

10 So, actually, let's go back to that slide for just a second. Marge in her 11 presentation actually had a list of items. The last two in one of her early slides, one was 12 innovate and one was innovation, and the next one was builder.

13 So you can see under that talent to drive and support change, the designers 14 are what I would call your innovators.

15 These are the ideas folks who are generating ideas of how things can be 16 different and then employing some of the discipline of human-centered design and design 17 thinking to test and prototype and really find those great solutions.

18 And then the great solution is there and then sometimes there's an 19 expectation that says, all right, great, organization, take it, I've done all the hard work. That 20 was the fun work. Honestly, the innovation is the fun work.

21 The hard work is the building. So we want to find the inventors, we want to 22 find the builders. We also want to find the community of helpers, these are your facilitators and 23 your trainers and your coaches, because it's hard.

33 1 It's hard to move the organization from where it is to where you need it to be.

2 Organizations are built for stability, they're not built for change. That's a good thing but that 3 means we have to figure out how to smartly work through that change.

4 We'll bring the slides up again and we will move to the next one, thank you.

5 You've heard the adage change or die, I think it's a little overkill. It's change or struggle, right?

6 We're going to get our operations and our programs to the point where we're 7 in a great position. Our programs are not generally going to die but we will get them to the 8 point where they're struggling because if we start -- I come from a nonprofit background and a 9 community organizing background so I use a lot of nonprofit pieces.

10 This is a maturity model for a nonprofit organization which I'm applying here.

11 So you have the startup mode, which our programs are long past that mode, right? We've 12 done that. You have the build-up mode, right?

13 And that was hard to get things going, figure out how it should work best, 14 made lots of changes, lots of mistakes along the way and you get to the point where it's mature 15 and you feel like you've arrived.

16 And this is why I like using the maturity curve instead of maturity models, 17 because maturity models assume that you climb to the next step and you've arrived there and 18 you're good to go.

19 The problem is that world is continuing to change around you at a rapid pace.

20 If you get comfortable in that mature state, you are losing ground because everybody else and 21 everything else is moving forward around you.

22 And hopefully, you're actually helping to drive some of that movement and 23 change forward. The folks that you regulate, they should be improving, and they should be

34 1 evolving and you've got to keep being a step ahead of them so that you're bringing them further 2 and further along.

3 So the maturity curve is about going back to do more and more of that 4 innovation and you're going to spend most of that time in the builder mode because you're going 5 to continue to reinvent and innovate and develop new solutions, test them, prototype them, and 6 you have to figure out how to build them up.

7 And then you reach that mature state again, have a few picnics, have a few 8 parties, but you've got to have that team of people. Someone asked the question about how 9 do you look ahead to 20 years from now?

10 What is the thing that you're working towards? Yes, you always have to 11 constantly be looking ahead to what you're looking towards.

12 Let's go back to slides and we'll go one further. I say the word entrepreneur 13 or intrepreneur and people say, yes, that's great, but we can't do that here, we're not a startup, 14 we're the Federal Government.

15 I hear that, I have my master's in public administration, I am a good 16 government person all the way. Government is different than private sector, always has been, 17 always will be.

18 However, there are times when we have some similar challenges and this is 19 where I'm suggesting we need to have some of the similar solutions. Profit motive versus 20 mission motive, yes, but we still measure things, we still all have bottom lines about knowing 21 how well we're doing.

22 We still have to measure that and we still have to figure out how to do better.

23 Startup mode versus new products, new programs, that's a similar need to reinvent. New

35 1 business models, yes, we have a need to reinvent our organizations frequently.

2 Let's jump down to the bottom. Mergers or takeovers, yes, we don't generally 3 have too many of those but reorgs happen pretty frequently. It's very similar.

4 Or in the environment that we're working in now, my home office is still in GSA 5 but it's the Office of Government-wide policy and our particular portion of that office supports 6 President's Management Agenda and cross-agency priority goals.

7 And it's all about how do you work at that multi-agency, cross-agency level 8 because that's where you're going to solve the problems. Well, now we need different tools 9 because our organizations are not set up to work across those organizational boundaries.

10 It means we need to instill those 21st century skills in folks so that they learn 11 to do that really well and very effectively. It's not something that we've been rewarding, it's not 12 something that our HR systems are even set up to support.

13 Let's go back to the slides. The blue box on the left, there are all kinds of 14 other reasons why I'm advocating for intrapreneurship inside of government on the heels of, and 15 in kind of overlap and overlay with, innovation.

16 Digital transformation and IT modernization, again, what our CoEs are 17 working on and what I'm doing in my job right now in assignment. But shift to shared services, 18 you mentioned shared services, that's a change and it's not an easy change for organizations to 19 adopt.

20 You've got to give up a little control, you've got to compromise a little bit.

21 Demographic shifts, industry changes, I'm sure that impacts you a lot, scientific discoveries or 22 technological changes. Legal decisions, case law comes down, wow, we've got to make some 23 change to our program because a court said we're not able to do it that way anymore. You've

36 1 got to figure out how to reinvent a little bit, you can't just tweak around the edges every time.

2 Sometimes you've got to dig a little bit deeper and reinvent. And of course, inside our 3 Federal environment we've got administration changes and priorities that change and transitions 4 that happen.

5 We have Congressional decisions that affect our budgets and legislation that 6 affects how we operate or the things that we need to do.

7 Next slide, please. So intrapreneuring in itself is not a new term. Gifford 8 Pinchot III actually invented this term back in 1978, wrote a book about it in 1983, and he even 9 said, I didn't invent the concept of it, I just put a word around it.

10 It's intrapreneurship inside an organization. Yes, it's a thing, it's more difficult 11 than being a full-on entrepreneur, right? Because there are no boundaries when you're an 12 entrepreneur except whatever legal operating environment you're operating in.

13 We have a lot of restrictions and constraints inside our Federal operations and 14 inside any organization. Entrepreneuring transferred to an organizational setting means you've 15 got to work even smarter about what that looks like and be even more strategic.

16 We put on, and again my whole office, an intrapreneurship summit. You 17 have them here at NRC I am sure of it.

18 If we go to the next slide, actually, there's a way to assess talent level of the 19 intrapreneurs you have in the organization.

20 Gallup actually has an instrument where we can test for that and we can look 21 at the likelihood of success of intrapreneurship, entrepreneurship, whatever sector it might be 22 in. It's a relatively new instrument.

23 If you've heard of StrengthsFinder, it's the same organization that does

37 1 StrengthsFinder. We're one of the first 50 or so people to be trained in this instrument and 2 we're really beginning to build it out and apply it in a variety of settings.

3 It's meant to be done in a context of coaching, right? Because a lot of these 4 things they don't want to talk about. In your organization I would imagine risk, people don't 5 want to stand up and say, yes, that's one of my top talents is risk, leaning into risk.

6 Yes, it's actually really important but in this environment we actually have to 7 figure out how to help that person use that talent well and safely.

8 Last slide, please. Actually, second to last slide. I'm not going to speak to 9 too much of this but innovation is great, we love innovation, but my focus is really on the 10 adoption of innovation, which again, that distinction is really important to your point.

11 And now the very last slide, and this speaks to the question that you had 12 earlier, in the earlier session about leadership development, and I think it was a spot-on 13 question. Harvard Business would agree with you.

14 There was a study that just came out in November, this past November, on 15 the state of leadership and meeting the transformation imperative. An organization is 29 times 16 more likely to be successful in their transformation when you lean in on a leadership 17 development.

18 And I'm not talking about your traditional CDP programs, I'm not talking about 19 sending someone away for a week. I'm talking about finding those intrapreneurs, finding those 20 innovators, finding those helpers.

21 How can leadership position itself to best support those people and allowing 22 leadership to understand what their role is in bubbling up that talent and supporting that talent 23 going forward?

38 1 So, it was actually an excellent point about investing in leadership and my 2 recommendation is how you invest in that leadership actually matters for innovation and 3 innovation adoption.

4 Thank you.

5 COMMISSIONER BARAN: Thank you. Dr. Atkinson?

6 DR. ATKINSON: Well, thank you. It's a pleasure to be here with you. I'm 7 going to start off with a caveat. I don't work in the Federal Government although I used to a 8 long time ago, and I know almost nothing about nuclear power.

9 But I do lead a think-tank that focuses on innovation and innovation policy, 10 and as part of that, one of the things we try to understand is how do organizations innovate and 11 what's the role of policy on them?

12 Secondly, we have a fairly robust project on clean energy innovation policy, 13 including advanced nuclear and so we think about that as well. My colleagues are focusing on 14 it.

15 Secondly, I want to commend you on doing this, it's super important. It's 16 important for every agency, particularly regulatory agencies. So why now? Most 17 organizations start to focus on innovation when there are fundamental shifts in technology that 18 affect really what their business model is all about.

19 As long as those shifts aren't happening, you can kind of just muddle along 20 and you can do okay as long as your management and your organization framework works.

21 But I would argue now that there's pretty fundamental changes going on in the 22 world, two major ones being climate and the impetus for that, for solving it, and secondly, pretty 23 radical new opportunities in nuclear power.

39 1 So, clearly that puts innovation really front and center at what you all are 2 having to do. I would add that with a caveat it's hard to do innovation well in big organizations, 3 it's harder to do it in Federal organizations, and it's even harder to do it in a Federal regulatory 4 agency.

5 So, you have your work cut out for you. It's harder to do in a regulatory 6 agency because your core mission is not innovation at one level, your core mission is making 7 sure that everything goes through the right process and there's no mistakes.

8 And how you get innovation in an organization like that is you have to end up 9 developing I would argue two separate kinds of parts of the organization, one part that's really 10 focused on innovation and another part that's focused on your core mission of quality and 11 getting the things right that you have to get right.

12 I'm going to base my comments really on my colleague Gary Pisano's book.

13 The people who are leading this work, you should really read Gary's book, it just came out 14 about a month ago. It's called Creative Construction.

15 Gary is a tenured faculty at HBS and this book is really all about how do large 16 organizations innovate? A lot of the innovation lessons we hear really sort of come up from 17 startups and creative entrepreneurs any all that.

18 I don't know that they're all that useful for how you help large organizations 19 and that's really what Gary's book is all about. And he focuses on three things, which I'll talk 20 about.

21 Number one, strategy, number two, organizational systems, and number 22 three, culture. I want to start with strategy because I think that's the most important question to 23 get right.

40 1 Innovation is not a goal in and of itself, innovation is a means. The question 2 is what are you trying to accomplish? What is your core strategy goal?

3 And that's what corporations that are undergoing innovation have to figure out 4 first and that's what government organizations have to figure out first.

5 I think, and again, this is my own opinion, that perhaps the goal should be 6 here to make the U.S. the best place in the world to bring innovative nuclear power designs to 7 the market safely. That to me is a pretty tangible goal and it really embeds innovation in there.

8 I did look at your mission statement and, by the way, I didn't see innovation in 9 that. It's easy to talk about innovation but it didn't appear that it was in your mission statement.

10 11 Once you decide on the overall strategic goal, then you have to I think then 12 focus on what are the kinds of innovation tools and organizational processes and how do we 13 allocate those resources?

14 And I would stress this, when you think about this next point, this is not about 15 applying a grab-bag of innovation best practices.

16 A lot of these people now use almost sometimes buzzwords, things like open 17 innovation, design thinking, rapid prototyping, autonomous decentralized teams, and internal 18 venturing.

19 These aren't magic bullets and some of them work better in some 20 organizations than others, and it's not to say that you wouldn't use those tools. But just sort of 21 grabbing a tool and throwing it against the wall and saying we're doing design thinking, maybe 22 design thinking is not the right tool for what you're trying to accomplish.

23 Let me suggest five particular ideas that you might want to think about in

41 1 terms of an organizational system. One would be what's called venturing outside your home 2 court.

3 I think that's probably one of the best ways to be able to be more innovative is 4 bring in new kinds of people, bring in the most radical, crazy people in nuclear power for a week 5 and do a workshop or for a day and do a workshop.

6 Encourage folks that you have on your teams to go out and work at different 7 organizations for a week or for a month, really begin to get people to be uncomfortable, if you 8 will, by having to be put in new situations.

9 Secondly would be experiment and iterate. It's not like you have to get 10 everything right the first time forever. A lot of this is all about failing in small ways, using test 11 beds, or as my colleagues at Google talked about, one of the guys there talks about 12 pretotyping.

13 In other words, you don't need a big prototype, you pretotype. You do 14 something, you work it out for three to four weeks and if it doesn't seem like it's going well you 15 throw it away and you start something else.

16 Third would be to design a system to challenge sacred assumptions. All 17 organizations are based upon internal assumptions that people have in their mind that 18 everybody knows are right.

19 Well, when you're in an innovation period of transition, many of those 20 assumptions no longer are right and that means that if you properly manage it, encouraging that 21 kind of debate can be a critical tool for innovation.

22 Why don't organizations do that more? One is that staff needs to be pretty 23 sure that if they're going to say something that might be seen as heretical, they're not going to

42 1 get punished, they're actually going to get rewarded.

2 Hey, that's a stupid idea but thank you for including your stupid idea because 3 one out of ten stupid ideas might actually not be stupid.

4 And so that's I think really important to figure out ways to challenge what 5 some call groupthink and that's hard to do, it's uncomfortable, but it's super important. I think, 6 Commissioner Burns, you alluded to that earlier.

7 Fourth would be train support functions how to say yes to innovation. Often 8 times support functions in government like HR and compliance and legal and accounting, they 9 just take a reflexive action to say the rules don't allow it.

10 I certainly saw that when I was in government. I quickly learned the best way 11 to get something done is don't ask them, just do it. Now, you can only do that on certain things 12 I get, but certain things it's better to do it.

13 And fifth would be innovation metrics. I don't know whether you're measuring 14 yourself against what your other leading counterparts around the world are doing but what are 15 other regulatory agencies, nuclear regulatory agencies, doing around the world to innovate?

16 How are you doing compared to them? I think that's very important.

17 Last would be changing the culture. I think there's a mythology out there that 18 culture can't be changed. Organizational cultures can be changed, they're a little bit like 19 software. What are the key aspects for organizational culture?

20 I think number one is tolerance for failure, willingness to experiment, flatness 21 -- in other words, people near the bottom feel that they have as much right to talk and express 22 ideas as you all.

23 And then incubate and protect innovative pockets. There are going to be

43 1 places in any organization where there's more of that going on, people want to do something 2 different, protect those. And fundamentally, top leadership is responsible for that, you are 3 responsible for that. At the end of the day, innovation can't happen if it's a middle management 4 function. It has to be top leadership.

5 And that leads to the importance of engaging front-line workers. I didn't look 6 at the NRC scores but when you look at virtually all these surveys of Federal agencies, like the 7 Accenture survey, only 30 percent of Federal employees rated their agency's support for 8 innovation as excellent or good.

9 I'm assuming NRC is in there and I would put that as one of your goals that 10 next year or every year that comes out, you ought to be going up, not down, and you ought to 11 be outperforming your peers.

12 Also, I alluded to this earlier, change in the culture. You don't want to change 13 your culture so radically that you don't do the things you're supposed to do really, really well.

14 There's a reason that you don't necessarily want to do innovation all the time.

15 If you're looking at a power-plant, you don't want to do it differently. If it's the same plant you 16 want to make sure that the same processes apply at the same time so that no mistakes are 17 made.

18 But at the same time, you want to create separate parts of the organization 19 here, I would argue, that enable innovation, enable, hey, let's think about a new way to do this 20 new thing.

21 So that's part of this big challenge is you have to create a new culture but, at 22 the same time, you preserve your existing culture. So you're thinking about sort of having two 23 parts of the organization.

44 1 Lastly, let me just say why do innovation efforts fail? The two big reasons 2 they fail is, number one, top leadership has too short a time horizon. They say, hey, we need 3 this in 18 months. Doesn't happen often times in 18 months, it might take 4 or 5 years to really 4 put this in place and get the results you need.

5 So you have to have patience, and that doesn't mean turning a blind eye to 6 not making progress but it does mean having reasonable time expectations.

7 And then secondly, managing inherent tradeoffs between current targets and 8 future innovation, and between quality and innovation. By definition, if you're going to do 9 innovation, that means there's going to be some mistakes made.

10 My only question is how comfortable are you with that? Because there are 11 the type 1 and type 2 errors. The problem in what you're doing is that whatever the type 1 error 12 is, the type 1 errors are pretty bad, they're really bad.

13 So, it's harder for you than it is for some agency that's doing something where 14 the risks aren't as bad.

15 And then finally, I'll just close by saying I think the major challenge really for a 16 Federal agency is compared to a private sector organization where the CEO is the one that 17 owns innovation and they usually will appoint somebody who will drive innovation and report to 18 them, it's harder in Federal agencies because who owns innovation?

19 And innovation really fundamentally has to be owned, it has to be something 20 that is embedded in the entire organization and that everybody in the organization feels they 21 have some stake in it and some responsibility for it.

22 So with that, I'll stop. Thank you so much.

23 COMMISSIONER BARAN: Well, thank you all for your presentations. We

45 1 really appreciate you being here. Our practice is to rotate the order of recognition for 2 questions. Chairman Svinicki would have gone first today but I'll start off instead.

3 In some ways, I think we at NRC have been using two terms somewhat 4 interchangeably that are actually different in important ways. Innovation I think is about new 5 ideas or ways of doing things.

6 For me, transformation implies more of a major departure from existing 7 processes or methods. I think coming up with innovative ways of doing our work is almost 8 good by definition, but for an organization that is generally high-performing, it's not going to 9 make sense to radically transform every process, procedure, and practice that we have.

10 Do you all have any high-level guidance for us about how we determine where 11 it makes sense to make big transformational changes and where we would be better off taking a 12 more incremental approach?

13 MS. BLAKEY: I would say maybe it's semantics but I think both of those -- I 14 think you're spot-on and I think you're absolutely right that transformation happens not -- we 15 would not recommend it be one big flash.

16 We're doing a massive transformation which is terribly disruptive and a little bit 17 dangerous. It's the cumulative effect of all those small innovations which actually creates the 18 transformation over time.

19 And having a really smart performance office that actually is helping you track 20 the organizational performance and the cumulative effect of all of those innovations is exactly 21 the way to go. So I feel like you're already doing it. I think this section is important, I'd say 22 keep doing all of the small innovations.

23 DR. ATKINSON: I would just say there's lots of internal organizational things

46 1 in any organization, how your HR department functions, what your website looks like, how 2 business processes -- those are all things that, sure, you should always innovate in them and 3 adopts new technologies like cloud or other things like that.

4 But I think the really fundamental transformational question is --

5 Commissioner, you talked about it -- really, to me have to do with how do you get fundamentally 6 new kinds of nuclear power into the marketplace?

7 And are your systems designed to enable that to happen in a reasonably fast 8 and efficient way while maintaining safety? SMRs and a whole set of other kinds of 9 technologies are here and, in our view, the U.S. needs to be the leader in that.

10 And that is to me where the transformational question comes about, I would 11 argue.

12 MR. WETKLOW: One thing that NSF is -- and I'm leveraging the OMB PMA 13 framework.

14 They have an initiative called Shifting from Low-Value work to High Value so 15 to get at your point, framing transformation, we kind of say, okay, in 2019 we're here, where do 16 we want to be in 2026?

17 And that's kind of like the spectrum of shifting from low-value work to 18 high-value. But then that framing has a little bit of a negative connotation because you ask 19 yourself, well, gosh, are we doing low-value work? So we kind of use -- and I resonate with 20 what you said earlier.

21 We try to change it a little bit to say how do we go from good to great, or even 22 better, to have more of a positive aspect?

23 And literally we just have this simple matrix that we have all the things we're

47 1 trying to change and then we're just gradually mapping out -- we're really diving heavy into 2 robots this year, starting to learn about blockchain.

3 You go to learn more and next thing you know, by 2026 we're going to be 4 fundamentally transformed in our finance function.

5 MR. DI FRANCES: We see that it's important often to really focus on culture 6 initially, fostering an environment that encourages collaboration and risk-taking, to be honest.

7 And we've gone into agencies, for instance, we've worked with VA for quite a 8 long time and early on I think a lot of what the fellows did when they first came in was to bring in 9 modern best practices, to create an environment where they could share ideas and they would 10 be heard and encourage those risks.

11 And now we're seeing a lot more transformation happening at the VA because 12 there is that foundation and I think there is a culture that's accepting of new ideas.

13 COMMISSIONER BARAN: When an organization makes a lot of changes at 14 the same time, some may be small, some may be bigger, those changes can have cumulative 15 effects and those effects could be good or they could be bad.

16 You alluded to this a little bit. Based on your experience, what's the best way 17 for us to make sure that we understand the cumulative impacts of potential changes so that we 18 can avoid negative unintended consequences?

19 MS. BLAKEY: Data. It's measurement. I've been in Federal Government 20 for 21, almost 22 years and it's in organizational performance management.

21 So, where you stand depends on where you sit and organizational 22 performance management and tracking against your goals and doing it in a very agile way 23 where you make an intervention, see what that did to send you on that trajectory to where you

48 1 want to go.

2 You make another invention and just measure as you go and you constantly 3 collect that data in a very agile way. And if it didn't do the uptick that you wanted it to make, 4 you figure out why and you try something else.

5 It's measurement.

6 COMMISSIONER BARAN: And if we're thinking about how -- I can imagine 7 maybe on the IT side, or in some other respects, the metrics are maybe more obvious about 8 what they would be.

9 If you're thinking more on regulatory licensing processes, those sorts of 10 things, we're talking about more about what Margie was talking about in the first panel, making 11 better decisions.

12 How do you measure that? Are there agencies out there who have 13 embarked upon this who have a sense of how do you make better decisions and measure 14 something like that, that's a little bit more amorphous seemingly?

15 MR. WETKLOW: I'll try. I guess to your question, how we're measuring it is 16 we are looking at compliance work becoming easier and workloads going down.

17 Because the nature of the work we do, we feel like we have so much 18 compliance work to do, you don't have the bandwidth to get to supporting decision-making.

19 In accounting we have a lot of things we do to reconcile things to get the 20 numbers right and now the technology is helping relieve that burden. And I mentioned shared 21 services, we have really one big lesson learned.

22 We had literally this work that we did for IT FISMA work, internal control work, 23 and then our external audit. We did this shared service report that just kind of combines it all

49 1 into one. And I measure it in terms of staff, what they say.

2 We could say that workload you used to do you don't have to do anymore and 3 now you have more opportunities to focus on upskilling and retooling yourself, and they find that 4 more valuable.

5 The robot stuff I talked about, I think another aspect is involving staff in 6 deciding which aspects of technology and starting very, very small. I liked the pretotype 7 comment earlier because as you try to do too much, you feel just working with your staff that it 8 makes them nervous. So really taking small, gradual steps to build up I would recommend.

9 COMMISSIONER BARAN: I think we all agree we want people at NRC to be 10 innovative on a daily basis, trying to find better ways to do their work. At the same time, when 11 processes, practices, organizational structures are frequently changing, it can be pretty 12 exhausting for people and it has the potential at least to create uncertainty for our stakeholders.

13 14 Based on your experience, do you have advice for us on how we encourage 15 innovation while avoiding people getting overwhelmed by constant change? This goes to kind 16 of the change management issue that you raised earlier.

17 MS. BLAKEY: I think it's part of having that helper network of folks, whether 18 it's coaches or trainers.

19 The change part is inevitable, it's just the world that we live in now and part of 20 it is getting people used to the fact that there will be constant change.

21 But being able to be in touch with how they receive the news of that change, 22 not necessarily expecting every single person and every single function to be an innovator. Not 23 everyone's going to do that. That's good, that's fine.

50 1 But really helping to set those expectations and finding that talent, finding 2 those innovators, and finding those intrapreneurs, finding those helpers and having that full 3 network of folks available.

4 Because it's evolutionary biology, our brains resist change, on purpose.

5 It's actually a natural human reaction but we know this, which means we can 6 expect it, which means we just have to get smarter about how to help people navigate through 7 it.

8 And make sure that change is not happening to them but that we're putting 9 them in the position of experimenting with and being the change-drivers themselves, which 10 helps to relieve the threat response in the brain that happens when people hear that a change is 11 coming.

12 And they immediately start worrying about what the impact is going to be on 13 their job or their work or their commute, or whatever it is they happen to be concerned about.

14 Because we know that about the human brain, it means that we can put 15 mitigating staff in place and communications in place to help that.

16 MR. DI FRANCES: I agree that not everyone's going to be an innovator and 17 you don't want to force people that don't want to but I think providing mechanisms for those that 18 do, to share ideas and for the ideas to reach the right people, and to break down organizational 19 siloes.

20 Because often people end up working within their small organization within a 21 larger organization. If there's not opportunities to learn about what other teams are doing, to 22 hear more, to share with those teams, I think it can be more challenging.

23 COMMISSIONER BARAN: Some of the discussion we've had so far is

51 1 focused on using new technologies to increase an organization's performance, and that applies 2 to NRC just as it would to any organization.

3 But as Dr. Atkinson noted, we also regulate private entities who are separately 4 developing new technologies. And it's our job to make sure that we can effectively license and 5 oversee the use of those technologies.

6 Based on your experience, are there strategies or organizational attributes 7 that help a Government regulator to maximize its effectiveness in regulating new technologies?

8 DR. ATKINSON: Sure, there are. We wrote a report recently, ten principles 9 for how regulatory agencies can better promote innovation as part of their mission while still 10 obviously completing with the most important mission which is safety and protecting the public.

11 And part of that is about thinking through things like anticipating innovation, 12 talking with your -- the challenge here is, and going back to your earlier question, how do you 13 know whether what you're doing is working?

14 One way to tell whether it's working is to go and talk to people who are trying 15 to bring new nuclear power systems to the market.

16 Now, you can't obviously believe everything they're going to tell you, but they 17 certainly can provide useful input to say it looks like this is going in the right direction or the 18 wrong direction.

19 But there are a set of things, one is, as I said, anticipating innovation.

20 Another is thinking through the difference between type 1 and type 2 errors, it's clear the entire 21 system is generally biased towards never making an error.

22 But all of the things that never get in the marketplace, that's never seen as 23 though that's a problem. And so bringing that out more, again, it goes back to in ten years is

52 1 the U.S. going to have newer and different nuclear power systems in place?

2 Because if it doesn't, to me that would be, it sounds to me, a failure. Obviously, it 3 fundamentally is about the technology, does it work, and is it economic enough to be in the 4 marketplace efficiently?

5 But assuming it is, then that to me would be an indicator that's saying, well, 6 wait a minute, maybe we should be doing better.

7 COMMISSIONER BARAN: Thanks, I'll leave it there. Commissioner Burns?

8 COMMISSIONER BURNS: Thank you very much for being here and for 9 sharing your thoughts this morning. You've given me a lot to think about and actually, I'm the 10 old guy on here.

11 I started here when Mr. Pinchot -- you mentioned 1978. So I started here at 12 the NRC in 1978, left for a few years and then was dragged back from France to become a 13 Commissioner.

14 But the interesting thing is, actually, as you spoke I sort of reflected a little bit 15 on that 40 years and you've mentioned a number of things, Dr. Atkinson mentioned the words 16 muddle through.

17 And I remember writing a paper, actually a paper on the Office of Technology 18 Assessment during law school and the notion, again, of this government and the way our 19 government is formed, I've got Mr. Hamilton and Mr. Adams and Mr. Washington all behind me 20 on this, is that we were not created for instant change.

21 It was this intention of the balance of power, the balances between the various 22 Branches that we would make decisions smartly, though often incrementally. So that's often I 23 see as a challenge on it.

53 1 Mr. Wetklow, you mentioned the reporting-type things.

2 We are burdened with reporting obligations and we burden our industry and 3 that's one of the things we've asked to look at, the types of reporting obligations, which in their 4 form may have made sense in 1978 or 1985 the way they were because you needed to send a 5 human being down there to make the check versus being able to use technology and things like 6 that.

7 So your thoughts there resonated. Again, I think Mr. Di Frances, what you 8 talk about, getting ideas from the outside, trying to generate, trying to get them across agency 9 cultures and all really resonated with me.

10 And then Ms. Blakey, I particularly liked the chart or the curve or whatever, 11 because I can think of things in this agency which sort of fit that curve.

12 And when Margie was talking, one of the things actually, with all due respect 13 to her, this isn't a criticism, but I'm sort of sitting back here and sort of hearing these words 14 about modernization and innovation and all that.

15 And I said, well, wait a minute, I can go back and think of various times across 16 this agency's history where it has done exactly that. And then I can think of, for example, we 17 talked a little bit about advanced reactors and potential for advanced reactor licensing.

18 When I think about that, some of the significant critiques over the last few 19 years have been about the ability of the Agency to move forward on that type of licensing 20 process. What is it doing with respect to the license?

21 This comes after what I can recall in the mid-1980s after Three Mile Island 22 when basically the call was greater standardization, greater innovation in terms of how we look 23 at the process for licensing. The Agency adopted those processes, started to move through

54 1 them, they were declared successful. They were in many ways not used until the early 2000s.

2 They started to work through those processes and so then to hear, okay, 3 you're too stodgy, not fast enough, things like that, they said but we're experiencing innovation.

4 I guess what comes to my mind then is, no, we have a different paradigm. So how you work 5 through that is different and I will give our Staff -- because I think this is one of the things in the 6 innovation or the modernization or the transformation paper that Margie reference, looking at 7 how we use within that process, there are opportunities that meet what new designers would 8 like to in terms of an early check, does this look like we're on the right path?

9 Can we get here? Can we get sort of approval that might not be the full, final 10 wrapped-in-gold type of approval that some may want but it gets you down the line.

11 So anyway, as I say, you've inspired a lot of thought on that.

12 But maybe I could ask sort of generally, because I think you all did allude to 13 this, is this notion that within the Federal Government, it can be difficult to do so and as I say, I 14 take that in part because of the way this government was established and for very good 15 reasons.

16 I don't know if any of you want to reflect on that?

17 MR. WETKLOW: I do. You had a lot of very powerful things you just said so 18 it's a lot to process.

19 I might not hit every one you just said but the one thing that really struck me or 20 reminded me of this case study about the last 40 years -- let me just ask you a general question, 21 and this is from a case study from U.S. Senator Ben Sasse that I read in a recent book.

22 When do folks remember when -- and this is an accounting story too so it's 23 very exciting -- when do you remember ATMs came out? Just anybody, can you throw a date

55 1 out? ATMs, when did ATMs come about?

2 COMMISSIONER BURNS: I'll say 1990s.

3 MR. WETKLOW: I remember 1980-ish but this very fascinating story I read 4 about the last 40 years, the first ATM came about in 1968 in London.

5 And my dad is an accountant too, he's a comptroller for Citizens Bank of 6 Maryland, if you remember that from years ago, day time, night time. So it's just the banking 7 industry I'm really familiar with, and accounting.

8 But the key point I'm trying to make to you or share with you is that technology 9 came out in 1968 and then there was all this scare with the bank tellers thinking we're not going 10 to be giving out money anymore, really frightened of losing their jobs. But it actually had the 11 total opposite effect. The ATMs created so many more jobs because bankers or tellers could 12 upskill and sell different types of services.

13 And as I watched my dad go through these phases, eventually banks start --

14 he weathered the disruption over a 30-year period but over time, banks did start shrinking.

15 Citizens Bank of Maryland became Crestar, then it became SunTrust, and then my dad 16 gracefully retired. He managed it well.

17 The point that really resonated with me earlier today was the sense of urgency 18 that I think the first NRC presenter spoke to, is the sense of urgency, and I think of myself.

19 I'm not a spring chicken anymore but I think I am not going to have 40 years to 20 manage this disruption like my dad did. Because the technology is coming at us so fast that it 21 is going to be the risk of change or struggle, or this is as significant and serious as you 22 mentioned, the fourth industrial revolution.

23 Because at the university levels, students are getting trained in all this stuff

56 1 and they're going to be coming into the workforce thinking about robots, blockchain. And then 2 you're going to have folks like me that need to really retool and upskill ourselves.

3 You've got this gap that is really significant and the technology is coming at us 4 so fast, we're really going to struggle.

5 So I really think this is a very important topic but also a very important 6 opportunity so I was really glad to share NRC's interest with this.

7 COMMISSIONER BURNS: Thanks. Dr. Atkinson?

8 DR. ATKINSON: So I'm glad you mentioned OTA. I used to be a Project 9 Director at OTA so it'd be nice if it was still there.

10 I think one way to think about this, although certainly not everyone agrees to 11 it, I think that big technological shifts come about in waves and we're in a wave now, essentially, 12 of new disruptive technologies that all sort of fit together in one way, shape or another.

13 And when you look at regulatory agencies, most of them are struggling with 14 that. So you think about how does FFA regulate drones or how does Department of 15 Transportation regulate autonomous vehicles, or how does Treasury or other financial regulate 16 what's called fintech?

17 And the gut reaction is to regulate them the way you've been regulating 18 everything else and to essentially put in place a lot of barriers, rather than tilt a little bit, I mean 19 you can't tilt all the way to put anything out in the market, but tilt a little bit towards let's enable 20 those innovations to come to market and let's iterate and test all the time.

21 And that's where I think the whole notion of regulatory sandboxes that I think 22 Treasury's doing here with fintech, a number of our competitor countries are using regulatory 23 sandboxes. And that may be something to think about in terms of these new areas.

57 1 I completely agree with you that when innovators are coming up with new 2 things, and SMR or whatever it might be, having to go through that regular same process is an 3 innovation killer and they need to be able to find ways that are a little more flexible and iterative 4 and responsive.

5 And I get that there are all these rules and regulations about how you can only 6 communicate in certain ways and all that, but to me, figuring ways through those or around 7 those so that you can really -- you want to frankly be helping these people, at the same time 8 while maintaining your core mission.

9 COMMISSIONER BURNS: And that's the thing. I think some of them have 10 seen this as this is like the Chinese wall, how do I get over that?

11 Whereas there are ways of working through that process that meet our 12 interest in assuring safety or security, and it's partly making that more transparent, more visible 13 and working through it. Anybody? Mr. Di Frances?

14 MR. DI FRANCES: I just wanted to share, Dr. Atkinson made me think of -- I 15 mentioned projects at the FDA and I think the FDA could maybe be an analog to a certain 16 extent.

17 And what they've done is there's two fellows working there but there's two 18 different groups. One is this Pre-Cert group and they really have created an incubator within 19 the FDA to look at how to regulate.

20 The FDA didn't really have an approach for regulating digital devices because 21 how it works right now is if you want to update software for a digital device, you have to go 22 through the entire process. And so they're pre-certifying certain companies and they're doing a 23 pilot program to enable these companies to then do regular software updates without having to

58 1 go through a rigorous process every time.

2 But just the idea of creating an accelerator group that's protected within a 3 regulatory organization that has a little more autonomy and flexibility to come up with maybe 4 new models I think could be useful.

5 COMMISSIONER BURNS: Thanks. My time is over, I want to thank you.

6 You added to my reading list, including I found at home a copy of Wired magazine on 7 blockchain so I've got to read that too. Thanks very much.

8 COMMISSIONER BARAN: Commissioner Caputo?

9 COMMISSIONER CAPUTO: I'm just going to start by saying wow, what a 10 panel. A lot to think about, very energizing, inspiring, I think about just changes possible and a 11 lot of input on how to go about it.

12 But given the nature of the panel, I'm going to focus my questions in the area 13 of financial management.

14 As I stated at a recent conference, allocation of resources is a major 15 instrument of policy for any agency. It's the statement of our priorities, it's the means to 16 achieve our objectives. So, while for the NRC our safety mission is top priority, that doesn't 17 mean our financial management should be excluded from transformative thinking and the 18 benefits of data analytics.

19 So, with that as a preface, I have a collection of observations I=d like to share 20 to provide a little context for my questions.

21 Mr. Wetklow referred to the Treasury Department's white paper, the Future of 22 Financial Management, which I really appreciated. The paper makes several points.

23 Pressure to reduce the size of Government will continue, detailed financial

59 1 data should be available to citizens interested in how their tax dollars are spent.

2 The use of data is crucial to the future of Federal financial management and in 3 the future, CFOs will play key roles in increasing the transparency of Federal financial and 4 performance data, thereby building trust in Federal financial management.

5 So, starting from those comments and in the spirit of what we call the NRC's 6 straight talk and realism, I'm going to share some observations that I've made just in my first 7 nine months at the Agency.

8 With regards to our financial operations, we have software systems 9 custom-developed by contractors for budget development and execution. Over time, our 10 budgets have evolved to contain less detail and transparency than in past years. We used very 11 little data on actual expenditures or performance to inform budget development.

12 Our custom accounting systems and complicated cost accounting codes get 13 reconfigured periodically so we can't make accurate spending comparisons over time and 14 discern patterns.

15 We have very little performance tracking with regards to whether we complete 16 work according to resource and scheduled plans, we are facing increasing criticism for our fees 17 and invoicing, we have yet to implement electronic billing, and our licensees have no 18 transparent real-time tracking of progress on their licensing reviews.

19 Another observation is when I ask questions about how much we've spend on 20 a task or an issue, it takes days or weeks to get an answer.

21 So, as I've been thinking about this as a Commissioner, I've been thinking 22 about how can we get to one system where we can compare actuals and budget, use data to 23 inform our decision-making?

60 1 But certainly after what I've read from what Mr. Wetklow provided and what 2 I've heard this morning, I feel like I'm stuck thinking about yesterday's solution when there's a 3 whole new paradigm out there that's unfolding.

4 So, what I want to ask the panel is based on what I've just shared, are we in a 5 position to make a technology leap here?

6 MR. WETKLOW: I think you are and I think all the problems or challenges 7 you just went through, NRC's not alone with that. We have that at NSF. Before NSF I spent 8 eight years at Homeland Security and your comments about what we did then versus now, I 9 was there right around -- and I had a very good career boom in terms of that Sarbanes-Oxley 10 stuff in the early 2000s.

11 But that was 2000, agencies didn't have audit opinions back then and now I 12 think the time is just now to move into these things.

13 So what we're doing at NSF, I think it's a really refreshing aspect of the new 14 administration because I think when they came on board, they looked at this compliance 15 infrastructure that has just built up over the '90s, the 2000s.

16 And what their policy directions are giving us now and the innovation 17 programs you have with the fellows and GSA, it's like there's so many opportunities to change 18 what we're doing.

19 So from a financial management perspective, because that's what I do, we're 20 looking at things and saying how can we reduce the number of systems that we do?

21 I don't know if you meant to say -- with electronic invoicing, we had two years 22 ago all these paper invoices coming in and we had people doing mail and stuff. And we're a 23 very small agency like NRC. We had two people that we lost and it really hurt, and we can't go

61 1 out and hire someone to open the mail.

2 It doesn't make sense to do that so we worked with the Treasury Department 3 with their electronic invoicing initiative and found that was something outside of our systems that 4 we could leverage. And we just don't have paper anymore, we didn't have to hire more people 5 to open the mail.

6 So we are systematically going through things and stopping it. And it's a little 7 scary to stop it because people wonder is something bad going to happen when you stop it?

8 But spending more of our efforts getting things outsourced to Treasury, not 9 maintaining -- I mean I have accounting systems in my portfolio but instead of building and 10 managing a system, it's like how can we oversee services so we can spend more of our time 11 getting to the data?

12 So what we're really struggling with is just years of compliance and policy 13 emphasis that now is a breath of fresh air. So I think now more than ever is the opportunity to 14 kind of move forward to these advancements.

15 But NRC's not alone, NSF has many of the challenges that you describe.

16 COMMISSIONER CAPUTO: So Mr. Di Frances, is this the kind of thing a 17 presidential fellow could help us with?

18 MR. DI FRANCES: Yes, certainly. I think that we bring experts from all 19 different types of industries. And one interesting part of our process is that we don't just assign 20 a fellow to an agency, the agencies get to interview the fellow as part of the process to 21 determine if they want to move forward with that particular candidate.

22 So, for instance, if we had someone with a financial management technology 23 background, that could be a senior advisor that could be very helpful potentially for you.

62 1 Also, there are other groups within TTS that I think could be beneficial.

2 There's a TTS Office of Acquisitions, which really helps agencies really better 3 understand how to bring in the best technology vendors and navigate the acquisitions process, 4 which can be very difficult and often doesn't result in the best technology vendors being chosen.

5 And then there's another group called 18F and it was an off-shoot of the PIF 6 Program and it's a digital consultancy internal to government.

7 But they bring in similar types of individuals and they go off in teams and 8 contract with agencies to really help them move to new systems, build new products, and 9 services.

10 So I think a combination of the different groups, or one of them, could be 11 beneficial in helping you transform and jump to some new technology.

12 COMMISSIONER CAPUTO: Thanks. Sure, go ahead.

13 DR. ATKINSON: I was going to say given the fact that almost no major 14 Fortune 100 or 500 corporations have their own software systems, they just don't anymore, they 15 all us COTS, commercial off-the-shelf technologies, and they all use cloud-based COTS, that to 16 me ultimately is where the Federal Government has to go.

17 Having custom-built systems is just really -- most of what you do is not unique.

18 It's just everybody else is doing that same thing. And so we don't use any of our own stuff, we 19 just go in the cloud and we do it, we customize it.

20 It takes a day to customize it and we're up and running. I just think ultimately 21 that's probably where you're going to need to go, and whether you can do it from a budget 22 perspective is a whole set of other questions.

23 But that's what these companies do that you buy from, they just spend every

63 1 day figuring out how to make the next version better than the current version.

2 COMMISSIONER CAPUTO: Okay. Ms. Blakey, so given how crucial data 3 and analysis are to informed decision-making, can you just describe maybe does GSA have 4 products or tips on how we can improve our data and analysis usage to better inform not just 5 financial decisions but programmatic decisions as well?

6 How to just harness metrics, the data that come from the metrics, how to 7 analyze that data to just give us a bed for learning how to improve the nature of what we're 8 doing?

9 MS. BLAKEY: I think it really -- there are lots of answers to that question so 10 I'm happy to follow up afterwards and make some connections there. There are lots of 11 wonderful data geeks at GSA actually in a variety of shops and it really depends on what you're 12 looking for.

13 In my home office, we've actually been here to NRC before, we've done a 14 whole day-long set of sessions on organizational performance management topics. And we did 15 touch base a little bit on some measurement processes, right?

16 How do you dig deep and make sure you're measuring the right things and 17 prioritize that work? So those are the kinds of things that that shop can do. But there are, 18 again, some data teams in GSA and other parts of the Office of Government-wide policy that 19 really are great at advising.

20 Also in my home shop, there is a person who really focuses a lot on 21 benchmarking and so we're looking at whether you're benchmarking financial information across 22 agencies or benchmarking human capital information across agencies.

23 But that's actually very helpful for you to figure out where you stand. In some

64 1 cases you realize you're not as bad off as you thought and you still want to move forward.

2 And in some cases when that benchmarking happens, they can point you to 3 some of the agencies who were doing it very well and make those connections.

4 And sometimes just getting that advice and being able to have that peer or 5 partner talk with you about what they did, how they did it, what to watch out for can be really 6 helpful.

7 So I'm happy to follow up after this if we have more specific questions and 8 maybe make some connections.

9 COMMISSIONER CAPUTO: Great.

10 MR. DI FRANCES: Just one thought on that. I think it could be helpful in 11 structuring the organization in a way that you can access the data. I know that there's a lot of 12 agencies now moving towards chief data officers or senior data advisors. Also one of our 13 fellows came from Microsoft and he was a senior data scientist.

14 But what they did there is there was all these individuals groups that own their 15 data and so they pulled it all out and they created a group that sat above all the different groups 16 and managed the control for all the data.

17 And so that's what some of the private sector companies, leading technology 18 companies, are doing.

19 But just thinking about your organizational structure to make sure the data is 20 accessible and then you have to think about more cleaning up your data, tapping into your data, 21 and making it useable.

22 MR. WETKLOW: One thing I'd offer to you and I think this is coming soon 23 any time now, the administration has as new Federal data strategy that's coming and it has

65 1 these principles to work towards. And we're using that, and I mentioned all the compliance, 2 financial management stuff we do. We're trying to aim our efforts more towards that Federal 3 data strategy.

4 I think my understanding is it's coming out in April. It's been shared in draft 5 but that is like a guiding star for us to look forward to.

6 MS. BLAKEY: Absolutely, and there are lots of pieces to that and lots of 7 groups that are participating.

8 One of my next projects, I'm going to have to have a foot in two worlds, is to 9 work with OMB on the data ethics framework, a government-wide data ethics framework so that 10 everyone has something to latch onto.

11 And we customize underneath that of course, but a government-wide data 12 ethics framework and a government-wide data science curriculum so that we make sure to find 13 all the data geeks in our government and make sure that they have career paths to move 14 forward from wherever they are into things like advanced analytics, because the field has 15 certainly grown.

16 And being able to look at where do people go to get training and information 17 and resources about how to do those types of better for themselves?

18 Some of those things you can handle internally, you're not going to want to 19 bring somebody in to do that, but in many cases, if you actually really are going to do a pretty 20 serious transformation, I think it is helpful to get some help.

21 But along the way there are lots of resources that we and others want to build 22 out so that you have those available to help guide you.

23 COMMISSIONER CAPUTO: Thanks.

66 1 COMMISSIONER BARAN: Thank you. Commissioner Wright?

2 COMMISSIONER WRIGHT: I share Commissioner Caputo's wow. So I get 3 the bat cleanup today. It's not a bad spot to be in.

4 I've had the benefit of listening to your presentations which I've enjoyed, and 5 they do make you think, and then getting to hear the questions from my colleagues.

6 So it's eliminated some of the things I might want to ask about but it calls me 7 to think of a couple of other things. So, really, I want to thank you for taking the time to travel 8 here today and to share with us.

9 And Ms. Blakey, I was looking at your bio, we shared a few things in common.

10 You come from a local and state government background as I do, and I know I learned some 11 things at that level that I try to carry forward here, and maybe that's been the same experience 12 for you.

13 And to my Commissioners, she is from the Federal Government today and 14 she really is here to help, so I'm very grateful for that.

15 So, you've heard the words innovation, transformation, reinvention, change 16 management among others today.

17 But I don't think it means the same everywhere, from agency to agency I just 18 don't think it means the same. And private business versus Federal Government, it can't be 19 the same either.

20 I'm not this smart that I thought of this on my own but I did look it up.

21 Facebook has a motto and their motto is move fast and break things. That wouldn't work here, 22 not at all.

23 And other technology-based innovators have that type of a mindset too, which

67 1 would not work here. So to go back to Margie's and our EDO's comment earlier, we've got to 2 identify what do we want to transform to?

3 Transformation just to transform doesn't mean anything unless you know what 4 you're really trying to get out of it. So, I'm real interested to figure it out.

5 Is it possible for an Agency this size to go through the efforts of transforming 6 using the people completely from the inside? Or do we need outside help to help us get there 7 along the way?

8 Not just an initial report or something like that, but do we need to use that 9 along the way? Any comment about that?

10 MS. BLAKEY: It requires cognitive diversity and I don't know enough about 11 the workforce at NRC to know if you have that kind of diversity that you need. If you don't, you 12 do need to bring in some partners.

13 But it's decision theory, right? There's a lot of we know about 14 decision-making and even though it's messy and uncomfortable and takes longer, you will 15 always get better decisions when you have a diverse set of people in the room coming at it from 16 different perspectives.

17 So I think that's the first assessment, looking at the cognitive diversity within 18 your workforce. If you've got a lot of nuclear geeks, I'm making this word up, and that's 80 19 percent of your workforce, you might not have enough cognitive diversity to get through that.

20 But I don't know the answer to that question, I think that's one of the first 21 things you need to ask.

22 MR. WETKLOW: I have one thought.

23 I think NRC, I'll just share a little bit with NSF, we are a lot alike, NSF and

68 1 NRC, but I think we're on different spectrums because I think the comments about type 1 versus 2 type 2 errors with NRC's mission -- I mean, we have the error of if we miss a scientific 3 innovation, so our failure acceptance might be calculated a little differently.

4 But to your comment about is it possible for NRC? Just coming from NSF, 5 our agency's size is very similar and very focused.

6 And then I mentioned earlier I came from Homeland Security where we had 7 22 programs, I think it was about a dozen agencies.

8 It was really hard to turn that ship. So we literally mapped out the pockets of 9 opportunities and then just grew small every time. Because I was there for the first eight years 10 of Homeland Security. And I have found at NSF that is our niche, that we're so small because 11 we're just so nimble and I just feel that coming in. You all are set up physically like NSF.

12 So I would just highly encourage not so much -- what we do at NSF is we look 13 at the folks we have but we are getting help from the people we have here.

14 With GSA, the 10X program, we are getting the venture- capitalist-type 15 aspects of it to help us build a blockchain pretotype.

16 And then I think there was a comment earlier about recognition with the 17 fellows and the President's Management, there's a Gears of Government award that we're just 18 very proud that one of our employees just won that last week.

19 So, this is what I'm doing at NSF, don't feel like we have to do it all ourselves 20 and figure it out, but go out and get help.

21 Because that wasn't completely there in years past with the GSA, OPM, the 22 central oversight agencies, if that helps.

23 COMMISSIONER WRIGHT: So Ms. Blakey, the model that you showed, the

69 1 jump off and get to the top and if you don't move you fall, and I get that and I understand it.

2 And the people here know and the people in the Agency know I'm a Stage 3 3 colon cancer survivor. Had they treated me the same way they did 40 years ago, I wouldn't be 4 here talking to you.

5 So, when you get to that innovative point, if you don't continue to innovate, as 6 somebody said, you don't die but you struggle, I think is the word you used.

7 So, is innovation as simple as sometimes recalibrating? Or does it include 8 recalibrating? Do you consider that innovative?

9 MS. BLAKEY: I think this would be a great conversation for Joshua and all of 10 us really to have, and sometimes it ends up being semantics. My definition is very, very broad.

11 12 If you find a way to reframe -- I also am a coach so for us we use a lot of 13 language around reframing -- if you can reframe what the problem is or even how you look at 14 the problem or reframe how you look at the opportunity, it sets up a different way for you to think 15 about how to solve for it or to do things differently.

16 And yes, I do think sometimes it's as simple as getting started there. Now, 17 there's a lot of work to do after that but I do believe very often it starts with reframing which is, 18 again, back to my comment about cognitive diversity.

19 If you've got folks who are operating and have been operating in a certain way 20 and they're trained in a certain way and they think a certain way, you're not going to have as 21 many opportunities for reframing.

22 Because you don't have somebody coming in from left field tilting things in a 23 different way so that you can step back a little bit from what you're used to doing and what

70 1 you're comfortable with and, frankly, what works, right?

2 We're protecting America from nuclear incidents. You've got to have that 3 cognitive diversity to do that.

4 Challenging, I think you talked earlier about just challenging assumptions and, 5 yes, I think it's as simple as reframing and rethinking things can help you do that.

6 Whether it leads directly to a dramatic innovation or not or a dramatic 7 transformation or not is not the point, but being able to challenge those assumptions and 8 reframe.

9 COMMISSIONER WRIGHT: So you just identified exactly one of the biggest 10 problems here and that is people who have been here doing things the same way because it is 11 safe, there's nothing wrong with it.

12 There's absolutely nothing wrong with it, but if you're going to try to get more 13 efficient and do things better, because of data, because of other things that are there, there is 14 the opportunity to recalibrate.

15 And I guess my question is looking at how do we do that? Where do we go 16 to get that help? Because we may not be able to do that from within initially to get -- I guess it's 17 a trust thing or whatever moving forward.

18 So I see that as an area where we're going to need some help.

19 I've got about a minute left. Do you have any recommendations about what 20 not to do when you're trying to transform? Are there pitfalls we should try to avoid?

21 MR. WETKLOW: Two quick things. One, I would not take on too much at 22 one time. I use a little silly joke about this, I would not try to build the Death Star because, 23 spoiler alert, it blows up. And then I don't think we talked about this enough, but the people

71 1 aspect of it is enormous because the change we're going through, it makes people nervous.

2 And helping them understand that they are going to have so many more 3 opportunities, career opportunities, learning about data science.

4 So two things, I would go really small, don't try to build a Death Star, and then 5 really hone in on the people aspect of it.

6 COMMISSIONER WRIGHT: Thank you very much.

7 COMMISSIONER BARAN: Well, thank you again to all of you for being here 8 today and sharing your experiences and thoughts with us. I think it was a great discussion.

9 Do any of my colleagues have any concluding questions or comments?

10 Okay, well, thank you, we are adjourned.

11 (Whereupon, the above-entitled matter went off the record at 11:11 a.m.)