ML18274A326

M180927: Transcript-Strategic Programmatic Overview of the Operating Reactor Business Line

ML18274A326
Person / Time
Issue date:	09/27/2018
From:	NRC/OCM
To:
References
Download: ML18274A326 (80)

Text

UNITED STATES NUCLEAR REGULATORY COMMISSION

+++++

STRATEGIC PROGRAMMATIC OVERVIEW OF THE OPERATING REACTORS BUSINESS LINE

+++++

THURSDAY, SEPTEMBER 27, 2018

+++++

ROCKVILLE, MARYLAND

+++++

The Commission met in the Commissioners Hearing Room at the Nuclear Regulatory Commission, One White Flint North, 11555 Rockville Pike, at 10:00 a.m., Kristine L. Svinicki, Chairman, presiding.

COMMISSION MEMBERS:

KRISTINE L. SVINICKI, Chairman JEFF BARAN, Commissioner STEPHEN G. BURNS, Commissioner ANNIE CAPUTO, Commissioner DAVID A. WRIGHT, Commissioner ALSO PRESENT:

ANNETTE L. VIETTI-COOK, Secretary of the Commission MARIAN L. ZOBLER, General Counsel

2 NRC STAFF:

MARISSA BAILEY, Director, Division of Security Operations, Office of Nuclear Security and Incident Response (NSIR)

DAVID CURTIS, NSIR/DPCP MARGARET DOANE, Executive Director for Operations RUSSELL FELTS, Deputy Director, Division of Risk Assessment, Office of Nuclear Reactor Regulation (NRR)

CHRISTOPHER J. FONG, NRR/DRA/APHB MICHAEL X. FRANOVICH, NRR/DRA CHRIS MILLER, Director, Division of Inspection and Regional Support, NRR HO NIEH, Director, NRR ROBERT TREGONING, Senior Level Advisor for Material Engineering Issues, Office of Nuclear Regulatory Research ANTON VEGEL, Director, Division of Reactor Projects, Region IV

3 1 P-R-O-C-E-E-D-I-N-G-S 2 (10:02 a.m.)

3 CHAIRMAN SVINICKI: Good morning, everyone, and 4 welcome to the Commission meeting this morning.

5 It looks like we have a lot of interest in the topic which is 6 always, it's kind of a content-rich program, but the Commission meets this 7 morning to hear from a panel of NRC staff regarding, well it's listed here as 8 discussion of strategic considerations associated with the operating reactors 9 business line.

10 This is, of course, the largest business line in the Agency's 11 budget and it involves principally the Office Of Nuclear Reactor Regulation, 12 but as we will hear today and is certainly is known by all of the NRC 13 employees in the room is that it is a business that is supported by, I think, in 14 some dimension basically every other part of the Agency beyond -- NRR is 15 also involved in supporting this business line.

16 And, of course, all of the important work carried out in the 17 NRC regions is an element of, or impacts in some way on this business line 18 as well. So, again, we will have one panel with a number of staff presenters 19 who are going to report on kind of a smattering of different topics and then 20 my colleagues and I will be able to engage in a question and answer period 21 with any topics that we care to raise.

22 And in looking at the topics I think that there is a lot to 23 cover today. Before we begin with the staff panel though do any of my 24 colleagues have any opening remarks that they would like to make?

25 (No response.)

4 1 CHAIRMAN SVINICKI: Okay, no. Well, I will just begin 2 by again thanking the staff for all the hard work to prepare for this meeting 3 and I will turn this over to our Executive Director for Operations Margie 4 Doane who is for the first time appearing in this capacity at a Commission 5 meeting.

6 This is also true of Mr. Ho Nieh as the Director of the 7 Office of Nuclear Reactor Regulations, but you are both veterans of 8 Commission meetings. So I will turn it over to you, Margie.

9 MS. DOANE: Okay. And I think the first time as general 10 counsel on the end of the table, too, so it's a lot of firsts.

11 CHAIRMAN SVINICKI: Yes, but if things go well she does 12 not need to intervene and say anything.

13 (Laughter.)

14 MS. DOANE: Good point. Good point. Yes, there will 15 be a lot of firsts if that happens, right. Okay, so good morning, Chairman 16 and Commissioners. Like the Chairman said this is my first meeting as the 17 Executive Director for Operations, but I spent many years at this table with 18 the Commission, six years at just about every Commission meeting that has 19 been held over that time as I was general counsel.

20 So I want to just talk a little about I have been in this job for 21 about three months now, just shy of three months, and what I have 22 confirmed in my relatively short time as EDO is that the staff has tremendous 23 technical expertise and commitment to our safety and security mission.

24 They have welcomed me into this position and provided 25 outstanding support to get me up to speed on the operational issues that I

5 1 had not had the responsibility for as the general counsel.

2 As you know I have been focused on activities that the 3 Agency needs to take to ensure that we are staffed with the right skills and 4 perspectives to address the current and future issues coming before the 5 Agency.

6 As the Chairman just said it is very fitting that this is the 7 Commission meeting that I would begin my tenure with because it does 8 touch just about every office in the Agency.

9 It involves employees from NRR, NSIR, NRO, Research, 10 the regions, and other offices. One thing that I want to stop and just take 11 some consideration of is that this business line is affected by the reduction in 12 the number of operating reactors in the nuclear fleet.

13 We are proactively managing the workforce and budget 14 adjustments appropriate to reflect this changing workload. We don't wait 15 until the plants close to plan this.

16 As the remaining operating plants look for new ways to 17 improve their performance they continue to request licensing actions that 18 challenge the way we have always approached our work.

19 New reactor technology raises similar challenges and that 20 it's new ways of looking at old things that we had been doing. In this 21 briefing we will highlight the activities we are undertaking to ensure we are 22 focused on the most safety and security significant issues.

23 It is essential that we make this turn to ensure that neither 24 we nor the industry miss out on the opportunities that these new approaches 25 or new technology offer to improve safety and security.

6 1 We look forward to discussing the important issues with 2 you about the NRC's role in making regulatory decisions in licensing and 3 oversight of these operating reactors and our role to protect public health 4 and safety and security and the environment.

5 Next slide, please. I would like to now introduce the NRC 6 team at the table who will talk about important aspects of this business line.

7 I would like to welcome Ho Nieh, as the Chairman said, our new Director of 8 the Office of Nuclear Reactor Regulation.

9 He started his post last month after serving as the Director 10 of Nuclear Safety Division at the Organization for Economic Cooperation and 11 Development's Nuclear Energy Agency in Paris.

12 But, of course, you likely know that Ho has spent most of 13 his professional career at the NRC with stints overseas at the NEA and at 14 the International Atomic Energy Agency in Vienna. Ho will provide an 15 overview of NRR's efforts.

16 Russ Felts, at the end of the table, is the Deputy Director 17 of NRR's Division of Risk Assessment and he will discuss the workload 18 management and risk-informed initiatives.

19 Next to him, Chris Miller, Director of NRR's Division of 20 Inspection and Regional Support, will talk about potential changes to the 21 reactor oversight process.

22 Tony Vegel, to my left, Director of Region IV's Division of 23 Reactor Projects, will discuss the focus areas of the regions and 24 implementation issues.

25 Marissa Bailey, Director of the Division of Security

7 1 Operations in the Office of Nuclear Security and Incident Response will 2 discuss the security program updates.

3 And, Robert Tregoning, the Senior Level Advisor for 4 materials engineering issues in our Office of Nuclear Reactor Regulatory 5 Research will discuss research infrastructure worldwide to support 6 regulatory decision making.

7 So, again, we appreciate the opportunity to discuss these 8 important issues to you and I would now like to turn it over to Ho.

9 MR. NIEH: Thank you very much, Margie. Good 10 morning, Chairman. Good morning, Commissioners. It's a real honor to be 11 here today before you in my first appearance as the Director of the Office of 12 Nuclear Reactor Regulation and the lead for the operating reactor business 13 line.

14 Even though we may be veterans in doing this it always 15 feels to me like I am preparing for an exam, so it's an unnerving feeling. I 16 don't know if that will ever change.

17 (Laughter.)

18 MR. NIEH: So I have only been in the job for one month 19 and I am still drinking from the proverbial fire hose and I really have to thank 20 the NRC staff and managers who have taken the time to really brief me on a 21 whole variety of technical issues to get me up to speed and to move me 22 further along the learning curve.

23 With that said I may have to throw a lifeline out if you give 24 me some really, really tough questions. It is really great to be back to 25 engage with and learn from many of my former colleagues and new

8 1 colleagues on the issues that are current in NRR.

2 And one thing that I have heard very frequently in these 3 discussions, this phrase "while you were away." So it's very clear to me that 4 things have changed while I have been away, and I have been away for a 5 little over three years, and after this first month I have certainly noticed that 6 there is an increased focus in improving our workload management and 7 prioritization, better use of risk information as a complimentary input to our 8 regulator programs and our decisions.

9 And I have also sensed a growing enthusiasm toward the 10 NRC's efforts to transform how we do business. I think that it's a really 11 exciting time to be here at the NRC and I really look forward to leading the 12 business line in light of this opportunity before us.

13 And while things have changed one thing that has 14 remained constant is the staff's strong commitment to the safety and security 15 mission. That commitment is just as I remember it before I left to go work at 16 the Nuclear Energy Agency almost three years ago.

17 And having worked in an international nuclear safety 18 organization, not only once but twice, I can tell you that the NRC is a 19 well-respected leader in nuclear safety regulation.

20 Our technical competence and regulatory expertise is often 21 looked at as a model by many of our international peers and I am very proud 22 to say that. I do want to thank the business line partner offices in the 23 regions and in headquarters for their excellent contributions to the operating 24 reactor programs.

25 Most recently the business line partners worked very

9 1 effectively together under the leadership of the NRC's Region II office in 2 Atlanta to prepare for and monitor Hurricane Florence as it made landfall in 3 the United States just a couple weeks ago.

4 And many of you know that the NRC was very 5 well-prepared for that event and that there were no, none of the reactors or 6 other nuclear facilities in the path of this storm experienced any significant 7 impact.

8 The business line partners have continuously sought to 9 improve how we are implementing their operating reactor programs. Most 10 recently we made available on the NRC's public website an operating 11 reactors performance dashboard that will provide up-to-date information on a 12 monthly basis on how we are performing in our licensing activities both in 13 terms of performance and timeliness as well as in other areas.

14 We also issued Inspection Manual Chapter 0611 that 15 provides an improved, an easier to read inspection reporting format. We 16 are going to combine that effort with some online tools we are developing to 17 help our inspectors more easily generate their inspection reports.

18 And by the way I am on Slide 3 if you are following along.

19 The strategic level the business line is continuing on its effort to enhance its 20 focus on safety, particularly those issues that are most safety significant, and 21 also on its mission of reasonable assurance through the better use of risk 22 information and also more consistent application of the principles of good 23 regulation.

24 As I mentioned before this is a really exciting time to be at 25 the NRC. I think there are a lot of opportunities to make some positive

10 1 changes on how we are using our resources.

2 The NRC staff do a lot of things very, very well and I am 3 really encouraged to learn that and find out that we all want to find ways to 4 do it better.

5 While these efforts to transform our work is underway the 6 business line is going to stay on top of several important programmatic 7 priorities. These priorities include implementing the Digital Instrumentation 8 and Control action plan to expand the use of digital technologies at operating 9 reactors, supporting domestic efforts in coordination with the Department of 10 Energy and the industry to safely deploy accident tolerant fuels, preparing for 11 the merger of the Office of New Reactors and NRR in 2020, and also better 12 aligning the business line budget and the workforce to the anticipated 13 reduction in the workload that we foresee over the next couple years.

14 So that concludes my introductory remarks. I would next 15 like to turn the staff's presentation over to Mr. Russ Felts who will give you a 16 little bit more about our risk-informed decision making efforts as well as 17 workload management. Thank you.

18 MR. FELTS: Good morning, Chairman. Good morning, 19 Commissioners. In the next several slides beginning on Slide Number 4, I 20 will be discussing staff efforts to leverage risk insights in our work.

21 Risk insights enable us to distinguish the more safety 22 significant aspects of a licensing review from the less safety significant and 23 can, thus, conform the level of rigor appropriate across the scope of review 24 aiding in workload management.

25 I will touch on challenges the staff has faced in leveraging

11 1 risk insights in our work and ongoing actions to address those challenges. I 2 will also discuss efforts to maximize efficiency of risk-informed licensing 3 reviews, including factors that complicate those reviews and actions to 4 address those complicating factors.

5 In a November 2017 Information Paper to the Commission 6 entitled "Plans for Increasing Staff Capabilities to Use Risk Information in 7 Decision-Making Activities" staff described six areas where we recognized 8 challenges to further progress in risk-informed decision making.

9 The challenges relate to staff knowledge of and attitudes 10 toward use of risk, process impediments embedded in procedures and 11 guidance, and issues with the state of probabilistic risk assessment 12 technology itself, such as unrealistic modeling assumptions and variation in 13 the maturity of PRA models across the industry.

14 In addition to describing these challenges the Paper laid 15 out five overarching strategies that we were implementing to enhance 16 integration of risk into regulatory decision-making practices and processes, 17 improve the technical bases for regulatory activities, and increase the 18 efficiency, effectiveness, and consistency of risk-informed decision making.

19 These strategies to evaluate and update guidance to 20 develop a graded approach to use risk information and licensing to enhance 21 mandatory training, to advance risk-informed initiatives, and to enhance 22 related communication were incorporated into an NRR action plan in August 23 of last year.

24 Slide 5, please. The action plan has two phases. In 25 Phase I staff focused on evaluation and update of guidance and

12 1 development of a graded approach for using risk information in licensing 2 reviews.

3 Phase I started with eight tasks, which were led from staff 4 at various divisions across NRR. Task leads and SES sponsors from the 5 Divisions of Engineering, Safety Systems, Operating Reactor Licensing, and 6 Risk Assessment led development and implementation of the plan. Phase I 7 was completed in June and resulted in 19 recommendations.

8 During Phase II, which is in progress now and will run 9 through December, we are implementing those recommendations grouped 10 into 13 action items. As part of Phase II we are using integrated review 11 teams for select submittals and applying a graded approach to using risk 12 information.

13 Consideration of risk insights may benefit a review even if 14 no PRA information is included in the submittal. So assignment of an 15 integrated review team is considered for all submittals.

16 Submittals lie along a spectrum defined by the utility of 17 probabilistic risk insights in the review. On one end of this spectrum where 18 a submittal clearly meets established deterministic criteria and can, thus, be 19 approved with no additional consideration of qualitative or quantitative risk 20 insights, use of an integrated review team would unnecessarily expand the 21 review effort.

22 On the other end of the spectrum an application that meets 23 established risk-informed criteria and can thus be approved without further 24 technical reviewer participation. An integrated review team would similarly 25 unnecessarily expand the review effort.

13 1 For all those applications in between these ends of the 2 spectrum bringing together risk analysts and technical reviewers from 3 various disciplines, such as safety systems, electrical, or materials, may 4 provide synergies in the review.

5 It is when we anticipate these synergies that an integrated 6 review team will be considered. Risk analysts working alongside technical 7 reviewers can develop improved understanding of plant systems and 8 operations, including system interactions and interdependencies.

9 Technical reviewers working alongside risk analysts can 10 develop improved understanding of the relative safety significance of 11 structure systems and components that should inform the focus and rigor of 12 their reviews.

13 When an integrated review team is employed the team will 14 collaboratively develop consolidated requests for additional information, 15 conduct consolidated audits, and develop consolidated safety evaluations.

16 Behind the integrated review team process lies a set of 17 tools associated with each step for efficiency and to provide consistency.

18 For example, safety evaluation templates tailored to the importance of risk 19 insights to the regulatory decision have been developed streamlining the 20 review documentation process.

21 We anticipate initial impacts to review hours and timeliness 22 as the teams form and storm before they norm and perform. For example, 23 acceptance review time may be affected due to the additional effort needed 24 to determine whether and how risk insights may be used, whether or not to 25 employ an integrated review team, and to reliably forecast review hours.

14 1 As part of Phase II of the action plan metrics specific to 2 risk-informed reviews are being developed to enable tracking and trending of 3 risk-informed review performance.

4 The strategies to advance risk-informed initiatives and to 5 enhance communication both cut across the entire action plan. Examples 6 of risk-informed initiatives include Technical Specifications Initiatives 4B, 7 which is risk informed completion times, and 10 CFR 50.69, risk informed 8 categorization and treatment of structure systems and components.

9 Advancing these initiatives is heavily dependent on 10 industry action and, thus, relies on the communication with industry. The 11 importance of effectively communicating is key to keeping stakeholders both 12 within and outside the NRC informed and to hear, understand, consider, and 13 sometimes influence their points of view as we move forward.

14 Finally, in addition to our focus within the office we are 15 working with operating reactor business line partner offices, including the 16 regions, to advance risk-informed decision making.

17 Slide 6, please. There has for some time been an 18 extensive catalog of risk training courses available to all staff and risk 19 training is embedded in licensing reviewer and inspector qualification 20 processes.

21 We want to raise the knowledge and acceptance of 22 risk-informed decision making at all levels of the staff. And while 23 employment of integrated review teams will increase review staff knowledge 24 of an appreciation for an acceptance of the value of risk insights.

25 It is also important to achieve these same outcomes in the

15 1 management ranks. Front line supervisors have tremendous influence, so 2 their buy-in is essential to any effort to achieve culture change.

3 Since the learnings that happened within the integrated 4 review team's may only tangentially influence first line supervisors and will 5 have less impact further up the management ranks we have developed 6 risk-informed decision-making training for managers.

7 This one day course was delivered in part by Dr. Michael 8 Golay from the Massachusetts Institute of Technology and draws some 9 elements of MIT's four day nuclear operational risk management course.

10 Against the brief historical backdrop of PRA the 11 complimentary nature of risk and deterministic perspectives is discussed 12 along with regulatory examples.

13 Also, industry representatives demonstrate risk 14 management tools as they would be used in real time in a nuclear power 15 plant main control room to aid decision making.

16 With Dr. Golay's assistance we are currently evaluating the 17 effectiveness of the pilot training course. Revision of the course and a 18 broader deployment will depend on the results of that evaluation.

19 Slide 7, please. We are optimizing review efficiency by 20 assigning where practicable the same staff to conduct the risk-informed 21 reviews and to conduct multiple risk-informed reviews for the same licensee.

22 We are using lessons learned from reviews to improve 23 future reviews and we are aware of a similar feedback loop in the industry.

24 We are disciplined in our use of requests for additional information.

25 Staff drafts the safety evaluation with gaps, then staff

16 1 drafts RAIs to fill the gaps, and then in most cases staff shares the draft 2 RAIs with the licensee and then conducts an audit.

3 In our experience this significantly reduces the number of 4 RAIs that actually have to be sent and responded to on the docket. Then 5 staff issues only those RAIs necessary to complete the safety evaluation.

6 The workload associated with risk-informed applications 7 has increased significantly and we are ramping up use of contract support to 8 manage the surge and to minimize impacts through new timelines.

9 Division managers from across NRR meet weekly to 10 discuss various risk-informed reviews, associated challenges and priorities, 11 and closely manage related issues.

12 The NRC's and industry Risk-Informed Steering 13 Committees which periodically hold joint public meetings provide a form for 14 discourse that is key in these communications and instrumental in advancing 15 risk-informed initiatives.

16 For example, some of the activities that are needed to 17 advance use of risk rely on industry actions, as I mentioned before. Some 18 of those related to the state of licensee PRAs and the risk-informed steering 19 committees provide a communication channel that is essential to progress.

20 Slide 8, please. For applications that are submitted as 21 risk-informed in accordance with Reg Guide 1.174 in order to determine the 22 technical adequacy of the applicant's PRA the staff reviews key assumptions 23 and sources of uncertainty along with relevant facts and observations from 24 industry peer reviews of the applicant's PRA that are submitted in the 25 submittal.

17 1 When licensees have followed accepted guidance, have 2 PRA peer reviews mapped to current standards and few or no open peer 3 review facts and observations staff reviews are streamlined.

4 For example, we recently issued a license amendment to 5 Limerick enabling the plant to risk-informed categorization of structure 6 systems and components under 10 CFR 50.69.

7 This was the first post-pilot 50.69 application and the 8 licensing review is completed ahead of schedule and using fewer than 9 projected review hours, largely because the licensee's PRA didn't have a lot 10 of open facts and observations.

11 On the other hand, deviations from accepted guidance, 12 open peer review facts and observations that call into question the technical 13 adequacy of the PRA to support the application at hand, or questionable key 14 assumptions or uncertainties make reviews more complex.

15 For example, about a year ago a licensee submitted an 16 application under 50.69. Their application relied on a PRA with a nearly 17 20-year-old peer review that didn't map to current standards.

18 In response to NRC reviewer questions this licensee 19 conducted four focus-scoped peer reviews that resulted in the generation of 20 many new facts and observations.

21 This happened during the conduct of staff's licensing 22 review of the 50.69 application complicating the review. And although the 23 licensee has stated that the overall PRA risk values didn't shift significantly 24 dominant contributors shifted and non-trivial changes are being made to the 25 PRA to address the peer review facts and observations.

18 1 Thus, the technical adequacy of the initial model the 2 licensee was using to support their application prior to these reviews was 3 questionable. For 50.69 it is important to get the dominant risk contributors 4 right to appropriately categorize structures, systems, and components.

5 Similarly, for Tech Spec 4B, which uses numeric values 6 from the PRA in real time to determine acceptable outage times for tech 7 spec equipment rather than the relative risk importance accurately 8 calculating quantitative risk of specific equipment. Being out of service is 9 even more critical.

10 If an application comports with accepted guidance, and is 11 based on a peer review against current standards with minimal facts and 12 observations, the staff's review is simplified.

13 Use of the industry F&O closure process can also reduce 14 the complexity of staff's review. We have been communicating these 15 messages to individual licensees and to industry in conferences and through 16 the Risk-Informed Steering Committee.

17 We have also recently seen the industry reinforcing these 18 messages in their own conference presentations. I am going to turn the 19 presentation over to Chris Miller who will discuss staff activities associated 20 with the reactor oversight process.

21 MR. MILLER: Thank you, Russ. Good morning, 22 Chairman. Good morning, Commissioners. It is my pleasure to brief you 23 this morning on the ongoing and potential upcoming changes to the NRC's 24 reactor oversight process.

25 We have some updates from our June 19th agency action

19 1 review meeting, Commission briefing, where we focused on ROP change 2 efforts. I will be starting on Slide 9.

3 The ROP now in its 19th year is a comprehensive 4 oversight program with elements aimed at making it objective, predictable, 5 transparent, risk-informed, and performance-based.

6 The ROP has been modeled in some fashion by a number 7 of regulators around the world. At the June 19th AARM meeting we 8 discussed how self-assessment and continuous improvement are 9 fundamental components of the ROP and we provided examples of our work 10 to improve the process.

11 Today I would like to update you on the current staff 12 initiatives that will help ensure the ROP is adapted to today's regulatory 13 environment and to further improve efficiency and effectiveness.

14 We have received valuable stakeholder feedback through 15 a variety of sources and continue our work with stakeholders to improve in 16 an open manner using our principles of good regulation.

17 Some of the staff's current efforts, taking into account 18 decreasing resources aim to approve efficiency while enhancing our 19 openness in the inspection and assessment areas.

20 For example, we have developed a shorter and more 21 consistent inspection report format now in use across all regions. We will 22 soon finish efforts with the replacement reactor program system to automate 23 the inspection report process, improve inspection scheduling to smooth out 24 the peaks and valleys of resource utilization across the regions, automate 25 the process for tracking inspection samples, and automate the availability of

20 1 licensee performance information.

2 Our operating experience team is using improved methods 3 to help inspectors get better information on risk-significant issues across the 4 industry while evaluating how data analytic techniques for large amounts of 5 operational data may be used to provide insights about equipment 6 performance and better focus our inspections.

7 The engineering inspection review initiative is wrapping up 8 a comprehensive examination of all our engineering-focused inspections.

9 Over the last two years through extensive interaction with the industry and 10 other stakeholders we developed options to increase the effectiveness and 11 efficiency of the suite of engineering inspections, which will be presented 12 shortly to you in a Commission Paper.

13 In addition, we are working with industry on a potential 14 review of their trial run for use of licensee self-assessments as part of the 15 oversight process for engineering inspections.

16 We are also finalizing our OP program documents to 17 improve the efficiency of our decision making for potentially 18 Greater-than-Green findings as part of the Inspection Finding Resolution 19 Management process, or IFRM.

20 The IFRM is helping to achieve early alignment both 21 internally and externally on important inspection issues and potentially 22 Greater-than-Green significance.

23 The IFRM will enable us to make more timely decisions, 24 improve efficiency of our resources, and maintain reliability within our 25 assessment process.

21 1 Slide 10, please. Next slide. When the NRC 2 transformation team began looking for more substantive ways to improve 3 NRC programs many changes were suggested in the area of improving the 4 ROP.

5 Seventy-two suggestions from internal and external 6 stakeholders were sent to the Office of Nuclear Reactor Regulation after the 7 completion of the transfer information team's effort.

8 In addition, the staff received specific suggestions on 9 possible changes to the ROP in separate communications from the Nuclear 10 Energy Institute and the National Regional Utility Group.

11 We also received feedback from the Union of Concerned 12 Scientists specific to the engineering inspection review and associated 13 licensee self-assessment efforts.

14 Next slide, please. The transformation suggestions were 15 binned into several main areas for ROP change consideration as 16 represented on this slide.

17 Some of the suggestions focused on organization and 18 staffing changes associated with performing inspections, such as changes to 19 the relationship between resident inspectors and region-based inspectors to 20 improve efficiency of inspections, and changes to the NRC organization with 21 respect to how the regional and headquarters offices could be better aligned.

22 Other suggestions focused on documentation of inspection 23 findings offering ways to improve efficiency and effectiveness, such as not 24 documenting green findings, simplifying report formats, reducing the 25 frequency of issuing inspection reports, and switching from inspection

22 1 reports to inspection documentation that is uploaded to the NRC website, 2 thus eliminating mailing out reports to licensees.

3 We also received feedback focused on numerous 4 components of the ROP program, including frequency and scope of 5 inspections, as well as improvements to the Performance Indicator Program, 6 crosscutting issues, the significant determination process, and the 7 assessment process.

8 One particular suggestion involved the NRC's recognition 9 of licensees with sustained Column 1 performance on the action matrix with 10 a reduction of inspections in the baseline inspection program.

11 Next slide, please. Nuclear Energy Institute's proposals 12 included recommendations that would focus NRC and licensee resources 13 more on issues with risk and safety significance, reduce unnecessary 14 regulatory burden, improve program efficiencies, and enhance 15 communication efforts for both the industry and the NRC.

16 The National Regional Utility Group recommended that 17 NRC give licensees more credit for effective corrective actions to address 18 performance issues that result in Greater-than-Green inspection findings.

19 And the Union of Concerned Scientists offered that as we 20 proceed with our engineering inspection enhancements we should use 21 caution and not sacrifice the benefits brought about by our independent NRC 22 inspection, including benefits from inspections that result in no findings of 23 Greater-than-Green significance.

24 Next slide, please. More interactions are needed with our 25 stakeholders to determine what ROP changes should be implemented. Our

23 1 initial assessment shows promise in a number of areas for additional fruitful 2 dialogue.

3 For example, efforts taken to improve efficiency and 4 provide for better integration of the suite of engineering inspections could be 5 a model for other inspection areas.

6 And the work used to develop the framework for the use of 7 licensee self-assessments in the engineering area could also be leveraged 8 in other inspection areas if a successful demonstration is developed with our 9 stakeholders.

10 We will also look at ways to better tailor inspection 11 frequencies and level of effort in the baseline inspection program to account 12 for varying licensee performance.

13 We believe this could provide a strong incentive for 14 licensees demonstrating effective corrective actions in sustained Column 1 15 performance. One potential change in this area, after consideration of a 16 number of licensee performance factors could lead to changes in problem 17 identification and resolution inspection.

18 Other inspections that we might consider evaluating to 19 determine if they are appropriately scoped and focused include special 20 inspections which take place after an operational event or degraded 21 condition, or supplemental inspections, specifically the 95001 inspection, 22 which is used to follow-up on corrective actions for White inspection findings 23 or performance indicators.

24 Changes to the ROP's use of performance indicators is 25 another area that could be leveraged to improve efficiency of the program.

24 1 The initial set of performance indicators used in the ROP was developed 2 using the best set available, taking into account plant risk a number of years 3 ago.

4 With the exception of the Mitigating System Performance 5 Index, or MSPI, these indicators have not changed substantially over the last 6 18 years. However, industry performance indicators used outside of the 7 ROP have improved.

8 Staff plans to discuss with industry whether modifying the 9 current set of PIs could be used in combination with the inspection program 10 changes to ensure adequate oversight while potentially reducing regulatory 11 burden.

12 Next slide, please. Numerous suggestions received from 13 both internal and external stakeholders addressing a wide spectrum of 14 potential changes to the treatment of White findings.

15 Examples include eliminating White findings altogether, 16 changing how the number of White findings in a cornerstone of strategic 17 performance area affects movement in the action matrix, reducing the 18 amount of follow-up inspection after a White finding, and changing 19 thresholds or practices for press releases.

20 We think a common theme that can be addressed is 21 ensuring that the significance of White findings or performance indicators is 22 properly communicated and inspection resources and the impact on licensee 23 assessment are applied appropriately.

24 As an example we are considering steps to improve the 25 incentive for licensees to correct White findings and to make the process for

25 1 White performance indicators more closely align with that of White findings.

2 This could involve the removal of White findings from the 3 action matrix once staff completes the supplemental inspection and once 4 satisfied that the licensee took the appropriate corrective actions without 5 waiting the currently required four quarters and we would consider treating 6 White performance indicators in the same way.

7 Staff were also looking to make improvements to a number 8 of significance determination process procedures to make them more 9 effective and address feedback from both NRC staff and our external 10 stakeholders.

11 The ongoing work to evaluate the emergency 12 preparedness SDP, which is currently being reassessed through a focused 13 review under the ROP self-assessment process, is just one good example.

14 Finally, we had previously identified that the crosscutting 15 issues program needs additional attention to make it a more effective 16 regulatory oversight tool.

17 Staff will continue to assess the CCI program in 2019 as 18 part of the annual ROP self-assessment and we anticipate that this may 19 guide us to a better approach for crosscutting issues.

20 Next slide, please. We received additional ROP-related 21 suggestions in a letter from NEI on September 19th and conducted an initial 22 public meeting with industry on September 20th where we heard more detail 23 about the industry recommendations.

24 We are also planning another public meeting on October 25 18th where we will have more detailed discussions and establish a path

26 1 forward with a project plan and associated schedule.

2 It is our goal to prioritize all the suggestions regarding the 3 ROP changes and then to evaluate them in 2019 depending on the 4 complexity of the proposed changes in the stakeholder feedback that we 5 received.

6 Any substantive changes to the ROP will require extensive 7 dialogue with our stakeholders and Commission approval in accordance with 8 the staff requirements memorandum COMSECY-16-0022.

9 We will continue our discussions with industry and other 10 stakeholders at the monthly ROP meetings and other focus meetings as we 11 need to.

12 We anticipate this work will require a significant amount of 13 time and staff resources but we are confident that we can make the needed 14 adjustments that yield efficiency benefits for staff, the public, and the 15 industry while maintaining adequate levels of safety and security oversight.

16 Now I will turn the briefing over to Tony Vegel of our 17 Region IV office for a regional perspective on implementation of the ROP.

18 Tony.

19 MR. VEGEL: Thank you, Chris. Good morning, 20 Chairman, Commissioners. Thank you for the opportunity to speak with you 21 about the reactor oversight process focus areas and the regional offices.

22 Slide 16, please. The key focus for the regional offices 23 and the operating reactor business line has been and continues to be 24 nuclear safety, specifically the thorough and independent verification of 25 nuclear plant safety.

27 1 Each day our inspectors in the field effectively conduct 2 risk-informed inspections to ensure operational safety of the nation's nuclear 3 power plants.

4 Some of the recent examples of the inspection findings 5 identified by NRC inspectors from each of the regions includes in Region I 6 an inspector identified inaccurate suppression pool level indication at Pilgrim 7 Station during an independent review of post-accident data.

8 Acting on the inspector's information the licensee staff 9 subsequently determined the cause to be air trapped in the instrument 10 sensing lines. Technicians corrected the condition and restored the 11 instrument to operation.

12 In Region II an inspector identified that Plant Vogtle had 13 not been establishing the maximum reactive load on an emergency diesel 14 generator when conducting a 24-hour surveillance run.

15 The station revised the test procedure and conducted 16 operator training to correct the problem. In Region III a senior resident 17 inspector at Palisades questioned indications received in the control room 18 during quarterly surveillance testing of the safety injection system.

19 Responding to the inspector's questions the licensee 20 subsequently determined that one train of safety injection actuation had not 21 functioned correctly since maintenance one year earlier when a relay wire 22 was improperly connected. The licensee corrected the problem.

23 And last but not least, a Region IV inspector identified at 24 Arkansas Nuclear 1 that the station failed to check initial conditions when 25 testing flood seals for leakage, which invalidated the test results.

28 1 The station subsequently revised the test procedure and 2 re-performed the test to ensure that flood seals were functioning properly.

3 All four of these examples of NRC inspector-identified 4 issues demonstrate the important and value-added contribution to safety 5 made by our inspectors in the field.

6 The regions are maintaining our focus on safety ensuring 7 best practices as we adapt to the changing times. Part of this change 8 includes recently announced early closures of plants.

9 Regional inspectors are focused on ensuring that plant 10 equipment and plant personnel remain capable of ensuring safe operation all 11 the way through to the premature shutdowns.

12 Specifically, our inspectors are ensuring that license 13 operator, command, and control of plant evolution remains robust and that 14 plant equipment is being adequately maintained through appropriate 15 preventative and corrective maintenance activities to ensure that safety 16 systems will function as designed if called upon.

17 In addition, the premature plant shutdowns, we have two 18 new plants, Vogtle Units 3 and 4 that are under construction, and regional 19 inspectors will continue to ensure a strong safety focus as these plants 20 transition from construction through acceptance testing to power operation.

21 Slide 17, please. To ensure we continue to be an efficient 22 and effective independent regulator we are continually assessing and 23 adopting our programs.

24 Actions we have taken to coordinate our programs across 25 the regions to ensure consistent inspection implementation include cross

29 1 regional panels to review potential findings from mitigating strategies, 2 temporary instruction inspections, cyber security inspections, target set 3 inspections, as well as implementation of training and regional peer audits of 4 reactor oversight process implementation.

5 This is only a small subset of activities that we are 6 supporting to ensure consistency of reactor oversight process 7 implementation across the regions.

8 Regional staff are also heavily involved in supporting 9 improvement initiatives related to the reactor oversight process.

10 Representatives from each region are active participants in working groups 11 formed to assess potential changes in the engineering inspection program, 12 inspection documentation improvement initiatives, emergency preparedness 13 significance determination process reviews, as well as risk-informed decision 14 making initiatives and other important activities.

15 We also recognize that the skills of our people are the 16 most important resource for achieving our nuclear safety goals. We must 17 keep a focus on our people during these changing times as well as to 18 continue to have the organization staff with highly skilled and knowledgeable 19 nuclear inspectors committed to ensuring nuclear safety.

20 To this end, the region has fully supported the resident 21 inspector recruitment and retention working group effort and are fully 22 involved in implementing the enhanced strategic workforce planning 23 initiative.

24 In summary, the regional offices continue to be focused on 25 the Agency's nuclear safety mission. We are dedicated to continuous

30 1 improvement and consistency between the regions and we are very involved 2 in improvement activities across the business line and the Agency.

3 Thank you very much for your time and attention and I will 4 turn the discussion over to Marissa Bailey from the Office of Nuclear Security 5 and Incident Response. Thank you very much.

6 MS. BAILEY: Thank you, Tony. Good morning. For the 7 security program update I will be covering our cyber security, physical 8 security, and Force-on-Force programs.

9 Slide 18, please. In cyber security operating power plants 10 implement the cyber security controls in two phases. Phase 1 provided 11 controls to address the most significant threat vectors and plant systems.

12 These controls were in place by December 2012.

13 Phase 2 provided controls for a larger number of systems 14 and processes that sustained the cyber protections. These were completed 15 at most plants in 2017 with the remaining plants scheduled to implement by 16 2019.

17 Cyber security full implementation inspections are 18 underway and will continue through 2020. So far the staff has completed 19 inspections at 14 sites.

20 Although inspection results to date have identified some 21 issues, in general we have found that licensees have adequately 22 implemented their cyber security programs.

23 Moving forward the staff is developing additional 24 implementation guidance based on generic issues identified during the 25 inspections. An example would be guidance on cyber security controls for

31 1 data transfer kiosks.

2 Also in 2019 we will perform a cyber assessment that will 3 look at the scope of the rule and our oversight program.

4 Slide 19, please. With respect to physical security last 5 year following Commission direction the staff completed a comprehensive 6 assessment of the baseline security inspection program.

7 As a result of this assessment we revised the baseline 8 security inspection procedures to eliminate redundancies, streamline the 9 inspection process, and also to ensure that the procedures are consistent 10 with the Commission's direction to be mindful that reasonable assurance for 11 safety and high assurance for security are equivalent.

12 We also revised the baseline security significance 13 determination process to align it with the ROP framework and also to ensure 14 that the significance of security findings are adequately characterized.

15 During the revision process we met with licensees and 16 other stakeholders to discuss and solicit input on changes to the inspection 17 procedures and the SDP.

18 Recently we issued a revision to the SDP that incorporated 19 a risk-informed approach to evaluating findings related to the unsecured 20 safeguards information.

21 Moving forward we plan to issue the revised inspection 22 procedures and the remaining SDP revisions later this fall.

23 Slide 20, please. Regarding Force-on-Force, NRC is 24 currently in the fifth triennial Force-on-Force inspection cycle. Over the last 25 four inspection cycles the staff has made continuous improvements to the

32 1 program.

2 For example, in Cycle 4 we reduced the number of 3 exercises conducted during a Force-on-Force inspection from three to two, 4 we expanded the formal Force-on-Force exercise critique process, and we 5 applied lessons learned to reduce direct inspection hours by 17 percent.

6 During our assessment of the baseline program we found 7 that the Force-on-Force program was effective, but we also identified 8 opportunities to further improve the program.

9 Options for improving the program are summarized in this 10 slide and also described in SECY-17-0100 which is before the Commission.

11 Each option is viable, however, as you know, we 12 recommended Option 3, which is to revise the Force-on-Force inspection 13 program to include one NRC-conducted Force-on-Force exercise and an 14 enhanced inspection of a licensee-conducted annual Force-on-Force 15 exercise.

16 We recommended this option because we believe it gives 17 us a different perspective from which we can assess the licensee's ability to 18 defend against the design basis threat and still allow us to continue our 19 regulatory responsibilities.

20 We believe that this option gives us the opportunity to 21 improve our oversight of the licensees Force-on-Force program and it puts 22 us in a position to make more transformative changes to the program if the 23 Commission wishes.

24 Also, this option provides the greatest, albeit still relatively 25 minor, reduction in staff inspection resources. As we await your guidance

33 1 on these options we continue to make gradual enhancements to the 2 program.

3 For example, we have made adjustments to our internal 4 processes to better ensure that exercise scenarios are realistic and 5 reasonable and we continue to engage our industry stakeholders to consider 6 their perspective as we make enhancements to the program.

7 Slide 21, please. Finally, I would like to address the mock 8 adversary force that is used in Force-on-Force inspections. Since 2004 9 NRC has used an NEI-managed composite adversary force, or CAF, to 10 conduct Force-on-Force inspections.

11 In April of 2018 the Commission approved the use of a 12 joint composite adversary force, or JCAF, for Force-on-Force inspections at 13 NextEra and Entergy sites but for this year and next year only.

14 Before deploying the JCAF NRC inspectors oversaw JCAF 15 selection and training to verify that their performance met the established 16 NRC standards. We also employed additional NRC inspectors to be 17 present during exercise preparations and during the exercises themselves to 18 verify separation and independence between the JCAF and the site's guard 19 force.

20 So far we have completed three Force-on-Force 21 inspections using the JCAF and we have not seen any JCAF performance 22 issues that would have inappropriate influenced the results of the exercises.

23 When the Commission approved the JCAF you also 24 directed us to provide a notation vote paper by April of 2019 with options for 25 a long alternative to the NEI CAF.

34 1 We are currently assessing a number of options and we 2 anticipate that some of them will take several months to put into place. Our 3 goal is to be able to implement the option that the Commission selects by 4 January 2020, which is the beginning of the sixth triennial inspection cycle.

5 Therefore, we plan to provide the vote paper to the 6 Commission by December of 2018. This concludes my portion of the 7 briefing. Thank you for your time and I will now turn it over to Rob 8 Tregoning.

9 MR. TREGONING: Thank you, Marissa. Good morning, 10 Chairman Svinicki and Commissioners. I am on Slide 22.

11 The principle focus of the Office of Research's operating 12 reactor business line activities is to assist the Agency by developing 13 technical bases and providing expertise to enhance effective regulator 14 decision making.

15 The office supports all aspects of the Agency's statutory 16 responsibilities, including safety evaluations, inspections, emergency 17 planning, security, and operational event assessment.

18 The importance of the operating reactor business line is 19 reflected in the budget as it comprises almost 85 percent of the office's FY 20 18 resources.

21 A few examples where the Office of Research contributes 22 significantly to important Agency actions are the development of the 23 guidance documents supporting subsequent license renewal applications 24 and the review and confirmatory analyses used to approve expanded BWR 25 operating ranges based on improved fuel performance.

35 1 Several current focus areas within the Office of Research 2 include working closely with stakeholders to ensure that the Agency can 3 support the implementation of accident-tolerant fuels, or ATFs.

4 The office is also assisting with the development of a 5 risk-informed regulatory structure for Digital I&C that will utilize a graded 6 approach to align regulatory requirements with safety significance.

7 Finally, the office is enhancing realism and important 8 probabilistic risk assessment tools, models, and guidance to better inform 9 the Agency's regulatory applications.

10 Slide 23, please. While the Office of Research 11 collaborates extensively with domestic stakeholders to fulfill its mission the 12 office also relies heavily on international partnerships to support operating 13 reactor business line activities.

14 For example, the office has implemented over 100 bilateral 15 and multilateral agreements with more than 35 countries, the Nuclear Energy 16 Agency, or NEA, and the International Atomic Energy Agency.

17 These agreements cover a wide range of activities and 18 technical disciplines and partnerships have been established with diverse 19 stakeholders, including regulatory organizations, research and technical 20 support organizations, academia, and vendors.

21 While this networking is diverse and extensive the office is 22 selective and only enters into agreements that most effectively obtain 23 information on safety issues to the use of unique test facilities, expertise, or 24 operating experience.

25 This cooperation typically produces substantial cost

36 1 savings. For NEA projects a 10:1 or better cost to benefit ratio can often be 2 achieved. Examples of areas that benefit significantly through international 3 cooperation include analytical code development and maintenance.

4 Current agreements generate nearly $2 million per year in 5 fees and provide additional in-kind contributions to help ensure that these 6 codes remain adequate for regulatory use.

7 NRC staff are also participating in several international 8 programs to gather critical data and operating experience for evaluating 9 damage in concrete structures, including an NEA program to analyze the 10 effects of concrete aging on containment performance.

11 The picture on the right in the slide shows the construction 12 in France of the containment vessel that will be tested under this program.

13 The office is also part of an NEA program to experimentally 14 assess fire propagation phenomena that will be used to confirm the 15 adequacy of existing guidance for fire probabilistic risk assessment.

16 Slide 24, please. As mentioned previously, one important 17 component of the Agency's international cooperative program is the use of 18 large scale flexible test facilities.

19 Such facilities have long been important in addressing 20 safety-related issues. For example, as shown in the picture on the slide the 21 PANDA facility at the Paul Scherrer Institute in Switzerland has been used to 22 address several phenomena related to containment integrity for both current 23 and new reactor designs.

24 These facilities will continue to play a critical role in 25 supporting long-term power plant operation by validating and identifying

37 1 limitations of analytical codes and helping understand the complex 2 multi-disciplinary nature of many of today's nuclear safety challenges.

3 For example, integrated test facilities have played a critical 4 role in demonstrating the adequacy of mitigation to ensure long-term reactor 5 coolability in the event of a loss of coolant accident.

6 However, such testing facilities in research reactors are 7 currently facing many of the same challenges as commercial nuclear power 8 plants. These facilities are often costly to build, maintain, and operate.

9 Many are also aging, which further increases cost and 10 decreases availability. Finally, demand for many of these facilities has 11 decreased as countries phase out or decrease their reliance on nuclear 12 power.

13 Slide 25, please. NRC staff are engaged in several efforts 14 that are addressing nuclear testing infrastructure challenges. Staff 15 participates on an expert panel chartered by NEA.

16 The panel's mandate is to identify large facilities that are at 17 risk to close and then recommend strategies to maintain critical 18 infrastructure.

19 The panel plans to complete its evaluation by the end of 20 2019. An example of a facility being considered by this panel is shown in 21 the slide, it's the Melt Coolability and Concrete Interaction Facility at Argonne 22 National Laboratory.

23 This is the largest such facility in the world and has 24 supported evaluation of reactor core melt events, such as those at 25 Fukushima Daiichi. However, it's future viability is somewhat uncertain.

38 1 NRC staff are also actively exploring strategies to mitigate 2 impacts resulting specifically from the closure of the Halden reactor. As you 3 are aware, the NRC jointly participates with 32 organizations from 20 4 countries in the Halden Reactor Project to conduct research and Digital I&C, 5 human factors, fuels, and materials aging.

6 The fuels and materials research are most significantly 7 affected by the reactor closure. While the Digital I&C and human factors 8 research are not directly affected, changes in the Halden Reactor Project 9 membership or funding levels may also impact this work.

10 As part of the project the NRC and other members are 11 considering revising the fuels and materials research plan while 12 simultaneous exploring the use of alternative facilities for completing some 13 of the current plan.

14 Separately, NRC staff are collaborating with both the 15 Department of Energy and the nuclear industry to address gaps in DOE's 16 ATF program that have been created by the closure of the Halden reactor.

17 The preliminary assessment has provided several 18 near-term recommendations that focus on transferring knowledge from 19 Halden personnel, exploring the potential to increase steady state fuel 20 testing capacity, conducting follow-on testing using materials irradiated in 21 commercial reactors, and evaluating options for performing in-pile fuel 22 experiments using flexible power conditions.

23 Similarly, in the materials area NRC staff are considering 24 cooperation with the nuclear industry, DOE, and other interested 25 stakeholders to conduct research effected by the reactor closure if the

39 1 revised Halden Reactor Project plan does not adequately support U.S.

2 needs.

3 Thank you for your attention. I would now like to turn the 4 presentation back over to Margie Doane for her closing remarks.

5 MS. DOANE: Okay. Thanks, Robert. So within this 6 amount of time we obviously couldn't cover all of the important work that this 7 business line does and just to remind everyone this business line covers or 8 oversees the operation of the nation's 98 reactors, operating reactors, and 9 31 research and test reactors, and, as you could hear from Robert, it's an 10 important component of the NRC's international program.

11 So what we tried to do today is just to highlight some of the 12 important issue and I think as you will see, as you could see from these 13 presentations this is not a static regulatory program.

14 There are old and new regulatory challenges that are 15 being addressed at headquarters and in the regions and in multiple offices, 16 including the ones highlighted at the table.

17 But we did not highlight everyone's work. For example, 18 we could not, we didn't have the time to tell you about all the good work that 19 we depend on from the corporate offices, condition offices, like SECY and 20 the Office of the General Counsel.

21 We count on these offices to help us address these 22 challenges that we face every day and we also have close collaboration with 23 our materials safety office, as you know.

24 So we will continue to look to use our resources 25 throughout the Agency to continue to ensure our decisions to ensure safety

40 1 and security are risk-informed and are more streamlined.

2 And in closing I just want to thank the staff at headquarters 3 and the regions for their service to the public, their professionalism, and their 4 day-to-day focus on and dedication to our important safety and security 5 mission.

6 This concludes our presentation and we look forward to 7 your questions. I think this is the most extraordinary part of me sitting here 8 instead of there, at least that's the way I feel at this moment.

9 CHAIRMAN SVINICKI: All right. Thank you, Margie, and 10 to all of the NRC presenters and also to your teams who prepared you to 11 give those very thorough and informative presentations.

12 And since you did mention the contrast of the general 13 counsel I will say this before I recognize my colleague for the rounds of 14 questions is that I regret, I think I might have been a little bit glib about my 15 reference to the role of the general counsel here at these Commission 16 meetings and the fact that they generally don't have to inject anything into 17 the meeting, but just sitting there silently with your legal gravitas is a great 18 contribution.

19 So, thank you very much for doing that and welcome to 20 Marian Zobler in her new role. I congratulate you again on that. So for our 21 question and answer period we will begin with Commissioner Wright.

22 Please proceed.

23 COMMISSIONER WRIGHT: Thank you. Good morning 24 and I thank each of you for the presentations, very informative, and your 25 roles are changing daily.

41 1 Margie, I am going to start with you, and before I delve into 2 my questions I just want to add my welcome to you in your first meeting as 3 EDO, and as well as to Ho and to Marian as well in your new roles.

4 I also want to thank you and your staff for the leadership 5 and efforts and the Agency's preparation for Hurricane Florence as she 6 came through.

7 I did experience the hurricane firsthand and I saw some of 8 the damage it caused. She came in like a weed-eater on the coast of North 9 and South Carolina and she dumped a historic amount of water that we are 10 still paying a price for down south.

11 So we want to thank you for responding and the response 12 to the storm was appreciated. I got a number of calls from people about, 13 you know, what was happening.

14 So turning to the presentation, in his remarks Ho 15 discussed the strategic direction for the operating reactors business line, 16 which is primarily to enhance, focus on issues of greatest safety 17 significance, and expand risk-informed decision making.

18 So the question I have got for you is could you speak to 19 your vision for the Agency and how this strategic direction aligns with that 20 vision, including transformation, and where do you see the Agency maybe in 21 the next 20 years, and are we really taking the steps necessary to get us 22 there?

23 MS. DOANE: So those are giant questions, and I will try 24 to make sure I keep this brief, because I know you have probably a couple 25 more questions within your time.

42 1 So, I think one thing that has pleasantly surprised me, I've 2 talked to everybody about kind of the learning curve that I expected when I 3 got into this job and I thought, and I knew it would be steep, and it is.

4 But fortunately, the General Counsel's Office actually sees 5 most, if not all, of the very important issues that are coming before the 6 agency. So when I got here, the issues were familiar.

7 What isn't as familiar is how we approach problems and 8 how we, on the operations side, and all the various day-to-day things that 9 they have to do.

10 And so, when I got into this job I had already had a lot of 11 mentees come in to me and tell me, when I was general counsel, that they 12 just couldn't understand, they knew we wanted to transfer, they knew we 13 wanted to innovate, but they didn't understand what their role, at the office 14 level was. How they were going to accomplish this.

15 So, the first thing I wanted to accomplish, and we have 16 completed this, it's a sit down with our executives and identify where we 17 think the agency needs to be in some important respects, so that the Staff 18 can see where we're going.

19 And I think the major thing that we've sort of been focusing 20 on is, even with the plans decreasing, we're still going to have probably the 21 largest nuclear program in the world.

22 We also know that the nuclear power fleet, they are also 23 considering the same issues that we are because they grew and they 24 weren't necessarily safety focused in the way that they thought they should 25 be. So they're making a lot of changes.

43 1 And what this is bringing to us is a lot of questions and a 2 lot of requests from licensees, both for new technology but also existing 3 technology, to try to look at it in a different way.

4 So, my vision is that we continue along this path, to be 5 able to answer those questions and to be risk-informed. We have many 6 tools to be risk-informed already.

7 And you heard from Russ about one effort that's going on 8 in the Office of Nuclear Reactor. So, we can do that but what we need to 9 continue to work on, and we can do this, is to have a systematic way of 10 doing this in a timely and reliable and predictable way.

11 That's a challenge for us, and so we'll continue to do that.

12 But like I said, I'm very pleased with the talent that we have.

13 And so, that's my vision, trying to set up a good place 14 where the agency is going to go, working with the executives and now 15 making sure that our actions are consistent with where, going in that 16 direction.

17 COMMISSIONER WRIGHT: Thank you. Ho, a lot of 18 people say welcome back, you know, and a lot has happened since you've 19 been gone, I know.

20 So, as you know, the risk-informed decision-making has 21 been the subject of a number of previous efforts by the agency. And these 22 efforts have had their success and they've had their challenges.

23 My question is, how does this current effort in our build on 24 the previous successes and address the challenges going forward?

25 MR. NIEH: Thank you, Commissioner. I think it's very

44 1 interesting that you point that out, that the agency, in the past, has taken on 2 a number of initiatives to be more risk-informed stemming from the 3 Commission PRA policy statement in 1995 and other risk-informed 4 rulemakings that have occurred.

5 And it's really interesting because I did some studying in 6 anticipation of coming back to the NRC. And it was almost like a back to 7 the future moment.

8 And I think, my feeling is that we've tried before in the past, 9 and we've had maybe inconsistent success in trying to implement these 10 risk-informed initiatives and we've learned from those experiences.

11 And I think as Margie mentioned, creating the space for 12 managers, branch chiefs, first line supervisors and most importantly, the 13 staff, to really understand what risk-informed decision-making means, how 14 we can apply it. I think that will help us perhaps achieve more success than 15 we have done so in the past.

16 I know that looking other tools that exist in our tool box, 17 such as the refresh training we've done on the back-fit rule, I think that also 18 complements how we could use risk information to assess how to address 19 issues in the regulatory process.

20 But I think really getting to the staff and really 21 communicating what this vision is for risk-informed decision-making is a key 22 aspect of that.

23 And in looking at some very discrete issues since I've been 24 back, in terms of where Staff may have had a particular view on how an 25 issue should have been addressed by the NRC, it's very clear to me that

45 1 risk-informed decision-making is not a black and white formula that we can 2 just put in some values and apply, it's really more of a holistic assessment 3 that we have to make in terms of when we're confronted with an issue, us 4 being able to answer the question, how safety significant is this issue we're 5 dealing with.

6 And depending on the answer to that question, I think we 7 need to use that as a guide to determine how much time we're going to 8 invest in resolving the issue. They're very well maybe important issues that 9 we're going to have to deal with and we know we're going to have to put a lot 10 of time in resolving and working with the regions, if they're involved in 11 addressing.

12 But then there are going to be some issues where people 13 will tell you, this is low safety significance, okay. So, it's up to not only the 14 management team but it's also up to the Staff to recognize that this is indeed 15 a low safety significant issue, let's address it with the right amount of 16 resources.

17 Again, I can't give you a black and white formula that says 18 this is exactly what you need to do that, but at least that's what's going 19 through my mind. And I've been sending that message to my leadership 20 team and NRR. And I think there are a lot of people that understand this 21 and that's the direction to move it.

22 COMMISSIONER WRIGHT: Thank you. In my time left 23 I'm going to go to Russ. So thank you for your presentation.

24 I know your group and many others in NRR and around the 25 agency have been working very hard to advance this initiative, so thank you

46 1 for that.

2 I have a question on the results of Phase 1, the 3 assessment phase. In the background information, there's a statement that 4 says, a culture change is needed to make more use of risk information and 5 risk insights and how we do our work on a daily basis.

6 Can you speak to that culture change and to the aspects of 7 the action plan that accomplish the desired change?

8 And as a follow-up to that, were there any surprises in the 9 working groups findings and subsequent recommendations, particularly 10 around any feedback received from staff and managers on the initiative?

11 MR. FELTS: So, I think the culture change that we were 12 referring to in the summary of Phase 1 really refers to, it sort of builds on 13 your recognition that we've had past efforts here that may not have been as 14 successful as we wanted.

15 And one of the things we did with this action plan is to try 16 to build it sort of as a grassroots thing where the efforts to understand how 17 we might improve use of risk insights and decision-making were being 18 developed in the divisions in NRR where those activities actually have to 19 happen rather than sort of originating in a division of risk assessment and 20 being pushed into the other technical offices.

21 So, the teams that worked on Phase 1 were all built with 22 that in mind. So that it would be more of a pull, right, into those divisions.

23 The recognition that in order for people's attitudes about 24 risk to change, they really need to understand that the risk insights will help 25 them make better decisions that will improve safety.

47 1 There's a fair amount of, in the agency there's a lot of 2 respect for precedent, there's a lot of respect for the way things have been 3 done. The way safety has been established through deterministic analysis.

4 There's a fair amount of, a sense of ownership. We want 5 people to be passionate about what they do, and they are. And that creates 6 a certain inertia that we have to overcome.

7 And in order to overcome that, people have to experience 8 a little bit of cognitive dissidence where they recognize, well, wait a minute, 9 maybe these risk insights will help me have a better understanding of what 10 level of defense-in-depth is necessary in this particular application with this 11 particular piece of equipment.

12 That's where we're trying to actually achieve culture 13 change by having people recognize that this risk information can be used to 14 improve their understanding of what's necessary for adequate protection.

15 CHAIRMAN SVINICKI: Well thank you very much. And 16 that's actually, Russ, I thank you for that because that's just philosophical 17 enough for me to use as a platform for some comments that I've been 18 thinking of as I listen to the presentations here this morning. First of all, 19 thank you for your presentations.

20 And, I have a long commute because I live in Virginia and 21 the traffic is pretty fierce so I listen to pod casts because I want to make use 22 of my time behind the wheel, and I was listening to a description of how do 23 people approach life connections with each other's general levels of 24 optimism or pessimism in life. And they talked about how formative your 25 youth is and kind of, what kind of upbringing you had.

48 1 But they described it as, that provides the lens. It doesn't 2 determine your destiny or do anything else, but it gives you a lens through 3 which you will see everything else.

4 And as I was listening to the topics and the presentations 5 this morning, we've mentioned transformation, we've mentioned innovation.

6 And maybe I'll speak a little bit as chairman for my colleagues in terms of 7 where we are with the Staff's paper but I'll also speak as an individual 8 member of the Commission.

9 Of course, we received a set of proposals that was an 10 outgrowth of a team chartered by Margie's predecessor, Victor McCree.

11 And so we have a paper that is pending before us.

12 And the time shortly after we got that paper, we were 13 joined by new members of the Commission. And so I'll characterize that 14 what we want to do right now is we're engaged in how do we get the 15 greatest benefit of the five member Commission on a proposal that was 16 already pending.

17 People are coming in new, they haven't been here for the 18 original of all of it. And I think that my experience on this Commission is 19 that it is very collegial.

20 And what we want to do is say, how do we position each 21 other best and ourselves best to make really informed feedback and then 22 decisions and issue a set of what I think will be initial set of direction on the 23 agencies transformation and innovation process.

24 But that being said, I think I get asked about, where is the 25 Commission on transformation. And it's almost as if we're going to kind of

49 1 design all the sets of actions.

2 And that isn't a paradigm that I predict is what's going to 3 happen here. I think that in these presentations I already see some of you 4 clicking a lens into place through which you're looking at the things you do 5 day-to-day.

6 And as I engage with NRC employees in the lobby or in the 7 elevator, I think that it's the beginnings of people feeling not only do they 8 have permission to but they're encouraged to think about, what did you do 9 today, did you have some things that you felt like added questionable 10 amounts of value to the important things that you did today.

11 So, I see it as, of course the Commission will take up an 12 act on the transformation proposals in front of it. But the way I come at it, 13 as a Member of the Commission is, this is really your NRC to design.

14 And that's what, I've heard just a little bit about the various 15 pieces and parts of what you're doing, and you're joined by the teams that 16 support you and they're looking at what they're doing. But it is a lens 17 through which you look at everything.

18 And so that's why it's hard to say, what is the NRC's 19 transformation and how is the Commission going to direct and control the 20 transformation of the agency. Listen, I've been here long enough to 21 observe the agency as a whole, kind of waiting out an individual Commission 22 Member's interest in initiative A or B, okay, we all know it happens because 23 that person brings a lot of energy to that.

24 But the agency is kind of like a river, you know, and it 25 moves forward and takes its own course on the evolution of Nuclear Safety

50 1 and Security Regulation for the United States.

2 And I agree with Ho that I engage our regulatory 3 counterparts from around the world and I hear it. I heard it.

4 The EDO and I were just representing the agency in some 5 engagements at the International Atomic Energy Agency last week. I'm 6 sure she heard it in the engagements as well as people say, oh, U.S. NRC, 7 you're the most expert and the most thorough regulator. And I compliment 8 that.

9 If I had that a personal vision for the agency and the 10 agency staff and experts designing their own future, it would be to marry that 11 praise that's heard around the world that NRC is the most expert, NRC is the 12 most capable, NRC has the most thorough and complete regulatory 13 framework.

14 I would complement that by praise at some future time for 15 a future chairman where people would say, U.S. NRC is the most agile and 16 innovative organization, it brings the most management and administrative 17 excellence to what it does.

18 Truth is, last week I heard a lot about our Canadian 19 colleagues, and we have a wonderful relationship with the Canadian Nuclear 20 Safety Commission, hats off to them, but they're getting a tremendous 21 amount of praise about bringing innovation to advance reactor reviews.

22 And the truth is, we're capable. And my colleague, 23 Commissioner Burns, has done a paper on the fact that we have all the 24 same flexibilities available to us under law and regulation.

25 And we're actually engaging advance reactor designers

51 1 with individual engagement plans and we have a very tailored thing about 2 how are you developing your design and how are you phasing the 3 finalization of your design. And we're able to bring regulator perspectives to 4 that process, but I don't know that Canadians seem to be winning the public, 5 or the marketing war.

6 And we can do a lot of the same things but I don't see us 7 out there getting the same acknowledgment for us. I know that an 8 organizational, an organization that is capable of the kind of reputational 9 standing that NRC has had globally for so many decades now, is also 10 capable of being the most innovative, of being the most affective, of being 11 able to be agile and dynamic.

12 I know we're capable of it, but change is uncomfortable.

13 And I don't kid myself, it is a choice. And saying that you can do something 14 doesn't mean that it guarantees that it will do it.

15 And I think what I'm watching happen, in NRC as a whole, 16 is that people are beginning to think about that choice, they're beginning to 17 consider both individually and collectively whether or not and in what ways 18 they want to engage in being a part of the choice that NRC makes in 19 marrying its, again, global reputation on the overall excellence of its system 20 with an organizational excellence of its process and the way that it can go 21 about doing things.

22 I was chatting with an NRC manager, who I won't name 23 because it was a one-on-one conversation said, you know, we do have folks 24 here that say, well, why does this need to change because I've worked here 25 20 years or 25 years and this is how we've always looked at this. And I

52 1 heard that I thought, you know, that is a good question that you need to 2 answer, but there's no aspect of my personal life that I say, gosh, you know, 3 25 years ago, I looked at this exactly the same.

4 And maybe the rebuttable presumption is, why would you 5 stand still. I mean, you really have to be moving forward. And so, I don't 6 know whether NRC will or it won't, but I know without a doubt that we are 7 capable of amazing things.

8 I think we could be as world class in everything else as we 9 are in our expertise and our capability. And I have no doubt about that 10 whatsoever.

11 But that doesn't mean that we will do the hard work of 12 change to get. It's a decision that NRC will have to make.

13 The Commission, and as Chairman I will be doing 14 everything I can to enable you to be able to carry through on the choice that 15 you make, but it is my individual view, as a member of the Commission, that 16 the Commission itself cannot design and propel you into the next evolution of 17 NRCs journey, that is something that NRC employees will have to do 18 individually and collectively in the teams and divisions and offices in which 19 they work.

20 So, I do appreciate all that we heard about today. It's 21 actually very encouraging to me because you're not waiting for that initial 22 commission SRM.

23 And that was baked into Victor's charter to the group, to 24 what they came forward with they said, how we can do a lot of things. And 25 they are doing it.

53 1 And I do think it's a little bit of back to the future. Because 2 our predecessors, I know I continue to be so gratified that my predecessor is 3 on the Commission, both as Commission Members and as Chairman, had 4 so much wisdom.

5 And they built a lot of flexibility into how we do what we do.

6 We've had a pretty enduring expectation about the excellence that we 7 expect on nuclear safety and security, but there's a lot of creative control to 8 even front-line workers, first level supervisors. And at every level, people 9 have a lot of creative individual thought processes that they can bring to 10 what they do.

11 And I know when I met with the innovation forum, which is 12 this other, this nomenclature confusion. We have transformation, we have 13 innovation, we have Project Aim. It's confusing and I get that.

14 And I do think the Commission and the senior executive 15 service need to paint a picture for people so they know at least where our 16 people think and we want to go.

17 But, when I met with the innovation forum I thought about it 18 and I thought, you know, they kind of know the NRC that they want to come 19 to and work at every day. Today and five years from now and ten years 20 from now and 15 years from now.

21 So that's, I think, what brings their energy to innovation.

22 And I think there are enough people that want to harness that.

23 We do need to have some collective idea of where we, 24 because in order to do it efficiently we have to have some vision of where we 25 want to head together.

54 1 But, I think everything, every topic you presented today 2 had elements of the beginnings of this next phase. And it is just a next 3 phase. We've always been a continuous learning organization.

4 My time is up. I'm three seconds over but I do feel like if 5 anyone wanted to react to that, Margie or Ho.

6 MS. DOANE: It was very eloquent and I think you've said 7 it all and you captured --

8 CHAIRMAN SVINICKI: Quite rambling but some --

9 MS. DOANE: Well no --

10 CHAIRMAN SVINICKI: -- there were some --

11 MS. DOANE: -- I think you captured --

12 CHAIRMAN SVINICKI: -- points in there.

13 MS. DOANE: No, for us, I think it's great for us to hear so 14 I appreciate that. Thank you.

15 CHAIRMAN SVINICKI: You know, it's all Russ' fault 16 because he started it.

17 MS. DOANE: With the philosophical, yes. He's very 18 good in that. I just want to highlight that, or maybe put a fine point on what I 19 heard from your messages and make sure that you know that we do 20 recognize this.

21 That, first of all, what's before the Commission is narrow 22 and it's looking at future, not being a barrier to future technology. And of 23 course, I mentioned that.

24 But there's lots coming before us that has to do with the 25 operating plants, it doesn't have to do with new technology. There might be

55 1 new approaches and new ways of looking at old things we were doing.

2 So, there are a lot of different things that we want to look at 3 and become risk-informed or approach them differently.

4 But, really what I wanted to say is that, and it behooves us 5 as the senior, me as, with my senior leadership team, to give our staff the 6 tools that they need, the training and a good understanding of where we 7 want to go.

8 Because without that, we can talk and talk about this, but 9 really, it will be up to us to make sure that we are giving them what they 10 need to make sure they know they have the flexibility that Ho was talking 11 about, and that you were talking about, and the tools and the training to 12 move to different areas.

13 Because, new technology and new issues will require new 14 skills. So, if you really want to be innovative in a healthy organization, it 15 takes all of these working pieces. So, I just wanted to highlight that.

16 CHAIRMAN SVINICKI: Okay, just quickly, Ho, yes.

17 MR. NIEH: Thank you, Chairman, I'll be very quick. Just 18 to build on one point you mentioned in terms of looking at other regulators.

19 I do think having worked with a lot of regulators, including 20 the Canadian Nuclear Safety Commission, I think one perception that is 21 seen around the world, in terms of such programs like Canada has is that 22 they're flexible.

23 And I think the U.S. is very well known to have a 24 prescriptive set of regulatory requirements, so one message that I've been 25 using with my team in NRR is that, we need to think about how we can be

56 1 flexible as regulator then still doing our job in terms of safety.

2 And then maybe just going back to the culture piece very 3 quickly and keep the movie theme going on, this culture chain, for me, when 4 I was thinking about transformation and what it means, and I use Yoda in an 5 all hands meeting to illustrate the point where he told Luke Skywalker to 6 unlearn that you have learned.

7 And for me, that really drove the point home for me 8 because I think we've done things for 20, 25 years and there's a real 9 opportunity for us to maybe look at it differently and more flexible. Thank 10 you.

11 CHAIRMAN SVINICKI: Okay, thank you. Thank you.

12 Commissioner Baran.

13 COMMISSIONER BARAN: Thanks. So I think this has 14 been a very good high level philosophical discussion and I want to turn to 15 that in a minute.

16 I want to start though with just a couple practical examples 17 of the NRC engineering inspections, with the staff discussing some of the 18 presentations earlier.

19 Last week I was at Millstone and heard from our terrific 20 resident inspection inspector team there about some of the findings at that 21 plant. They explained that a recent design basis assurance inspection 22 discovered missing flood seals in the emergency diesel generator and 23 auxiliary buildings that were previously identified by the licensee but never 24 corrected. It turns out 22 other penetrations were not sealed.

25 A separate environmental qualification inspection found

57 1 that auxiliary feedwater valves had components that were not replaced after 2 their service life had been exceeded, casting doubt as to whether those 3 valves would have performed their intended safety function.

4 Tony, you see the results of engineering inspections from 5 a lot of plants, based on your experience, are these engineering inspections 6 catching important safety issues, are they providing a real safety benefit?

7 MR. VEGEL: Thank you for that question. I'll provide my 8 perspective from Region IV from my experience. But I will tell you that yes, 9 the design basis assurance inspections are adding value.

10 We are identifying issues, I think, in all four regions, of 11 issues, or potential issues, to prevent them from occurring. An example that 12 comes to mind is Region 3.

13 They identified an issue with freeze protection up at the 14 Perry Plant. And it does get cold, that could have affected the swap-over 15 from condensation storage tank to suppression pool.

16 You know, another very good example that actually 17 prevented a problem, that an issue was identified and the licensee was able 18 to take corrective actions. So, based on my experience, the design basis 19 inspections, the other engineering inspections do provide value and do 20 identify issues.

21 COMMISSIONER BARAN: That matches really what I'm 22 hearing from the inspectors I've talked to from all the regions and when I'm 23 out at plant visits or at regional meetings or just even talking with folks at 24 headquarters.

25 I think these engineering inspections are valuable, and I

58 1 strongly believe that NRC inspectors need to be conducting them. So I am 2 really concerned about all of this talk about licensee self-assessments.

3 And this gets back to, what is our lens. And we have to 4 have the lens of a regulator.

5 Right now, if a licensee self-identifies a safety issue that 6 would otherwise result in a Green finding, we basically provide credit for that 7 self-identification and don't cite the Green finding. And I can see the 8 rational for that because a licensee is stepping up and self-reporting a safety 9 problem of low safety significance.

10 But that's completely different from turning NRC 11 inspections over to the licensee. And that's what people are talking about 12 when they talk about self-assessments.

13 Allowing licensees to inspect themselves in lieu of NRC 14 inspections. Not only is the Staff talking about allowing that for engineering 15 inspections, you're talking about allowing industry self-assessments to 16 replace NRC inspections in other areas.

17 What's next, radiation protection, emergency 18 preparedness, security? I understand why industry would prefer to inspect 19 themselves, but that doesn't make them a good idea for the safety regulator.

20 I think we need to step back, as we have in the discussion so far today, and 21 think about first principles.

22 Why does NRC conduct inspections? Because our 23 independent inspectors find problems that licensees don't.

24 Because licensees perform better and more safely with us 25 performing rigorous independent oversight.

59 1 Because the public has entrusted NRC, a public agency 2 that works for them, with the responsibility of establishing standards to 3 protect their health and safety in enforcing those standards impartially.

4 None of those purposes are meet when licensees are allowed to inspect 5 themselves.

6 There's nothing wrong with licensees performing 7 self-assessments for their own purposes. In fact, they do them right now 8 when they prepare for major NRC inspections.

9 But when we then conduct those inspections, our 10 inspectors still identify issues that the self-assessments did not. Isn't that 11 right, Tony?

12 I mean, your folks, when they go out there and do these 13 inspections, it's not that the, the licensee is prepared for it ahead of time, 14 they've done the self-assessment, but your folks are still finding things.

15 MR. VEGEL: That is correct.

16 COMMISSIONER BARAN: We need to stop and think 17 about what we're doing on self-assessments.

18 Another industry idea that the Staff is considering, based 19 on what I'm hearing today from these slides, is conducting fewer baselines 20 inspections for plants that are performing well. Chris, isn't the whole 21 premise of baseline inspections that these are the minimum inspections that 22 should be performed at every plant?

23 MR. MILLER: Commissioner, yes. The baseline is the 24 minimum. How you treat perhaps number of hours, the number of samples 25 in that baseline can vary and that's something that we're looking at.

60 1 COMMISSIONER BARAN: Well, if we conduct less than 2 minimum oversight of plants that had been performing well, I worry that we're 3 going to see the performance of those plants decline.

4 Rigorous independent oversight improves performance.

5 We all know that. That's why we perform oversight in the first place. That's 6 why NRC exists. We should not do less than the minimum on inspections.

7 I want to touch on one more set of proposals that we're 8 hearing from industry, which focuses on minimizing the importance of White 9 findings.

10 Some stakeholders are arguing that only a Yellow or Red 11 finding should result in a column change in the action matrix, and a resulting 12 increasing in NRC oversight. But one of the fundamental premises of the 13 reactor oversight process is that Green and White findings can be leading 14 indicators of larger, more safety significant problems.

15 Pilgrim is a textbook example of that. Pilgrim has been in 16 Column 4 since September 2015 and it got there from three White findings.

17 It didn't have any Yellow or Red findings in recent years, but the White 18 findings made us take a closer look at performance at Pilgrim. And when 19 we looked more closely, we found major problems.

20 Under a recent industry proposal, Pilgrim wouldn't have 21 even moved to Column 2, let alone Column 4. Does anyone on this Panel 22 disagree that Pilgrim should be in Column 4 right now?

23 Well, I think that tells us all we need to know about the 24 risks of discounting the importance of White findings.

25 Look, stakeholders are going to have a lot of ideas about

61 1 changes we can make to the reactor oversight process. Some of them are 2 going to be good, some of them are going to be bad.

3 The public is depending on the staff, is depending on all of 4 you, to remember why we are all here and to avoid pursuing ideas that are 5 going to weaken our safety oversight.

6 And so as we think about possible changes, and there 7 could be a lot of good possible changes to either ROP or other areas of the 8 agency's work, we've got to have that lens. We've got to have the lens of 9 the rigorous independent regulator who understands why it is we're doing 10 oversight and the importance of that oversight.

11 So, I'll stop there. That gains back the couple of minutes 12 that we lost throughout, and I'll leave you with those thoughts. Thank you.

13 CHAIRMAN SVINICKI: Thank you, Commissioner Baran.

14 Commissioner Burns, please proceed.

15 COMMISSIONER BURNS: Again, thank you for the 16 presentations and the discussion this morning to, rather than sort of waive 17 banners and all that, I want to get into, onto on the ground, how some of this 18 is going to work and what we're really evaluating. Because I don't take it, 19 anything that we have decided that we are walking away from inspection 20 programs, that we are sort of manipulating the oversight program.

21 But I think there are a number of areas where, and this is 22 where I would agree with the notion, I think as the Chairman has articulated, 23 this regulation is always a journey. There are things we do today, in the 24 ROP for example, that are much better than we did 20, 30 years ago under 25 systematic assessment of licensee performance program.

62 1 There are ways we look at enforcement that are better 2 today than they were when I started out in the agency. But part of that is 3 creating a balance, and there is a tension between what is the regulators 4 role and the basic principle reflected in the convention on nuclear safety that 5 the operator is primarily responsible for safety.

6 The regulator is there to help hold that operator 7 accountable. But ultimately, those who operate are responsible for safety, 8 and that's our job to make sure that they do that.

9 So, a couple of the things, and where I see some of the 10 tension. So let me start with you, Russ.

11 We talk a lot about risk informing. And as I said, we've 12 got the ideas. We got the banners flying and all. Some of the difficult work 13 is, I think, is the, is basically the development of things like the guidance.

14 And so where are we on that? How are we integrating 15 that into the program itself?

16 MR. FELTS: Okay, so in Phase 1, the evaluation phase, 17 we developed some guidance using anecdotal, I don't want to call them 18 anecdotal --

19 COMMISSIONER BURNS: Yes.

20 MR. FELTS: -- examples where we had successfully 21 used risk in sort of an ad hoc way in the past. We developed some tools for 22 the licensing reviewers to work in these integrated teams using a graded 23 approach.

24 And those are being, basically it's a trial run right now 25 through December. We may make some adjustments to that guidance.

63 1 But then, if it proves that the guidance works well in the 2 trial phase, we'll go ahead and start to put that guidance into durable 3 guidance. It's interim right now, we'll put it in durable guidance, like in the 4 standard review plan and so forth.

5 And so that's how we're going to ultimately put that 6 process in a more formal setting.

7 COMMISSIONER BURNS: Okay. Do you sort of foresee 8 it as sort of a time frame? You may have just mentioned something and 9 that passed through my head.

10 MR. FELTS: Well, I mean, so the trial phase ends at the 11 end of December.

12 COMMISSIONER BURNS: Okay.

13 MR. FELTS: How long it takes to actually incorporate that 14 into the formal guidance and the SRP there's a process for that. And I can't 15 tell you right now how long that will take but there is --

16 COMMISSIONER BURNS: One of the other things, and I 17 looked at the chart you all created in terms of, as you said, you had, on two 18 ends, on a couple of these ends where you wouldn't do the integrated team 19 --

20 MR. FELTS: Right.

21 COMMISSIONER BURNS: -- have you all sort of thought 22 about, in terms of the, so that the volume of license amendment requests or 23 licensing action requests that we get, how many you think might fall in, 24 whether it's just sort of a Kentucky windage or any kind of assessment of 25 that?

64 1 MR. FELTS: I couldn't give you statistics, but I think that 2 the vast majority of licensing actions are going to be in the middle of the 3 spectrum. They're going to be, the ones where deterministic criteria are 4 clearly met and there's really no added value in doing an integrated team are 5 probably going to be infrequent.

6 And then the risk-informed applications that they have a 7 strong technical basis that don't require additional technical review or 8 participation are probably a relatively small subset of the population.

9 COMMISSIONER BURNS: Okay. Okay, good. Chris, 10 let me move on to you. One of the things, as you say, you've been 11 engaging various stakeholders on the ROP and I know we've had, I think 12 we've had some discussions at some of the earlier meetings this year on the 13 ROP itself and with respect to its, I think it's overall success.

14 So to some extent, sometimes I find myself in the range, of 15 just tweaking it for the sake of tweaking it or are there other things we have?

16 I've looked at some of the industry proposal. I haven't 17 really fully digested some of them.

18 But I know you got, as mentioned, we got some feedback 19 from both industry and from UCS, and I think some others. Where there 20 any areas where you might have seen that sort of at-range of folks were 21 aligned on potential suggestions or is it just a dichotomy of various types of 22 proposals and things like that?

23 MR. MILLER: Thanks, Commissioner Burns. There was 24 a very wide spectrum so there was some on one side that would counter the 25 proposals on the other side obviously.

65 1 COMMISSIONER BURNS: Yes.

2 MR. MILLER: But, including with what with Staff was 3 already working, with what industry is proposing and some of the things we 4 got from transformation team and from NEI and NRUG, I think there is a 5 couple of areas that we could focus on.

6 First of all, and I would put it in the tweaking realm, but 7 take an overall look at really the things that are given us benefit and where 8 we might make adjustments.

9 I think the emergency preparedness SDP, but not just the 10 SDP but looking at the inspections, can we get to a more risk-informed area 11 there?

12 And they've got a good assessment process going right 13 now. I think they're expecting to deliver a paper in the end of November, 14 somewhere at the end of this year, and we'll assess that.

15 But that's an area, some of the SCPs can use some work.

16 I think there is a feeling that our public and internal members of the NRC and 17 industry have some feeling that a White finding, which is, right now, 18 characterized as low to moderate risk significant, has too much emphasis.

19 In other words, there is too much public emphasis on it for 20 an item of that risk significant. So, can we do some things to actually look 21 at what we do with White findings and, first of all, communicate that properly.

22 And then look at see what is, what are ways to incentivize licensees that 23 take the corrective action, which is really what we want in that areas, for 24 them to take the corrective action.

25 COMMISSIONER BURNS: Right.

66 1 MR. MILLER: And then as you've mention, put it in the 2 licensee's hand rather than the inspectors. If we already know what their 3 corrective actions are or where they're going, maybe we don't need to focus 4 on that area quite as much.

5 The other thing that I think that I've heard from industry, 6 and we're also looking at, is there is some performance indicators that we 7 can get better value out of that aren't serving us well. I know one that has 8 been highlighted in the industry's paper is, the mitigating systems 9 performance index and whether we're getting any value, is it giving us any 10 insight as regulator and is it worth the effort that everyone is putting into that.

11 So maybe there is other performance indicators that can 12 help supplement our inspection program. So I think that's kind of where the 13 staff is trying some of our efforts, get a few items that we can address in the 14 short-term and see how we can improve in those areas.

15 COMMISSIONER BURNS: Okay. All right, thanks.

16 Marissa, I'm going to turn to you.

17 You talked about efficiencies in identifying the baseline 18 security inspections, including streamlining the inspection process. I think it 19 was about two weeks ago or so I went down to Calvert Cliffs to observe a 20 licensee conducted force-on-force and I think one of the interesting things 21 there, our inspection team was there, they were doing some of these other 22 things there and I think that they still obviously engage in looking over the 23 preparations and then conduct of the exercise itself was very interesting.

24 And I think I got a lot of benefit out of, from our NSIR 25 staffers there in terms of the insights, in terms of the program. What I call

67 1 overall program, both inspection and force-on-force in terms of it's the 2 maturity that it's gained over the 15, 17 years that it's been more in this type 3 of form.

4 And I think the observations I heard from our team is that 5 basically where licensees are in terms of their, the fidelity, quality of conduct 6 of these exercises is much higher than say ten years ago overall.

7 And that's, I think, partly a benefit of the NRC conducted 8 force-on-force. It's also, I think, I think when I say stability, it doesn't mean 9 that we don't adjust and things like that, but we know kind of what the 10 program is and what we're looking for, certainty in doing that so that it does 11 give a real value in terms of assessment where licensees are and for our 12 ability to give the evaluation that we need. So, I thought that was an 13 interesting thing so I appreciate the work there from NSIR.

14 Finally, I want to, and this may be for Robert or Ho, there 15 was some background material with respect to the leveraging research, and 16 I agree, Ho and I having been former NEA people and I had more of the 17 legal end because I had to look at all the agreements and certify that it was 18 Brian Sheron's signature on it or Mike Weber, or not Mike Weber, it was only 19 Brian at that time.

20 But, one of the things, a couple of the things we got a 21 background, there are relatively, what I'll call old NEA reports, 2007, 2002.

22 Is there something going on, the reports, these were provided to the 23 Commission as background material, they're publicly available reports, but 24 they were about sort of an international assessment research needs, or 25 actually, one of them talks about vulnerabilities. I think the older one, the

68 1 2001.

2 Is there something going on now at NEA where the 3 member states are looking at that again? I didn't get a good feel about that 4 question.

5 MR. TREGONING: Do you want me to take it or do you 6 want to take it?

7 MR. NIEH: Go ahead and go first and then I can 8 supplement.

9 (Laughter.)

10 MR. TREGONING: The short answer is yes. There is a 11 current activity that actually, serendipitously actually started about a year 12 ago, even before the first rumblings of the Halden closure actually were 13 being heard.

14 But the idea of this effort is internationally. It's a group of 15 participants from all the nuclear countries, in both Asia and Europe and then 16 the Americas.

17 And we have, NRC has representation on this Panel. And 18 the Panel's mandate is to look at nuclear infrastructure worldwide, identify 19 facilities or capabilities that are important to safety that are at risk and then 20 provide recommendations for either sustaining them if they are viable, or 21 perceived to be viable in the future, or if they're not perceived to be viable or 22 needed in the future, possibly just letting them close if that's an option.

23 So yes, that's an activity that's currently underway. It 24 started when Ho was at NEA. He chaired a couple of those early meetings 25 and sort of articulated his vision for the effort. And it's scheduled to, I think,

69 1 finish up about the end of 2019.

2 Now, unfortunately, we saw with Halden, things can move 3 very quickly in an international community and the Halden decisions were 4 made over a relatively short amount of time. So I think the panel is 5 cognizant of that fact.

6 So, we're interested in even getting some preliminary 7 recommendations well out in advance of the end of the project. Even 8 starting as early as end of this year, early 2019.

9 COMMISSIONER BURNS: Okay, thank you.

10 MR. NIEH: I think Rob gave a very good explanation.

11 COMMISSIONER BURNS: Okay. And just a 12 clarification, is this the senior expert group on safety research support 13 facilities for existing advance reactors?

14 MR. TREGONING: That's correct.

15 COMMISSIONER BURNS: Okay.

16 MR. TREGONING: You got the acronym correct.

17 COMMISSIONER BURNS: All right, I'm done. Thank 18 you.

19 MR. TREGONING: If you go further, it's SFEAR2.

20 COMMISSIONER BURNS: Oh, there you go.

21 MR. TREGONING: So there, you delve into acronyms.

22 COMMISSIONER BURNS: No, I'm not buying a nuclear 23 acronym report. Thanks.

24 CHAIRMAN SVINICKI: All right, thank you very much.

25 Commissioner Caputo.

70 1 COMMISSIONER CAPUTO: Well, I'm in the unenviable 2 position of being all that stands between everyone and lunch. But I will start 3 by echoing the Chairman's, some of the Chairman's remarks.

4 I do believe NRC can strive, innovate and excel. I also 5 agree that it is a choice, but it's also a matter of leadership.

6 And I'd like to just say I am very pleased that Margie and 7 Ho have stepped into the positions that they have. I have high expectations 8 but also high confidence that you will do very well, and I appreciate the 9 dedication and willingness to take on the challenges in front of you. So 10 thank you very much for taking on your new positions.

11 Ho, I will start with you. To me, budget execution should 12 be risk-informed more or less and resources should be spent on the most 13 safety significant work.

14 How will you use budget execution from 2018 to inform 15 resource allocation in 2019, to ensure that this is the case?

16 MR. NIEH: Thank you, Commissioner. So, one thing 17 that I've been working on with my team in NRR is to really get a clearer 18 picture of how we're spending our resources in terms of how much we're 19 charging to specific activities. I mean, very, very specific activities in how 20 the budget itself was even broken down across the business line.

21 So, very recently, I would say within the last week or so, 22 everything is kind of running together, I had a really good meeting with the 23 CFO staff. And they gave a very comprehensive picture about the NRC's 24 operating business lines 2018 budget and how it's distributed across the 25 partner offices.

71 1 The next level of detail I'd like to get to is to really look at 2 areas that were doing work, such as licensing, oversight, how much we're 3 actually spending. Because I think I need, for me at least, I need that 4 reference to look at actually what we're doing to better inform what we 5 should be doing in the out years.

6 Another area that I'm looking at, and I'm exploring this with 7 the NRC's leadership team, NRR's leadership team is to, okay, when we 8 have the whole picture of all the things we're working on, I'd like to get a 9 good sense of, in this universe of work, what are the issues that are really 10 the most important, okay.

11 And if there are issues that we judge that are not high 12 safety significance, does the amount of time we're spending on that 13 synchronize with, is it commensurate with the safety significance of the 14 issue. So, to me I need the data to really see what the staff is charging time 15 to, how much effort they're putting into the whole host of issues that are 16 happening in NRR.

17 So, for me I see that getting the actual data in 2018 is very 18 important. Again, we have the tools available and the staff are working on it 19 right now to really get that clear picture.

20 There is an effort also underway to even look a little bit out 21 to the future beyond 2019 and 2020, to really get a sense of what's 22 happening in the environment that we're regulating and to try to match that 23 up with what we would anticipate asking for in terms of resources and how 24 much we need to carry out our mission.

25 COMMISSIONER CAPUTO: Thank you. Ms. Bailey, in

72 1 SRM-16-0073, which was before I was confirmed, regarding force-on-force 2 inspections, the Commission directed the Staff to assess whether crediting 3 of operator actions, the use of FLEX equipment and response by local state 4 and federal law enforcement would make the force-on-force exercises more 5 realistic.

6 The Staff was directed to submit, and I'm quoting, "submit 7 a notation vote paper to the Commission within 12 months, from the date of 8 this staff requirements memorandum, with recommendations on 9 improvements to the security inspection program."

10 17-0100 currently before the Commission is that paper.

11 And the 12 month deadline was October 4, 2017. However, the Staff is only 12 partially responsive to the Commissions direction in that SRM 16-0073.

13 With regard to crediting operator actions, including FLEX 14 equipment, the NRC Staff indicates it will use the change management 15 process to credit for any additional operator actions, but no mention is made 16 to bring this matter back to the Commission as a voting paper.

17 With regard to credit for law enforcement response as to 18 how states that it has evaluated, whether to give credit and will provide 19 options and recommendations to the Commission, in a subsequent paper.

20 But with no indication of a time frame.

21 So, clearly, it was the Commission's direction in 16-0073 22 that the Staff submit a notion vote paper with recommendations on these 23 matters by October 4, nearly a year ago. When will the Staff complete the 24 remaining work as directed?

25 MS. BAILEY: The Staff is currently evaluating a proposal

73 1 from NEI on a methodology for crediting local law enforcement. That's been 2 an iterative process.

3 NEI first submitted that proposal in the 2017 time frame.

4 We provided questions and feedback and they've revised that proposal 5 based on the questions that we've asked.

6 Our latest interaction with them, they've indicated that they 7 plan to provide another vision to their proposal in the October time frame.

8 We still have to align with the Office of the EDO in terms of 9 when we would submit the SECY paper or the vote paper to the Commission 10 with options for crediting local law enforcement. One of those options would 11 be the methodology that's presented by NEI, but there will be other options 12 in there.

13 COMMISSIONER CAPUTO: And how about credit for 14 operation actions and FLEX equipment?

15 MS. BAILEY: I think that's going to be part of that. I'll 16 need to, if you would allow me, I could ask Dave Curtis to provide more 17 details on that one?

18 MR. CURTIS: Good morning, Commissioners, Chairman 19 Svinicki. So, they are intertwined.

20 And in addition to that integrative response, which is a 21 voluntary program, has provided insights to the staff over a number of years 22 about local law enforcement and how they respond. And so we also intend 23 to provide insights into how we intend to proceed with the integrated 24 response program in addition to the other two items that you mentioned.

25 COMMISSIONER CAPUTO: All right.

74 1 MR. FELTS: Can I offer just a thought? And it's probably 2 not in my swim lane but I used to be in NSIR, I used to be a force-on-force 3 team leader.

4 Licensees can currently credit FLEX and associated 5 operator actions as long as they basically demonstrate that they're viable, 6 reasonable in the context of an advisory tech. So, that might help as 7 context.

8 COMMISSIONER CAPUTO: Mr. Miller, 19 years of 9 experience with the ROP should offer a wealth of information and support of 10 risk-informing of the ROP. I believe this is something we should pursue.

11 In your work to improve the ROP, what are you doing to 12 risk-inform it?

13 MR. MILLER: So, thank you for the question, 14 Commissioner. There is a number of areas we're looking at, first of all, just 15 the tools for risk informing or for coming up with risk, which would be the 16 significance determination process.

17 There's a number of areas where that has been improved.

18 I mentioned the Mercy Preparedness effort.

19 And it really is a holistic effort because what are they 20 focusing on, the risk, the four risk significant planning standards, so why do 21 we need to focus a lot of effort in some of other areas when they can focus 22 them on those specific areas. But they're also carrying that on to 23 inspection. So, you got to look at how we reach our risk significant 24 decision.

25 Another area is recognizing that industry performance has

75 1 improved in some areas. So, perhaps in areas where we were worried 2 about them in the inception of the program, maybe there has been enough 3 improvement that we can look at another way.

4 We could look at perhaps performance indicator that 5 captured things better. I'm thinking about areas like radiation protection, 6 other areas where there has been some significant improvement in, for 7 example, overall dose. So, we know that there is indicated.

8 Overall, SCRAMs have reduced across the last 18, 20 9 years. So, is there a way of recognizing better performance in initiating 10 events and other areas to say, well, maybe there is less inspection needed 11 in some of those areas. Maybe the follow-up from the SCRAMs may be 12 samples in the inspection program.

13 So, we're not looking at it as an overall transformational 14 change to the ROP, because it has a lot of good elements. In fact, in the 15 recent letter from the industry they said, hey, the ROP is very sound, we'd 16 like some moderate adjustments is what they're looking at.

17 So, we don't want to throw the baby out with the bath water 18 but we do think that there is elements that we can do.

19 And I think, as I mentioned before, some of the 20 performance indicators, industry has some, a better suite of performance 21 indicators that we don't currently use or hold in our ROP.

22 But maybe there is some tradeoffs that can be made 23 where we actually get better performance indicators. And ones that aren't 24 really helping us much, helping the public much and are difficult to maintain, 25 like the MSPI as I mentioned earlier.

76 1 I think those areas we can use, put our efforts somewhere 2 else where it's providing more value.

3 COMMISSIONER CAPUTO: Thank you.

4 MR. MILLER: Thank you.

5 CHAIRMAN SVINICKI: All right, well, thank you all. And 6 I will just note that I'll give an opportunity for my colleagues to ask a 7 lightening round question before I do the following, but we didn't talk too 8 much about accident tolerant fuels.

9 There's a lot of activity in the regulated community and 10 then there's a lot of interest in accident tolerant fuels. One of the precursor 11 activities to considering new fuel types is the insertion of what are called 12 Lead Test Assemblies in currently operating reactors.

13 The agency was approached over the last couple of years 14 about, with various proposals to insert these Lead Test Assemblies so that 15 we can test materials, properties, nucleonic assumptions, things like that for 16 the development of new fuel types.

17 My characterization is that we discover a little bit of 18 procedural ambiguity when we were in receipt of these proposals. My 19 question is simple.

20 If we put some measures in place so that we would be 21 able to provide procedural clarity going forward if we received additional 22 proposals for Lead Test Assemblies or were aware of licensee proposals to 23 do so, is the Staff confident that we have, or are taking the right measures to 24 avoid procedural ambiguity should we receive additional proposals or learn 25 of licensee initiatives to insert Lead Test Assemblies, including what requires

77 1 the license amendment request and what does not?

2 Are we taking the steps so that we will have the necessary 3 clarity going forward?

4 MR. NIEH: Thank you, Chairman. Yes, I believe we're 5 moving in the right direction to provide the clarity.

6 As you may know, my predecessor, then the acting NRR 7 director earlier this year, issued by letter some guidance that attempted to 8 clarify this ambiguity in terms of what would require a license amendment 9 request versus what could be done via a process like 50.59 at a site.

10 So, that letter was made publicly available. We received 11 hundreds and hundreds of comments that we're responding to. It is our 12 goal to finalize that letter.

13 It's my understanding that we're taking into account the 14 comments we have received. I don't anticipate that the letter is going to 15 dramatically change, but we're working on it and we just want to make sure 16 we give a very thorough review of the comments and we hope to get that 17 letter out in draft form.

18 There are some administrative aspects of finalizing this 19 letter that we're going to need to go through, such as review by OMB and 20 things of that nature. But we are targeting first quarter of 2019 to get the 21 final letter out.

22 And again, I think that should provide some sufficient 23 clarity to the industry and for LTAs.

24 CHAIRMAN SVINICKI: Okay, thank you for that update.

25 Do any of my colleagues have a final quick question? Commissioner --

78 1 (Off-microphone comment.)

2 CHAIRMAN SVINICKI: Okay. Commissioner Caputo.

3 COMMISSIONER CAPUTO: All right, I'll take this 4 opportunity to squeeze one last question in then. Mr. Felts, I'd like to 5 understand how you have used the results from the SOARCA report to 6 support risk-informed decision-making, one, if SOARCA's key results is that 7 all modeled accident scenarios progress more slowly and release much 8 smaller amounts of radioactive material than calculated in earlier studies?

9 In other words, the consequences are less.

10 How do you use this knowledge to improve realism and 11 risk-informed decisions?

12 MR. FELTS: I might need some help from either my 13 research colleagues or DRA staff to answer how we are using SOARCA 14 results now to improve risk-informed decision-making.

15 I know that we have recently received, from EPRI, a report 16 talking about margins to the safety goals. And we are anticipating a 17 follow-on letter from NEI talking about how we might leverage the insights in 18 the EPRI report operationally to take advantage of those margins.

19 And I know SOARCA was, in large part, part of the 20 analysis that EPRI did in talking about the safety goals.

21 COMMISSIONER CAPUTO: So, I guess just to clarify, 22 so, the SOARCA report was 2012, 2013? So, our assessment of whether 23 or not we are going to use that knowledge in everyday decision-making is 24 still in a research state?

25 MR. FELTS: No. My point was that, the details of how

79 1 SOARCA was used in the intervening years between then and now.

2 Someone else might know, have more --

3 CHAIRMAN SVINICKI: I would note for you, Russ, you 4 have your very capable boots on the grounds, PRA individual C.J. Fong, 5 who's really going to help us out there.

6 MR. FONG: Thank you, Chairman. No pressure with 7 that intro.

8 (Laughter.)

9 MR. FONG: A couple other examples. We are revising 10 some of our appendices in the significance determination process, including 11 Appendix H, Appendix Hotel, which looks at large early released frequency.

12 So we are leveraging some insights there.

13 I'll also point to the seismic PRA reviews. They've used 14 insights from SOARCA, in particular, how ice containment, ice condenser 15 containment rather would perform under certain conditions. So, we are 16 using those results to make better risk-informed decisions.

17 MR. FRANOVISH: Mike Franovich, I'm director of risk 18 assessment, NRR. A couple of historical points here on SOARCA.

19 SOARCA has been applied in a number of regulatory 20 decisions. If you'll look back at some of our post-Fukushima work, in 21 particular with the BWR filtered vents, the containment protection and 22 released reduction ruled a SOARCA tools were applied extensively in 23 coming out with that recommendation that we not proceed further with the 24 rulemaking effort in itself, and that helped enable the Commissioner to make 25 a decision there.

80 1 Other places we've used it, and it's probably been a little 2 more anecdotal than systematic. Again, in the Fukushima work we were 3 looking at a number of plants that needed to do a seismic PRA. It's a 4 subset of the fleet.

5 A few of the plants came in and said, in particular, we have 6 some insights from the SOARCA study, we are particular containment type, 7 can we leverage those insights in particular with the ice condenser plants 8 and see if we can do a less of a seismic review short of doing a full SPRA.

9 And we actually granted relief for four units to say we didn't 10 really need a full SPRA to get the extra insights to make our final regulatory 11 decisions. So I wanted to share that with you.

12 And then, what Russ was alluded to is, the industry has 13 said, can we do more, can we leverage these insights more systematically in 14 our decision. So we're now expecting, or anticipating, a paper from NEI to 15 come sometime in the late October time frame, that will be presented before 16 our executive committees, for risk-informed steering committee in particular, 17 to see what more can we do in that arena to bring it into the RIDM arena for 18 our decision-making and our processes down the road.

19 So there is work going on in that area it's just not labeled 20 SOARCA per say. So I'm going to leave that with you for thought.

21 COMMISSIONER CAPUTO: All right, thank you.

22 CHAIRMAN SVINICKI: All right, well, thank you all. And 23 thanks again to the Staff. It's obviously a very, very active content rich area 24 so it's a tough business line to cover, but you did so very admirably, so thank 25 you again. And we are adjourned.

81 1 (Whereupon, the above-entitled matter went off the record 2 at 12:06 p.m.)

3 4

5 6