ML22089A010
| ML22089A010 | |
| Person / Time | |
|---|---|
| Issue date: | 03/23/2022 |
| From: | Office of Nuclear Reactor Regulation |
| To: | |
| Bhagwat Jain | |
| References | |
| NRC-1873 | |
| Download: ML22089A010 (269) | |
Text
Official Transcript of Proceedings NUCLEAR REGULATORY COMMISSION
Title:
Digital Instrumentation and Controls (DI&C):
Licensing and Inspection Lessons Learned From Recent DI&C Modernization Projects Docket Number: (n/a)
Location: teleconference Date: Wednesday, March 23, 2022 Work Order No.: NRC-1873 Pages 1-268 NEAL R. GROSS AND CO., INC.
Court Reporters and Transcribers 1716 14th Street, N.W.
Washington, D.C. 20009 (202) 234-4433
1 1 UNITED STATES OF AMERICA 2 NUCLEAR REGULATORY COMMISSION 3 + + + + +
4 NRC WORKSHOP ON DIGITAL INSTRUMENTATION AND CONTROLS 5 (DI&C): LICENSING AND INSPECTION LESSONS LEARNED 6 FROM RECENT DI&C MODERNIZATION PROJECTS 7 + + + + +
8 WEDNESDAY, 9 MARCH 23, 2022 10 + + + + +
11 The workshop convened via Videoconference 12 at 9:00 a.m. EDT, Bhagwat Jain, Meeting Facilitator, 13 presiding.
14 15 NRC STAFF PRESENT:
16 BHAGWAT JAIN, NRR/DEX/ESEB 17 SAMIR DARBALI, NRR/DEX/ELTB 18 DAVE DESAULNIERS, NRR/DRO 19 GREG GALLETTI, NRR/DRO/IQVB 20 BRIAN GREEN, NRR/DRO/IOLB/HFT 21 KIM LAWSON-JENKINS, NSIR/DPCP/CSB 22 SHIATTIN MAKOR, R-IV/DORS/EB2 23 MICHAEL MARSHALL, NRR/DORL/LPL1 24 LAUREN NIST, NRR/DRO/IOLB 25 CHARLEY PEABODY, NRR/DSS/SNSB NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
2 1 RICHARD STATTEL, NRR/DEX/EICB 2 DINESH TANEJA, NRR/DEX/ELTB 3 JUSTIN VAZQUEZ, NRR/DRO/IOLB/HFT 4 MICHAEL WATERS, NRR/DEX/EICB 5
6 ALSO PRESENT:
7 RONALD L. BORING, Idaho National Laboratory 8 JACOB CHAMPAGNE, Entergy 9 JOHN CONNELLY, Constellation 10 WESLEY FREWIN, NextEra 11 PAREEZ GOLUB, NEI 12 JEFFREY C. JOE, Idaho National Laboratory 13 WARREN ODESS-GILLETT, Westinghouse 14 15 16 17 18 19 20 21 22 23 24 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
3 1 P R O C E E D I N G S 2 9:00 a.m.
3 MR. JAIN: Okay, we can get started, it's 4 9:00 a.m. Good morning to all, welcome to the digital 5 I&C workshop. Today's public meeting is an information 6 meeting with a question and answer session.
7 The workshop is posted by the NRC Staff by 8 the industry and licensee to discuss licensing and 9 inspection lessons learned from recent digital I&C 10 upgrades, presubmittal activities, and regulatory and 11 technical issues related to human factor engineering, 12 or HFE, reviews.
13 The discussion and exchange of information 14 today will assist the NRC in approving our regulatory 15 infrastructure and in communicating the statutory 16 expectations realized in the inspection of digital I&C 17 upgrades.
18 My name is Bhagwat Jain and I am a senior 19 Project Manager in the Office of NRR, along with me is 20 Michael Marshall, and together we performed the 21 project management function for all things digital in 22 NRR.
23 I need to cover a few logistics points and 24 then I will turn it over to Mike Waters, the INC 25 Branch Chief, in the Office of NRR for opening NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
4 1 remarks.
2 The duration of the morning session is 3 three hours or three and a half hours and it will 4 include Staff's presentation on licensing and 5 inspection lessons learned with a ten-minute break in 6 between at around 11:00 a.m.
7 After one-hour lunch, the afternoon 8 session is three hours long and it will include 9 presentations on Staff's HFE reviews, NEI's approach 10 to HFE, industry reflections on licensing and 11 inspection lessons learned and then an opportunity for 12 an open discussion.
13 In this public meeting, no regulatory 14 decision will be made. This is an open public 15 interaction and as such, we will not be discussing any 16 proprietary or sensitive information.
17 We are using Microsoft Teams to conduct 18 this meeting. Please ensure you are muted when you 19 are not actively speaking and do not speak over each 20 other.
21 To facilitate the question and answer 22 portion of the meeting, we recommend that you utilize 23 the raised hand feature in Teams so we can more easily 24 identify who has a question or comment and call on the 25 individual to ask their questions.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
5 1 The presentation will be shown via 2 Microsoft Teams meeting link that was provided in the 3 meeting notice. You can also the presentation slides 4 in our ADAMS system at accession numbers, for Staff 5 presentation the ML number is 22077A409.
6 For NEI presentation, the ML number is 7 22080A021, repeating, ML22080A021, and for the 8 industrial presentation the ML number is ML22076A191.
9 Again, ML22076A191. Now, the slides have also been 10 posted on the meeting notice on the NRC's public 11 website.
12 Given the number of participants on the 13 call, we are going to forego introductions. Instead, 14 I would ask that as a person speaks, the first 15 introduce themselves, please stand and speak.
16 Just for your information, today's meeting 17 is being transcribed.
18 The Staff contacted stakeholders to 19 provide comments and their feedback. If you have 20 comments or feedback on any aspect of the meeting, 21 please contact me or Michael Marshall and we will 22 provide you the necessary forms.
23 Our contact information is provided in the 24 public meeting notice posted on the NRC website. With 25 that, I'll turn it over to Mike Waters for opening NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
6 1 remarks. Mike is going to make opening remarks.
2 Mike?
3 MR. WATERS: Thanks, BP, and again to 4 everybody, welcome to our 2022 digital I&C workshop.
5 We are very to get everybody together again in this 6 forum. We truly do consider this as an important 7 aspect of our continual effort to improve our 8 regulatory processes and our regulatory efficiency.
9 It has been a little bit more than a year 10 since our last workshop and since then, there have 11 been a number of accomplishments and let me recount 12 them. NRC has made good progress in both exercising 13 and honing the licensing and inspection guidance.
14 Entergy has successfully provided a sound 15 design and licensing basis for the planned digital 16 upgrade of Plant Waterford.
17 Both NextEra and Constellation have put a 18 lot of energy and work into a major design license 19 amendment request for Turkey Point and Limerick, 20 planned to be submitted to the NRC later this year, 21 and as we all know on a broader scale, industry, NEI, 22 DOE, EPRI, IEEE, and other licensees have made some 23 good passenger in the digital I&C arena, especially in 24 developing statewide practices and standards from 25 digital upgrades to support your long-term operations.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
7 1 So, truly our goal today is for everyone 2 to share these insights, build upon streams and 3 experiences we have gained from a regulatory 4 perspective, and of course, identifying challenge 5 areas where more attention may be needed to address 6 fact of life changes in how these digital I&Cs are 7 implemented.
8 We are confident this workshop will 9 succeed for a diversity of stakeholders that we have 10 here today. Finally, because I am a regulator, I do 11 have to make one disclaimer and fall upon what BP 12 said.
13 No regulatory decisions will be made, we 14 do have two pending applications before us in pre-15 application space. Indeed, part of this workshop is 16 to discuss their experiences and lessons learned from 17 the security application interactions.
18 However, we cannot use workshops to make 19 decisions on specific issues associated with those 20 applications that are coming to us. Those types of 21 discussions of course will be made in separate 22 application interactions.
23 So, again, welcome, we do have a packed 24 agenda. There are some impressive presentations and 25 I'm excited to get started. With that, BP, any issues NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
8 1 to communicate before opening remarks?
2 MR. JAIN: Yes, I'm requesting to NEI if 3 they would like to make any opening remarks. Alan?
4 MR. CAMPBELL: Thank you, BP, can you guys 5 hear me okay?
6 MR. JAIN: Yes.
7 MR. CAMPBELL: Good morning, everyone, I 8 am Alan Campbell and I'm a technical advisor with NEI 9 and we are the Digital I&C Work Group. I want to 10 thank the NRC for hosting this workshop today.
11 I think we have a lot of great agenda 12 items and great discussions to look forward to.
13 As Mike was just mentioning, ISG-06 Rev 2 14 and the associate alternate review process were a 15 great step forward for the industry and really helped 16 provide licensees with the confidence to move forward 17 with a fresh wave of digital I&C applications.
18 These digital modifications are imperative 19 for our industry as we continue to strive for safe and 20 long-term operation of our existing fleet. As 21 expected, we have learned a great deal through these 22 initial projects using this approach.
23 And we appreciate the opportunity today to 24 discuss the lessons learned using this process and 25 garner insights from the inspections. In addition to NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
9 1 gaining a good understanding of the lessons learned, 2 NEI and its members are looking forward to the 3 discussion on human factors engineering, expectations 4 during the LAR process.
5 We appreciate the NRC's efforts to 6 communicate its concern with the availability of 7 information and the potential path forward using 8 multi-stage validation and earlier public discussions.
9 Our goal today is to further this 10 discussion with an alternative approach that aligns 11 with the intent of the alternate review process and 12 uses a research-based approach that has proven to lead 13 to successful integrated system validation results.
14 Ms. Pareez Golub will be presenting on 15 behalf of NEI and its Members, and we've allocated 16 part of the NEI timeslot for Idaho National Labs to 17 present some research pertinent to the discussion.
18 Our goal today with that discussion is to 19 identify any considerations or significant challenges 20 to our alternative approach. We're also looking 21 forward to discussing the NEI member comments provided 22 to the NRC on the inspection plan 52003.
23 While we understand inspection procedures 24 do not typically require public comment periods, we 25 felt that it was necessary to identify key items that NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
10 1 we believe can be resolved generically to avoid any 2 misunderstandings or misapplications in the future.
3 And we appreciate the NRC's willingness to 4 engage with the comment resolution process on this 5 matter. Again, I want to thank the NRC, specifically 6 BP and Michael Marshall, for arranging this workshop.
7 And we look forward to a great dialog through today's 8 meeting.
9 MR. JAIN: Thank you, Alan. Now, I'll 10 next introduce Samir Darbali, he's the Staff Team Lead 11 for this morning's session on the presentations, 12 licensing and inspection lessons learned.
13 Now, we have several other NRC Staff 14 online and as they contribute to the meeting they will 15 introduce themselves. Please wait for your questions 16 until the end of each presentation.
17 With that, I will now ask Samir to make 18 the Staff's presentation. Samir?
19 MR. DARBALI: Thank you, BP. My name is 20 Samir Darbali, I'm an I&C technical reviewer in the 21 Division of Engineering and external hazards in NRR.
22 And I'll be going through the licensing and lessons 23 learned slide.
24 Like was mentioned, we had the first 25 lessons learned digital I&C workshop back in February NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
11 1 of last year, which although it was early in the use 2 of the alternate review process or ARP, it proved to 3 be very productive.
4 We have covered the background and history 5 of ISG-06 before and so we're not looking to repeat 6 that today. We can think of the theme for this 7 presentation to be where we started, where we are, and 8 where we're going when it comes to ISG-06 usage for 9 digital I&C modernization.
10 So, following that theme, we'll start 11 looking at what was the intent of the alternate review 12 process when it was developed. We'll then move into 13 some of the deviations we have encountered and how the 14 overlap of the licensing and development schedules is 15 somewhat shifting.
16 We'll also identify some of the lessons 17 learned related to information submittals, licensing 18 audits, the vendor oversight plan, and the VOP 19 summary.
20 We'll also talk about the review scope of 21 other NRC Branches that are outside of the INC grout, 22 and finally, we'll look at some of the things that we 23 can do to achieve licensing success path for digital 24 I&C modernization.
25 As Mike mentioned, the current digital I&C NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
12 1 licensing activities that are exercising key portions 2 of ISG-06. The Waterford III core petition calculator 3 short-term replacement was the first application using 4 the alternate review process and it was approved in 5 August of last year.
6 The Staff also completed the inspections 7 for the factor acceptance testing and set acceptance 8 testing, SAT, and we're currently preparing for the 9 site installation inspection.
10 We've also been engaged in pre-application 11 meetings since 2020 with the licensees for Turkey 12 Point and Limerick and we're expecting their 13 respective digital modernization LARs to be submitted 14 this year.
15 So, the main intent of the ARP was to 16 allow for issuance of a license amendment prior to 17 completion of the system's implement and test 18 lifecycle phases. This would provide licensees with 19 much needed regulatory certainty.
20 An application based on the ARP would 21 focus on the system design, the development process 22 for the system's software and hardware, a summary of 23 the vendor oversight plan, and additional regulatory 24 commitments for those lifecycle activities that take 25 place after each license amendment.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
13 1 And system implementation and testing 2 including factor acceptance testing is subject to 3 oversight through NRC inspections as well as the site 4 inspections.
5 This slide shows the ISG-06 licensing re-6 process options provided to licensees for developing 7 license amendment requests. Each lane is for a 8 specific review process and there are prerequisites 9 for using each lane.
10 The tiered lane provide the benefit of 11 supplemental information as well as options for 12 referencing a pre-approved topical report.
13 The ARP lane provides the benefit of an 14 earlier approval but doesn't allow for supplemental 15 information to be submitted and doesn't allow for 16 deviations from the topical report.
17 That means that not all applications are 18 suited for use in the ARP as it is described in the 19 ISG. We typically compare the tried process with the 20 ARP and the ARP shouldn't be seen as being easier for 21 either the licensee or the Staff, it just means that 22 it's packaged in a way that supports an expedited 23 review.
24 So, for a licensee that wants to use a 25 platform that hasn't been approved previously through NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
14 1 a topical report, let's say they identify a vendor and 2 they want to use this platform that hasn't been 3 approved, that's something that is accommodated in the 4 tiered process.
5 They just need to provide details on that 6 platform in order for the Staff to perform its review.
7 For a Tier 2 application, we're looking at a licensee 8 referencing an approved topical report but there are 9 deviations in the application.
10 So, for example, it could be new modules, 11 a change in technology, some new cards or a change in 12 software that wasn't approved during the topical 13 report evaluation.
14 So, the licensee would need to add that 15 information into the application for the Staff to be 16 able to review the system.
17 And then the Tier 1 process references the 18 topical report without any changes. So, out of the 19 three tiered options, that provides the more efficient 20 review process.
21 And the Staff's focus in the evaluation is 22 based on just those plant-specific aspects of the 23 modification, including plant-specific action items 24 from the topical report.
25 Now, the ARP also references a previously NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
15 1 approved topical report but like we said, it doesn't 2 allow for deviations to that topical report.
3 So, you wouldn't expect an application 4 that resembles a Tier 1 with some supplemental 5 information or Tier 2 with deviations to the topical 6 report to follow through or use the ARP lane.
7 So, the ARP lane does provide that 8 expedited path but an application has to meet the 9 prerequisites for using that lane.
10 And as we'll cover later, one of the key 11 aspects and benefits of the ARP is the earlier 12 issuance of the license amendment followed by 13 inspections of implementation and testing.
14 So, the ARP was developed to provide 15 licensing review guidance to quickly support upcoming 16 digital I&C modifications.
17 This intent was that all information was 18 to be provided in a single high-quality submittal, 19 meaning that there would not be a need or room for 20 RAIs or supplemental information.
21 The emphasis was given on determining how 22 past one-for-one digital modifications like Oconee, 23 Diablo Canyon, and Hope Creek could be reviewed and 24 improved before completion at FAT.
25 Emphasis was not given specifically to NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
16 1 modifications that will be envisioned in the future 2 like those that incorporated a combination of systems, 3 the elimination of sensors, measure of control room 4 changes, changes to plant defense in-depth and 5 diversity, or crediting of self-diagnostics to 6 eliminate surveillance requirements.
7 We knew those applications would 8 eventually come but the expectation was that they 9 would come in after the ARP had had enough run time 10 and lessons learned.
11 So, we've covered the intent of the ARP 12 and now we're going to look at what we're seeing 13 currently, which is that actual applications deviate 14 in some manners from the ISG-06 guidance in terms of 15 the LAR contents, the use of supplemental information, 16 and the licensing and development timelines.
17 The scope of LAR modifications is more 18 expansive and goes beyond the one-for-one digital 19 replacements that we were expecting.
20 And they include changes to plant 21 architecture, operations, and tech specs, which 22 requires added NRC resources to address more complex 23 applications.
24 Now, minor or modest deviations are not 25 necessarily impediments to the review if they're NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
17 1 addressed early in pre-application meetings. The 2 challenge really is in attempting to apply a specific 3 ISG-06 licensing process when the application doesn't 4 align with that process.
5 To address these deviations and maintain 6 regulatory certainty, the Staff and the licensee may 7 need to adjust the licensing review scope and 8 schedule, as well as their expectations.
9 This includes adjusting for the potential 10 for additional information to be audited or docketed, 11 changes to the licensing review and license amendment 12 issuance schedules, and changes to the licensing audit 13 and inspection scopes and schedules.
14 Flexibility on the part of the Staff and 15 licensees will need to consider other aspects of the 16 application, such as the level of detail in the vendor 17 oversight plan, the use of regulatory commitments, and 18 even the safety significance of the modification.
19 Now we're going to talk about the 20 scheduling deviations.
21 The ISG-06 describes what can be 22 considered as the model cases for Tier 1 and ARP in 23 the sense that it assumes that specific licensee and 24 vendor lifecycle development activities will overlap 25 with specific NRC licensing review and inspection NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
18 1 activities.
2 The Tier 1 review schedule overlaps with 3 the system design implementation and testing lifecycle 4 phases, whereas, the ARP compressed the licensing 5 schedule to overlap with the system design and early 6 implementation.
7 What we're seeing is that the actual 8 system development schedule appears to have also been 9 compressed and so the implementation phase is taking 10 place during the later half of the licensing review, 11 also supplemental information submittals like 12 equipment qualification summary reports have an impact 13 on the review schedule.
14 And this has resulted in an overlap of the 15 development and licensing schedules that are neither 16 the ARP or the Tier 1 as it was envisioned in the ISG.
17 So, this slide shows the model case for the ARP 18 licensing and development timelines.
19 Most of you have seen variations of this 20 timeline before, it keeps evolving as we gather more 21 lessons learned.
22 But the top part in blue are licensee and 23 vendor activities, going through the different 24 lifecycle phases, and the bottom part is the NRC 25 licensing inspection activities.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
19 1 And typically, what happens is a licensee 2 will identify an installation date based on a specific 3 refueling outage, when they want to install the 4 system, and that's going to determine, as we'll see in 5 a later slide, when they really need that license 6 amendment to be issued and approved and also that's 7 going to determine when they're going to be able to 8 submit the license amendment request.
9 As you can see in the model case, ARP 10 supplemental information is not provided. The Staff 11 performs the licensing review and regulatory audits 12 based on the information in the LAR which includes 13 requirement specifications and a level of detail of 14 the system design in order to demonstrate that the 15 regulatory criteria is met.
16 I do want to point out that the date when 17 the license amendment is issued doesn't mean it's the 18 time when the Staff completes our safety evaluation.
19 The Staff actually completes the safety 20 evaluation a month or more before, which means that 21 after the Staff completes their draft SE, that SE goes 22 through internal reviews and concurrences, no legal 23 objection review by OGC.
24 There may be ACRS meetings taking place in 25 this time and of course, preparation of the license NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
20 1 amendment itself. So, there are other activities that 2 take place after the Staff has completed their draft 3 safety evaluation.
4 Because at this point, you could say the 5 Staff has put their pencils down, they're not 6 reviewing any implementation activities that are 7 taking place so at that time, if necessary, we would 8 expect vendor oversight plan implementation and vendor 9 inspections to begin.
10 Those would be carried out until factor 11 acceptance testing and they would be followed by 12 regional inspection side activities, including side 13 acceptance testing.
14 This slide shows how an actual ARP 15 licensing and development timelines have shifted.
16 There's no easy way of having this timeline show the 17 effects of multiple deviations without becoming too 18 complicated and hard to read.
19 So, these examples show the effects that 20 multiple supplemental information submittals have on 21 the licensing review schedule.
22 So, you can see supplemental information 23 is being provided so that extends the licensing review 24 schedule and compresses the VOP and vendor 25 inspections.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
21 1 Now, a similar effect may happen when the 2 review schedule remains the same and there isn't 3 supplemental information submittals, but the lifecycle 4 development phases are compressed because for some 5 reason, it doesn't take the vendor as much time to 6 complete each lifecycle phase.
7 So, those would be compressed but it would 8 have a similar effect that the license amendment is 9 getting issued somewhere in the testing phase. Now, 10 in both cases, that license amendment is still being 11 issued before completion of factor acceptance testing.
12 So, this slide shows the factors that 13 determine the licensing and development schedules.
14 So, the factors are, like I mentioned earlier, when is 15 the license amendment needed?
16 That is going to be determined by when the 17 modification is going to be installed. That's going 18 to feed into when the project starts and so when are 19 the development lifecycle phases started and 20 completed?
21 Some of the early lifecycle development 22 phases like concept design requirements and early 23 portion of the design phase are going to feed into the 24 contents of the LAR.
25 So, the schedule for those lifecycle NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
22 1 phases is going to determine when the LAR is going to 2 be submitted and depending on if additional 3 information is going to be provided, when will all of 4 the information be submitted.
5 These three factors are dependent on 6 licensee and vendor resources as well as complexity of 7 the design. So, once all that is incorporated and we 8 have a LAR, then the question is on the NRC side, when 9 can the Staff complete the draft SE?
10 And that's going to be determined on what 11 is the time needed to perform the licensing review?
12 Again, that's going to be dependent on NRC's Staff 13 resources and the complexity of the design.
14 After the Staff has completed their SE, 15 the question is when can the license amendment be 16 issued? And that's going to be dependent again on 17 those post-draft SE activities and those NRC internal 18 processes.
19 So, what we do during pre-application 20 meetings is the licensee will come in and say we are 21 thinking of submitting the LAR at this particular date 22 and we would like approval by this particular date.
23 So, we go through all of these exercises 24 to understand the scope of the modification, the need 25 for supplemental information, and looking at resources NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
23 1 that both the licensee, the vendor, and the Staff 2 side.
3 And what we're trying to achieve during 4 pre-application meetings is can the license amendment 5 issuance date meet that license amendment request 6 date? So, that's an exercise that we do.
7 This slide shows the first table of 8 Enclosure B, which identifies the typical information 9 to be submitted depending on the applicable review 10 process. And it points to the respective sections of 11 the ISG for the detail criteria.
12 So, the information in the Enclosure B 13 tables is based on the applicable lifecycle 14 information and outputs that are subject to Staff 15 review and audit. The tables assume a model case 16 application and did not account for deviations from 17 the ISG-06 guidance.
18 So, given any deviations or application-19 specific aspects, we can expect the different 20 information from that identified in Enclosure B may 21 need to be provided.
22 In the end, the information submitted for 23 an actual application and the timing of submittal may 24 end up resembling something in between the ARP and a 25 Tier 1 application.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
24 1 So, for example, when it comes to the 2 crediting of self-diagnostics for the reduction or 3 elimination of surveillance requirements, there is 4 certain information that needs to be provided besides 5 what is identified in the ARP.
6 This is because even though such 7 surveillance reductions had been approved before under 8 the TR review process, this was not a review topic 9 that was considered when the ARP was developed.
10 However, licensees do want to credit self-11 diagnostics for the elimination of some surveillance 12 requirements and during pre-application meetings, we 13 have identified that an FMEA needs to be provided to 14 support a technical basis.
15 Licensees will need to provide analysis to 16 justify the crediting of self-diagnostics for a tech 17 spec surveillance requirement reduction and we'll need 18 to perform periodic functional tests and the self-19 diagnostic features to satisfy BTP 7-17 guidance.
20 Licensees will also need to provide a 21 description of plant administrative control that will 22 provide assurance that falls are captured and 23 investigated. And here are some lessons learned 24 regarding the licensing audits.
25 During the pandemic we relied on virtual NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
25 1 audits of undocketed material and living documents 2 like the VOP.
3 And this has proven to be very effective, 4 using open items during audits and providing questions 5 to the licensee in advance of virtual audit really 6 improve the effectiveness and use of the audit time.
7 We did find that in-person audits of the 8 vendor should still be performed during the licensee 9 review and this helps Staff and inspectors familiarize 10 themselves with the system and interfaces.
11 And the scope of the information to be 12 audited should include those vendor and licensee 13 documents developed during the licensing review.
14 If any audit information is needed to 15 support the Staff's draft safety evaluation, then the 16 Staff will ask the licensee to place that information 17 on the docket.
18 Now we'll talk about the integrated 19 licensing reviews, a digital modification encompasses 20 various technical review disciplines, and depending on 21 the application, the Staff responsible for these 22 branches may be involved in the licensing review, the 23 inspections, or both.
24 Each branch involved in the review will 25 use the criteria found in the SRP for the respective NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
26 1 review area, the guidance in ISG-06 is the digital I&C 2 portion of the review.
3 The ISG is structured in a way to 4 reference or interface with other guidance such as the 5 SRP, BTPs, and other disciplines such as human factors 6 engineering.
7 And now I'll turn it over to Charley 8 Peabody from the Reactor Systems Branch in NRR, who 9 will go over the reactor systems review of diversity 10 and defense in-depth. Charley?
11 MR. PEABODY: Thanks, Samir. Depending on 12 the application, like Samir said, you can get either 13 INC Branch reactor systems or human factors 14 engineering staff involved in the defense in-depth 15 portion of the review.
16 That involves a systematic approach used 17 to analyze a proposed digital I&C system for common 18 cause failures which can occur concurrently within a 19 redundant design.
20 Branch Technical Position 7-19 discusses 21 how you address those common-cause failures and 22 provides three potential paths to deal with them.
23 You can either eliminate the common-cause 24 failure from consideration, you can use diverse means 25 to mitigate the common-cause failure, or you can NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
27 1 determine that the consequences of that common-cause 2 failure may be acceptable and still be able to meet 3 the requirements of the accident analysis.
4 Some of the reviews may require re-5 analysis of Chapter 15 accidents. Next slide, please.
6 For reactor system reviews, the following 7 items are expected to be included in the D3 analysis, 8 identification and selections of transients and 9 accidents that are to be considered in combination 10 with a common-cause failure.
11 So, basically, that requires addressing 12 all Chapter 15 events, both anticipated occupational 13 occurrences as well as the postulated accidents.
14 Other critical events such as those initiated by 15 spurious actuation that are not analyzed in Chapter 15 16 may also have to be considered.
17 Description of what systems are lost 18 during a common-cause failure, either whether that's, 19 for example, in the past reactor trip system or 20 engineer safety features and then you identify and 21 describe the credited diverse equipment which can be 22 either existing or new equipment and doesn't 23 necessarily have to be safety-related if you can show 24 that it's of sufficient quality.
25 We'll also consider identification for NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
28 1 credited operator actions in lieu of systems that are 2 lost. And like I said, that factors into the 3 evaluation and analysis of each event.
4 Next slide, please. Actually, I see 5 somebody has a hand up?
6 MR. HERB: Yes, this is Ray Herb from 7 Southern Nuclear, I really had a question on the 8 previous presentation. It can wait until you're done 9 and when we get ready to go to break, then ask it.
10 But it was on Samir's presentation.
11 MR. PEABODY: We'll circle back to that.
12 MR. HERB: Thank you.
13 MR. PEABODY: Like I said, that gets into 14 the event characterization and analysis. So, when 15 you're determining what level of detail needs to be 16 provided, if the event where the CCF has no adverse 17 effect, those events require no reanalysis.
18 If the events are terminated by a diverse 19 system, you need to discuss what those diverse systems 20 are and describe any changes or a side-by-side 21 comparison of the diverse system actuation timing 22 versus the original based timing.
23 And depending on the results of that, a 24 new analysis may need to be performed if there's a 25 significant time difference between the base and the NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
29 1 new case. Next slide, please.
2 If the event is dominated by another 3 event, you can just refer to that, that for instance, 4 in the past, an inadvertent steam generator relief 5 valve opening may be bounded by a steam line break.
6 And other than identifying that it's 7 bounded by the other event, it doesn't require any 8 initial analysis, just the analysis of the bonding 9 event.
10 The events that the analysis is required 11 to demonstrate that acceptance criteria are met, these 12 are basically what were not eliminated by other 13 categories.
14 And they may be analyzed using either 15 best-estimate methods or conservative methods. And an 16 analysis must be done to demonstrate they meet the 17 acceptance criteria in Section B3.3 of Branch 18 Technical Position 719.
19 Next slide. The D3 analysis is considered 20 for all relative events and then we're going to 21 determine that they were categorized correctly, 22 whether it's any of the three outcomes that were 23 previously mentioned.
24 And we'll also verify that it has 25 demonstrated the consequences of common-cause failures NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
30 1 remain acceptable, so should the common-cause failure 2 occur, the facility will remain within the appropriate 3 acceptance criteria for the limiting events applicable 4 to the proposed digital I&C system or component.
5 Next slide. This is just an example of 6 some previous ones that we've looked at and what 7 methods are credited in each one.
8 I guess I'll either turn it back over to 9 Samir or I don't know if you want to go back and 10 address that question now, or whether you want to go 11 onto human factors?
12 MR. DARBALI: We'll go on with Human 13 Factors and we'll get your raised question when we 14 finish. I think we're close to the end. Thanks, 15 Charley.
16 MR. PEABODY: Thanks, Samir. I'll now 17 turn it over Brian Green from the human factors team 18 to give a high-level overview of human factors 19 engineering review of digital modifications.
20 MR. GREEN: Thanks, Samir, good morning, 21 everybody, can you hear me okay?
22 MR. PEABODY: Yes.
23 MR. GREEN: I'm going to just speak real 24 quickly here because I do have some time on the agenda 25 later where we'll go into these points in more detail NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
31 1 but just some observations we've had here related to 2 the human factors engineering, a point to start with 3 is ISG-06 points out the HFE review guidance area is 4 not within the scope of ISG-06.
5 Instead, rather that's handled by Chapter 6 18 and that's in concurrence with NUREG 0711 Rev 3, 7 the human factors engineering program review model.
8 One observation we've had is that some of 9 the recent modifications we've been discussing have 10 been more significant than what were considered at the 11 time that ISG-06 was developed and revised. So, we're 12 kind of adapting and working out ways to adapt to 13 that.
14 We have also notice there have been some 15 scheduling challenges with regards to the review of 16 the integrated system validation testing, it's one of 17 the key human factors points of contact and there are 18 some challenges as far as how and when to schedule 19 that in order to make the modifications to simulators 20 while still maintaining the ability to do other 21 functions in the simulator such as operator licensing 22 training and other activities.
23 To address that, the NRC Staff have been 24 considering some possible alternatives that may help 25 provide a suitable regulatory checkpoint that's a NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
32 1 little bit earlier in the process to break the log jam 2 at the back end there, so to speak.
3 One of the methodologies we're proposing 4 for that is to use a multi-stage validation test 5 program, which effectively spreads out validation 6 activities throughout the course of the design work 7 into different discrete steps.
8 And we believe we can, depending on how an 9 MSB program were designed, we could probably find 10 enough support for the design a little bit on the left 11 of the schedule when the ISV would happen, and that 12 may free up some scheduling opportunities there.
13 We've also been considering various 14 alternative test beds that would allow testing to 15 happen without necessarily modifying the plant 16 simulator. At the same point, one of the really 17 promising tools for that is the glass-topped 18 simulators.
19 But our discussion has not been limited to 20 just those, there are other methods that may also be 21 acceptable to use within an MSV program. And I 22 mentioned earlier that I'm going to discuss this in 23 detail later so I think that's all I've planned to 24 cover this morning.
25 Back to you, Samir.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
33 1 MR. DARBALI: Thank you, Brian. I'll now 2 talk about the secure development and operational 3 environment and the cybersecurity considerations.
4 Clause 5.9 Control of Access of IEEE 603 1991 is the 5 basis for SDOE reviews.
6 Section D8 of the ISG discusses the SDOE 7 reviews and refers to the guidance in Regulatory Guide 8 1152 Revision 3. And so the ISG also provides 9 guidance to NRR Staff to coordinate with NSIR Staff on 10 matters related to cybersecurity.
11 Now, the licensing review of a digital I&C 12 modification does not include a cybersecurity review 13 for compliance with 10 CFR 7354, which is the 14 cybersecurity rule.
15 The regions with NSIR's support perform 16 cybersecurity inspections of the digital INC 17 modification and the licensing review is independent 18 from the cyber inspections and the cybersecurity 19 inspections did not impact the licensing review 20 schedule.
21 And so NRR, NSIR, and Regional Staff 22 worked together to ensure adequate coverage and 23 understanding of the SDOE and cybersecurity aspects of 24 the modification and this includes security 25 requirements for controls to be implemented by the NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
34 1 vendor, supply chain requirements of the cybersecurity 2 plant, and security impact analysis of the 3 modification.
4 So, the combination of SDOE and 5 cybersecurity programming provisions address the 6 secure design, development, and operation of digital 7 I&C safety systems.
8 So, you can see in this table that there 9 is a difference when it comes to the focus of SDOE and 10 cybersecurity, the focus on SDOE is on safety, and 11 we're looking at aspects that are non-malicious acts 12 where cybersecurity is focused on prevention of 13 radiological sabotage and malicious acts.
14 Regulations are different for SDOE. It's 15 10 CFR Part 50, for cybersecurity 10 CFR 7354, and the 16 Regulatory Guides are different. Again, for SDOE is 17 Reg Guide 1152 Revision 3 and for cybersecurity it's 18 Reg Guide 571.
19 And almost all licensees use NEI 0809 20 Revision 6 as the basis for their cybersecurity 21 plants. And I'll now turn it over to Deanna Zhang 22 from the Vendor and Quality Inspection Branch to talk 23 about the VOP.
24 Deanna, are you there? You might be on 25 mute. Just give us a minute, folks.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
35 1 MR. JAIN: Is Greg going to do the 2 presentation?
3 MR. DARBALI: Greg was doing the 4 inspection portion of the vendor inspection 5 presentation. So, hopefully Deanna can join us and 6 I'll go over the VOP slides.
7 So, the Staff recognizes the need for 8 guidance for developing the VOP and VOP summary. This 9 is not just an industry comment or feedback, we also 10 understand these internally.
11 We do plan on developing this guidance 12 after the licensing actions for Turkey Point and 13 Limerick and use those lessons learned in the 14 development.
15 For the Waterford III review of the VOP, 16 the Staff evaluated the licensees' oversight 17 activities as described in the VOP summary.
18 And we did that again following the 19 criteria for Appendix B 10 CFR Part 50, Criterion 3, 20 inline control, Criterion 5, instruction and procedure 21 on drawing, Criterion 7, control of purchased 22 material, equipment, and services, and Criterion 16, 23 corrective action.
24 The VOP framework should supplement the 25 licensee's overall QA program descriptions with NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
36 1 specific system, hardware, and software development 2 activities, including a description of the proposed 3 development lifecycle, development documents to 4 produce, and management activities that will be 5 implemented in the design and the development of the 6 I&C safety-related systems.
7 The VOP and VOP summary should address how 8 the licensee's oversight activities will verify the 9 software development processes and the lifecycle 10 design outputs meet the software development process 11 descriptions that are summarized in the LAR or any 12 reference software program manual.
13 If the full VOP is not a lengthy document, 14 it may be beneficial for a licensee and the Staff if 15 they submitted the full VOP in the LAR instead of a 16 summary, that of course is up to the licensee.
17 Engineering procedures that are used to 18 implement the VOP should be described in the VOP and 19 VOP summary, including how they fit into the overall 20 QA program for the site.
21 The critical characteristics that will be 22 verified by VOP activities should reflect system and 23 architecture design that is specific to the 24 application being reviewed.
25 Identification of all documents that will NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
37 1 be reviewed and approved as an engineering design 2 document should be identified. And identification of 3 all lifecycle activities including V&V activities 4 should be described.
5 And the VOP and VOP summary should 6 describe the VOP changed controls. So, we need to 7 understand what is the process and the controls the 8 licensee has for changing the VOP after the license 9 amendment has been approved.
10 Regarding the VOP audits, they serve 11 several purposes. First, if the full VOP hasn't been 12 submitted in docket, then the VOP provides for the 13 Staff to review the full VOP and this can be done as 14 part of a virtual audit.
15 VOP audit also helps the Staff to verify 16 how the licensee is implementing the VOP and to 17 determine if there is reasonable assurance that the 18 licensee will implement the VOP after the license 19 amendment has been issued.
20 For Waterford, the VOP audits were 21 conducted throughout the log review timeframe. The 22 focus was on VOP activities described in the VOP 23 summary plus details of implementation provided in the 24 VOP itself to ensure consistency.
25 Like I mentioned, the pandemic required NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
38 1 these activities to be performed virtually.
2 A lesson learned or some feedback that we 3 got internally is the VOP audit should take place 4 after the audit of the vendor or if a vendor 5 inspection does take place during licensing, then 6 after the vendor inspection.
7 And this helps the Staff to focus on 8 specific items that will be audited. The licensee's 9 implementation of VOP activities may not occur 10 sequentially in accordance with the development 11 lifecycle.
12 This is something that should be explained 13 in the VOP. For example, oversight activities or the 14 implementation phase may take place after completion 15 and observance of the factor acceptance test phase.
16 One thing we also notice is that the 17 licensees' oversight audit reports may lack the actual 18 observed vendor activity by as much as a month or over 19 a month.
20 So, this creates a challenge to the Staff 21 reviewing the licensee's VOP audit reports as they are 22 seen as issues identified months in the past and may 23 not be aware of the resolution in a timely manner.
24 If the licensee's VOP audit report will be 25 issued after completion of the draft ANC, the Staff NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
39 1 may need to consider if an NRC audit of the vendor is 2 more practical to support the licensing review.
3 So, we've gone over the intent of the ARP, 4 we've gone over some of the deviations that we're 5 encountering, how the overlap of the licensing and 6 development schedules are shifting, and we've gone 7 over some of the lessons learned and some of the human 8 factors engineering and reactor systems' 9 radioactivities.
10 So, now how do we apply these lessons 11 learned and move forward to achieving a success path 12 for digital I&C licensing?
13 If an application comes in and follows the 14 tiered or ARP guidance according to the ISG, then 15 great, this is going to maximize the regulatory 16 certainty and the certainty of the review schedule.
17 But if an application doesn't follow the 18 tiered ARP guidance according to the ISG, then the 19 Staff and the licensee will need to consider what the 20 licensing review schedule is going to look like.
21 And what is a reasonable review time and 22 does the schedule support the requested installation 23 date?
24 What information needs to be submitted and 25 depending on the modification and the review schedule, NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
40 1 it could be a mix of the ARP Interior 1 and what's 2 going to be the availability of the information, what 3 is going to be submitted and when.
4 This way of moving forward provides for 5 some efficiencies and flexibilities that allow both 6 the Staff and licensees to navigate through licensing 7 process deviations.
8 Both the Staff and licensees need to be 9 flexible to allow for consideration of other aspects 10 of the application such as the level of detail in the 11 VOP, the use of regulatory commitments, and safety 12 significance of the modification.
13 It's likely that moving forward in this 14 manner can result in a license amendment being issued 15 before it would have been issued under the Tier 1 16 process. And when compared to the ARP, it's likely 17 that some inspection activities will be covered 18 instead through licensing audits.
19 Now, the goal for industry is to get the 20 digital modifications license approved in time to be 21 installed and to do so with regulatory certainty.
22 The goal for the Staff is to have all the 23 necessary information to make a safety determination 24 on the adequacy of the digital modification.
25 If the necessary information can be NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
41 1 provided in a timely manner to support the licensing 2 review and the license amendment is issued in time to 3 support the planned installation date, then I think we 4 can consider that to be a success.
5 But in order to do that, the Staff and 6 licensees have to be flexible and adapt to the 7 licensing process deviations and be accountable for 8 those changes.
9 We have to continue having productive 10 communications during pre-application and licensing 11 review meetings, maintaining openness on the scope of 12 the modifications and what the deviations are, and 13 having realistic expectations of what the licensing 14 review process is going to look like if the 15 application doesn't meet the guidance in the ISG.
16 Finally, we have to continue to leverage 17 lessons learned, the use of innovative tools like open 18 items or virtual audits, and licensing successes we've 19 had both using Tier 1 and ARP.
20 And with that, I think we're ready for the 21 questions so we'll start with you, Ray.
22 MR. HERB: This is Ray Herb from Southern 23 Nuclear and first of all, I want to say I appreciate 24 the NRC's willingness to be adaptable and flexible in 25 the licensing process, and I think this works out as NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
42 1 a great idea.
2 But I had a question back on Slide 11. I 3 wasn't sure if you were taking questions in the middle 4 of the presentation or if I had to wait until the end, 5 so that's why I raised my hand.
6 But here I seem to have heard that you say 7 those vendor oversight parameters and vendor 8 inspections and activities could not really start 9 until the draft SE was complete.
10 And my question was if some of those 11 activities in my VOP happened to happen before that 12 draft SE is complete, would the NRC still support 13 those? Would those move into a different phase?
14 Or would we be sitting around just waiting 15 to hold those activities before you guys are done?
16 MR. DARBALI: Now, those would be part of 17 the VOP audits and I'll jump back to the VOP audit 18 slide. So, like we said, there's a few purposes for 19 the VOP audits.
20 The first one is to review the full VOP 21 and then to see the implementation of those VOP 22 activities that take place during licensing.
23 And the idea of performing those 24 implementation audits during licensing review is for 25 us to get a better idea of, hey, the licensee is NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
43 1 implementing the VOP, they are performing their 2 inspections with being able to audit the licensee 3 oversight audit reports during the licensing review.
4 And that gives us a level of assurance 5 that, okay, we feel confident they're going to 6 maintain that same level of vendor oversight after 7 we've completed our draft safety evaluation.
8 MR. HERB: That makes me feel much better 9 because I was thinking that we wouldn't be able to 10 even start those activities on our end until after 11 that green box was complete, and the yellow box.
12 But they would still happen, they would 13 just be in a different space for you guys, and because 14 those activities may be planned out years in advance.
15 MR. DARBALI: Right, and to add to that, 16 the licensee's vendor oversight activities shouldn't 17 depend on when the Staff is reviewing them, whether 18 it's on licensing or it's on inspection.
19 That's your oversight, you control that, 20 what you're providing us is the vendor oversight 21 planned summary and a lot of the plan itself and the 22 implementation, what takes place during the licensing 23 review ideally becomes a licensing audit.
24 And what takes place after that would go 25 into inspection.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
44 1 MR. HERB: Thank you for clearing that up.
2 MR. DARBALI: Thank you for the question.
3 MR. JAIN: I see another raised hand.
4 MR. CAMPBELL: Good morning, Samir, this 5 is Alan Campbell. Back on Slide 6, it stated that the 6 ARP is intended to have no RAIs or requests for 7 additional information. This seems to be some new 8 information or a new expectation.
9 Even when I'm reading through ISG-06, it 10 acknowledges the RAIs as part of the ARP process. Can 11 you help provide a better understanding or a basis for 12 that?
13 MR. DARBALI: So, when we were developing 14 the ARP, again, we're looking at what was reviewed and 15 approved before under the TR process and the idea is 16 how can this be reviewed and approved in a shorter 17 timeframe, get that approval before completion of 18 factor acceptance testing?
19 And what was looked at was, well, we're 20 going to need an application that's complete. And the 21 time that was being requested was much less for the 22 Staff to perform their review.
23 So, we had these conversations with 24 industry during the development of the ISG. If we're 25 looking at adding RAIs, that's going to push the NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
45 1 review schedule and it's not going to be as early as 2 what was being requested.
3 Now, I'm curious to see which section of 4 the ISG, you say it's referring to RAIs for the ARP?
5 MR. CAMPBELL: Yes, it's Section C.3.2.2.
6 It's a fairly generic statement that just alludes to 7 an RAI process.
8 So, the NRC Staff should draft the safety 9 evaluation and issue RAIs for information or 10 clarifications necessary to finish the review of the 11 docketed material.
12 MR. DARBALI: So, we understood that it 13 might not be realistic to have an application that's 14 perfect but the goal for that LAR was that it has to 15 be as complete.
16 RAIs should be the exception, not the 17 expectation, in order for us to meet that schedule.
18 MR. WATERS: Hey, Samir, this is Mike 19 Waters, I do want to supplement. And I think, yes, 20 RAIs are part of the licensing process, no doubt about 21 it, but our goal at NRC is never to have RAIs, right, 22 but that does happen.
23 I think the point being made is based on 24 the history of previous digital I&C modifications, 25 there's a mentality and thought process on both sides NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
46 1 that multiple-phase submittal, NRC details what we 2 need later on through an RAI process.
3 We'll plan to have an RAI, we'll plan to 4 have a supplement to address these gaps that we know 5 exist today. So, the more fundamental point for ARP 6 is no, the idea is to submit everything up front.
7 We've aligned clear expectations of what's 8 needed up front.
9 There are no plans to supplement later 10 through an RAI process for example, and that was the 11 paradigm we had before the ARP and I think what we're 12 trying to communicate is the ARP paradigm now is the 13 idea at the time was to submit everything at once.
14 We're not playing at supplements later, 15 we're not playing to purposely ask RAIs because we'll 16 know a supplemental meeting later. We know what we 17 need up front and I think that's what was meant here.
18 MR. DARBALI: Right, the flow chart for 19 the alternative process does incorporate RAIs. I 20 think the intention was to -- again, this is guidance 21 for the reviewers to understand.
22 You always have the option to ask RAIs but 23 I think Mike explained it well.
24 MR. CAMPBELL: I think just my 25 understanding here is through the presubmittal phases NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
47 1 and through the communications up front, we want to 2 aim for the best and highest quality submittal 3 possible.
4 We just wanted to make sure that from the 5 expectation point of having a perfect submittal every 6 time, that's a really high bar to achieve but our 7 intent is to minimize the RAIs as best we can to keep 8 within the review windows that we're aiming for.
9 MR. DARBALI: I see Rich and then Ted?
10 MR. STATTEL: They key point here is the 11 supplemental information and I also want to explain 12 that when we first developed ISG-06 in the previous 13 revision, the NRC and us worked together and we 14 recognize that we're performing these evaluations in 15 parallel with the systems being developed so it's a 16 moving target.
17 So, back in 2007, 2008 timeframe, industry 18 asked us to have this two-phase submittal process to 19 accommodate that. And in a sense, we're evaluating a 20 moving target because the system is being designed and 21 evaluated at the same time.
22 So, that was the original premise of ISG-23 06. What happened with the revision was when we 24 created the alternate review process industry also 25 asked us to provide this early issuance of the license NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
48 1 amendment.
2 And that basically threw the whole process 3 out of whack because in order to do that, in order to 4 accomplish that, the early issuance, prior to testing 5 of the system, we really needed to have a once-and-6 for-all submittal right up front.
7 And we could no longer adopt the Phase 2 8 submittal that was later in the process, we just 9 couldn't do that. So, that's how that evolved to 10 where it is and that's certainly being challenged with 11 the applications we're receiving today.
12 MR. DARBALI: Thanks, Rich. Ted?
13 MR. QUINN: Samir, can you hear me okay?
14 Good morning, and thank you, NRC, for doing this and 15 providing this workshop. It follows up on the 16 February one that I thought was so well done.
17 It's Ted Quinn from Paragon. My question 18 relates really to the Vendor Inspection Branch. And 19 there's a second question on eRoom that may have to be 20 done at a later time today.
21 On Vendor Inspection Branch and the VOP at 22 Slide 27 and 28, that series that you presented, 23 Samir, I wanted to talk about the division of 24 responsibility between NRR and the Vendor Inspection 25 Branch.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
49 1 I think it's a great addition to the Staff 2 on this new process to us relative from after Diablo 3 Vendor Inspection Branch but the ideas on, say, Slide 4 28, do you have lessons learned on the division of 5 responsibility between vendor inspection and NRR in, 6 for example, software development activities?
7 Did you during, say, Waterford or previous 8 license reviews address that the Vendor Inspection 9 Branch focused more on procedural guidance and that 10 the NRR focused more on evidence, on did that 11 incorporation -- what did you see of this overlap, how 12 it was handled?
13 MR. GALLETTI: Samir, this is Greg, I'll 14 try to respond to that. This is Greg Galletti with 15 the Nuclear Regulatory Regulation. I'm in the Quality 16 Assurance Vendor Inspection Group and was responsible 17 for leading some of these inspections.
18 Ted, to answer the question, what 19 typically happens when we engage a vendor on a vendor 20 inspection, we're going to have a combination of 21 quality assurance personnel staff from the NRC as well 22 as some of the technical staff that we're responsible 23 for doing the licensing reviews.
24 So, those individuals would have an 25 in-depth knowledge of the systems being modified or NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
50 1 changed and the processes being used, the technical 2 requirements for those systems.
3 So, when we would go out to do the vendor 4 inspection, it would be a combination of the two 5 things that you brought up.
6 One is we would be looking at the vendors, 7 Appendix B quality assurance program implementation, 8 looking at the procedures and implementation of those 9 procedures and focusing more primarily on those 10 important system requirements that the technical staff 11 has identified and evaluated as part of the licensing 12 review.
13 So, that's typically how we would go about 14 doing these. Does that answer the question, Ted?
15 MR. QUINN: It does, I guess you would sit 16 side by side with NRR and as they do their reviews of, 17 say, the different documents that are being developed 18 in the software lifecycle, for example, then you 19 perform your review and they perform their reviews as 20 well?
21 MR. GALLETTI: Yes, and it's a 22 collaborative effort, quite frankly. We, the vendor 23 group, are in NRR also. So, they are a sister branch 24 --
25 MR. QUINN: Sorry, you're right.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
51 1 MR. GALLETTI: No problem, but as we go 2 through the licensing review, we and the quality 3 assurance staff also have people involved in the 4 licensing amendment review.
5 And so we are familiar with the VOP, the 6 VOP summary, the application in general and we work 7 together with the technical staff to develop RAIs and 8 open issues, open questions and we support and are 9 involved in the auditing that goes on before the 10 safety evaluation is drafted.
11 So, there's a very, very strong 12 collaborative effort.
13 MR. QUINN: Thanks, what I should have 14 said is the I&C branch and I like what the VOP is 15 doing. So, let me go on, Samir, to the eRoom. Where 16 today are the potential benefits of eRoom?
17 It was discussed well in the February 18 workshop so is that on the agenda today?
19 MR. DARBALI: What do you mean by eRoom, 20 Ted?
21 MR. QUINN: So, the eRoom is the post-LAR 22 acceptance and the start of review process, and the 23 integrated actions we did, say for example, during 24 Diablo where there was the exchange of information 25 between the utility and the NRC on the many documents NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
52 1 that were available for the review cycle.
2 It was very detailed and there's many 3 documents on the order of 40 or 50 documents. And 4 there was an eRoom discussion that did occur during 5 the workshop in February and I'm wondering if that's 6 on the agenda or is that a separate item?
7 MR. DARBALI: I'm understanding you're 8 talking about the acceptance review that we do once we 9 receive the LAR.
10 MR. QUINN: No, it's beyond that.
11 So, the issue is there are docketed and 12 undocketed items and the issue part of ISG-06 Rev 2 13 was the streamline the process so that it would be 14 clear on the availability of information to NRC and 15 the three potential docketed audits and inspections.
16 And the purpose of an eRoom was to help 17 optimize that so that the NRC would request specific 18 docketed information. But it would be done in a 19 tailored manner to the information you really need in 20 order to write the SER.
21 So, the eRoom was a clear access of 22 information to the Staff in all of these many, many 23 documents that are developed as part of a hardware or 24 software lifecycle. Is that clear? Is that not 25 clear?
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
53 1 MR. DARBALI: So, you're talking about the 2 virtual portal?
3 MR. QUINN: Yes.
4 MR. DARBALI: We don't have digital 5 information on this presentation from what we provided 6 in February of last year but for Waterford, we used 7 that virtual portal throughout the licensing review.
8 So, we used Certrec and what we did is we 9 developed an open items list similar to what we have 10 done for the Diablo Canyon review and we developed and 11 tracked those open items.
12 And those open items were carried into 13 Certrec.
14 And we were able to look at those living 15 documents that were being developed and then we would 16 perform virtual audits to determine if this is 17 something that we need to have docketed or if it just 18 becomes part of the audit scope.
19 But yes, the eRoom or the virtual portal 20 has been tremendously useful for the Staff and 21 licensees in reduction of docketed material and just 22 speeding up the process of getting that information.
23 MR. QUINN: That was tremendous, thanks.
24 MR. WATERS: This is Mike again, I just 25 want to add on just for everyone's awareness, that NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
54 1 open IM process and portal that was formally done in 2 our licensing auto process.
3 We had a long process and procedures and 4 we had I believe a consumer-defined audit plan that 5 defined that process. So, it's transparent and open 6 to everybody what we're doing here.
7 Of course, a lot of it is proprietary but 8 that was technically under the licensing audit process 9 and how we defined that.
10 It did take a lot of effort for us to get 11 started, I'm looking forward to Entergy's comments on 12 that because I know the first few months was hard to 13 get the fireworks on.
14 But once it all got going, it was 15 extremely beneficial, Samir.
16 MR. QUINN: To us on both sides as well.
17 I'm really appreciative for both of your discussions 18 here. Thanks.
19 MR. DARBALI: And Ted, going back to your 20 question on the VOP activities and interaction between 21 licensing and inspection, in the slides for last 22 year's workshop, I believe there is a table that 23 identifies the VOP activities during the licensing 24 review and that falls into inspection.
25 MR. QUINN: Thank you.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
55 1 MR. DARBALI: I see Ron Jarrett, go ahead, 2 Ron.
3 MR. JARRETT: Can you hear me?
4 MR. DARBALI: Yes.
5 MR. JARRETT: There was a trigger word in 6 one of your slides on VOP, the word all. I don't know 7 what slide number it's on but it said something about 8 listing all the engineering and approved documents.
9 Could you I guess be more specific on what 10 all means?
11 MR. DARBALI: We'll ask Greg if he can 12 take it.
13 MR. GALLETTI: Sure, this is Greg again.
14 I guess all is probably not the proper term to be used 15 there but the intent of that comment is in the VOP 16 summary itself it talks about various vendor oversight 17 activities of the licensee.
18 And what we have learned from the 19 experiences is that the implementing procedures that 20 you're using for various activities and typically, 21 these are your 10 CFR Appendix B quality assurance 22 program criteria implementing procedures are needed by 23 us to understand how you're going about doing these 24 oversight activities.
25 So, what we saw in the past is the VOP NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
56 1 summary didn't have that level of detail in it, and 2 then when we looked at the VOP itself there was some 3 additional detail.
4 But again, it wasn't a complete picture of 5 the various activities that you were performing, the 6 criteria of Appendix B that you were using and 7 leveraging to perform those activities, and the 8 governing procedures to do those activities.
9 So, that's really the intent of that 10 bullet.
11 MR. JARRETT: Thank you for the 12 clarification.
13 MR. DARBALI: Thanks for the question.
14 Any other questions?
15 MR. JAIN: Any more questions on Samir's 16 presentation this morning? We've got a question from 17 Waterford.
18 MR. ODESS-GILLETT: For the Waterford 19 license submittal, they submitted a draft LAR before 20 the actual LAR. Did you find that improve the 21 licensing review process?
22 MR. JAIN: I'm going to let Mike Waters 23 take that one.
24 MR. WATERS: Thanks, Samir. We tried to 25 use a different approach.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
57 1 I think it was helpful, in one aspect I 2 think it gave us an early jumpstart of the VOP and I 3 think there's been a lot of effort on that because 4 Entergy is the first one to develop the VOP and 5 summary.
6 And I think it jumpstarted on asking 7 questions there. It is a struggle because the review 8 of that draft was a very short period of time and it 9 was always any significant red flags.
10 But as a regulator, it's not part of the 11 formal review process and we didn't communicate any 12 findings to that degree. It did give us a jumpstart 13 in looking at it but we are rethinking does that make 14 sense from a regulatory standpoint, from that 15 standpoint?
16 I look forward to hearing your comments 17 and Entergy's comments on that as well. It's a lot of 18 effort for a licensee to put together a draft and then 19 have to resubmit it a month later. I think that's 20 what Entergy said.
21 We have to balance regulatory process 22 uncertainty with the benefit of something that may be 23 provided for us.
24 MR. JAIN: Are there any more questions to 25 Samir on the presentation? Hearing none, Samir, NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
58 1 should we continue because our break is almost a half-2 hour away?
3 Maybe you want to continue with your 4 presentation on inspection lessons learned and then 5 we'll break in the middle?
6 MR. DARBALI: I think maybe we should take 7 the break first because we do have a series of 8 presentations and different presenters who might not 9 be ready until a bit later.
10 MR. JAIN: Okay, that'll be fine so we'll 11 break for 10 minutes and this is 10:22 a.m. so we'll 12 restart at 10:30 a.m., everybody. So, a 10-minute 13 break. Thank you.
14 (Whereupon, the above-entitled matter went 15 off the record at 10:22 a.m. and resumed at 10:32 16 a.m.)
17 Hello, everyone, now we will start the 18 Staff's presentation on inspections lessons learned 19 and Greg will lead the presentation.
20 MR. GALLETTI: Good morning, everybody, 21 again, this is Greg Galletti from the Office of 22 Nuclear Reactor Regulation. I'm one of the inspection 23 team leaders in the Quality Assurance Vendor 24 Inspection Branch.
25 And I was responsible for two of the NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
59 1 inspections that we've performed to support the 2 Waterford digital modifications. Next slide, please.
3 So, I just wanted to give a quick 4 background. As we discussed back in February, the 5 intent was for the NRC's inspection framework to 6 really be focused in two areas, the first being the 7 vendors' development activities and design outputs.
8 Again, the inspection framework was really 9 twofold, the first area was looking at the vendors 10 programs for the development and the activities and 11 support of the design itself.
12 And the second area for inspection is 13 looking at the licensee's implementation of their 14 vendor oversight plan activities and that, as you'll 15 see, will be covered in a later presentation by our 16 regional representative.
17 As I mentioned, we had an opportunity to 18 implement the inspection framework for the Waterford 19 Digital I&C modification project and I'd like to go 20 over -- sorry, Joe, that you can't hear me -- some of 21 the lessons learned from the inspections as well as 22 originally we were going to go over the lessons 23 learned here for the audits but you've heard that 24 earlier from Samir.
25 Next slide, please.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
60 1 So, we conducted two inspections at the 2 vendor's facility, one was actually a virtual audit, 3 unfortunately, due to the pandemic but that occurred 4 back in March of 2021.
5 It was a virtual audit and we focused on 6 the requirements phase of the development process that 7 the vendor had been implementing.
8 We issued an inspection report with no 9 findings on May 10, 2021 and I provided the ADAMS 10 accession number for you. Just to give you a little 11 bit of background, for both of these inspections we 12 used our routine vendor inspection procedures that we 13 have in place.
14 The first procedure is 43002, which as 15 titled is a typical routine vendor inspection 16 procedure that we use that covers the applicable 17 portions of the 10 CFR Part 50 Appendix B 18 requirements.
19 And the second more focused inspection 20 procedure is IP 35710, which again, focuses more on 21 the software development lifecycle and the quality 22 assurance application to the development of that 23 software.
24 In concert, we used both of these 25 procedures typically whenever we do a digital I&C or NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
61 1 software-related vendor inspection. So, further 2 requirements phase inspection, we focused primarily on 3 looking at both the software and hardware requirements 4 phase lifecycle development documentation.
5 We verified adequate translation of the 6 design information provided in the LAR into the 7 hardware and software requirement specifications, we 8 looked at and we evaluated the traceability controls 9 that the vendor had in place to ensure the 10 requirements were incorporated directly into those 11 specifications and carried through to later phases of 12 the development lifecycle.
13 We looked at their configuration 14 management and design control processes specifically 15 to address these areas and we also looked at the 16 independent verification and validation processes that 17 the vendor had in place to evaluate the outcomes of 18 the various phases, and in this case the requirements 19 phase.
20 And finally, we focused on the non-21 conformance reporting system as well as the corrective 22 action program reporting system that the vendor used 23 during the development lifecycle phase in order to 24 capture any non-conformances or discrepancies, 25 evaluate those, and disposition those.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
62 1 The second inspection that we performed 2 was the factory acceptance testing phase that was done 3 in July of 2021. Again, we issued the report about a 4 month later, September 10th, the ADAMS accession 5 number is provided there.
6 Again, as a result of the inspection, we 7 did not have any significant findings. Again, for 8 this particular inspection we focused on and reviewed 9 documentation associated with both the design and the 10 implementation phases as well as the factory 11 acceptance testing phases.
12 So, as part of that, we looked at design 13 documentation and verification reports associated with 14 design and implementation, we looked at the inspection 15 and testing procedures that were being performed.
16 And in this case, we had an opportunity to 17 physically be on site to do the inspections so we got 18 to look at the test facility, the system as developed 19 for the testing, and then observe certain aspects of 20 the testing that were going on during the week that we 21 were on site.
22 Next slide, please. What did we learn 23 from all of these things? I'll characterize these 24 five or six bullets in three simple terms, 25 coordination, communication, and scheduling.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
63 1 And those three areas were the ones that 2 were the most challenging for these particular types 3 of inspections and things that as our opportunity to 4 do these sorts of inspections continues, processing 5 will be more mature and we'll get better results.
6 So, if we go through some of these, 7 certainly depending on what type of process you're in, 8 whether you're using the alternative review process or 9 a traditional Tier 1 process, those will dictate when 10 and what types of information are going to be 11 available to the inspection staff to conduct those 12 inspections.
13 And that'll be based on the maturity of 14 the design at the time of the submittals. Secondly, 15 the I&C systems as we've seen, the development 16 activities don't necessarily follow the traditional 17 Waterford lifecycle phases.
18 What we had noted when we did these 19 inspections is that certainly, certain design changes 20 may occur during the lifecycle that would affect the 21 design and therefore, require some additional 22 regression analysis that may not actually occur until 23 after our inspection was completed.
24 Additionally, during the inspections of 25 the testing, whether it be factory acceptance testing NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
64 1 or at the site level, site acceptance testing, those 2 may trigger some additional design changes that again 3 would occur after the inspections had been completed.
4 Regression would have to be done by the 5 licensee and the vendor and then subsequent evaluation 6 may need to be evaluated by the NRC at that point.
7 What we also noted was that the 8 expectation when we went out and we were looking at 9 the fact testing, our expectation going in there was 10 that the implementation and design phase activities 11 would have been complete at that point in time and 12 then we would have been looking at the result reports 13 in concert with the FAT inspection that we were doing.
14 In fact, what we found out is that there 15 were certain design artifacts associated with both of 16 those lifecycle phases that were not completed at the 17 time we were on site for the FAT testing and so again, 18 that made the inspection somewhat limited and we had 19 to evaluate what to do in those cases and move that 20 over towards the system level reviews that were done.
21 The scheduling, this comes down to that 22 the scheduling really needs to be somewhat dynamic.
23 As you're approaching the design, the 24 vendor is implementing the various lifecycle phases 25 and we have to have formal communications between NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
65 1 ourselves, the vendors, and the licensees so that we 2 understand what's going on with the development 3 lifecycle, the dynamic aspects of that, and any 4 potential changes or delays or accelerations to that 5 lifecycle activity.
6 The frequency of the vendor inspections 7 also, as we pointed out back in February, will 8 probably be dependent on both the complexity of the 9 digital I&C system under development as well as 10 potentially our expertise in understanding experiences 11 with the vendors and the licensees that may have 12 already gone through digital modification upgrades and 13 we have some familiarity with their programs.
14 As I pointed out, probably the most 15 significant thing that came out of the inspections and 16 certainly the FAT inspection is that there needs to be 17 really strong coordination between the vendor 18 inspection group, the licensees themselves, and the 19 vendors in order to ensure that adequate coverage can 20 be accommodated at the testing facility.
21 In the case of Waterford, we've had the 22 vendor inspection team, we've had the licensee 23 oversight team. Obviously, we had the vendors testing 24 team and we also had support from our regional 25 inspection team.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
66 1 So, there's a lot of moving parts to these 2 inspections and coordination is really extremely 3 important to ensure that we have effective and 4 efficient inspections.
5 And communications, it's clear that as 6 early on in the development lifecycle that the earlier 7 we can open up the channels of communication between 8 the licensees, the vendors, and ourselves in terms of 9 developing schedules for doing inspections, 10 establishing the resource allocation that we need 11 internally to support the inspections as well as the 12 vendors staff in terms of being able to support those 13 types of inspections is really critical.
14 I think that pretty well sums it up in 15 about 10 minutes or less. So, I'll open it up for 16 questions.
17 Was there a hand or did it go away?
18 MS. GOLUB: There was a hand, it was me, 19 it's Pareez Golub. First, thank you, Greg and the NRC 20 of course in general, for doing this workshop, we 21 really appreciate the opportunity to hear the lessons 22 learned.
23 Greg, on your Slide 4, I didn't quite 24 understand when you were talking about something that 25 was not complete at the time of FAT.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
67 1 MR. GALLETTI: Yes, when we were out at 2 the facility looking at the FAT testing that was in 3 progress, one of the areas that we wanted to focus on 4 was to look backwards as part of the lifecycle 5 development activities.
6 And I should preface this by saying when 7 we did the FAT inspection we also had support of the 8 quality assurance vendor group, we had support of the 9 technical staff that's responsible for the LAR review 10 and approval, and we had a member or members of the 11 regional staff that were performing vendor oversight 12 inspection activities in parallel.
13 So, while we were out there looking at 14 some of the activities associated with design and 15 implementation, certain of the oversight activity 16 evaluations that were done by the licensee and IV&V-17 related activities done by the vendor were not 18 completed at the time we were there on site.
19 And so we couldn't obviously evaluate 20 those final reports associated with those development 21 lifecycles.
22 MS. GOLUB: Thank you.
23 MR. JAIN: Any more questions for Greg?
24 MR. GALLETTI: Ted?
25 MR. QUINN: Thank you, it's Ted Quinn. I NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
68 1 wanted to ask and again, I'm going to focus in on 2 docketed and non-docketed, and it has to do with your 3 request.
4 When you go through the procedure 5 lifecycle at a vendor and you have a set of 6 procedures, are you able to address that in the 7 inspection process or do you have to docket those?
8 I didn't know what lessons learned you've 9 had from past work.
10 MR. GALLETTI: I should say in this case 11 the vendors' implementing procedures that implement 12 their Appendix B program typically are not at a level 13 of information that we need docketed.
14 So, we will review those implementing 15 procedures typically during the inspection itself, 16 looking at whether those implementing procedures carry 17 out the requirements that are documented in their 18 quality assurance program manual, and then looking at 19 the implementation of those procedures to ensure that 20 they were implemented properly and the results of that 21 implementation were documented properly.
22 And the fact that we can review that 23 documentation to see what the outcomes of those 24 activities were.
25 MR. QUINN: Perfect, end of comment.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
69 1 MR. JAIN: Samir, do you have questions?
2 MR. DARBALI: Yes, I just need to 3 reiterate something that Greg had mentioned earlier, 4 the fact that Greg and Deanna were also part of the 5 licensing review team, I think that made the 6 transition into performing those inspections much, 7 much easier because they were already familiarized 8 with that modification and the contents of the LAR.
9 So, I think that's something we need to 10 continue to do in the future.
11 MR. GALLETTI: Samir, I fully agree.
12 I think the continuity between 13 understanding the design and looking at the VOP and 14 the VOP summary and following that through to the 15 completion of the LAR documentation, and then carrying 16 that beyond into the inspection is important, 17 absolutely.
18 MR. JAIN: Any more questions for Greg?
19 If not, then we can move to regional inspection 20 lessons learned.
21 MR. DARBALI: Thanks, Greg, so Shiattin 22 will take over.
23 MS. MAKOR: Yes, I'm going to take over.
24 I'm going to switch it to mine, give me one second.
25 Is it showing now?
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
70 1 MR. DARBALI: Yes.
2 MS. MAKOR: Good morning, everyone, my 3 name is Shiattin Makor, and I'm the team lead for the 4 regional inspection activities associated with the 5 Waterford III core protection calculator system, 6 better known as CPCS and, what I'll be referring to it 7 as throughout the rest of this presentation, digital 8 upgrade.
9 And I will be presenting our lessons 10 learned regional processes and the application of the 11 inspection procedure.
12 As I just mentioned, Waterford III is 13 implementing a digital I&C modification to upgrade 14 their CPC and their CEAC system using the previously 15 approved common-queue platform, This Spring.
16 Because Waterford III volunteered to be 17 the first site to you the alternate review process, 18 which relies heavily on vendor oversight and is 19 inspected using the revised inspection procedure 20 52003.
21 Region IV of the NRC Office has the 22 privilege of being the first to implement the revised 23 inspection procedure. To organize our assessment we 24 used key inspection milestones marked by the 25 completion of major deliverables.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
71 1 So, for this presentation I will cover how 2 we implemented the new inspection procedure, how we 3 inspected the alternate review process, and the 4 lessons that we've learned so far.
5 The first area that I would like to 6 discuss in the regional inspection process that we 7 chose to employ is how we decided to go about this 8 inspection. So, what we did is we divided the 9 regional project into these manageable steps to easily 10 control the inspection project.
11 We wanted to make sure we were able to 12 control the quality of the output and identify major 13 deliverables. So, the initial phase, which you see 14 blocked in the red was more of a familiarity-type 15 activity for us.
16 This is where in the licensing review 17 process we had to obtain regional administrator 18 approval.
19 And so for this part, during this phase, 20 NRR evaluated the proposed license amendment by 21 reviewing the design and capabilities of the 22 modification as part of the normal review process.
23 The region during this phase reviewed 24 specific documentation to gain familiarity with the 25 system which made it easier downstream to perform NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
72 1 documentation and functionality review once the system 2 left the vendor.
3 And it allowed us to be familiar with the 4 licensee's administrative programs for designing, 5 installing, testing, and maintaining modifications.
6 So, for the license review process, we would sit in on 7 the meetings, we weren't active participants, we were 8 very passive in it.
9 We listened in, we saw the type of 10 questions being asked with the request for additional 11 information. We saw the discussion for safety 12 evaluation and pretty much just shadowed the approval 13 of the license amendment.
14 For the regional administrator, we had to 15 obtain approval because this is, especially, an 16 infrequently performed activity and so this required 17 approval. So, for us to be able to get approval we 18 had to be able to brief management and stakeholders at 19 each phase and to get approval to do the project as a 20 whole.
21 So, in order to do that, we had to be 22 familiar with what the modification was. The next 23 phase of the regional process is the planning and 24 scoping essentially.
25 So, for this phase, the team used the NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
73 1 inspection procedure requirements to identify what 2 documents were necessary to perform the inspection.
3 We use the license amendment review in the 4 inspection procedure to be able to determine our 5 inspection scope, and due to the dynamic nature of 6 these inspections, the team had to be flexible.
7 So, pinpointing dates, team composition, 8 and inspection in some cases was a moving target. Our 9 request for information was heavily based upon the 10 inspection procedure requirements.
11 And in our inspection planning and scoping 12 we identified our team composition, we used objectives 13 in the inspection procedure, scheduling we obtained 14 from the licensee for their master scheduling to see 15 when major activities were occurring.
16 And then specific activities and 17 assignments, those were based upon our team 18 composition, which as was mentioned before, we want to 19 maintain some of the continuity so our team 20 composition included a lot of the individuals who were 21 involved at the beginning.
22 So, the vendor inspections, the technical 23 reviews, our team had team members from that, so they 24 had an idea and understanding of what was already 25 going on.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
74 1 The next part of the process is the 2 implementation phase and what this essentially is is 3 the licensee was doing a factory acceptance testing, 4 the site acceptance testing, and the CPC modification 5 installation.
6 So, for this phase, the inspections are 7 all conducted by inspectors knowledgeable in the areas 8 of digital I&C and operations.
9 But because these inspections occur over 10 several weeks, actually months, the decision was made 11 to perform the inspection in phases which allowed the 12 team to keep track of what was assess and afforded a 13 level of transparency.
14 So, for our factory acceptance testing, 15 that was performed in July of 2021, and we were there 16 at the same time, as mentioned before, as the vendor 17 inspection. And so we were providing oversight of the 18 licensee, who was providing oversight of the vendors.
19 So, it was a lot of watching one another.
20 But there was value added in that phase because it 21 allowed us to see the equipment at the vendor facility 22 and started to get familiar with the process, the 23 testing, everything that we would need to address it.
24 If we would have waited until the very 25 final activity, which is installation, we wouldn't NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
75 1 have some of the lessons learned and the awareness 2 that we do have.
3 In January of 2022, we performed the site 4 acceptance test activities and that was performed at 5 the facility. So, that was performed at Waterford and 6 then in May of 2022 we are preparing right now to 7 perform the installation inspection activities.
8 At this point, I want to pull up the 9 inspection procedure application and the two major 10 updates that affect and apply to regional inspections 11 particularly. So, Inspection Procedure 52003, digital 12 instrumentation and control modification inspection, 13 was issued July 1, 2021.
14 There are two major updates in here that 15 particularly affected us.
16 One of them was the revision allowed for 17 the confirmation of the digital I&C and software 18 development, integration, and testing to be stripped 19 from licensing activity to inspection activities as a 20 result of the ultimate review process.
21 So, under the new guidance, the licensee 22 has an option to use the alternate process or 23 traditional approach. Under the traditional approach, 24 the license amendment review would be completed after 25 the completion of a system design.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
76 1 Under ARP, the license amendment approval 2 relies heavily on vendor oversight and allows approval 3 prior to the completion of the design. This update 4 shifted some activities that fell under the licensing 5 activities to inspection.
6 So, in the past, digital mods such as 7 Oconee, Diablo Canyon, the FAT did not fall under 8 regional purview but this time it did.
9 Because of the reliance on the vendor 10 oversight plan, when using the ARP it's important to 11 note that when executed the VOP can be used to ensure 12 the vendor, one, executes the project consistent with 13 the LAR and, two, uses an adequate software or quality 14 assurance program.
15 The VOP when executed helps ensure the 16 vendor will meet both the process and the technical 17 regulatory requirements. But one thing to note is 18 that execution of the VOP is the responsibility of the 19 licensee.
20 So, this includes performance of oversight 21 activities on the design artifacts and development 22 activities for the requirements, design, 23 implementation, integration, and factory acceptance 24 test phases of the system.
25 So, as I mentioned before, this was NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
77 1 something that was different in my previous 2 experience.
3 I was usually more of an observer during 4 the factory acceptance testing for the other two 5 facilities that I mentioned whereas this time around, 6 our team, we were active participants and it was an 7 actual team.
8 Usually, it was just the team lead that 9 participated in the FAT but now you'll have a full 10 complement that's actually providing oversight. Those 11 are the two major -- I did leave out the second one, 12 I apologize.
13 The second major change to the inspection 14 procedure 52003 was the formal inclusion of the 15 verification of commitments in the licensee's 16 cybersecurity plan.
17 So, I'm going to talk about that a little 18 later after I talk about the applications and I do 19 have my cybersecurity team member who will also add to 20 that, because we did feel there was some value in 21 talking about that a little bit more since it is one 22 of the major updates.
23 When it comes to the application of 24 inspection procedures 52003, it's divided into five 25 major areas that are listed below. Here I just NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
78 1 included major or key takeaways for each of the 2 sections.
3 For the inspection objectives, the 4 expectation is that the licensee has performed and 5 will verify that the licensee has performed enhanced 6 vendor oversight activities as defined by the vendor 7 oversight plan.
8 So, as I mentioned before, the vendor 9 oversight plan is very significant and very important.
10 That's something that we're looking at to make sure 11 that it's being applied and performed correctly.
12 The next part that we wanted to point out 13 is the inspection requirements, 52003 Section 2, all 14 requirements are not needed to be completed.
15 The team lead has the autonomy to look at 16 the applicability of items that are in the inspection 17 procedure and then if we're unable to do it, then they 18 don't have to be completed.
19 So, that section, Section 2, was pretty 20 extensive but some of the activities it may not apply 21 to the site.
22 So, that depends on the particular 23 inspection. For the Waterford inspection, what we did 24 is if it didn't apply, we note that so that there's no 25 wondering or no confusion later on.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
79 1 That was an important aspect for us. For 2 inspection guidance, I did say see the reference page.
3 I have a reference page at the end of my presentation 4 and some resources that will include the ML extensions 5 for all of the FAT and SAT and all of the documents 6 with ML numbers so that you can go back and look at 7 that.
8 But the main thing that we wanted to make 9 sure was emphasized when you look at the inspection 10 guidance section is that this is guidance for 11 inspectors. So, the reason why we wanted to emphasize 12 this is because we were in a training class for DGE 13 and there was an impression that this was the 14 requirement.
15 Section 3 is not the requirements, the 16 requirements are found in Section 2. Section 3 is 17 just guidance for inspectors, it includes references 18 and things that you can look at and use to inspect the 19 requirements in Section 2.
20 The next section in Inspection Procedure 21 52003 is in regard to resource estimates. So, the 22 resource estimates is 120 hours0.00139 days <br />0.0333 hours <br />1.984127e-4 weeks <br />4.566e-5 months <br /> onsite activities and 23 if you're doing the ARP, then it's an additional 120 24 hours2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br />.
25 So, 240 hours0.00278 days <br />0.0667 hours <br />3.968254e-4 weeks <br />9.132e-5 months <br /> total if it's using the ARP NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
80 1 process. I'm going to discuss this later in lessons 2 learned so I won't go or delve into it right now but 3 I did want to point out that information.
4 And then the last section discusses 5 procedure completion. And so it's conducted to 6 demonstrate the modification is implemented in a safe 7 manner.
8 So, satisfactory review of documentation 9 verification, testing, operations and training, and 10 plans for maintenance and repair are in accordance 11 with the inspection plan, and that will constitute 12 completion of this inspection procedure.
13 There's two things that I did want to 14 point out, one is that it may be completed over 15 similar inspections so there may be concern or 16 wondering why do we do it over several inspections.
17 And the inspection procedure does allow 18 for this. The other thing that I wanted to point out 19 which I'll talk about later is how do we call the 20 inspection or procedure complete?
21 How do we call it good? And so for this, 22 we have looked at using the major activities such as 23 the FAT, SAT, and installation as our methodology for 24 saying, okay, this portion is completed and moving on 25 from there.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
81 1 As I mentioned before, formally including 2 cybersecurity requirements in Inspection Procedure 3 52003 was another major update that did affect the 4 regional inspection team's inspection scoping, 5 planning, and composition.
6 Additionally, there are sections of the 7 cybersecurity plan that are particularly applicable to 8 the digital equipment upgrades and things the 9 inspection team saw during the FAT and the SAT.
10 So, my cyber team member, Kim, will 11 specifically discuss this area. Kim?
12 MS. LAWSON-JENKINS: Hi, I'm going to 13 briefly discuss the cybersecurity area where we 14 participate in these inspections and the factory 15 acceptance testing, and we will be participating in 16 the upcoming installation inspections.
17 The basis for almost all of the site 18 security plans and also for Waterford, which includes 19 Waterford, is NEI 08-09.
20 Appendix A, which talks about Section 313, 21 with identification of critical digital assets, 22 specifically in there it discusses technical security 23 requirements for security controls.
24 And those security requirements are 25 usually sent to the vendor because then the vendor NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
82 1 implements those security requirements and the 2 licensee will take credit for those as a part of the 3 cybersecurity plan.
4 So, one of the main things that we were 5 looking at as we went into the inspections were the 6 requirements that were sent down to the vendor and the 7 vendor received this information and it will be 8 information that is sent back to the licensee.
9 Also, we look at NEI 08-09, which is 10 Appendix E of 08-09 security impact analysis 10.5, and 11 that is verifying when the vendor determines that 12 they're going to use a new CDA or system, what would 13 be the impact to the security posture of the plant?
14 And that may be an ongoing process 15 documenting this information but most certainly it 16 should be done as you're sending those requirements 17 down to the vendor that certain mitigations are put in 18 place and technical requirements are implemented.
19 And that comes back up into going into the 20 overall CSP. But we looked at the security impact 21 analysis to get an idea of how the licensee is looking 22 at implementing this new equipment from a security 23 aspect.
24 A lot of the controls may leverage safety 25 protections that are put into the system but they are NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
83 1 very specific security impacts that we are looking at 2 because as Samir mentioned earlier in his 3 presentation, we're looking at malicious harm that may 4 occur with the system, not just safety impacts.
5 Also, we looked at a series of controls 6 and in NEI 08-09, there should be Appendix E11 for 7 controls for the supply chain.
8 There's five controls that are in there 9 and specifically for the vendor, we were looking at 10 the security test and evaluation plan, which says how 11 did they test these security requirements that were 12 implemented based on the instructions they've gotten 13 from the licensee.
14 And the last control that we look at is 15 NEI 08-09, which is Appendix D 1.4, information flow 16 control, because when you install the new equipment, 17 there's going to be attack pathways that have to be 18 analyzed and addressed and mitigated if there's a 19 possible way to attack the system.
20 And that control, Appendix D.1.4, is very 21 important in the cybersecurity plan when you're 22 implementing new equipment.
23 The plan covers many more controls than 24 these but these are the base, I would say the 25 foundational controls that we look at going into a NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
84 1 cybersecurity inspection.
2 So, the plan isn't in place until the 3 equipment is installed but the actual requirements, 4 especially when it comes to technical security 5 controls, have to be relayed to the vendor so they are 6 properly implemented.
7 And then that information is, like I said, 8 sent back to the licensee and then we will come and 9 perform oversight to verify that the complete 10 cybersecurity plan has been implemented for this 11 digital system.
12 That is patient in this. In the early 13 phases, this is for the factory acceptance testing and 14 the site acceptance testing, we may have had 15 observations but that wasn't the formal inspection.
16 The actual inspection will occur at the 17 installation when the plan is actually in place.
18 That's all I have today. Later on if there are any 19 questions I will be definitely glad to answer any 20 questions.
21 MS. MAKOR: Thank you for that, Kim.
22 MR. JAIN: Are there any questions for 23 Shiattin or Kim on inspection lessons learned at this 24 point? A question from Ted?
25 MR. QUINN: Hi, I'm really pleased at the NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
85 1 reissue of the 5203 and the expanded role in the 2 region. The first question is does Region II take 3 care of all of them in the country or is it spread out 4 among the regions?
5 And then the second question is you have 6 what's called site-specific follow-up items that are 7 part of a safety evaluation report and they come from 8 the different divisions.
9 They come from I&C, they come from the 10 Vendor Inspection Branch, they come from Human Factors 11 Branch, they come from NSIR.
12 And I just saw what Kim did is a great job 13 in presenting the overview of where the cyber is done 14 in the breakdown between the role of I&C and the role 15 of NSIR in evaluating. You have a funnel and all of 16 your items come into the Vendor Inspection Branch, is 17 that correct?
18 Do you see your role as closing those 19 items in addition to completing your other actions in 20 5203?
21 MS. MAKOR: Yes, that would be an accurate 22 statement. We're more of a confirmatory follow-up-23 type activities so I think your first question was in 24 regard to -- there was three of them, I lost the first 25 one.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
86 1 MR. QUINN: Region II did it for new 2 reactors and I was wondering are they also doing it 3 for current fleet?
4 MS. MAKOR: Region II, as far as it would 5 go in regard to Inspection Procedure 52003 and the 6 implementation, that's done regionally for your plant.
7 So, Region IV has everything west of the Mississippi 8 and so our facilities that are doing digital I&C, we 9 would implement IP52003 for it.
10 We would implement it for there and Region 11 I and Region III.
12 MR. WATERS: Region II there's 13 construction specifications for new build, I don't 14 know if that's where Region II gets involved often.
15 MR. QUINN: Right, that's what we saw 16 earlier.
17 And then the second one, really, was 18 you're a compendium, a collection of those site 19 inspection follow-up items and you relate what each of 20 the NRC divisions that have open items to close.
21 I assume you do that, right?
22 MS. MAKOR: What we do to handle that is 23 our inspection procedure or scope comes from the 24 IP52003 and then input from the technical reviewers 25 from the safety evaluation. So, items that weren't NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
87 1 closed, they're not required for us to do it but these 2 suggest items to follow up on.
3 In our case, a lot of it was already items 4 that are in our requirements in the inspection 5 procedure requirements and then our team also has a 6 complement of an inspector that was in the vendor 7 group, an inspector who was the technical reviewer on 8 this, and then we had the cybersecurity role.
9 And then operations is followed up by 10 using our regional onsite inspectors. So, our 11 complement is how we maintain continuity and make sure 12 that we capture everything throughout this process.
13 So, we've built a team that included 14 people who were very aware or involved with the other 15 areas.
16 MR. QUINN: Thank you, you answered my 17 question. As more utilities go through SLR and 18 modifications, this is really a benefit that you're 19 expanding your role. Thank you, end of comment.
20 MR. JAIN: Samir?
21 MR. DARBALI: Yes, I just wanted to add to 22 what Shiattin was saying. What we do is we provide 23 the region with recommended inspection activities, and 24 like Shiattin said, those are optional.
25 So, they're not technically open items NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
88 1 from the review. Our review stands on itself and we 2 don't depend on inspections or inspection results to 3 determine the acceptability of a modification.
4 So, when Shiattin develops the inspection 5 plan, she's going to use IP52003, which is the 6 inspection procedure, and she's also going to use 7 those recommended inspection activities that we 8 provide as more targeted activities for the review.
9 And also I think Shiattin might cover this 10 in the lessons learned. She was involved throughout 11 the review so she was aware of the types of 12 modifications that were being performed.
13 So, it's a combination of different 14 factors that go into that inspection plan.
15 MR. JAIN: Jarrett, I saw your hand 16 raised?
17 MR. JARRETT: Yes, I know you haven't 18 performed the inspections for Waterford as far as 19 installation, but we have detailed outage schedules 20 and has there been the thought of planning and 21 integrating with the plants outage schedule and that 22 interface during the outage?
23 Has there been any thought on how to 24 interact during the outage? Because it's critical for 25 the plant and so how that will actually occur and what NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
89 1 will you all be looking at?
2 MS. MAKOR: We do this just like any other 3 inspection process, what we've done is we've been 4 communicating with Waterford since the FAT.
5 We found that communication was really 6 important and so we communicate frequently, we have a 7 master schedule so we know what is going on during the 8 outage, and then when we put our inspection plan 9 together and do our request for information, we 10 essentially announce our inspection with request for 11 information and that'll give an idea of where we're 12 going to be looking at.
13 And then also, with the inspection plan 14 for this part of it, the plan is to go ahead and have 15 that actually formally issued.
16 So, the inspection plans for the factory 17 acceptance testing and the site acceptance testing was 18 not formally issued but we figured this would be a 19 good resource not only for Waterford, but also for 20 other plants that this is coming down the pipeline 21 for.
22 So, we've been in communication with 23 Waterford, it's not an unannounced inspection, they 24 know we're coming. We're not going to disturb work or 25 bother them in those ways.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
90 1 We understand it's critical to have them 2 and we're making plans as we speak to be able to 3 perform our duties, our regulatory requirements, 4 without interfering with implementation and 5 installation.
6 Does that answer your question?
7 MR. JARRETT: Yes, thank you.
8 MR. JAIN: Ted had a follow-up question?
9 MR. QUINN: After Samir spoke, I just 10 wanted to ask a follow-up, in the back of the Diablo 11 LAR there was a section called site inspection 12 follow-up items and I just wanted to clarify, I 13 assumed that is a continued process for NRC that is 14 part of the SER so that those are followed up.
15 And I didn't quite understand if those are 16 followed up by Headquarters or if they're followed up 17 by the region, are you able to clarify?
18 MS. MAKOR: They're followed up by the 19 region. There was a change in how those items were 20 documented, so it was decided we would no longer put 21 it into -- since it's optional, we would not put it in 22 or recommend it, we would not include it in the 23 license amendment.
24 Because the license amendment should 25 standalone, it should be a standalone document.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
91 1 And Samir, if I'm misspeaking please 2 correct me but the decision was made to do it how 3 we're doing it now versus having it in the license 4 amendment.
5 So, there are changes from Diablo Canyon 6 until now.
7 MR. WATERS: This is Mike Waters, I'll 8 just jump in, you're absolutely correct.
9 There are some intense changes at Diablo 10 Canyon and stakeholders were concerned that having the 11 recommended inspection items, just having even the 12 optics of somehow being a basis for licensing 13 decisions.
14 So, we decided not to do that anymore and 15 it's not going to appear in the SE.
16 What we do is internally communicate in a 17 similar manner with the region and recommend 18 inspection items like Samir did to help on the 19 inspection plans.
20 MR. QUINN: So, the licensee, Mike, would 21 hear that in a different manner than was done for 22 Diablo, is that correct?
23 MR. WATERS: As part of the inspection 24 planning from the region, or if there's been 25 inspections after the license issuance, that could NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
92 1 possibly happen, we don't know but that's where you'd 2 hear it from, yes.
3 MR. QUINN: Thank you.
4 MR. JAIN: Dinesh, do you have any 5 comments to make?
6 MR. TANEJA: No, I'm just unmuting.
7 No, I think Ted's question and concern, I 8 think it's been addressed. What I was going to say is 9 when a LAR is issued, licensing is complete, we made 10 our safety finding.
11 And so the follow-ups and inspections are 12 confirmation that the implementation is being done in 13 accordance with the approved LAR. And there is not 14 really part of the ongoing licensing act and I think 15 Mike just clarified that, and that's what I'm going to 16 do, that's why I took my hand down.
17 MS. MAKOR: Yes, and if you look at the 18 Inspection Procedure Section 2 it's very general. So, 19 the information that we're provided is just more 20 specific.
21 Instead of it saying digital equipment, 22 now it's saying the CPCS, make sure the CPCS is doing 23 X, Y, Z versus make sure the digital equipment is 24 doing XYZ.
25 So, it just specifies what's already in NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
93 1 the requirement section and it gives us a little bit 2 more pointed or detailed information.
3 MR. JAIN: Thank you, Shiattin.
4 MS. MAKOR: You're welcome, are there any 5 other questions? I can't see the hands.
6 MR. JAIN: Are there any more questions on 7 inspection lessons learned?
8 MS. MAKOR: I do have two more slides.
9 MR. JAIN: Go ahead.
10 MS. MAKOR: I broke the next two slides 11 down into one on lessons learned and the other on best 12 practices, so I'll try to get through this pretty 13 quickly. I see that we did take a little bit more 14 time.
15 But the first lesson learned is not really 16 a lesson learned, it's more of a challenge that was 17 only specific to the region and Waterford. So, 18 Inspection Procedure 52003 was actually not issued 19 until July 1, 2021.
20 And then the LAR was not approved until 21 after the FAT inspections. So, the challenge that it 22 presented to us was that, one, we didn't have an 23 inspection procedure until we were on site.
24 So, it was actually issued I think the 25 Friday before the Monday we were on site.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
94 1 And so I think that presented some 2 challenges to both the inspection team and to 3 Waterford, just because there wasn't the level of 4 detail or understanding of what's in the inspection 5 procedure.
6 So, for instance, the vendor oversight 7 plan expectations for that also just the licensee 8 didn't have that inspection procedure yet so you had 9 an inspection team come on site that's using the 10 inspection procedure and they didn't have the 11 opportunity to prep with that inspection.
12 So, that's really something that just 13 happened, it was the artificiality of having the 14 inspection procedure issued at that time but it was a 15 challenge and a lesson learned for us.
16 So, the three main lessons learned that I 17 wanted to cover quickly is, one, all the work is in 18 progress and documentation reviews such as test 19 results, procedure changes, all those are going on 20 throughout the entire process.
21 And it doesn't really catch up until the 22 installation and even at installation, there was not 23 any regulatory hooks or anything that we can address.
24 All we can do is observe and so until we return to 25 service, there's not going to be violations, only NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
95 1 observations at this time.
2 So, I wanted to point that out, that is a 3 challenge for us but it puts the onus to stay on top 4 of everything and get everything done because at the 5 end, when you return to service, then any issues or 6 concerns that we express that we had been looking at 7 can actually be addressed once you return to service.
8 The next item that I think is a really big 9 lesson learned is just inspection scheduling, how to 10 account for when to be on site, what activities we're 11 looking at, a lot of times we're not speaking the same 12 language, terminology is different between the 13 licensee and the region or just the NRC in general.
14 So, we struggled because initially we saw 15 the factory acceptance testing and the site acceptance 16 testing as when the testing is complete that's it.
17 So, we can call everything good and so we 18 struggled because what we tried to do initially was 19 call everything good or complete for that inspection 20 phase based off of the inspection report. Not the 21 inspection report, but the report that's performed by 22 the licensee.
23 So, Waterford would do their report after 24 the FAT was performed. They looked at the test data 25 and everything but we were trying to be complete NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
96 1 immediately when the testing was done.
2 And so that afforded some challenge and so 3 for the installation, the team is going to consider 4 return to service as complete. So, our inspection 5 won't end when the testing is done whereas for the FAT 6 and SAT, we did end when the testing was done.
7 Since we didn't see any anomalies or major 8 anomalies or issues, we were able to feel confident 9 that we could go ahead and say, okay, for the factory 10 acceptance testing this is done when testing was 11 completed and the same for SAT.
12 Whereas for installation, we'll be using 13 return to service as our metric for marking it 14 complete. And that's not clarified or detailed in the 15 inspection procedure so that's something that we 16 learned and decided to implement as a result of doing 17 the FAT and the SAT.
18 The other item lesson learned is 19 inspection procedure resource estimates. So, the 20 estimates, as I mentioned before, it was 120 hours0.00139 days <br />0.0333 hours <br />1.984127e-4 weeks <br />4.566e-5 months <br /> 21 plus an additional 120 for ARP.
22 What we found is that with the team 23 composition that we have, one, it's a struggle to stay 24 under 240 hours0.00278 days <br />0.0667 hours <br />3.968254e-4 weeks <br />9.132e-5 months <br />.
25 If you have five inspectors in operations, NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
97 1 a vendor, a technical reviewer, a cybersecurity, and 2 a team lead for even just one week that 40 hours4.62963e-4 days <br />0.0111 hours <br />6.613757e-5 weeks <br />1.522e-5 months <br /> or 30 3 hours3.472222e-5 days <br />8.333333e-4 hours <br />4.960317e-6 weeks <br />1.1415e-6 months <br /> for 5, 6 people, that already eats a big chunk 4 of your 240 hours0.00278 days <br />0.0667 hours <br />3.968254e-4 weeks <br />9.132e-5 months <br />.
5 One thing that we did notice is during the 6 factory acceptance testing, since it's more of a 7 familiarity activity, we didn't need as much resource 8 during that as far as time.
9 A lot of it we were able to charge to 10 prepping and just getting familiar with the system, 11 which I think the inspection procedure had it in 12 reverse, where the expectation is that you would spend 13 a lot of time and charge more during the FAT and 14 during the all the time less.
15 But we found it's the opposite. Because 16 everything is in progress and still being worked on 17 and changed, we just weren't able to call anything 18 done. So, we're going to be able to look at most of 19 it during the installation.
20 So, those are the later lessons learned.
21 For best practices, some of these I think the vendor 22 inspection, and Greg mentioned, but we found the 23 phased inspection approach is a better approach for 24 us.
25 It allowed the team to divide the project NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
98 1 into manageable pieces and then also it allowed us to 2 identify lessons learned and best practices as we went 3 along.
4 So, when we did the factory acceptance 5 test, we learned a lot and we were able to implement 6 that with the site acceptance testing.
7 And then now we're able to combine what 8 we've learned and practices to the installation that's 9 upcoming.
10 And we wouldn't have had that value or 11 that opportunity if we would have just waited, not 12 documenting and not formally doing it and waiting 13 until the installation.
14 During the FAT, the next best practice 15 that we notice is that during the FAT, I mentioned 16 this slightly before but for the major milestones, it 17 introduced overlapping priorities and it required 18 significant coordination from the licensee.
19 So, this is one of those things that 20 brought awareness for the licensee because during the 21 FAT you'll have both the vendor and the regional 22 inspection teams on site.
23 And so the regional team really tries to 24 focus on their interaction with and observations of 25 the licensee. We're not really trying to interact NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
99 1 with the vendor.
2 If we are asking quality assurance 3 questions, they're questions that are going directly 4 to the licensee, not the vendor. So, it may look like 5 we're providing oversight of the vendor but our focus 6 is really on how the site is interacting with the 7 vendor and providing oversight.
8 And one of the themes is just that 9 awareness so that you know when we're on site, that's 10 what we're doing. Because I think there was some 11 confusion on the roles and a little lack of clarity 12 there.
13 The next item is that best practice for 14 the SAT, we found that during the FAT our observations 15 weren't as substantial whereas during the SAT we did 16 Pareez substantial observations.
17 And so the documentation for that area, 18 the documentation in the FAT, is really just a summary 19 of the team's overview.
20 The documentation for the SAT will include 21 observation so if there are any significant 22 observations or things that we expected to already be 23 in process or completed, then we would document it in 24 the SAT.
25 It's not a finding, it's not a violation, NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
100 1 it's just an observation and that way, we're able to 2 look back at that report and the previous one and see 3 what we've looked at and prep for the installation.
4 And the last best practice is just the 5 communication between NRC, all of the groups, that was 6 very significant and valuable. So, regional, 7 licensing, vendor and cyber, all of that interaction 8 helped to make this go smoothly.
9 And then communication with the licensee 10 is also critical just because of the dynamic aspects 11 and the fact that the biggest thing is this is a long 12 evolution, we started way before July of 2021 and we 13 won't be near completion until, essentially, June of 14 2022.
15 So, communication, staying on top of it is 16 very important, very significant and one thing that we 17 did is as we got closer to inspections, we would have 18 meetings with the site every two weeks.
19 And so we would have a biweekly call and 20 just a status, this is where we're at, this is what 21 we're doing.
22 And that helped to facilitate our 23 inspection activities.
24 So, those were best practices and the key 25 messages from our regional inspection activities is NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
101 1 that, one, I mentioned this before but Inspection 2 Procedure 52003 had two notable changes, so that's in 3 adding the ARP option, and formalizing the 4 cybersecurity plan which we discussed earlier.
5 The next key message that I wanted to 6 emphasize is just that the execution of the VOP is the 7 responsibility of the licensee. And that's the focus 8 of what we're looking at, the inspection team.
9 So, this includes performance of oversight 10 activities on the design artifacts and developmental 11 activities for requirements, design, implementation, 12 integration, and factory acceptance testing.
13 And so this is the crux of it, this is 14 pretty important and significant and the focus of 15 inspection procedure 52003.
16 And then the last key message that I 17 wanted to deliver is that lessons learned during the 18 FAT, the SAT and installation are used in real time in 19 our inspection planning and scoping.
20 So, once we get enough run time from 21 Waterford, Limerick and Turkey Point, we can combine 22 lessons learned to improve their process, procedures, 23 identify best practices, learn from missteps, and make 24 overall improvements.
25 So, everything that I did cover I try to NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
102 1 keep it focused on Waterford and the regional 2 inspection activities because we do have the NEI 3 comments and some of the observations and 4 recommendations and lessons learned that we had also 5 fall under there.
6 So, some of that will be covered. Is 7 there any questions or comments about my presentation?
8 And I apologize for taking up a little bit more time.
9 MR. JAIN: Ted has a question.
10 MR. QUINN: This is Ted, very good 11 presentation and I liked all of the work that's been 12 done to increase the coordination, for example, 13 between yourselves and NEI and C Branch, the Vendor 14 Inspection Branch, the Cyber Branch, there have been 15 real improvements.
16 Now my question is to the next phase which 17 is currently underway, the Human Factors Branch. How 18 do you see your interface? You just described the FAT 19 and the SAT and your activities, which in some of the 20 larger modifications will include a significant role 21 in cyber.
22 I know it will be addressed by Brian and 23 Pareez this afternoon but I wanted to ask how you see 24 it in your increased role in that area.
25 MS. MAKOR: So, I would say the way I see NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
103 1 that we are implementing is not formal but we went 2 ahead and we added our resident inspector as a team 3 member so you get that operations, that human 4 interface.
5 It ties it together.
6 And so even though we're not formally 7 performing those assessments, like the whole human 8 factor engineering looking at the procedures in that 9 way, we are looking at it from an implementation 10 aspect.
11 So, operations being included I think 12 covers it.
13 MR. QUINN: Thank you.
14 MS. MAKOR: And if anyone else on the team 15 has a comment contrary to that?
16 MR. WATERS: Shiattin, this is Mike. I 17 would offer I think for those human factors, the 18 regional coordination would be similar to the 19 coordination with I&C or other technical disciplines.
20 So, we would expect it to continue and I 21 think that's maybe part of your question, Ted.
22 MR. QUINN: Yes, thank you.
23 MR. WATERS: Samir, is that what you were 24 going to say?
25 MR. DARBALI: Yes, I was going to add that NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
104 1 for Waterford there wasn't a major human factors 2 modification. We do have some recommended inspection 3 activities related to the operator modules and some 4 signals that are trained.
5 But it wasn't a major part of the review, 6 we do expect for the upcoming LARs that would be much 7 more involved as far as our recommended inspection 8 activities.
9 MR. JAIN: With that, Samir, do you want 10 to continue with your presentation and response to NEI 11 comments on Inspection Procedure 52003?
12 MR. DARBALI: Yes, Shiattin, you're done, 13 right?
14 MS. MAKOR: Yes, I am. Thank you, 15 everyone.
16 MR. DARBALI: I think that's a good segue 17 from Shiattin's presentation. We're going to take a 18 look at some of our responses to the comments that NEI 19 provided on IP 52003.
20 And Shiattin already talked about the 21 development of the revision -- let me turn my camera 22 on -- to IP52003 which was issued in July in 2020, 23 just in time to support the factory acceptance test 24 inspection of Waterford.
25 Now, public comments, like Alan mentioned NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
105 1 in his opening remarks, are not really part of the IP 2 development process but NEI felt the need to provide 3 those comments in August of last year and we're going 4 to be going through those comments in the following 5 slides.
6 Some of the key things to understand about 7 the IP, and Shiattin already went through some of 8 these, is that IP52003 is strictly for modifications 9 associated with a license amendment request, which 10 means it's only for safety-related modifications and 11 does not apply for modifications performed under 5059.
12 Like Shiattin said, Section 2 is 13 inspection requirements and identifies the inspection 14 activities to be performed as applicable. That means 15 that if there's a specific inspection activity that 16 doesn't apply to the modification, it doesn't have to 17 be performed.
18 And Section 3 is inspection guidance for 19 the inspectors in preparation for the inspection. So, 20 if there is an activity in Section 3 that calls for 21 review of a specific document, it is for the 22 inspectors to become familiarized with the 23 modification.
24 So, what I propose is as I go through each 25 individual comment, we can take questions on each NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
106 1 comment or comments. That way, we don't have to wait 2 until the end.
3 So, Comment 1 refers to the IP activities 4 that may have already taken place during the licensing 5 review, like activities that refer to the requirements 6 phase and the design phase, which again, they are 7 parallel with the licensing review.
8 So, why does the IP refer to these 9 activities?
10 Well, although a specific vendor oversight 11 activity may take place during the licensing review, 12 for example, the licensee could be performing an audit 13 of the vendor during the design phase, that audit 14 report from the licensee may not have been completed 15 until after the license amendment has been approved.
16 In this case, the licensee's audit report 17 is something that can be part of the VOP inspection.
18 Note that the inspection activities that rely on 19 direct observation will be as applicable and they will 20 be detailed in the inspection plan.
21 So, we don't see a change made to the IP 22 on this comment. Any questions or feedback?
23 I'll jump ahead and let me know if I need 24 to go back. Comment 2 has to do with SDOE and there's 25 two parts to the comment.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
107 1 The first part notices that there are two 2 sections in the IP that are related to verifying the 3 adequacy of a secure development in operations or 4 environment.
5 And NEI recommended these two sections be 6 combined. And the second part of the comment notes 7 that the wording in Section 22B8 ties SDOE to 8 cybersecurity requirements instead of software quality 9 requirements.
10 And NEI recommends the wording be 11 clarified to reference other quality requirements.
12 So, we agree with the intent of the comment, in 13 particular, Item 22B8. We think it fits better under 14 Subsection A rather than under Subsection Bravo of the 15 IP.
16 So, we'll consider rearranging that in the 17 next update to the IP.
18 Regarding the second part of the comment 19 on the term secure environment, as used in Section 20 02.02.b.8, the term being used is secure environment, 21 not SDOE and secure environment could apply to both 22 SDOE and cybersecurity requirements.
23 So, the way it is used in Section 24 02.02.b.8 is specific to cybersecurity. We'll try to 25 make that clear when we eventually merge those NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
108 1 sections.
2 I'll go to the next one. So, Comment 3 3 has to do with the inspection activity related to 4 power quality, which focuses on the verification that 5 harmonic distortion is measured.
6 NEI notes that plants don't typically 7 measure harmonic distortion and recommends removing 8 this criteria from the IP and instead refer to DGC 17 9 or an applicable IEEE standard.
10 We'll consider this comment in the next 11 update of the IP. I think the focus here is that it's 12 for the inspectors to verify how the licensee has 13 performed power quality measurements.
14 We'll consider that comment. On Comment 15 4, this has to do with two sections of the IP that 16 deal with the VOP and NEI recommends the two sections 17 be consolidated into a single section to avoid 18 duplication of inspection activities.
19 So, we don't see a change being made based 20 on this comment because Section 21C is focused on 21 inspecting the licensee's oversight of the lifecycle 22 activities, so more of a direct observation.
23 Whereas, Section 22E is focused on 24 inspection of the documentation generated by the 25 licensee as a record of their oversight activities.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
109 1 So, for example, their audit reports or 2 their letters reviewing vendor documents. So, we do 3 see those as separate activities that don't overlap.
4 So, moving onto Comment 5, this one refers 5 to some inspection items related to site acceptance 6 testing, installation, and startup plans that are 7 phrased using future tense.
8 And it gives the impression that the NRC 9 is approving the test plants, the comment also points 10 to some language that is some subjective, like using 11 terms like sufficiently or adequately.
12 And it calls for objective criteria for 13 inspections to be provided. So, regarding the future 14 tense, this is used because it's referring to the 15 inspection review of the planning documents which have 16 yet to be implemented.
17 The inspectors are not acting as approvers 18 of the plan, that's either a vendor or a licensee 19 activity. We are reviewing the plans to verify that 20 the plans are detailed and clear enough to perform 21 those activities.
22 This also helps the inspectors to confirm, 23 once the plan has been implemented and completed, that 24 the plan was carried out accordingly.
25 Regarding the use of the term sufficiently NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
110 1 and adequately, which could seem to be subjective 2 terms, they provide a range of acceptability but 3 please note that the inspections are performed using 4 an inspection plan, which is besides the inspection 5 procedure.
6 So, like we mentioned earlier, the 7 inspection plan is going to be more detailed, is going 8 to incorporate some of those detailed recommended 9 inspection activities and so the inspection plan will 10 carry the acceptance criteria for what needs to be 11 accepted and their acceptability.
12 So, we don't see a need for making an 13 update to the IP based on this comment. So, Comment 14 6 through 9 are regarding Section 3 of the IPEE, which 15 we already clarified is guidance for the inspectors to 16 prepare for the inspection, and they're not really 17 activities to be performed during the inspection.
18 So, Comment 6 reads the ARP was intended 19 to disconnect the license amendment, request approval 20 from completion of the factor acceptance test, and 21 that to a degree, the IP reconnects the factor 22 acceptance testing to the modification.
23 So, please note that the LAR approval is 24 still independent from completion of FAT under the ARP 25 and the sentence in question, what it does, it points NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
111 1 to inspection activities of FAT which are independent 2 from the licensing review.
3 And again, this is for the inspectors to 4 be prepared for the inspection. So, we don't see any 5 changes to the IP based on this comment. Comment 7 6 notes that a software development capability maturity 7 model is readily used in the U.S.
8 And it asks what would happen if such a 9 document doesn't exist? If it doesn't exist, it 10 doesn't exist. Again, this is in Section 3 so it's 11 one of the many documents listed for an inspector to 12 be aware of and familiarized with as they prepare for 13 the inspection.
14 So, if it's not listed specifically as an 15 item, it's not listed specifically as an item that is 16 going to be inspected. So, we don't see a chance of 17 the IP. Comment 8, it infers that the language in 18 Section 33B inserts the NRC into the implementation 19 planning activities.
20 Again, this is guidance for the inspection 21 team to help with the inspection planning. So, 22 understanding of the licensee's implementation 23 schedules helps the inspection team to prepare for and 24 schedule the inspection activities and balance 25 resources.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
112 1 So, we don't see a change to the IP based 2 on this comment. Comment 9 refers to another 3 preparation activity calling on the inspectors to 4 review the human system interface design as part of 5 their familiarization with the digital modification.
6 The comment points to this as an example 7 of the risk of combining non-safety-related, 8 safety-related, LAR and 5059 inspections all into one 9 IP. Again, this is guidance for the inspection team 10 to become familiarized with the modification.
11 And again, this IP is applicable to the 12 modifications associated with license amendment 13 requests only and only to safety-related 14 modifications. So, we don't see a change to the IP 15 based on this comment.
16 Comment 10 is a general observation and 17 refers to the terms verify and inspect. The comment 18 notes that ARP is inspection guidance and should be 19 mostly be used in the word inspect instead of verify.
20 So, a couple of things to note here, the 21 first is that the IP is inspection guidance for the 22 inspectors. The IP does not contain any licensing 23 review guidance and inspectors don't perform licensing 24 reviews.
25 So, any guidance in the IP, regardless of NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
113 1 the term used, is referring to the inspection 2 activities. And secondly, the term verify is a 3 confirmation activity so all NRC inspections are 4 verifications.
5 So, the word verify doesn't imply a 6 licensing review. So, if we are reviewing a document, 7 we review a document to verify but if it's all done 8 under an inspection, you're reading a document to 9 confirm something.
10 You're not tying that to the licensing 11 review. So, we don't see a change being made to the 12 IP based on this comment.
13 Comment 11 is a general observation that 14 the structure of IP52003 combines IP52001 for digital 15 modifications that require a LAR and IP52002 for 16 digital modifications that do not require a LAR.
17 And it combines them into a single 18 procedure. And the comment states that the IP doesn't 19 differentiate between safety-related and non-safety-20 related and doesn't differentiate between 21 modifications that require a LAR or those that are 22 under 5059.
23 So, please note that IP52001 and 52002 are 24 no longer applicable and IP52003 only is used for 25 modifications associated with a LAR. And we'll NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
114 1 consider making this more clear in the next update.
2 Comment 12 is a very good comment that 3 ISG-06 Revision 2 provides limited guidance on what 4 constitutes an acceptance vendor oversight plan.
5 And that given that the IP contains 6 guidance for performing VOP inspection, the IP 7 effectively establishes what the VOP content and 8 acceptance criteria are.
9 And so NEI recommends the Staff develop 10 regulatory guidance on the expected content and 11 acceptance criteria for a VOP.
12 So, we are considering developing VOP 13 guidance after we've completed the review and 14 inspection of these first few applications so that we 15 can incorporate those lessons learned.
16 The VOP guidance in whatever shape or form 17 it is developed, it will not be included as part of 18 the IP and therefore, we're not making changes to the 19 IP based on this comment. But it is well taken, we 20 understand the need for VOP guidance.
21 MR. CAMPBELL: Samir, this is Alan. I 22 just wanted to thank you for acknowledging that and, 23 you know, this is one of the more significant comments 24 that we had, and so appreciate the consideration there 25 and understanding our intent was not to have that NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
115 1 included in the IP. It would be a standalone.
2 MR. DARBALI: Thanks.
3 MR. CAMPBELL: Thank you.
4 MR. DARBALI: Okay, thank you. Comment 13 5 is a general observation that the IP provide specific 6 cybersecurity criteria to be inspected and it requests 7 some clarity regarding the scope of cybersecurity 8 inspections specifically when using the ARP.
9 So, the cybersecurity inspection items in 10 the IP are there to verify the licensee's 11 implementation of those applicable cybersecurity 12 controls that are providing their cybersecurity plan 13 associated with acquiring a digital safety system.
14 Earlier, Kim described the NEI 08-09 15 appendices that are applicable to digital upgrades, so 16 it's just a confirmation that those cybersecurity 17 controls have been implemented according to the 18 cybersecurity plan, so we don't see a change to the IP 19 based on this comment.
20 Comment 14 is a general observation 21 regarding the use of terms like effective, properly, 22 and correctly, which the comment notes are subjective 23 terms and it recommends that a reference or objective 24 acceptance criteria be provided instead of using such 25 terms. This is similar to an earlier comment.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
116 1 Again, terms like effective, properly, and 2 correctly do provide a range of acceptability, but 3 again, the inspection plan will be containing the 4 detailed criteria for the inspection, so the language 5 provides that range of acceptability in the inspection 6 procedure. The plan will contain more detailed 7 criteria, so we don't see a change to the IP based on 8 this comment.
9 And then the last comment is a general 10 observation that calls for training to be provided to 11 regional inspectors on the intent of the ARP and it's 12 a comment that's also well taken.
13 You know, as we discussed earlier, you 14 know, Shiattin has gathered all of these great lessons 15 learned, but she's only in Region IV. You know, what 16 about the other regions? So, we understand the need 17 for training and uniformity for when those inspections 18 are carried out.
19 So, an introductory training was provided 20 to inspectors when the IP was revised and we are 21 considering providing more detailed training to the 22 regional inspectors, both on the intent of the ARP and 23 the use of the IP, and hopefully licensing and 24 inspection lessons learned helped to shape that 25 training.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
117 1 So, again, the comment is well taken. We 2 don't see a change to the IP based on this comment, 3 but we are considering that more detailed training.
4 And that is it for the IP comments. Any questions?
5 Alan, you still have you hand up.
6 MR. CAMPBELL: Yes, Samir, again thank you 7 guys for acknowledging the comments and responding as 8 you did. I think just the one, we had those two 9 comments regarding the subjective terminology there.
10 Our intent here is at the highest level 11 possible to kind of tighten in the bands of that range 12 of acceptability. When we get into regional 13 differences or differences with inspectors, you know, 14 that's what we're trying to minimize to the extent 15 practical.
16 As we move forward and as we have more 17 runtime with the inspection procedure, this is 18 something that, you know, we're sensitive to and will 19 be tracking, and as it matures, we'll continue to be 20 communicating with you as we see either examples or 21 additional comments.
22 Otherwise, you know, again, thank you for 23 acknowledging these. We just got the hard copies of 24 these with the meeting material, so if there's any 25 additional follow-ups that we have, we'll communicate NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
118 1 that to you.
2 MR. DARBALI: All right, thank you, Alan.
3 MR. CAMPBELL: Thanks.
4 MR. DARBALI: So, if we don't have any 5 other comments, I'll turn it over back to Shiattin.
6 Oh, Ted, go ahead.
7 MR. QUINN: Okay, thank you, Samir, and I 8 hope -- this was a very good summary. I'm sure NEI 9 will review and provide feedback.
10 Okay, so my question needs to go back to 11 the turnover of the SER to this 52003 series, and 12 maybe the word turnover is not appropriate, but what 13 I want to address is the lessons learned from Diablo.
14 One of them was that the site-specific 15 follow-up actions had a tracking mechanism, and I 16 heard what Mike said and what you said, and I'll take 17 a very simple example.
18 The very first one from Diablo said ensure 19 that a new key switch used for setting subsystem 20 operating mode is added to the key control procedure, 21 so there is an interface between the design team and 22 the site construction team that is also changing 23 station procedures.
24 If I just use that simple example, Samir, 25 how would that be tracked by the NRC as a turnover NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
119 1 from SER to, you know, to the inspection branch?
2 MR. DARBALI: Right, so what we did for 3 Waterford, we had similar recommended inspection 4 activities, and so we created that table, and I think 5 it was like an internal memo we sent from the 6 technical group to the respective region, so it went 7 to Region IV and said these are the recommended 8 inspection activities.
9 Now, what happened also is we were also 10 involved in the development of the inspection plan.
11 So, we provided feedback and input into the inspection 12 plan, both for FAT and SAT, for those activities we 13 had identified as recommended inspection items, and 14 that's how we've been able to maintain tracking, that 15 all of those recommended inspection items made it to 16 those plans.
17 MR. QUINN: Are those visible to the 18 licensee?
19 MR. DARBALI: So, for Waterford, those 20 were shared as an internal memo. It wasn't a public 21 document or a letter to the licensee.
22 MS. MAKOR: And Samir, can I add? The 23 plan for the installation is to share that with the 24 public, so we did recognize there is value in sharing 25 that publicly.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
120 1 So, typically for regional inspections, we 2 don't share our inspection plans, but for this one, 3 we're going to share our inspection plan for the 4 actual installation, so that will have the recommended 5 items as an attachment in the inspection plan.
6 MR. QUINN: But for the future, I really 7 think it's a tool that is of benefit to the different 8 portions of NRC staff. I think it's a valuable tool 9 and the question is how your licensees know what is on 10 those items so they can address them in a formal 11 manner.
12 (Simultaneous speaking.)
13 MR. DARBALI: Go ahead, Mike.
14 MR. WATERS: This is Mike. This is a good 15 comment to consider, and again, it's a multi-16 dimensional thing here. One is, you know, the 17 wording, we don't want to imply that, again, that the 18 licensing decision is somehow dependent or even linked 19 to those specific recommended inspection items.
20 Now, the licensee has already committed to 21 do those, right? That's part of the plan and part of 22 the design, so they're obligated to do everything you 23 say, right?
24 But we didn't want to word it in a way 25 that made it sound like it was, you know, like ITAAC NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
121 1 or, you know, like a site-specific action item you see 2 in topicals. We wanted to get away from that because 3 the licensing decision is independent.
4 But we do communicate those recommended 5 items, and as Shiattin noted, those are considered by 6 the region and directly incorporated into the 7 inspection plan.
8 And your example of, what was it, key 9 switch, and I think when we recommend things, hey, we 10 know the design and we know what attributes, features, 11 elements of the plan that are more significant to us, 12 more important to us, more significant to the extent 13 we can do that, and those are the type of things we 14 tell the region that given your inspection times, here 15 is what we recommend the focus to be on.
16 Ultimately though, it is a region's 17 decision on what the scope is and how to inspect it, 18 but we are, as we've communicated, highly integrated 19 and we communicate on this and, you know, the I&C is 20 usually part of that inspection team to some degree.
21 MR. QUINN: Thank you. End of comment.
22 MR. DARBALI: Thank you, Ted. All right, 23 so, Shiattin, I'll turn it over to you.
24 MS. MAKOR: Okay, so this is just, it's 25 one slide, so I'll just go ahead and jump into it NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
122 1 because it goes in line and kind of parallels the 2 conversations that we've been having as far as lessons 3 learned and then the NEI comments, and then this area, 4 I wanted to just briefly touch on it just because it 5 has a relationship.
6 So, with the implementation of digital 7 technology and safety in security systems, it's been 8 useful in resolving Ops lessons issues, but it also 9 presents challenges.
10 So, this workshop focused on inspection 11 procedure 52003, but there's also digital upgrades 12 that do not require a license amendment and those fall 13 under regional reactor and resident inspector baseline 14 inspections.
15 So, there is a working group that is out 16 there that's working on digital instrumentation and 17 control operating experience smart sample.
18 So, the purpose of this is to recommend 19 inspection activities and provide support for baseline 20 inspections. The working group has members from the 21 regions, headquarters, and we all came together to try 22 to work on drafting this guidance.
23 So, it kind of parallels IP 52003, but 24 it's more used with Inspection Procedure 71-111.17, 25 which is evaluation of changes, tests, and NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
123 1 experiments.
2 So, the modification inspections are 3 typically where the regional team will go and look, 4 and so we're having more and more digital activities 5 come up.
6 So, when we review them, one thing that we 7 are checking for is that should this have had a 8 license amendment, you know, request, or is this 9 something that the site can do on their own?
10 So, we're working on additional guidance 11 and, you know, resources to help inspectors, regional 12 inspectors, resident inspectors understand the 13 differences between digital I&C as it would be under 14 baseline Appendix A versus digital I&C in the 15 inspection procedure, the non-routine type inspections 16 that fall under Appendix C.
17 That's really all I want to emphasize with 18 that since we had a lot of conversations about 19 training for inspectors. This is another tool that 20 inspectors will have. And that will conclude my 21 comments on that. Any questions or comments? Ted?
22 MR. QUINN: Do you have a schedule?
23 MS. MAKOR: So, we don't have a schedule.
24 We were trying to get it out I would say late last 25 year, but we have had some delays in trying to include NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
124 1 lessons learned. You know, we just did the Inspection 2 Procedure 52003, so that's one that -- I think that 3 the focus shifted to that.
4 And so, the working group is still in 5 process and I think we'll have to revise the expected 6 date because we did want to try to complete it before 7 the end of last year, but that didn't occur.
8 MR. QUINN: Okay, thank you.
9 MS. MAKOR: You're welcome. And then I 10 think there was a question that I saw pop up on the 11 inspection procedure, and the inspection procedure is 12 71-111.17 and that's evaluation of changes, tests, and 13 experiments. That's typically where digital 14 modifications that don't require a license amendment 15 are covered.
16 One thing to note is that it could fall 17 under design basis. Just depending on what inspection 18 is being done, it could be covered in other areas, but 19 the main core one is the evaluation of changes, tests, 20 and experiments that we usually look at digital 21 upgrades or modifications, sorry. Any questions?
22 That would conclude that portion.
23 MR. JAIN: If there are no more questions 24 for Shiattin or any of what we have covered so far, we 25 are slightly ahead of our schedule. We will take an NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
125 1 early lunch break for one hour and reconvene at 1:05 2 this afternoon with a session on HFE. See you then.
3 Thank you.
4 (Whereupon, the above-entitled matter went 5 off the record at 12:04 p.m. and resumed at 1:05 p.m.)
6 MR. JAIN: Hello, everyone. Good 7 afternoon. We'll start the afternoon session with a 8 presentation by the NRC staff on HFE reviews. Brian 9 Green of the NRR Operator Licensing and Human Factor 10 Branch will lead the presentation. Brian?
11 MR. GREEN: Thanks, BP. Hi, everybody.
12 Good afternoon. I'm Brian Green. I'm the team lead 13 for human factors.
14 Dave Desaulniers, the senior technical 15 assistant for human factors, is on the line with us, 16 and I know my branch chief, Lauren Nist, will be 17 joining us shortly, and I believe Jesse Seymour will 18 be here as well from the human factors staff.
19 I look forward to having a productive 20 discussion today so we can continue to keep making 21 progress with regards to human factors for modernizing 22 control rooms. Next slide, please.
23 So, here is a quick overview of what I 24 plan to present on this afternoon. I'm going to give 25 a kind of a rundown of some of the challenges we've NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
126 1 seen from our pre-application interactions with a few 2 licensees at this point.
3 I'm going to present a few potential 4 solutions, including a brief description of multi-5 stage validation, it's a relatively new concept, and 6 also a discussion of using alternate test beds in that 7 process.
8 And I'm also going to discuss some of, you 9 know, our expectations for how the application will 10 come together in an LAR submittal and kind of go 11 through some of the guidance about what we have 12 available and what the expectations are there. Next 13 slide, please.
14 So, Chapter 18 of NUREG-0800 provides the 15 staff with guidance about how to conduct various types 16 of human factors reviews. Chapter 18 references 17 several guidance documents to consider when doing 18 various sorts of HFE-related reviews.
19 And of particular relevance to large-scale 20 control room modifications is NUREG-0711 which 21 describes a risk-informed and performance-based 22 approach to human factors design which is based upon 23 the systems approach to engineering.
24 So, in accordance with the guidance in 25 0711, the integrated system validation, one part of NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
127 1 the human factors process, has been the accepted means 2 for validating the HFE of significant control room 3 modifications.
4 ISV is typically conducted on a full-5 scope, high fidelity plant-referenced simulator, such 6 as those used for licensed operator training and 7 examination.
8 And ISV is perhaps, some of us would 9 argue, the most important aspect of the 0711 process 10 because it includes the use of performance-based 11 metrics to demonstrate that the HFE design is an 12 effective and safe means for the operator to monitor 13 and control the plant.
14 In addition, the ISV presents an 15 opportunity to identify issues with the HFE design, 16 and any issues that may have safety implications can 17 then be resolved in the final design. Next slide, 18 please.
19 Maintaining the availability of a 20 simulator meeting ANSI and ANS 3.5 for operator 21 training and examination limits the availability of a 22 modified simulator to use as a testbed for HFE 23 validation. So, one of the big challenges we've seen 24 is the timing of when the modification to the 25 simulator can actually occur.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
128 1 This is complicated when we look at the 2 design development and testing schedules that place 3 ISV near the time at which the actual control room 4 will be modified and it may provide insufficient time 5 for the NRC staff to consider ISV results when making 6 a safety determination.
7 It's really back loading -- it had the 8 potential to back load things a lot. And this is 9 something that's kind of come to our attention in the 10 last, you know, several months or so as we've seen the 11 more detailed schedules, and when we compare it to the 12 staff, the planning events that we do internally to 13 figure out how we're going to actually conduct our 14 review, we find that, you know, the timing of the ISV 15 is just not necessarily at a time when we can look at 16 it and conduct that.
17 So, that's -- the key challenge we're 18 finding is finding an ability to look at validation 19 results that works for both of the schedules for the 20 licensee and for the NRC. Next slide, please.
21 The alternative review process discussed 22 in ISG-06 is intended to be used if the NRC staff will 23 decide whether to issue or deny the license amendment 24 before completion of the detailed design, 25 implementation, and/or testing activities.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
129 1 And this poses a challenge where HFE is 2 concerned in that ISV testing would likely not be 3 performed until after the NRC would be expected to 4 issue a licensing decision, and the ISV results 5 typically relied upon would not be available for the 6 staff to consider while making their decision.
7 So, in other words, delaying the ISV until 8 after the staff needs to make a safety determination 9 robs the staff of key information needed to make that 10 safety determination.
11 And I'd just like to take a quick little 12 aside here to note that the NRC staff have substantial 13 experience conducting license reviews in which 14 detailed design of the control room is not complete at 15 the time of licensing.
16 The staff successfully licensed the AP1000 17 using DAC or design acceptance criteria, a special 18 form of ITAAC. Staff also completed a review of the 19 NuScale control room using a similar process, but was 20 able to avoid using DAC because the applicant 21 completed the design work while the staff was 22 conducting the licensing review.
23 So, in that case, the applicant submitted 24 a series of supplements, or result summary reports in 25 this case, that superseded and/or supplemented the NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
130 1 implementation plans that had been docketed with the 2 application.
3 In both cases, the applicants and the NRC 4 staff were able to utilize the flexibilities inherent 5 in the NUREG-0711 process to evaluate the detailed 6 design while still providing the NRC staff an 7 opportunity to review final results.
8 Unfortunately, Part 50 does not have a 9 similar provision for the use of ITAAC. However, the 10 staff has been working to identify solutions that will 11 support the needs of the industry while still 12 providing a meaningful regulatory checkpoint for the 13 NRC to review human factors validation results needed 14 to make a determination of reasonable assurance of 15 safety.
16 Next slide, please. I think we can move 17 onto the next one and we'll discuss some solutions 18 here.
19 So, one potential solution that the staff 20 have identified to address this challenge is called 21 multi-stage validation or MSV. Multi-stage validation 22 entails a staged approach to validation testing where 23 results are obtained at stages throughout the design 24 process. ISV is still a part of MSV. It's just the 25 last portion of it would be the way we typically think NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
131 1 of just the final stage.
2 So, MSV incorporates successive, 3 coordinated validated efforts performed at multiple 4 points or periods during the development of a control 5 room design or design modification.
6 So, we think that MSV may provide an 7 opportunity to collect that performance-based data 8 that we would normally get during ISV earlier in the 9 process by looking at other testing that a licensee is 10 already planning throughout the design process.
11 So, in other words, using MSV may help to 12 move the opportunity for regulatory oversight to the 13 left and would make it more likely that the NRC could 14 support the types of project schedules that we're 15 seeing requested.
16 And I think it's important to point out 17 here that the goal of MSV is not to really make, you 18 know, a significant more amount of work. We're 19 thinking that there are lots of design activities that 20 are happening with the designers that typically do not 21 have the, you know, the regulator doesn't necessarily 22 look at.
23 But if we were to reconceive of it, we 24 might say hey, you know, we're going to run this 25 usability testing early on and maybe we can use this NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
132 1 as -- a licensee could reconceive that and say hey, we 2 can start to use the results of this test to start 3 building our safety case to convince the NRC that 4 things are going to be safe as we progress through.
5 So, we're hoping to find those, you know, 6 those correct checkpoints where we can find those 7 efficiencies without creating a lot of extra work in 8 there as well. Next slide, please.
9 So, this is just a little, a picture of 10 the standard. The IEEE standard 2411 was released in 11 2021. So, if you need this as a reference, feel free 12 to look that up, but I want to point out that this 13 standard was based on the work conducted by NEA, and 14 several NRC staff were involved in a series of 15 workshops that were supporting the NEA work and Dave 16 Desaulniers was one of the key players in developing 17 the IEEE standard as well.
18 So, our staff are fairly well familiar 19 with the concepts in here and we do see this as --
20 we're well aligned that this is a valuable tool even 21 though it's not necessarily called out by name in our 22 guidance yet such of 0711. Next slide, please.
23 So, this slide summarizes at a high level, 24 and I don't want to dwell on this too long here, but 25 the defining characteristics of multi-stage NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
133 1 validation, and these key points include, you know, 2 the following considerations here.
3 The activities that comprise the MSV are 4 validation activities as opposed to simple tests and 5 evaluation, so there is some sort of goal to provide 6 a validation result at the end of it.
7 The thought is that they're conducted in 8 series and that they are constructed and coordinated 9 in a way that they build upon each other to support a 10 final validation conclusion.
11 We can come back to that more if we need 12 more discussion on that, but it kind of helps to frame 13 the concept of MSV to see some of these 14 characteristics. Next slide, please.
15 The IEEE standard describes a series of 16 guidelines, which you see on the slide here, that we 17 think may be suitable as acceptance criteria. We may 18 need to have some more additional discussion on that, 19 you know, that this may work in lieu of more specific 20 criteria.
21 So, let me read through these real quick.
22 The first one is that validations are conducted from 23 early, AKA conceptual, to detailed stages of the 24 design development and operations.
25 B, the subjects of validation comprising NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
134 1 an MSV include design concepts. Examples are 2 operations and automation, system elements such as 3 subsystem designs, and the integrated design.
4 C, results from each validation stage 5 contribute to an accumulated body of evidence for the 6 validation of the final design.
7 D, design changes made subsequent to a 8 stage of validation are addressed through testing or 9 analysis in the subsequent stages.
10 E, at each stage, validation methods, 11 controls and rigor are commensurate with the intended 12 use of the associated results and findings, and Annex 13 A of IEEE 2411 gives some examples of that, how that 14 can be done.
15 And validation testing of design elements 16 that are novel, complex, or critical to safety is 17 initiated early in the design process and confirmed in 18 the integrated testing.
19 I know from some of our discussions, you 20 know, that there is not an NRC guidance document that 21 describes how to do multi-stage validation, so in lieu 22 of that, this may be our best guideline. And we 23 recognize that there is some consternation about how 24 some of these criteria could be filled and I think 25 that would be an area we could have, you know, some NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
135 1 further discussion on.
2 You know, my thought is personally that 3 point B here, the validation methods, controls, and 4 rigors, that's certainly one that I think, you know, 5 licensees would want to have some more discussion on, 6 and I believe we have the tools to do that within pre-7 application meetings.
8 You know, NUREG-0711 currently has lots of 9 validation information in it about how to do it, and 10 I think one of the guiding principles is that to --
11 We are not necessarily insinuating that 12 every validation activity should be controlled the way 13 ISV is described in 0711, that there is room for some 14 reasonable grading, and we think that that would be 15 something that we could have more discussions on 16 perhaps in pre-application meetings so that we can 17 avoid any sort of misalignment between the 18 expectations of a licensee and the NRC. Next slide, 19 please.
20 So, some things to consider when we're 21 thinking about the early-stage MSV results, 22 recognizing the constraint that ISV testing can place 23 on the scheduling for significant and complex control 24 room modifications, staff sees that the potential to 25 leverage early-stage MSV results in evaluating an LAR NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
136 1 submittal for a proposed I&C modification.
2 Assuming we have sufficient information, 3 it's possible the NRC could reach a determination to 4 support approval of a proposed license amendment based 5 on the early-stage MSV results without the 6 availability of final results from ISV testing.
7 So, we would consider these results in 8 congruence with the final ISV testing. However, the 9 review of ISV testing may be relatively, you know, 10 relatively brief on the back end, assuming that we had 11 the opportunity to build the confidence using the MSV 12 data earlier on. Next slide, please.
13 To be considered as providing a sufficient 14 basis for the NRC's safety determination, early-stage 15 MSV testing should provide a reasonable assurance that 16 the eventual ISV testing will be successful.
17 Here are some examples of things to 18 consider, the conformance to the MSV guidelines that 19 I discussed earlier, the sufficiency of the testbed 20 scope and fidelity to support to the validation 21 objectives, the adequacy of the test scenarios that 22 would be used should address both the normal scope of 23 operations and emergency operations for which any HSIs 24 are to be used, and the extent to which the results 25 demonstrate the changes to HSIs will not have an NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
137 1 adverse impact on the ability of operators to safely 2 conduct the risk-significant activities. Next slide, 3 please.
4 So, I just want to reiterate here that, 5 you know, if we move forward with an MSV approach, it 6 does not eliminate the need for ISV testing. It would 7 still be necessary within the program to show that the 8 final system actually performance as it's described.
9 However, we're just going to build our 10 case based on the implementation plans in addition to 11 the MSV data, so we still have kind of a method, a 12 balance between methodology and results to rely on 13 here.
14 We may also elect to conduct follow-up 15 inspections of the final ISV testing results along 16 with the resolution of any identified HEDs. Those 17 might be of particular importance if the ISV were to 18 run and any significant issues were to be addressed, 19 we would want to make sure those are closed. Next 20 slide, please.
21 One of the tools that we expect, you know, 22 that we have heard a lot from different licensees 23 about and we think that many will probably follow 24 suit, is an interest in using an alternative testbed.
25 So, alternatives to full-scope, high-NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
138 1 fidelity simulators as validation testbeds may be 2 acceptable depending on the stage of the validation 3 being credited towards the NRC's safety determination 4 and the nature of the control room modification.
5 The glass top panel, the glass top 6 simulators are one that we think are going to be 7 particularly popular. However, I want to note that 8 with MSV, there are -- you know, you can define 9 different stages in different ways.
10 So, you may be doing some sort of 11 conceptual, validation during a conceptual stage of 12 design in which, you know, a glass top simulator may 13 be overkill or may be unnecessary or unavailable. So, 14 in that case, you may use mockups, and given that it's 15 early stage, you know, that very well may be an 16 appropriate testbed.
17 So, we'd want to have some level of 18 alignment just to make sure we're not heading towards 19 a misalignment later on, to make sure that the scope 20 and fidelity of the testbed are adequate for the 21 purposes of the particular validation that we're 22 looking at in any particular stage. Next slide, 23 please.
24 If the credited validation testing will 25 not be performed on a traditional full-scope, high-NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
139 1 fidelity plant-referenced simulator, we think that 2 some justification should be provided as to why the 3 alternative testbed being used is adequate.
4 So, again, this would be an area that we 5 would probably want to have additional discussion, 6 perhaps in pre-application space, just to make sure 7 that we're aligning on what some of these tools are 8 and how they might be used.
9 If certain components or subsystems within 10 the testbed will be modeled using a low-fidelity 11 simulation, without full haptic fidelity per se, 12 sufficient justification should be provided to 13 conclude that the performance is not to be expected to 14 be adversely impacted when operators are interfacing 15 with the actual components or subsystems within the 16 integrated system. So, those are the sorts of 17 considerations we might have in that discussion. Next 18 slide, please.
19 Okay, so some things to consider, you 20 know, as a licensee puts together an application that 21 uses MSV is that staged testing, if it's used, should 22 be consistent with the MSV guidelines we discussed 23 earlier.
24 And the remainder of the HFE program 25 activities should be conducted consistent with the NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
140 1 0711 process and the Standard Review Plan Chapter 18 2 with one note in that NUREG-0711 does not describe 3 MSV. However, we believe that MSV is consistent with 4 the validation activities described in that document, 5 and we believe some level of grading would be 6 acceptable, you know, based on the particular plan.
7 So, reading through that guidance would 8 provide, I'd say, some breadcrumbs about how to put 9 together an application that has MSV if you were to 10 pare down the ISV material to a level that was 11 sufficient to prove, sufficient for the objectives of 12 a particular stage of MSV, I think we would find 13 ourselves in the right place.
14 And the validation activities and results 15 should be assessed within the context of other HFE 16 program activities. Next slide, please.
17 So, this table here is a summary of -- in 18 the first column, you see a description of various 19 human factors program activities. These are described 20 in the 12 elements of 0711, and NUREG-0711 describes 21 basically two types of submittals.
22 There's an implementation plan and there 23 is a result summary report. An implementation plan as 24 we see it typically describes a methodology to conduct 25 one of the analyses, or design activities, or NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
141 1 evaluations described in 0711.
2 And an RSR is, the corresponding RSR that 3 goes with an implementation plan describes the results 4 of that activity when it's carried out, and usually 5 has some level of summary of the process as well. And 6 0711 describes that a licensee has a couple of 7 different options as far as how to put the submittal 8 together.
9 The first one, and this one may not be 10 practical, but I'll start with it because it's the 11 easiest to explain, is that the licensee could come in 12 a result summary report at the time of a license 13 amendment and say hey, we've done the design work. We 14 have the design work. This is the results of that 15 design and the supporting analyses. Review it, NRC, 16 and move forward.
17 That is the easiest path through because 18 it is the most descriptive and it's easiest for the 19 staff to wrap their head around. However, we don't 20 expect we're going to see a lot of license amendments 21 that come in with that path given some of the 22 challenges and the timeframes needed to design these 23 sorts of mods.
24 So, the alternative approach is to submit 25 the implementation plans with the license amendment NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
142 1 which describe the methodologies about how the 2 analyses and design work will be done, and then follow 3 at some later point in time the RSR that describes the 4 outcome of those activities.
5 And that's how we end up closing the loop 6 here, and the real challenge, as I described before, 7 is that the human factors verification and validation 8 activity tends to occur outside of the time when the 9 staff will be conducting the license amendment review.
10 So, the challenge is how do we get an RSR 11 into the hands of the staff in the time they can look 12 at it? And that's why we think MSV allows us that 13 opportunity, to give the opportunity to the staff to 14 look at that data of the results of that important 15 element to us.
16 Let me see. There are a few exceptions.
17 Not all IPs have RSRs and those are dictated here. I 18 don't want to go into those a whole lot. If we have 19 questions on that, we can have those later. The ones 20 that we do see RSRs are clearly marked here and we can 21 get into that if we need a more specific meaning or in 22 the question section if we want to here. Next slide, 23 please.
24 So, this slide basically summarizes some 25 of the key points from the table on the previous NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
143 1 slide, so I'm going to run through these real quickly 2 verbatim here.
3 In accordance with NUREG-0711, applicants 4 should include within their LAR submittal an 5 implementation plan for each of the 12 elements of the 6 HFE program review model.
7 IPs, and I know we, the NRC likes to use 8 IPs as the inspection procedure, so I apologize for 9 having an inconsistent acronym here. When I use it, 10 it's implementation plans.
11 The implementation plans should describe 12 the methodology for conducting the activities 13 associated with that program element. The IPs are 14 subsequently expected to be followed up with a result 15 summary report once the activity is completed during 16 the licensing review period.
17 Alternatively, if the applicant already 18 has a completed RSR for a program element at the time 19 of the initial submittal, they should include that in 20 their LAR application. Next slide, please.
21 So, if an application relies on early-22 stage MSV results to inform the NRC's safety 23 determination, the IP discussing the verification and 24 validation should include details as to how early-25 stage test results will provide reasonable assurance NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
144 1 that subsequent ISV testing will be successful and 2 will not result in HEDs with safety consequences, so 3 that would be an appropriate place to put that level 4 of description.
5 Also, we should be confident that any HEDs 6 that come out of this process are going to be 7 resolved, well, any HEDs with safety consequences.
8 Not all HEDs necessarily need to be resolved, not 9 according to the 0711. Next slide, please.
10 If an application relies on early-stage 11 MSV results, information should be provided to 12 describe that those early-stage results are performed 13 under a sufficiently robust test program to ensure 14 credible results from early-stage testing.
15 Applicants should consider providing a 16 discussion of their program's conformance with the six 17 guidelines for effective MSV testing address in the 18 IEEE standard, and applicants should consider 19 discussing the 0711 criteria for ISV testing that are 20 applicable to the MSV program being used.
21 I said it before, but perhaps I wasn't 22 real clear about it, but the ISV criteria in the 23 staff's opinion are a very robust method, so if you 24 were planning an early-stage validation and you wanted 25 to scale that down, that would be one way to do it NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
145 1 would be to pick those criteria that are of particular 2 importance.
3 For instance, the ISV criteria have some 4 discussions about the qualifications of the human 5 factors team, and maybe there is some early design 6 work where maybe you don't need a human factors expert 7 to be involved in it, or perhaps you don't need a 8 licensed operator to do some early testing, that you 9 might be able to do it with somebody who is not 10 licensed to do some early, you know, HSI usability 11 testing or something like that.
12 So, there may be some arguments to say 13 hey, we're going to reduce to a subset of these ISV 14 criteria, and we think that these particular ones help 15 to ensure that the results of the test are credible in 16 the long run. Next slide, please.
17 The initial LAR submittal should provide 18 an estimated timeline regarding when the V&V 19 information will be available for review.
20 After I looked at this slide, you know, it 21 says here they should be in the LAR submittal, but 22 really it would be preferable to have these 23 discussions in pre-application space at the earliest 24 point possible.
25 I know there is some frustration that this NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
146 1 topic is coming out as late in the process as it is, 2 and, you know, I think that had we had more 3 discussions, substantive discussions about the 4 contents of the human factors program and how and when 5 those things would be submitted, we might have found 6 that earlier.
7 So, I just kind of want to undercut what 8 I have on this slide here to say the pre-application 9 space is really the best option to start this 10 discussion.
11 But we should have some consideration 12 about a timeline that clearly demonstrates what 13 information will be available at what points 14 throughout the NRC review period because we may need 15 to get in and do an audit or to look at some of this, 16 and with the short timeline, you know, we're shooting 17 at a very small window typically, so anytime we get to 18 plan is preferable.
19 The submittals should also discuss 20 contingency planning and expectations is there are 21 possible delays in the completion of validation 22 testing. For instance, you know, if a late-stage MSV 23 stage identifies a safety-critical HED or human 24 engineering discrepancy, that would be something that 25 we would expect to be fixed.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
147 1 Now, there's lots of ways to fix HEDs, but 2 in some cases, it may cause a design change, so we 3 would be looking to make sure that there is some sort 4 of ability to do that. Now, not all HEDs, you know, 5 warrant design changes. Sometimes they can be fixed 6 with procedures, or training, or other methods, but 7 sometimes design changes are necessary.
8 So, there is a concern that if an HED 9 comes up real late in this process, it's going to ruin 10 a lot of schedules, and clearly our position will be 11 leaning on the side of safety in those cases.
12 Such a discussion will help to support the 13 NRC consideration and acceptance of the proposed 14 approach discussed within the LAR submittal and 15 development of the NRC review timeline, and licensees 16 are encouraged to discuss the timing as early as 17 possible. Next slide, please.
18 So, this final slide here on, you know, 19 the LAR submittal is that the RSRs submitted for NRC 20 review should include the information indicated 21 throughout the 0711 for each of the associated HFE 22 review program elements.
23 As discussed in 0711, RSRs may include 24 summaries for the indicated information, provided that 25 the references given are for more detailed documents.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
148 1 Sometimes RSRs, you know, if there was an 2 implementation plan submitted, you know, on the early 3 side, an RSR can say hey, you know, refer back to the 4 IP. We followed everything in it, or, you know, we 5 had to modify it. We went through a change process 6 and made a change to the IP and this is what that 7 change was.
8 So, the RSR should have some level of the 9 final methodology tied to it, but if that's 10 incorporated by reference, that's typically okay.
11 And I know there was some discussion about 12 the electronic reading rooms. You know, to the extent 13 possible, the IPs should be on the docket. If there 14 are supporting documents, and this very well may be 15 the case, that other licensee procedures or documents 16 are to be used, you know, certainly those would be 17 things that we would look at in electronic reading 18 rooms.
19 We are open to finding appropriate 20 documents to be in there, into an electronic reading 21 room. One that is, you know, quite typically looked 22 at in there is the detailed scenario guides for 23 running validation testing tend to be things that the 24 licensees put a lot of time and effort into, are 25 usually proprietary, and they're often under exam NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
149 1 controls, so there is a lot of good reasons not to 2 docket that.
3 So, we have in the past been willing to 4 look at those in an ERR, and we can consider other 5 expectations, but typically the IPs themselves and the 6 RSRs would be something that would be docketed.
7 Anything aside from that, we would take under 8 consideration and willing to have that discussion.
9 So, those were the key points I wanted to 10 bring up. I want to just open the floor for Dave or 11 Lauren, if there was anything that I missed that they 12 wanted to add before I open it up for questions here.
13 MR. DESAULNIERS: Thanks, Brian. I don't 14 have anything to add. I'll just wait for questions.
15 MR. GREEN: Great, I see we've got a hand 16 here. Sorry, my computer is a little slow loading the 17 names here for some reason.
18 MS. GOLUB: Brian?
19 MR. GREEN: Pareez, yes, hi.
20 MS. GOLUB: Hi, first, thank you for your 21 presentation, appreciate that. I guess one of my 22 questions is we appreciate all of the dialogue on 23 multi-stage validation, but from an industry 24 perspective, it's difficult just looking at all of the 25 slide material and the NEA document to really NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
150 1 understand what would need to be done.
2 You know, it feels a bit overwhelming, 3 frankly, and so I guess I'm wondering is there any 4 examples of precedents or anything out there that 5 industry could take a look at so that we would have a 6 sense of what is necessary if anybody wanted to go 7 this path?
8 Is there anything out there or is it 9 really just guidance and then, you know, folks are 10 going to have to take their best shot at it?
11 MR. GREEN: So, I'll start to answer that 12 and Dave may want to jump in. I don't know of a 13 particular precedence where this has been applied yet.
14 I do know that, I believe it was the NEA document, and 15 Dave, correct me if I'm wrong on this reference here, 16 provides some examples. I know there was one example 17 about how the MSV stages might be considered.
18 When we were putting that together, we 19 were hesitant to prescribe what stages should be used 20 because we recognized that different designers 21 approach design in different ways and we did not want 22 to, you know, tell a designer that uses one approach 23 that their approach is somehow not adequate or not 24 appropriate because it's inconsistent with the 25 prescribed stages that we had put together.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
151 1 So, we chose to put an example in that 2 report and that particular one, it was kind of a four-3 stage MSV that started with a conceptual design stage, 4 moved to a subsystem design stage, and did some 5 integrated system design stage, and then, you know, 6 worked itself into a deployment and operation stage.
7 And then, you know, as we would envision 8 it, and my memory is kind of rusty about how this gets 9 built out, would be that, you know, the designers 10 would consider the objectives within those stages and 11 propose the validation tests that would conclude each 12 of the stages.
13 So, and that, you know, that particular 14 example, I had suggested, you know, the integrated 15 system design stage might be a test very similar to 16 the ISV that we see, but in the stage that precedes 17 that, the subsystem design stage, there might be, you 18 know, perhaps the full simulator is not available yet, 19 but large-scale or large semi-functional level 20 simulations of different systems might be available 21 and you might be able to run some desk tests, or, you 22 know, maybe not with a licensed operator.
23 Maybe you've only got some design 24 engineers at that point or, you know, so there is some 25 kind of wiggle room to make those determinations about NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
152 1 who the appropriate people are and what that test 2 looks like.
3 And, you know, I think the guiding 4 principle as I see it is to understand what the 5 objective is within the defined stage that a licensee 6 makes, and then to, you know, design an appropriate 7 test to conclude that stage, you know, using some of 8 the ISV tools that are available or similar.
9 You know, it doesn't necessarily have to 10 be the ISV criteria from 0711, but put that together 11 in a way that creates a creditable, sorry, not 12 creditable, a credible result.
13 You know, from a regulatory standpoint, 14 when I go and look at one of these stages, I want to 15 believe the results and I want to see that, you know, 16 if you say that, I don't know, the updated rod control 17 system works, I want to see a test that gives a good 18 flex to that using whatever simulation or tool is 19 available at the time, and I want to understand what 20 limitations there are to that test.
21 You know, maybe the rod motion doesn't go 22 all the way or you don't have all of the interfacing 23 systems connected yet. You know, if I understand some 24 of those limitations, I can say okay, well, this is a 25 good early-stage test. It might not be enough to NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
153 1 convince me that, you know, we don't need to look at 2 anything else down the line.
3 So, you know, it's a matter of do you have 4 enough control that, you know, a reasonable person is 5 going to look at it and say, yeah, I believe the 6 results that came out of that test because you 7 controlled the methodology that went into it.
8 I know that's still a little vague and I'm 9 going to turn to Dave here to see if he has a better 10 or maybe a more concrete example to answer your 11 question.
12 MR. DESAULNIERS: Okay, sure, Brian.
13 Well, I think that an example that we can point to, 14 while not identified specifically as multi-stage 15 validation, may not have all of the, meet all of the 16 check boxes of multi-stage validation, but I believe 17 is just really quite consistent with what we're, the 18 notion that we're trying to put forward here is a 19 recent review example we had with NuScale and the 20 design certification for their small modular reactor 21 where early on in the process, NuScale engaged with us 22 on their staffing and it was a unique staffing 23 approach.
24 And therefore, they undertook efforts to 25 do preliminary testing to show that indeed the NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
154 1 staffing concept that was being used there was a 2 viable approach and gave confidence for both the 3 applicant and the NRC that we were proceeding down a 4 path towards success.
5 So, you know, those tests were being 6 conducted, not with licensed operators and with their 7 fully developed procedures, but they were robust tests 8 consistent with the level of design development that 9 was available at the time and controlled in ways, as 10 Brian noted, such that the individuals going through 11 the scenarios for the staffing validation didn't have 12 prior knowledge of what those test scenarios were 13 going to be.
14 And so, I think if you're looking for an 15 example within the NRC and, you know, that's been 16 reviewed, within the U.S. that's been reviewed by the 17 NRC, that, you know, I think gives some indication of 18 the type of thing that is done under a multi-stage 19 validation type approach.
20 There, we're not talking about a 21 subsystem, but really it was a staff concept that was 22 being validated, and it aligns with, you know, the 23 early-stage validations of design concepts.
24 MR. GREEN: Yeah, that's a great example, 25 Dave. Thank you for bringing that one up. I should NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
155 1 point out that NuScale provides a unique approach that 2 may work for some licensees in that they submitted the 3 implementation plans with the, along with the license 4 application.
5 And there was an agreement that we had 6 come to with them at one point in pre-application 7 space where, you know, they were not interested in 8 having a lot of DAC as part of their strategy, and 9 their proposal to do that would be to provide an 10 implementation plan with a license amendment, but then 11 conduct all of the validation activities, all of the 12 design and evaluation activities before the staff 13 would be done reviewing that implementation plan.
14 So, it occurred within the scope of a 15 licensing process, in which case the staff were able 16 to go and audit the validation testing that we saw, 17 and it provided a high level of confidence and was 18 relatively easy to move forward with.
19 Now, that may not work for all licensees, 20 but it might work for some and it was a fairly 21 successful strategy, I think.
22 MR. JAIN: With that, I see three hands 23 raised. I will go with first Bill Hannaman. Please, 24 go ahead with your question.
25 MR. HANNAMAN: Hi, can you hear me?
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
156 1 MR. GREEN: I can hear you, Bill. Thanks.
2 MR. HANNAMAN: Okay, thank you. I was 3 really impressed with your slide 17 where you actually 4 kind of laid out all of the 0711 activities and then 5 you can kind of see a pathway in there depending on 6 the complexity of the design and so forth, but it 7 looks like if you put the IP plans in with the 8 submittal, with the LAR submittal for a change, that 9 that would allow you to do the approval of the LAR at 10 that point in time, and then I don't know the role 11 exactly of the regional inspectors, but would they do 12 something related to the HFE plans?
13 MR. GREEN: Yeah, see the challenge that 14 we have with what you say about making the licensing 15 determination on the IPs alone, you know, that's kind 16 of -- Bill, I've never had you over for dinner, but 17 I'm not much of a cook, but if I showed you a nice 18 recipe, you know, you may or you may not get a good 19 meal out of it, and so that's why this validation part 20 of it is a key part. Sorry, we're getting some -- we 21 got somebody on the line, Mike. Okay, there we go.
22 Thank you.
23 So, yeah, you know, that's why the 24 verification and validation activities has always been 25 a really key part of our decision, and that's one of NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
157 1 the challenges here is that for the staff to write a 2 safety evaluation, we have to make our conclusion with 3 the information we have in hand, and we have a -- we 4 can't really bump that into the regional inspectors to 5 confirm that. It leaves us lacking on a key bit of 6 information needed for the safety evaluation.
7 So, I mean, I think, you know, NEI is 8 going to give a presentation here about how, you know, 9 perhaps a license condition might work. We have had 10 some discussions on that and, you know, we're 11 skeptical, but we'd like to hear it through and see if 12 there is a path there, but that's generally been, you 13 know, something we've not been able to pursue, to 14 delay --
15 MR. HANNAMAN: Okay.
16 MR. GREEN: -- that part of the decision 17 into that level of inspection, but I appreciate your 18 question.
19 MR. HANNAMAN: Okay, well, thank you.
20 MR. GREEN: Thanks.
21 MR. JAIN: The next question is from 22 Richard Paese. Richard?
23 MR. PAESE: Yes, can you hear me okay?
24 MR. GREEN: Go ahead, Richard. Thanks.
25 MR. PAESE: Yes. Thank you. So I'm not NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
158 1 sure if this topic has come up before, so my apologies 2 if it has, but my question is related to the basis for 3 the LAR expectations. So my understanding in reading 4 the rule for 50.34(f) is that this regulation isn't 5 applicable for many of the operating plants that are 6 doing these plant upgrades. And as I understand it, 7 NUREG-0800 Chapter 18 and NUREG-0711 were largely 8 written to give guidance for review and compliance 9 against this regulation of 50.34(f). So what's 10 confusing to me it's not clear why NUREG-0711 is 11 expected if 50.34(f) doesn't strictly apply.
12 And the reason I ask this is because many 13 utilities' licensing basis say don't reference NUREG-14 0711 or 50.34(f) but they go into a lot of discussion 15 about the orders and generic letters that were issued 16 to them back in the 1980s and how their design 17 complies with those orders and letters including 18 things like NUREG-0737 Supplement 1. For example, 19 many of them go into a lot of detail about how they 20 address the control room design review requirement 21 from NUREG-0737 Supplement 1, but these types of 22 requirements from their license and from these orders 23 and generic letters, they're not really specifically 24 discussed here in this LAR expectation.
25 So my question is how these two ideas are NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
159 1 being addressed together, namely, the expectation to 2 use 0711 and the commitments that were made in the 3 licensing basis for these plants which don't 4 specifically include NUREG-0711 or have expectations 5 to meet 50.34(f).
6 NUREG-0711, obviously, is a good process 7 and I'm not advocating for -- for not using it, but my 8 question is really going back to the actual regulatory 9 and licensing requirements that are -- that we're 10 using to tie back to the safety case. This seems to 11 be a slight disconnect to me. So maybe the assumption 12 is that 0711 is to be used to show how these licensees 13 are determining the acceptability of the changes that 14 they're making to how they previously -- to the 15 previous requirements that the orders and letters that 16 they have referenced in the license. But this idea 17 isn't clear to me. So that's my question.
18 I was wondering if you could speak to 19 that?
20 MR. GREEN: Dave, do you want to take a 21 first stab at this or would you prefer for me to?
22 MR. DESAULNIERS: You can go ahead, Brian.
23 MR. GREEN: Okay. So, yes, it is -- it's 24 a great question, Richard. So the first place I would 25 start is GDC 19 does say that they have a control room NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
160 1 where they can safely monitor the plant. Not all 2 plants are GDC plants, but those that are, you know, 3 that is another -- there is a position, I believe it's 4 -- I believe we wrote in the advanced reactor world, 5 but you know, these principles are the ones that help 6 us confirm that the operators can, and safely, operate 7 the plant under the normal and emergency conditions.
8 50.34, as we know, does have some 9 limitations about the dates in which the applicability 10 is there. I know there is some rulemaking going on to 11 fix that. I'm not sure what the correct interim 12 answer is to those that fall outside of that scope 13 though.
14 MR. MARSHALL: So Brian, would you mind if 15 I took a stab at this? This is Michael Marshall.
16 MR. GREEN: Oh, yes. Please do, Michael.
17 Thanks.
18 MR. MARSHALL: You're right about the 19 applicability of 50.43(f), but the thing to keep in 20 mind and you're also correct in pointing to the TMI 21 orders as being the basis for the requirement for the 22 many of the operating plants being the human factors 23 requirements there.
24 50.34(f) wasn't backfitted on the existing 25 fleet because of the TMI orders that are in place.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
161 1 And in some ways, depending on the plant, the TMI 2 order might be a little more restrictive or -- not 3 restrictive -- has more specifics in it than 50.34(f).
4 But generally, the NUREG and the guidance that NRC 5 staff is using applies equally to both 50.34(f) and 6 the plants to have TMI orders as to requirements for 7 human factors activities.
8 Dave, Brian, did I get that correct?
9 MR. GREEN: I appreciate you jumping in 10 with that, Michael. Thank you.
11 MR. MARSHALL: Okay.
12 MR. PAESE: Just a quick follow up. So 13 the way I understand it is that -- so these licensees 14 have committed to these orders, these generic letters, 15 these are the -- these are their actual requirements.
16 These are the things that are referenced in their 17 license. They've performed actions according to those 18 orders and those letters.
19 And the idea here is that 0711 is the 20 document that is expected to be used to show how their 21 changes to those -- to their compliant -- potential 22 changes to the compliance to those requirements, 0711 23 is being used as the means to show why those changes 24 are acceptable.
25 MR. MARSHALL: Yes, but one thing to be NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
162 1 mindful of, even though most of the plants got similar 2 TMI orders, there might be some uniqueness and an 3 individual's plant TMI order that might bring into 4 question the NUREG applicability, but generally the 5 NUREG-0711 should be applicable to the plants that 6 received the TMI orders.
7 But again, there was a lot of 8 correspondence after the initial order, I would say 9 modify what licensees are expected to do or required 10 to do. But you would have to recreate that paper 11 trail to make a determination that the NUREG-0711 12 wasn't applicable.
13 MR. PAESE: Yes, I guess the concern that 14 I have is that I think -- it seems to me that we might 15 be getting lost in what the actual requirements are in 16 focusing so much on 0711. Again, I'm not saying 0711 17 isn't an important document and a good document to 18 use, but I think it's important to always remember to 19 go back to what the licensing and regulatory 20 commitments are and how 0711 might be used to satisfy 21 showing continued compliance to those -- to those 22 regulatory licensing requirements.
23 MR. MARSHALL: Yes. I agree. And again, 24 it's good to acknowledge that even though the orders 25 went out to everybody, again, there was subsequent NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
163 1 correspondence that might have customized how 2 individual licensees responded to the requirements in 3 the order.
4 So that would be wise for licensees to 5 make sure they understood what their human factors' 6 requirements were with regards to the TMI order, as 7 they're assembling their amendment packages for 8 submittal to the NRC.
9 MR. JAIN: Okay, with that, the next 10 question is from Jacek Nowakowski.
11 Jacek, you can unmute.
12 MR. NOWAKOWSKI: Yes, good afternoon.
13 Yes, this is Jacek Nowakowski. I'm with Framatome, 14 Inc. And my question was regarding the definition of 15 validation in your 0711 and in IEEE 2411.
16 Does the staff consider that word 17 validation means the same thing in both documents and 18 it will be interpreted the same way, meaning it can be 19 used interchangeably as meaning as used in integrated 20 system validation and as used as part of the multi-21 stage validation where it's broken up into somehow, 22 substance of a validation. Is that word -- would you 23 advise that we use that word when referring to the 24 components of MSV or would you advise that we use a 25 different word like analysis or test or something of NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
164 1 that sort?
2 MR. GREEN: I would have to go back and 3 compare those definitions. I don't have them in my 4 memory enough to contrast the two of them right now.
5 And if maybe Dave is more familiar with them, but I 6 guess I would question is this -- is it a matter of --
7 I'm trying to understand like the nature of the 8 question. Is it that they perceive there to be some 9 difference between the two of them? And you think we 10 may be getting disaligned or is this more so a matter 11 of how to communicate with the staff?
12 MR. NOWAKOWSKI: I think it's mostly about 13 how to communicate not to cause any problems, you 14 know, during the process. So just the question up 15 front -- the validation definitions are slightly 16 different. One of them kind of reads, it's kind of 17 leading you into a degraded system validation and the 18 IEEE one it talks about, like you said, components, 19 subsystem, and system, so implying that there is 20 multi-stages to validation.
21 So it's kind of a -- can we assume that 22 the word validation is well understood and it's kind 23 of colloquial at this point and we can use it for 24 components of MSV, as well as we can use this for ISV, 25 or would you advise that we use alternative words for NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
165 1 that?
2 MR. GREEN: You know, there might be 3 something worth some more discussion. I'd want to 4 think on it a little bit, but Dave, if you have 5 anything to say -- I mean I don't see a problem with 6 it, but I've not had a chance to digest it either.
7 MR. DESAULNIERS: Right, unfortunately, I 8 haven't had an opportunity actually to bring up the 9 differences in definitions, but as I seem to recall, 10 I believe, 0711 provides a definition for integrated 11 system validation whereas the IEEE standard provides 12 a definition for validation.
13 MR. GREEN: That's right.
14 MR. DESAULNIERS: Am I correct? And so I 15 guess where you see the differences in the definition 16 may be that -- I think we're getting a little feedback 17 there.
18 Okay, integrated system validation is a 19 specific type of validation. So the IEEE definition 20 I expect was written to be broader and incorporate ISV 21 whereas ISV is going to be more specific to a 22 particular stage of validation where your focus is on 23 validating integrated operations.
24 So the bottom line is probably would not 25 want to consider the two definitions -- I would want NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
166 1 to pull them up again to be certain on this, but I 2 wouldn't necessarily see them as interchangeable in 3 that ISV, or integrated system validation, is a more 4 specific concept than validation.
5 MR. JAIN: Are there any more questions 6 for Brian on his HFE presentation or the staff?
7 MR. NOWAKOWSKI: Sorry, I was muted and I 8 was trying to reply to the answer if that's okay.
9 MR. JAIN: No, you go ahead.
10 MR. NOWAKOWSKI: Okay, so if I understand 11 it correctly that the two definitions of validation 12 should not be understood as interchangeable? I guess 13 that was my question. So thank you very much for that 14 answer.
15 And I have one more quick question 16 regarding slide 18 that's currently up on the screen.
17 Does staff have any preferences to seeing individual 18 implementation plans or is it standard practice to 19 combine them into documents that include multiple 20 elements and multiple implementation plans in one?
21 MR. GREEN: That's a great question. It 22 is convenient for us to have them separate, but 23 there's nothing that requires that. And to some 24 extent it may make good sense to combine some of them.
25 For instance, and I'm just speaking off the cuff here, NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
167 1 this isn't anything that anybody has to do, but the 2 HFE program management one, that's one that really 3 stands off well by itself and I think from what I hear 4 from licensees, that seems to be a stand-alone 5 document.
6 It's unclear to me if we're going to get 7 a single document that's going to -- to submit the 8 operating experience review function analysis and 9 allocation, task analysis, staffing equals, and 10 treatment of important human actions. It could be --
11 this could be separated into six documents or in a 12 single document. There's not a big difference to us.
13 However, given the tight time lines that industry 14 seems to have, it might be better to send them 15 individually and get them into the staff's hands to 16 look at them as quickly as possible. If they're all 17 going to come at once, then I don't know that it makes 18 a big difference though.
19 MR. NOWAKOWSKI: Okay. I appreciate that.
20 Thank you.
21 MR. JAIN: Pareez, you can go ahead with 22 your question.
23 MS. GOLUB: Thank you. Just on this same 24 slide, I guess I want to make sure I understand what 25 the expectations represented on the slide are. I had NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
168 1 thought from reading 0711 that the IPs being submitted 2 with the LAR was sufficient. If you could address the 3 V & V element to make sure that ISV was addressed 4 either as part of the -- during the review stage or 5 after the review stage if you had a sufficient 6 argument for why that was so and your MSV was being 7 one example of that argument.
8 But when I look at the slide, it almost 9 seems like the RSRs are still required prior to the 10 staff issuing the SE. I just wasn't clear on the 11 answer that was provided when that question was asked 12 earlier.
13 MR. GREEN: Yes, so thank you for the 14 question if that was not clear. So the staff would 15 expect to -- we need to come to a determination of a 16 reasonable assurance of safety. And within the 17 validation aspect, you know, we're still having the 18 discussion right now on if -- let's just say, you 19 know, a licensee was going to conduct a relatively 20 sophisticated multi-stage validation and they were 21 going to say, Brian, come out, you can watch us do the 22 testing. I can go and watch that and I can write an 23 audit report that says, you know, I saw that they were 24 using the good controls, the right sorts of controls, 25 and the results that I saw were positive and support NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
169 1 safe operation. And I can cite that in my safety 2 evaluation. So that gives us the performance-based 3 part of the process.
4 Now it would be preferable to have the 5 RSRs that has the licensees' perspective on that.
6 We're still discussing internally can we live without 7 that and the answer is maybe right now. But I don't 8 know that we've determined that.
9 One of the reasons, the justification 10 behind that is the licensee would run their tests and 11 draw their conclusions and their conclusions might be 12 different or there might be some HUDs identified that 13 we don't see in the week that we're there. So it's 14 just a matter on how much of that information -- how 15 much confidence do we have at the time that we make 16 that observation.
17 And you know, if we go in and we see a 18 test that has lots and lots of issues come up, and the 19 operators can't safely control the plant, then we're 20 not going to have lots of confidence at that point.
21 But if we get in there and we do see that things are 22 working as expected and the HSIs are conforming with 23 the guidelines and things like that, then perhaps our 24 audit report is enough to get us across the finish 25 line. That might be something for some additional NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
170 1 discussion on both sides, but we are considering that 2 right now.
3 As far as for the other elements go, you 4 know, we can get in and audit those. We don't 5 necessarily look at all of them. For instance, task 6 analysis is one that we typically find to be an 7 important part of that process. Specifically, because 8 it feeds into the verification and validation 9 activities real nicely. It tells exactly what the 10 operators have to do and those get built into the 11 procedures and those procedures are the ones that the 12 operators tend to run in the V & V. So there's a 13 direct nexus there.
14 You know, function allocation could be 15 really important if a particular modification is going 16 to be automating lots of things that were not 17 previously automated or vice versa, making lots of 18 manual actions instead of things that were automated.
19 But it may not -- you know, depending on the mod, it 20 may not be as important. So we would have to figure 21 out how we would handle that RSR as well. So those 22 discussions there.
23 I don't think those are necessarily going 24 to be challenging ones, but there is some room for 25 discussion there.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
171 1 MS. GOLUB: Okay. Thank you.
2 MR. JAIN: Are there any more questions 3 for the staff on HFE? If none, then we can ask NEI to 4 make its presentation on HFE.
5 MR. GREEN: Jacek has one more.
6 MR. JAIN: Go ahead.
7 MR. NOWAKOWSKI: Sorry to be monopolizing 8 your time. Just quickly, do you understand that HFE 9 analysis elements of NUREG-0711 to be prerequisites to 10 the human system integrate design element? And then 11 also to the MSV components or do you think that MSV 12 components can happen prior to the completion of HFE 13 analysis, you know, like its function analysis, task 14 analysis, and all those other elements?
15 And do you think is there a precedence of 16 going through the HFE analysis as the design of the 17 project matures or is it something that should be 18 locked down before any kind of MSV element is 19 attempted?
20 MR. GREEN: That's a great question. I'm 21 really happy you brought that one up. I probably 22 should have put that on the slides although I didn't 23 think of it. So this is how we've handled this in the 24 past because this is meant to be an iterative process.
25 So, for instance, let's say you're going through your NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
172 1 design process and you know, you identify some issue 2 with the way the operator is interacting with the 3 automation. You may need to go back and revisit the 4 function allocation and make changes to it so -- or 5 perhaps the task analysis changes or the HSI design.
6 All of these things have the potential to circle back, 7 if and when issues are found.
8 So how we have treated this in the past is 9 not that you necessarily need to lock down those 10 analyses and have them be complete and never touch 11 them again, but they need to be relatively 12 sophisticated and you should have at least done a 13 first pass on the task analysis and these other 14 analyses. If you need to go back and revisit them, by 15 all means. I think that's the expectation that you 16 will, but it's not -- you don't tie up all around it 17 and never come back to it.
18 Now that being said, you know, you can 19 especially with a compressed scheduled, that may make 20 it tempting to start some of the design work very 21 early and that may cause you problems, but there's 22 nothing that says you can't do these in parallel.
23 However, the MSV guidelines do say that those 24 validation activities should happen in discrete 25 stages. So you would want to propose how you were NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
173 1 going to do that, you know, and how you were going to 2 control for that and there are multiple ways in which 3 that could be done.
4 But I want to let Dave jump in here in 5 case he sees that any differently than I do, but I 6 think that's a great question.
7 MR. DESAULNIERS: No, Brian. I think you 8 answered it well in that you commented on the analysis 9 stages are not necessarily prerequisite to the design.
10 And there's an iterative process here where there will 11 be naturally feedback between the design and analysis.
12 And the other part of the question was with respect to 13 multi-stage validation. I think the question was does 14 analysis need to precede multi-stage validation? Did 15 I hear the question correctly?
16 MR. NOWAKOWSKI: That is correct.
17 MR. DESAULNIERS: And no, that would not 18 be the case. The general guidance for multi-stage 19 validation is to start your validation work as early 20 as possible such that again, you know, you could 21 consider doing validations of conceptual design in 22 which case you're clearly still in the analysis 23 process. And so we see these processes, the design 24 development and multi-stage validation as processes 25 that go hand in hand and should be complementary.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
174 1 MR. NOWAKOWSKI: Okay.
2 MR. DESAULNIERS: I'd only note that I 3 will agree with Brian that the strict definition of 4 multi-stage validation is taking it from early in the 5 process and doing it in discrete stages. The bottom 6 line criteria that I think is important to our 7 discussions today is keeping in mind the notion that 8 these validation efforts could use some performance-9 based information that can support the final 10 validation conclusions.
11 So there's quite a bit of flexibility 12 there in terms of how it's constructed for a 13 particular design modification or new design. So I 14 wouldn't see that as necessarily being a strict 15 constraint if that's a source of concern.
16 I did want to take -- I'll stop, pause 17 there. I wanted to make sure I answered your question 18 and then I wanted to go back and modify an answer I 19 provided previously to whomever asked the question 20 about the definitions of validation.
21 MR. NOWAKOWSKI: Yes, that was me.
22 MR. DESAULNIERS: That was you, okay. Did 23 I answer your question here about MSV?
24 MR. NOWAKOWSKI: Yes. Thank you, 25 appreciate it.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
175 1 MR. DESAULNIERS: And going back to the 2 question on validation, I did pull up the two 3 documents to see where there might have been some 4 point of confusion. And I just wanted to clarify that 5 under 0711 we provide definitions for both validation 6 and integrated system validation. And so right there, 7 that should show that we consider those as two 8 separate concepts.
9 I think where the alignment is is that 10 IEEE defines validation. It does use different 11 terminology for validation. It does specifically 12 refer to performance-based testing and to various 13 aspects of a design. I think, in fact, NUREG-0711 14 definition of validation is a little bit more open.
15 So just be sure you're comparing the two different 16 definitions of validation right now. For NRC 17 purposes, we use the 0711 definition, although I don't 18 think in a practical perspective that there's much 19 difference between the two definitions.
20 MR. JAIN: Okay, we have a question from 21 David Hooten.
22 David, you go ahead.
23 MR. HOOTEN: Yes, thank you. When the 24 industry was working with the I&C Branch staff to 25 develop ISG-6 Rev. 2, one of the ideas that was NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
176 1 discussed related to what level of detail needs to be 2 achieved in the conceptual design to support the LAR 3 submittal. Was this idea of the -- the conceptual 4 design needed to be bounding relative to the future 5 detail design. In other words, whatever was done in 6 conceptual design, the detail design, as long as it 7 stayed within the boundaries that were defined by 8 conceptual design, that that would be okay.
9 Does the Human Factors Branch view that 10 similarly relative to the HSI design that you would do 11 the conceptual design of the HSI to be bounding 12 relative to the detail design phase activities?
13 MR. GREEN: Let me think on that before I 14 respond here. My initial thought is that with an MSV-15 type approach that, you know, if you were to have a 16 conceptual design and work through the design process 17 and find yourself in a place where that design has 18 evolved to something different, you know, if you were 19 following the data on that that would seem to be the 20 reasonable and prudent path forward there, but I'd 21 have to think a little bit more on that. I'm not sure 22 how that's all necessarily used within the ARP. I'm 23 not sure if that's creating an inconsistency there.
24 But I believe the MSV standard does say 25 that if you find yourself in that place where you're NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
177 1 moving away from those concepts that you know, you 2 would measure that and take -- you would try to -- I 3 don't know if you necessarily go backwards, that's 4 probably the wrong way to say it, but you would 5 reevaluate where you are with the changes to the 6 design.
7 MR. DESAULNIERS: Brian, I'll just add on 8 to I think the notion that you were trying to capture 9 there. I think it's an interesting question in terms 10 of starting with a conceptual design. I think what's 11 underlying that question is would a conceptual design 12 be sufficient for the staff to be able to complete its 13 HFE review is what I'm reading into the question. And 14 so maybe I'll pause there and make sure that I'm 15 answering the right question.
16 MR. HOOTEN: Well, the idea with the I &
17 C Branch folks was that a LAR could be submitted based 18 on the completion of conceptual design, but not the 19 completion of detail design. So the idea was that if 20 during detail design you change the design to the 21 point that it was no longer bounded by the conceptual 22 design, then you would need to either alter the LAR, 23 pull it back and resubmit it, or possibly do a 50.59 24 on it, whatever the various regulatory options might 25 be there. But I guess I just wanted to see if the NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
178 1 human factors folks would do things the same way as 2 the I & C Branch folks along those lines and I guess 3 so far I've heard we're not sure. We have to think 4 about it, maybe.
5 MR. GREEN: I guess if I could -- it seems 6 to me that the conceptual design is kind of a 7 different place than where we are or at least as I've 8 conceived on it thus far and maybe there's room for 9 different opinions here.
10 But the conceptual design happens at a 11 very early stage and our thoughts are that the MSV 12 would have some sort of point later on into that, into 13 the detail design portion. Perhaps the detail design 14 is maybe not -- not 100 percent complete at that point 15 which is the benefit of the MSV approach is that we'd 16 be getting some preliminary validation prior to the 17 ISV. But if you were to get to the ISV and find that 18 you had significant changes from the final MSV stage, 19 then I think that some -- the change process would be 20 the right way to look at it to see if there's a 21 significant difference.
22 One part of the human factors process that 23 gives us some insight on this, although it might read 24 a little funny in the MSV world is the design 25 implementation program activity. It does have a NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
179 1 change control process built into it that is meant to 2 capture any changes to the design that happened after 3 ISV, but before the design is implemented. So there 4 are a couple of criteria in there that help evaluate 5 that. And I think maybe the way to conceive of that 6 is to open up the scope between -- instead of marking 7 ISV at the entry point into that, that the final MSV 8 point would be the entry point.
9 MR. DESAULNIERS: And Brian, previously 10 you started alluding to a process with an MSV. One of 11 the criteria that's in the IEEE guidance essentially 12 speaks to if your early testing of a design results in 13 changes to the design, subsequent testing has to take 14 a look at have you changed the design to the point 15 where you've basically invalidated the earlier 16 testing?
17 And so, I guess that's the closest I can 18 think about how we've thought about whether you've 19 somehow gone outside the bounds of the conceptual 20 design. We've thought about it in terms of, not the 21 conceptual design as the boundary, but the extent to 22 which you can credit the testing, and whether you've 23 changed the design in such a way that, as I said, the 24 prior testing done would no longer be able to be 25 credited as a valid test of that design.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
180 1 MR. JAIN: Any more questions for the 2 staff on HFE? If not, then I would once again wave to 3 NEI to make the presentation on HFE.
4 MS. GOLUB: Yeah, thank you, I appreciate 5 that. So, first, again, thank you to the staff for 6 giving us the opportunity to participate in this 7 workshop. As you can tell from the questions, and all 8 of the interest, this is a big topic for industry, 9 especially for some of the earlier adopters of the 10 alternate review process trying to get protection 11 systems, do digital upgrades for their systems.
12 So, very important, I appreciate the staff 13 taking the time to really explain in that last 14 presentation as much as they did. Next slide please.
15 We're going to go through these slides, and actually 16 a number of the topics we've already started 17 discussing, so that's kind of perfect. We created 18 these slides prior to seeing the NRC slides, so while 19 this presentation follows the NRC's, they're not in 20 response to the NRC's.
21 They were really done independently just 22 based on industry's first attempt to try to get their 23 arms around this issue. So, just a little bit of 24 background, and I think we've covered some of this, 25 but NEI members acknowledge that when doing the NRC's NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
181 1 integrated action plan, modernization plans, in 2 particular MP4A, when that work was going on in 2017, 3 there wasn't a lot of discussion on the part of NEI 4 members regarding significant main control room 5 modernizations.
6 Those discussions primarily focused on 7 digital I&C, and we're with the digital I&C branch of 8 the NRC, so we do recognize that that was the focus 9 area of industry at the time, and the staff was very 10 responsive to industry input, and industry concerns, 11 and we appreciate that. However, as time has moved 12 on, five years later, and these larger modifications 13 are being scoped, and being undertaken, there's also 14 recognition that to continue to improve plant safety, 15 reliability, we do need to address the main control 16 room, that modernization is needed there as well.
17 And so while we didn't think about it as 18 much in 2017, or focus on it as much in 2017, we are 19 where we are, and we do need to make sure that we're 20 addressing equipment reliability, and other concerns 21 in that area as well, again with a focus on continuing 22 to improve safety. A key industry objective for the 23 industry in providing input to the staff for MP4A was 24 about earlier issuance of license amendments.
25 This was really to support overall NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
182 1 reduction of project risk for industry in terms of 2 getting these larger modifications underway, and 3 making sure that we keep that budget risk in a place 4 that was acceptable. And the NRC was very responsive 5 to that. And so we're hoping in that same light of, 6 again, trying to improve safety, and trying to manage 7 risk, that we can find a solution for this HFE, this 8 ISV issue as well.
9 Next slide please. Okay, so again, this 10 is just a little more background. So, under the 11 alternate review process, the licensee is submitting 12 design information earlier in the project life cycle.
13 We recognize that the LAR includes an NRC approved 14 software life cycle process, or information provided 15 in the LAR that describes life cycle activities, 16 there's allowances to that.
17 The pilot LAR relied on the vendor 18 oversight program, and regulatory commitment for some 19 of those later life cycle activities, so there was 20 acknowledgment within ISG6, and this has to do with 21 the question Dave Hooten asked as well. Dave was part 22 of those discussions under MP4A about making, about 23 providing staff information that was bounding, and 24 then making sure as the detailed design was being 25 undertaken, that the design was either staying within NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
183 1 those bounds, or if they were changes, those were 2 evaluated, and communicated.
3 We're hoping that the same process can 4 apply here in the world of HFE, and that if there are 5 more significant design changes which impact the 6 information that was submitted, or even information 7 that was part of the NRC's SER, so the approval of the 8 LAR, that the same process we would apply for the 9 design information for digital I&C, would also apply 10 for HFE.
11 So, we're really looking at it not as two 12 discrete areas, but for industry, it's all one 13 project, it's all one system. So, we would view them 14 in a similar lens. And just when we were going 15 through the MP4A work with the staff, providing input 16 to the staff, this was a caution that the NRC did 17 raise. That you're providing this information, this 18 bounding, if it changes, you need to be aware that 19 there could be an impact.
20 So, I think that's a risk that industry 21 has already acknowledged is a possibility, whether 22 it's in digital I&C, or in the HFE area. The next 23 major bullet is on -- we recognize of course the NRC 24 intends to use NUREG-0711 for the review of 25 information, and this kind of ties into the question NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
184 1 that Rick Paese asked about what is the underlying 2 regulatory requirement?
3 And so, in a previous public meeting, we 4 thought we had heard the NRC say that the underlying 5 regulation is 50.9 for completeness, and accuracy, and 6 that's why that bullet is on there, because we thought 7 we had heard that in a previous meeting. Brian, I see 8 you have your hand up, do you want to ask a question 9 right now, or?
10 MR. GREEN: I thought you were looking for 11 a response on that, because I saw that in the slide.
12 I can hold the question if you want.
13 MS. GOLUB: No, that's okay, if you have 14 a response, go ahead.
15 MR. GREEN: Yeah, so we did discuss 50.9 16 at a recent meeting, and that was in response to there 17 seemed to be some confusion about the implementation 18 plans, and when they get submitted. So, that was how 19 we have treated it before, is that in order to submit 20 an LAR, an LAR needs to be complete when it's 21 submitted. And one way that we have interpreted that 22 is either with an IP, or an RSR at the time that it 23 comes in.
24 So, that's the basis for why it would be 25 a part of the application, because in that particular NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
185 1 case, there was thought that the IPs would not be 2 submitted, and they would be audited in an ERR at some 3 other point. And that begged the question on if we 4 would be able to accept the application when it came 5 in for the acceptance review. So, that was where the 6 50.9 discussion came in, and I see how that would be 7 a confusion.
8 So, that's more of an administrative 9 requirement than some stand up to the argument.
10 MS. GOLUB: Yes, okay, and that's helpful 11 Brian, I appreciate that, because I guess we were --
12 and that's why Rick asked the question as well, just 13 struggling a little bit with what is the underlying 14 regulation, because as we try to find a path forward, 15 it's always helpful to know what target you're 16 shooting at. So, okay, that's helpful. The next item 17 is, and this kind of ties back to what Dave Hooten was 18 referring to, where we were trying to tie the content 19 we provide in the area of HFE to a similar place for 20 what we're providing in the area of digital I&C.
21 Again, from an industry perspective, these 22 are not separate areas. HFE, digital I&C, these are 23 really all the same projects, same system. And so we 24 were trying to draw kind of a similar place, and 25 similar line in the sand perhaps, that we provide this NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
186 1 type of information in the area of digital I&C, of 2 guided information. We'd like to provide something 3 similar for HFE.
4 And so these items here tie to the 5 framework in the EPRI Digital Engineering Guide, which 6 as you know, the bulk of the industry is using as the 7 design framework. And under that framework, these 8 activities already experience functional analysis, 9 task analysis, staffing qualifications tied to this 10 same area that the conceptual design information, 11 requirements information, which forms the basis for 12 the design information provided with the LAR edits.
13 And so that's why these activities were 14 chosen here. And then similar, again to the way the 15 I&C is handled, for later HFE life cycle activities, 16 the LAR would describe that process for completion of 17 those activities. So, there could be implementation 18 plan type information, program plan type information, 19 that's what we were looking for. And then the NRC 20 could decide whether that was acceptable, how we were 21 handling those later life cycle activities.
22 Okay, next slide. Okay, topic of the 23 hour2.662037e-4 days <br />0.00639 hours <br />3.80291e-5 weeks <br />8.7515e-6 months <br />. So, for the integrated system validation, first 24 NEI members are concern that the MSV approach provided 25 by the staff doesn't really have a clear precedent, NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
187 1 and it's not clear to us exactly how it fits into the 2 regulatory framework. Certainly appreciate, Brian, 3 especially the presentation you made, it's clear that 4 the NRC is also looking for a solution to try to 5 preserve maybe some of the objectives that were 6 achieved as part of the alternate review process.
7 And we appreciate that, really appreciate 8 the fact that the NRC is also offering solutions to 9 industry, not just kind of saying it's your problem, 10 figure it out. So, appreciate that. Having said 11 that, MSV is difficult, especially listening to all of 12 the different criteria at the different stages, it's 13 unclear how that would work. Now, you're going to 14 hear in the presentation that follows my few slides, 15 Idaho National Labs has kindly supported this call.
16 Supported providing some results from 17 their research in this area in determining what leads 18 to a successful integrated system validation. And I 19 will say while the terminology is different, multi 20 stage validation, from some of the terms that you're 21 going to hear in the INL presentation, I can't help 22 but feel there's a lot of similarities in the two 23 approaches.
24 And one other thing that's kind of become 25 clear to me through talking to both I&C people, and NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
188 1 HFE people is that the term validation may be used 2 differently between these two sets of technical 3 people. And so that could also be a source of maybe 4 -- I don't want to say some confusion, but perhaps a 5 perception that the two approaches are not as aligned 6 as they are.
7 And so I ask the staff, when you hear the 8 INL presentation, I don't know, maybe look at it from 9 the lens that it may be closer to the approach that 10 the staff has offered, than maybe the initial glance 11 would tell. So, I'm looking forward to staff's 12 feedback from that. And so one option for 13 consideration, since the ISV is such a critical 14 activity, and as has been acknowledged, the RSRs for 15 all, or some of the earlier activities will likely not 16 fit within that window of the license amendment review 17 approval requested by industry in accordance with the 18 alternate review process.
19 And I recognize that's a staff decision to 20 determine what that schedule really looks like. But 21 understanding that those RSRs may not fit within that 22 category, and that ISV is so important, one option 23 that industry is kind of offering is potentially 24 including a licensed condition that the ISV will be 25 completed in accordance with the process laid out in NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
189 1 the LAR.
2 So, trying to make it clear that it's not 3 that industry does not intend to do the ISV, an ISV is 4 part of these projects. But simply that the time 5 lines may not align with what was discussed, or what 6 was included for the alternate review process in ISG6.
7 We do want to acknowledge that we do understand that 8 the NRC's safety determination cannot depend on a 9 license condition.
10 That unlike say ITAAC, or DAC, or some of 11 the regulatory mechanisms in Part 52, a license 12 condition is different. But the NRC does have to 13 conclude in their SER that adequate safety has been 14 achieved. And so, we want to be clear that we're not 15 proposing that the safety determination depend on the 16 license condition. But simply that it's one way to 17 show that industry is going to complete this activity 18 in accordance with the process described.
19 The strategy for addressing the VNB 20 element will be included in the LAR, as I mentioned, 21 and that's what the INL presentation is also going to 22 cover, what does that strategy look like, and the INL 23 presentation is one attempt to provide that strategy, 24 but an attempt that has behind it, research showing 25 that it would be successful. So, I think that's very NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
190 1 positive.
2 But we're going to keep the NRC apprised 3 of any ongoing activity, so recognizing again that 4 because there's not finality of all of these 5 activities at the time of LAR submittal, if there are 6 activities that the NRC wants to audit, or even review 7 results of in some kind of an e-room, industry 8 understands that we do need to continue to communicate 9 when these activities are taking place.
10 And make sure we have that good 11 communication so staff can participate, and do their 12 audits, or inspections as the LAR review continues, 13 and some of these other activities are continuing, or 14 even ones outside of that LAR review scope. And then 15 the last bullet here, that we request that the NRC 16 consider making their safety determination for these 17 alternate review process LARs based on the early HFE 18 activity.
19 And then the plan based on INL research, 20 which describes how completing these activities, 21 performing these activities could lead to a successful 22 integrated system validation. Okay, next slide 23 please. Okay, so this is a little bit of kind of a 24 summary. So, the LAR, and I put 50.9, because just 25 from our earlier discussion we were trying to pick out NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
191 1 what the underlying regulation is.
2 So, we could show the staff that we 3 thought we were meeting that underlying regulation, so 4 that's why 50.9 is here. It would include these early 5 HFE activity IPs, RSRs, or similar information. It 6 would include the program plan based on INL research 7 describing that the rate of HFE activities would 8 assure a successful integrated system validation. It 9 would include a license condition to perform this ISV 10 in accordance with the LAR.
11 We note that the implementation plan for 12 the ISV would be available for audit prior to the 13 execution of ISV, industry of course would perform the 14 ISV to close the license condition, and that the 15 result summary report for ISV would be available for 16 inspection. So, that's just a little bit of a summary 17 for those activities. Next slide. Is that the end of 18 mine?
19 Yeah, okay. And then, so we could pause 20 here, or we could, maybe I would suggest we have 21 Jeffrey, Joe, and Ron Boring go through their 22 presentation first if it's okay, BP, and then take 23 questions from the staff. How does that sound to you 24 guys?
25 MR. JAIN: Sure, we can do that.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
192 1 MS. GOLUB: Okay, all right, thank you.
2 And I'm going to turn this over to INL.
3 MR. JOE: Okay, this is Jeffrey Joe, can 4 you hear me?
5 MS. GOLUB: Yes.
6 MR. JOE: Okay, great. Yeah, so I am a 7 senior human factors researcher at Idaho National 8 Laboratory. I've been at INL for just about over 22 9 years now. I also want to thank NEI, and the NRC for 10 the opportunity to talk about the human factors 11 engineering, or HFE. R&D that my colleagues, and I 12 have been doing at INL for the Department of Energy's 13 light water reactor sustainability, or LWRS program.
14 Next slide please.
15 Yeah, so the LWRS program conducts 16 research to develop technologies, and other solutions 17 to extend the operation of our nation's fleet of 18 nuclear power plants primarily by improving their 19 economic competitiveness. The part of this though, is 20 that this is the programmatic context that undergirds 21 both what Ron Boring, and I have to say in the next 22 few slides. And so with that, I'm going to hand it 23 over to Ron, and have him introduce himself.
24 MR. BORING: Thank you Jeffrey, can 25 everybody hear me okay?
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
193 1 MR. GREEN: We can hear you.
2 MR. JOE: Yes.
3 MR. BORING: Excellent. If we could 4 proceed to the next slide then? My name is Ron 5 Boring, so I am the manager of the Human Factors and 6 Reliability Department at INL, and I was heavily 7 involved in the LWRS program in some of the earlier 8 stages, primarily 2012 through 2018 where we were 9 doing a lot of this foundational control, and 10 modernization work.
11 Most of this work has to be understood in 12 the context of different pathways of LWRS, and this 13 work falls under the plant modernization pathway, 14 which has a focus really to engage human factors 15 researchers to support those types of I&C control room 16 upgrades that we're talking about in this general 17 meeting. So, really we're looking to do modernization 18 that obviously is technically sound, that supports 19 regulatory requirements, and is safety focused.
20 And really what we were doing in this 21 process is trying to help utilities to have a roadmap 22 for how to go about these activities. And you can see 23 there on the right, we have this figure that depicts 24 the four quadrants, where we're really taking it from 25 initial engagement with utilities to find out their NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
194 1 needs, all the way through deploying it as a control 2 room modernization activity.
3 And along the way, we were working on 4 developing principles, many of those, which are found 5 in the DHG that EPRI released to back end state of the 6 art guidance specifically on control room 7 modernization. Collaborating with industries, and 8 that means many partners, owners, operators, and 9 suppliers. And more importantly then, disseminating 10 those lessons learned back to industry, and the 11 regulators.
12 So, we would work with particular industry 13 partners to develop the template for how we would go 14 about control, and modernization, and then we would 15 publish guidance on that to make sure that that would 16 be a benefit to everyone. Next slide please. And so, 17 really this was, if you go back to 2012 as we were 18 kicking these activities up, this was a chance to 19 address some of the technological obsolescence issues 20 that we were experiencing in the industry 21 specifically.
22 As we extended the life of reactors, and 23 had a need to come up with upgrades for the then, 24 mainly analog systems. Along the line of course, 25 there was the opportunity to do more than for life.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
195 1 So, there's the opportunity to figure out what, with 2 the advent of digital technologies, what could be 3 introduced that might actually be a benefit towards 4 efficiency of the plants for example?
5 And so the industry in particular 6 expressed a need for human factor support for 7 modernization, and so we developed guidance on 8 applying NUREG-0711. We captured the experience on 9 how to incorporate human factors in design, and then 10 kind of one of our capstone activities, you can see 11 there in our photograph, was the development of the 12 human systems simulation lab, which is really a test 13 bed for design, and evaluation.
14 It's an engineering simulator that can be 15 tied reconfigurably to the existing plant simulators, 16 but gives plants an opportunity to stage those types 17 of validation exercises, design validation exercises.
18 And then next slide. And so, Jeffrey, and I are going 19 to talk about a few of the examples, just very high 20 level to captures -- apologies, it was very quiet on 21 the street before I started speaking.
22 So, Jeffrey, and I are going to talk about 23 some of the examples of these activities. And really 24 what you can see here, you see our steadfast NUREG-25 0711 in the top right, and we developed some tools to NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
196 1 align design to ensure that it was meeting NUREG-0711 2 guidance. And so there was a lot of emphasis in here 3 on how to go about doing those evaluations.
4 And in particular then, we found as we 5 were doing those early stage evaluations on conceptual 6 designs, a lot of the information was qualitative.
7 This is not your traditional thing that makes its way 8 into a LAR. It's qualitative information that helps 9 to inform the design of the system, and then as you 10 move along in the 0711 stages, which are on the upper 11 right axis there, you can see that you get to that 12 point of ISV, at which point you're looking at 13 confirming the design.
14 And really what we found was this high 15 level of confidence, as we did those early evaluation, 16 and design activities, we would go into ISV with high 17 confidence, because we had already tested all of the 18 elements, and refined them. And with that, I believe 19 that's my last slide Jeffrey, so I turn it over to you 20 to continue the story.
21 MR. JOE: Yeah, thanks, we can go to the 22 next slide please. So, Ron just described the 23 iterative design evaluation processes that we created, 24 these design processes align with the NUREG-0711 25 phases. This side is a continuation of what Ron was NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
197 1 describing before. This slide just shows some 2 examples of R&D activities that we have done in that 3 context.
4 So, the top picture shows ergonomic, and 5 HFE analysis that we performed of I&C hardware, and 6 its human system interface, or HSI to help identify 7 the anthropometric, and other design issues. And so 8 by improving the ergonomics, and human factors of the 9 HSIs, we were helping improve the performance of the 10 new digital control system, or DCS, digital control 11 system.
12 So, we're improving the performance of the 13 DCS, and the overall performance of the plant. The 14 two pictures on the bottom half of this slide show how 15 we conduct operator workshops to iteratively evaluate 16 the HSI design, and other HFE elements of the DCS.
17 So, the photo on the left shows a tabletop review of 18 the HSIs by operators, other plant personnel, HFE 19 experts, and the DCS vendor.
20 And so this is a static evaluation on the 21 screen, something that we can do earlier in the 22 upgrade process, rather than later of course. But the 23 photo on the right shows how we developed a functional 24 prototype of the upgraded control system, which we 25 then iteratively evaluated with operators in the loop NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
198 1 as they dynamically interact with the system on the 2 full scale simulator that Ron mentioned in our human 3 systems simulation laboratory.
4 So, these dynamic operator in the loop 5 workshops give the operators, and give the human 6 factors engineers, the evaluators, an opportunity to 7 evaluate, and also work with the new DCS currently in 8 the upgrade process. And this, as Ron mentioned, 9 provides the opportunity to modify the design, and 10 further improve it. And in general, improve the plant 11 performance prior to implementation.
12 And so these are the things that we can do 13 that can be thought of as a lead in to the ISV aspects 14 that was also a point that Ron had made. So, next 15 slide please. So, this slide is perhaps a little bit 16 redundant, but this slide shows where the HFE R&D 17 activities that we have done under the LWRS program, 18 how those activities fit within the existing HFE 19 phases, and elements of 0711.
20 So, in this will be the items in black are 21 the HFE elements of up to NUREG-0711, they should be 22 of course very familiar. But of course the way these 23 elements are listed in this table is a little 24 different than how they're typically portrayed, but 25 all of the elements of 0711 are shown in black. The NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
199 1 blue items are what we have done, and continue to do 2 in our research to enhance human centered design.
3 These items are somewhat duplicative of 4 the examples I showed on the previous slide, so I 5 won't repeat them in detail. What I'm pointing out 6 here though is INL researchers have adopted an 0711 7 framework, and our collaborations with our utility 8 partners, we've done all these elements, but in 9 addition to that we've done some other things that 10 we've found to be very helpful.
11 So, for example, the 3D modeling, the 12 development of prototype HSIs, these activities have 13 proven to be beneficial from our view in the research 14 to improve the overall HFE effort, and contributions.
15 So, combined, these activities provide a staged, or 16 iterative approach that helps provide confidence in 17 the HFE aspects of the final design. Next slide 18 please.
19 So, that really takes me -- that's really 20 the take home message for me. The LWRS program, its 21 goals, and objectives undergird the work that we've 22 done to research, develop, demonstrate, and deploy HFE 23 methods, and to make HFE design recommendations for 24 all of the industry to use. And I just want to 25 mention there are reports that we have, these NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
200 1 additional -- and additional details on what Ron, and 2 I have presented can be found in these technical 3 reports that are publicly available.
4 There's actually a website address cited 5 on the next slide where you can find these reports.
6 And these reports show how collectively the HFE 7 methods, and recommendations address all elements of 8 NUREG-0711, and increase the confidence in the final 9 design of the upgrade control room. And that's really 10 where I wanted to stop. There are some other slides 11 where we provide some information. But I think the 12 plan at this point was to open this up for questions.
13 MR. JAIN: So, there are questions for 14 NEI's presentation?
15 MR. GREEN: Yeah, this is Brian, sorry, I 16 was just finishing taking a note here. Yeah, if you 17 wouldn't mind Jeffrey, do you mind going back to, I 18 think it was one of the slides that Ron had spoke to, 19 the formative, summative one. Next slide actually.
20 Yeah, I like this slide a lot, and I find this to be 21 very helpful, because I like the evaluation continuum 22 that you have here between the formative, and 23 summative.
24 And I think it shows, what we're trying to 25 find with the MSV suggestion is some appropriate NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
201 1 checkpoint within the summative activities to get the 2 confidence -- I think we agree that together, both the 3 formative, and the summative process give us high 4 confidence in the results, and that's why 0711 is 5 structured the way that it is. We're just trying to 6 find that right checkpoint to get some of the 7 summative information for us to consider.
8 And keep it so that we don't just have 9 formative information. And I was wondering if you 10 could speak to -- maybe on the next slide, if there 11 are places in the next table -- I'm sorry, one more 12 slide, the one with the blue, and the black. Are 13 there activities on this table that might provide us 14 with opportunity for the NRC to observe, or see some 15 of this?
16 My initial thought was maybe those initial 17 prototypes, or the iterative evaluations of the I&C 18 seem really reasonable to me. But I don't know if 19 those are within the time frames, or if we have any 20 thoughts on that. Is there a good checkpoint from 21 your perspective?
22 MR. JOE: Yeah, Brian this is Jeffrey. I 23 can certainly give an initial answer, I would also 24 invite Ron Boring to also jump in as well. So, I 25 understand that the question is where in these NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
202 1 additional R&D activities might there be good 2 checkpoints for the NRC? Off the top of my head, it's 3 a good question, and I'm not sure -- I'm not as 4 familiar with what a licensee would need to do, just 5 because I don't work for a licensee.
6 But I would say that as researchers, for 7 every activity that we do, we always want to write a 8 technical report, and that has always been the goal of 9 the LWRS program, is to research these activities, and 10 then write a report that not only describes what we 11 did, but reports out on the findings of that 12 particular activity. I would think offhand, that 13 those technical reports could be a way for the NRC to 14 accomplish this checkpoint thing that you're 15 requesting.
16 But that's just what came to my head, Ron, 17 do you have anything that you want to say in response 18 to that?
19 MR. BORING: Well, it would be up to the 20 individual utilities how they want to do that. Most 21 of the blue here is still at some phase of the 22 formative. So, you can see things like the ergonomic 23 analysis is probably at the planning, and analysis 24 stage, right? And that would obviously have 25 documented artifacts coming from that, initial NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
203 1 prototypes, you can see how that moves from the 2 conceptual getting it very close to that finalized 3 design.
4 So, again, I would refer to Pareez, or NEI 5 as to when might the appropriate checkpoints be. When 6 we've done this more generally, of course we publish 7 reports at each stage, and part of that dissemination 8 was to get the ideas out there, and see are we 9 capturing the right information? So, from my 10 perspective, it would always be welcome, but utilities 11 would have to figure out when the best intervals are.
12 Pareez, any comments on that?
13 MS. GOLUB: Yeah Ron, and I think that 14 makes sense. Certainly as we go through this process 15 with the NRC, we can talk about what those checkpoints 16 are, and determine the right ones. And that was part 17 of what I was saying earlier about making sure we have 18 good communication as we go through this, to make sure 19 that we provide that confidence to the staff.
20 MR. GREEN: Yeah, I think, and I'm just 21 kind of working through these for the first time here, 22 but my initial thought is that if we were able to 23 identify a checkpoint, or two, and that may be, for 24 the way we had described it earlier, that may be the 25 early MSV stage. I mean these may not be terribly NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
204 1 different in concept. So, there may be some room to 2 align there, but I'd like to -- maybe that's a 3 discussion for another time.
4 But if we could find a couple of those, I 5 think Dave may have thoughts on it as well, but that's 6 my initial impression.
7 MR. JAIN: There are three questions here.
8 One, I'll have Bill, you go ahead with your question.
9 MR. HANNAMAN: If you can hear me, this is 10 Bill Hannaman, and I'm with Paragon. I thought this 11 was a very good presentation, and it brought to mind 12 -- this may be a little bit redundant, but you said 13 that you would produce research reports when you did 14 an activity. Would that research report fall into the 15 category of a validation?
16 That's where I think you can do a lot of 17 early stuff that would fit into the MSV approach. But 18 you would call, instead of a research report, can you 19 put the term validation on it?
20 MR. JOE: This is Jeffrey, I'll jump in, 21 if anyone thinks that question was directed to them as 22 well, please speak up. I think of this as a 23 researcher, and I think of these as potential 24 artifacts for referenceable, technical documents that 25 could be included in whatever additional documentation NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
205 1 that might need to be provided to the NRC. So, for 2 example, little that I know about license amendment 3 requests, there's a license application that a 4 licensee would submit.
5 These technical reports, along with either 6 results on reports, or implementation plans could be 7 referenceable artifacts. But that would again, as Ron 8 had mentioned, that would be up to the licensee to 9 make that call as to what it would be. I actually 10 can't really speak to whether, or not we would call 11 these research reports validation reports, or not.
12 I don't think I am versed enough in the 13 nuanced differences in terms of how one discipline of 14 engineering might define validation, versus how it's 15 defined in 2411, versus how it might be defined in 16 0711, or just at large in human factors. That's all 17 I needed to say, anybody else on the human factors 18 side want to jump in? Yeah, go ahead Pareez.
19 MS. GOLUB: So, I probably am not on the 20 human factors side, but I guess I just wanted to go 21 back to something I had mentioned in my presentation, 22 which is that I think that there's a lot in this 23 approach that's being put forward by INL, and industry 24 of course, that isn't that far different than the MSV 25 stuff. Even -- like letting go for a moment on the NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
206 1 exact terminology of validation.
2 But more on this idea that there are these 3 earlier insights that show that this approach would be 4 successful. And so I think if we can try to come to 5 some alignment for these different licensees on what 6 this approach would be, and describe it appropriately 7 in the LAR, that's why I'm hoping there is a success 8 path. The terminology is only one aspect, but I know 9 it's more of a general approach.
10 MR. HANNAMAN: Thank you. I think that 11 gives a broad picture of a kind of report that fits 12 into the MSV framework. So, I like the ideas.
13 MR. JAIN: Okay, so the next question is 14 from David. David, do you want to go ahead?
15 MR. DESAULNIERS: Thanks BP, actually it's 16 not a question, and the time is perfect.
17 MR. JAIN: A comment?
18 MR. DESAULNIERS: Yes, it's a comment, and 19 it follows directly on this conversation here, and the 20 comments of Pareez, and Bill. With respect to where 21 we see commonality between what INL is proposing as an 22 HFE program, and what might fit into multi stage 23 validation. So, there certainly is commonality in 24 terms of types of activities. There are differences 25 though, and Bill, you asked a good question.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
207 1 Could you call these validation? What 2 I'll try to summarize briefly here, but I'll point the 3 audience to as a resource to consider that question, 4 is that was an important aspect of what we considered 5 when we were working on the NEA effort to further 6 develop the MSV concept, and tried to distinguish 7 multi stage validation from some similar, but 8 competing concepts.
9 Because in developing multi stage 10 validation, we often went back to this discussion of 11 well, what in fact is the difference between what we 12 are proposing to develop in terms of multi stage 13 validation, and what you might call a design test in 14 the general sense. And I believe that what you have 15 here on the screen, and prior to getting into VNB is 16 design testing.
17 And if you go to the NEA report, 18 specifically section 2.4, there is a section 19 specifically dedicated to describing the relationship 20 between multi stage validation, and design testing.
21 They are different concepts, but complementary. And 22 the distinction gets into largely the controls that 23 are put in place on the nature of the testing, and the 24 objectives of the testing.
25 Design testing has some very broad -- can NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
208 1 have many different types of objectives. As for an 2 example, you might be looking at comparing concept A 3 against concept B, and trying to decide which is the 4 more effective design to use for your particular 5 application. Whereas invalidation, really your focus 6 is -- your objective is more focused, and that's 7 determining will this design successfully meet its 8 intended use?
9 And the way you go about the testing may 10 require, in order to have results -- and this is where 11 I'll take a moment to just maybe correct my colleague 12 Brian's -- he actually corrected himself, and I'd say 13 he corrected in the wrong direction relative to how 14 the terms are used say in the IEEE document with the 15 difference between credible, and creditable. What is 16 being produced perhaps in design testing, if done 17 properly, is producing a credible result.
18 But for purposes of having something that 19 we can use for a regulatory decision, what we want is 20 something a little more that's a creditable result, 21 and that means it has the pedigree associated with it 22 in order to be able to support our decision. And that 23 may mean for instance whereas a test under design 24 testing is done by the individuals that are directly 25 involved in the development of that design under a NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
209 1 validation test, it might be the same test.
2 But it's being done by an independent 3 group of individuals developing that test, and 4 evaluating the results, as just one particular 5 difference. I think in the end though, when you 6 appreciate those differences, there are ways to take 7 what is being done by INL, and with appropriate 8 adjustments, elevate design testing to something that 9 would be considered a validation.
10 So, this is where we said -- and Brian 11 alluded to earlier with respect to not trying to add 12 a whole lot more work to it, but looking at what's 13 being done. Does it actually -- what portions of your 14 testing actually support the final conclusion, as will 15 this design support its intended use? Not just 16 improve it in some way. And provide the level of 17 rigor, and documentation to that testing.
18 Such that it can then be part of the basis 19 for the application. I'll pause there to see if I 20 rambled on too much, and confused everybody, or made 21 some clarification.
22 MR. HANNAMAN: Thank you for that 23 response.
24 MR. JOE: That was helpful to me David, 25 this is Jeffrey, and that's all I'll say, very NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
210 1 helpful, thank you.
2 MR. JAIN: Our next question from Warren.
3 Warren, you want to go ahead with the question?
4 MR. ODESS-GILLETT: Thank you, and I'll 5 lower my hand here. So, this is -- so Pareez, on 6 slide three of the presentation, I just want to sort 7 of simplify in my own head, the proposal. So, 8 basically the idea on -- actually slide three of the 9 NEI presentation, yeah. The idea is that for the 10 operating experience, functional analysis, and 11 allocation, task analysis, staffing qualifications, 12 both an IP, and an RSR would be provided during LAR 13 review?
14 MS. GOLUB: In this case, the RSR would be 15 provided, yes. Yes, I shouldn't speak, so yeah, it's 16 really either one, yeah. The licensee would have to 17 decide, yes.
18 MR. ODESS-GILLETT: Okay, and then 19 afterwards, those later life cycles, if you go to 20 slide -- I'm on the wrong -- let's see.
21 MS. GOLUB: I think it's slide four of the 22 NEI presentation, I think is where Warren is going.
23 Here that bottom bullet Warren, is that where you're 24 going?
25 MR. ODESS-GILLETT: Yes. That the --
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
211 1 actually, it was I think on the next slide. Yes, and 2 the major bullet, and the second sub bullet says the 3 HFE program planned based on INL research describing 4 the later HFE activities assuring a successful ISV.
5 My understand is that what would be submitted would be 6 implementation plans based on the INL research that 7 was presented that have demonstrated that the 8 activities that INL had discovered contributed to a 9 successful ISV.
10 These would be incorporated in 11 implementation plans as part of the -- either as part 12 of the LAR, or LAR review, is that true?
13 MS. GOLUB: Yeah. So, I guess I want to 14 be clear, part of the implementation plan, or that 15 content. Whether it's called an implementation plan, 16 or it's just described in the LAR, but yes. Because 17 the INL research of course has been -- I mean it's 18 been, this is not sort of -- and please, Jeffrey, 19 correct me if I'm wrong, but I mean that research has 20 a basis, it's been proven.
21 There's been -- I think we have good 22 confidence that that methodology leads to a successful 23 ISV. And so that's -- what we're trying to say is 24 that based on that research, not just something that 25 industry is making up in an attempt to find some way NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
212 1 to resolve this issue with the staff, but based on 2 that research, we believe that's the right approach.
3 Did I answer your question?
4 MR. ODESS-GILLETT: It did, thank you.
5 MR. JOE: Pareez, did you want me to jump 6 in on that as well, or?
7 MS. GOLUB: Please do, yeah, I would 8 appreciate that.
9 MR. JOE: I would only speak in general 10 terms. Our research has shown in certain cases steady 11 improvement in the overall design from the conceptual, 12 to detailed, to final implementation, and we have 13 confidence that the research that we do in support of 14 0711 activities, as well as additional things that we 15 have added to the HFE process contribute to that 16 steady improvement.
17 I don't want to get into any specific 18 results, or describe any detailed methodologies in 19 detail. I would just only comment that I think it's 20 a fairly common anecdote in the industry where some 21 upgrades are implemented, and operations, and human 22 factors are only brought in at the very end, when the 23 plant is going into -- or the simulator is being 24 modified.
25 And the common finding from that approach NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
213 1 is that that's too late to get operator input, and 2 oftentimes when the operators, who are quite frankly 3 the end users of these new systems, when they want to 4 make design change recommendations, it's too late in 5 the engineering process, it becomes very expensive, 6 and then we find that we have to solve those problems 7 in other ways.
8 And what I think our research generally 9 points to is that we have a different way that gets 10 around that, I think particularly common anecdotal 11 finding.
12 MS. GOLUB: Yeah, and thank you Jeffrey, 13 I appreciate you clarifying. Because I don't want to 14 draw conclusions from the research that aren't there.
15 But I also just want to point out this slide is kind 16 of perfect. It's certainly not that industry is 17 proposing to do the ISV. This is simply trying to 18 provide the staff confidence that we're doing 19 everything that needs to be done to make sure it's 20 successful, so that the license amendment review can 21 conclude without the ISV results being necessary.
22 And that's exactly why a license condition 23 is being offered in addition, for that very reason, 24 yeah.
25 MR. JAIN: Okay, Michael, you have NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
214 1 comments to make? Michael Marshall?
2 MR. MARSHALL: Yes, I had one question, 3 and a comment. And I'll start with the comment first.
4 I think this has been addressed, but I just want to 5 make sure. With regards to 50.9 not being a technical 6 requirement for a licensing decision, and that's just 7 not limited to human factors, it's I&C, and others.
8 50.9 is a pretty broad requirement that governs a lot 9 of correspondence between the NRC, and the licensee, 10 and the documentation they maintain on site.
11 It's not a licensing requirement. But I 12 think Brian addressed that by referring to it at the 13 end as an administrative requirement. But someone 14 else mentioned earlier before, especially for all the 15 plants that were operating at the time of the TMI 16 accident, your human factors requirements, your 17 control room requirements, those are in TMI waters, 18 and those aren't clean in a sense that it's in a 19 single document.
20 Or single piece of paper, or single piece 21 of correspondence. And it would be advisable for 22 licensees to go back, recreate that if they have not 23 done so already, to see what the requirements are for 24 them when it comes to control rooms. I've only done 25 it for a couple of plants, but again, I don't think NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
215 1 anything that we have been discussing with regards to 2 0711 is inconsistent with at least a couple orders.
3 Or the paperwork that comprise those 4 requirements for the two plants I'm familiar with.
5 The actual question I had was with regard to the 6 license condition. I agree with the statement that an 7 MSV approach is untested, but I would say the same 8 would be true with a license condition approach, that 9 would also be untested. Brian mentioned that there 10 are challenges with license conditions.
11 And typically it comes down to how the 12 license conditions are worded, and what exactly are 13 governed by the license conditions. And right now, I 14 don't know if you have specific language you were 15 planning on sharing with us today, but I'm not sure we 16 can give you a lot of meaningful feedback on the 17 concept of a license condition versus the actual 18 requirement as its written.
19 So, I don't know if you guys had thought 20 as far as specifically what's the language that NEI is 21 proposing to put in this license condition, because 22 I'm not sure -- folks here on the NRC can argue both 23 ways, because some of us have seen poorly written 24 license conditions, which would be totally 25 unacceptable. And then there are more better written NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
216 1 license conditions that tip toe that fine line of 2 deferring actions that the staff are required to 3 complete before it makes a regulatory finding.
4 Versus it being a confirmatory action just 5 to ensure what was communicated, and what was reviewed 6 by the staff is actually implemented by the licensee.
7 MS. GOLUB: Yes, and Michael, you're 8 raising a very good point. And to be clear, this is 9 not a small thing industry is offering, right? A 10 license condition is a pretty significant regulatory 11 mechanism. And so yes, the wording of that would 12 definitely need to be something carefully thought 13 through, and probably discussed in presubmittal 14 meetings phase.
15 Similar to what we did for Waterford with 16 the regulatory commitments. There was discussion 17 between that licensee, and the NRC in presubmittal 18 meeting space to make sure that the language was 19 appropriate, that it was achieving that goal. And so, 20 that should be the same case here. If a license 21 condition is necessary, then -- necessary for industry 22 to achieve the goal of getting the early approval in 23 accordance with the alternate review process as we're 24 trying to.
25 Then that language would need to be NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
217 1 carefully discussed in advance, and not just put into 2 the LAR.
3 MR. JAIN: Next question is from Steve 4 Kenney. Steve Kenney, can you unmute?
5 MR. KENNEY: Thank you JB. I'm a human 6 factors consultant supporting the Dominion team 7 currently. I have a question for Mr. Desaulniers, and 8 Green. In the discussion of the iterative nature of 9 the process, and the multi stage approach to 10 validation, what are the margins of safety that would 11 be considered relevant to the testing that needed to 12 be demonstrated as part of acceptance criteria for the 13 testing that would be part of such a multi stage 14 process?
15 MR. GREEN: So, I'll start with that.
16 It's hard to prescribe it, because the stages would be 17 handled differently by different licensees. But I 18 think some of the things that we would be concerned 19 about -- or a good place to start of the discussion, 20 I guess maybe that's the best way to put it. Is if 21 there are any credited operator actions, or important 22 human actions that are credited, some level of 23 consideration on the feasibility of those within any 24 sort of relevant time frames.
25 Could I tell you what kind of margin to NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
218 1 put on that? Not immediately, but you would put some 2 thought into creating a validation that would test to 3 see what those margins were, perhaps would be one way 4 to do it. That's the first one that comes to mind, 5 I'm sure we probably could think of some more. But 6 again, it would all depend on what your stages were, 7 and what your modifications were getting into, the 8 sorts of modifications that were being made. Dave, if 9 you wanted to expand on that?
10 MR. DESAULNIERS: Yeah. I don't know if 11 we want to get down into too much detail here, if this 12 is what you're really going for sir. But Brian 13 commented on credited operator actions, and so a way 14 to -- let me back it up. Under validations, we only 15 have so many opportunities to test a certain operator 16 action, and determine whether, or not it can be 17 completed in the time required to -- before you have 18 adverse consequences to the plant safety.
19 So, in conjunction with those, yeah, if 20 we're taking that as a particular example, we're 21 typically looking for margins associated with that.
22 And as Brian said, it's difficult to prescribe a 23 margin that might determine. Factors that go into 24 determining that margin tend to be -- first off, how 25 short of a period of time from onset of the event to NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
219 1 time that the operator needs to take action?
2 Are we talking about an action that needs 3 to be taken in the realm of a couple minutes, or 30, 4 or more minutes? And the further you go out, the 5 nature of the margin consideration can change. One 6 common guideline that we've used in looking at margins 7 is to consider is time available to recover from 8 credible errors associated with that event? In other 9 cases, percentage time frames are used. But it does 10 vary according to the case.
11 MR. JAIN: Thank you David.
12 MR. KENNEY: As a follow up then, is it 13 reasonable to expect some clarification on this 14 subject matter area as NRC proceeds with the 15 formulation of these ideas around the multi stage 16 validation?
17 MR. GREEN: I think that right now we're 18 still evaluating options. So, I don't know that multi 19 stage validation is the sole answer. So, if this 20 license condition approach is the way that goes 21 forward, then we may have to rethink that. There may 22 be other options on the table as well. So, I don't 23 know that we'd be putting forth guidance imminently on 24 that.
25 However, there is, with regard to operator NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
220 1 actions, we do have existing guidance that has 2 information on that. NUREG-1764 talks ad nauseam on 3 the topic, and there is an appendix to NUREG-1852 that 4 actually, it's fire human actions guidance, but it 5 actually does a really nice time line analysis. That 6 if we were to put forward some sort of guidance, I'm 7 going to say we'd probably lean on one, or both of 8 those documents.
9 MR. KENNEY: Thank you gentlemen.
10 MR. JAIN: David, do you have something to 11 add?
12 MR. DESAULNIERS: Well, I have a question 13 that goes back to thinking about the INL HFE program.
14 And gaining clarity on what the NEI proposal is with 15 regard to that program. But just as this last 16 question raised, the devil is in the details on some 17 of these things, on many of these things. And I think 18 one concern that I see, and just thinking about the 19 program as described at a general level, as we have in 20 the slide, and I realize there's only so much you can 21 do in the course of a brief presentation, and a 22 PowerPoint.
23 But I guess the concern that I have is 24 maybe a potential expectation that what we could come 25 up with is a generic set of tests, or processes that NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
221 1 might apply that would say okay, this is the NEI 2 endorsed INL program, and it will work for all 3 applications coming in. And I think the -- what tests 4 are necessary to show that a design is in fact going 5 to be acceptable is going to depend from one design to 6 the next.
7 And what level at which you need to do 8 this testing at. So, I don't know if the expectation 9 is that this would be modified from one application to 10 the next, then this is just an example. I'm a little 11 confused as to maybe what's exactly being proposed.
12 But I am concerned that you might be missing the point 13 of looking at the particular modification, and what's 14 necessary for that modification in terms of testing.
15 These are good, all good HSI design, 16 development activities. But frankly, pretty much what 17 would have been expected under 0711, just drawn out as 18 particular instances of the types of activities that 19 could be done. So, I'm not sure it's adding new 20 information, just maybe some specificity to a 21 particular approach that may, or may not be applicable 22 for a particular modification.
23 MS. GOLUB: I guess I'm not sure what to 24 do with that feedback David. But you know, we're 25 trying to put something forward, because there are NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
222 1 people who are in the middle of -- licensees in the 2 middle right now of trying to get these mods done, and 3 so we're trying to put something forward to get staff 4 feedback. So, this is not information that NEI made 5 up, we're trying to base our approach on research 6 that's been done.
7 And so of course, all of this has to be 8 adapted, the same way the MSV approach would have to 9 be adapted to whatever the modification is. I mean 10 everything has to be adapted to that modification.
11 So, I guess I'm not sure what you are saying. Are you 12 saying that you're not sure that this approach applies 13 to the modifications, all of them, or -- could you 14 please clarify a little bit?
15 MR. DESAULNIERS: My question was whether 16 you're looking for NRC to endorse some general high 17 level approach, or --
18 MS. GOLUB: Yeah, I don't think this is --
19 this is not endorsement, this is an approach we would 20 like to describe in individual license amendment 21 requests for particular designs. So, the scope is, 22 this is not a generic scope, licensees will describe 23 this in their LARs for their designs, and use that as 24 the basis for the NRC to draw their safety conclusion 25 if this process is followed.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
223 1 But this is not a generic endorsement 2 conversation, this is not like an NEI guideline that's 3 being put forward. We're trying to find a way to help 4 these early movers, early adopters find a success 5 path. So, it would be LAR-specific, does that help?
6 MR. DESAULNIERS: Thank you, that does 7 help. So, it would be LAR-specific, and yeah. Again, 8 I think that it does -- we end up in the same place 9 though in the end, when we're talking about MSV, or 10 something, more guidance, and detail with respect to 11 the focus of the testing, and the rigor with which 12 it's applied would be needed in order to be able to 13 determine if this is a viable pathway.
14 MS. GOLUB: Yeah, and I appreciate that.
15 But I guess, I offered this to the agency in general, 16 that we're here because we're trying to get feedback.
17 We're trying to find a path forward, trying to offer 18 things that are not always easy to offer, like license 19 conditions, and try to enlist INL to help us find this 20 path forward. So, yeah, I would say that --
21 MR. JAIN: Pareez, this is BP. We are 22 running late, and perhaps do you want to schedule 23 another public meeting on this topic? This very 24 interesting topic. I get why --
25 (Simultaneous speaking.)
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
224 1 MS. GOLUB: I don't know if there's enough 2 information here for licensees to move forward. So, 3 I agree, I think we may need one. I don't want to 4 speak on behalf of NEI, Alan can certainly speak to 5 that, but just in my opinion as a member of industry, 6 it doesn't seem needed.
7 MR. JOE: BP, this is Jeffrey, I'm sorry, 8 if you could provide just a little more latitude, I 9 noticed Bruce Hallbert had his hand up, I would very 10 much like to hear what he wants to say here, 11 recognizing that we are over time.
12 MR. JAIN: Okay, go ahead.
13 MR. HALLBERT: Thanks. Hi, my name is 14 Bruce Hallbert, I'm the national technical director of 15 the LWRS program, and I appreciate being invited to 16 listen to this interesting discussion. I wanted to 17 just respond to, first of all, say hello again to Dave 18 Desaulniers, and I'll respond to the comment, or the 19 question that he had. Our presentation wasn't so much 20 to propose an approach, or even a path forward 21 necessarily resolving whatever the perspectives are on 22 this issue of 0711, or multi stage validation.
23 But really just to say we've been 24 addressing human factors engineering issues as a part 25 of sustainability efforts sponsored by the Department NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
225 1 of Energy that are needed to address the aging, and 2 obsolescence of control room systems in place today.
3 And we've been using 0711, I think that was one of the 4 main points we wanted to just communicate. We've been 5 using 0711, and 0711 does tell us the kinds of things 6 that NRC is looking for.
7 It's good information, but it doesn't 8 always tell us exactly how to do things for example.
9 It says you need to do an operational experience 10 review, but it doesn't say what you should do to do 11 that. So, we've been working with a number of 12 vendors, suppliers, and utilities owners, operators, 13 and working out some of the details on a variety of 14 different design projects fundamentally.
15 And what we've learned is that 0711 is 16 very complete with regard to the guidance it provides.
17 We know what NRC expects from a regulatory review.
18 There are sufficient human factors methods out there 19 for us to be able to conduct these activities, and 20 overall, like Jeffrey was saying, it really does 21 enhance the confidence you have in the final design 22 once you get to integrated system validation.
23 So, I think that was the intent of the 24 remarks that we were trying to provide today. I don't 25 think there's a way for us to really resolve the NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
226 1 question of what is validation? We're using the 2 definitions that are supplied in 0711 currently, but 3 it sounds like there is a lot to be discussed as well.
4 MR. JAIN: Thank you Bruce, thank you.
5 So, perhaps, like Pareez was saying, industry needs to 6 offer if they want to discuss any further, a public 7 meeting where we can discuss all this stuff. With 8 that, I'll ask Entergy to make its presentation on the 9 Waterfall experience.
10 MR. CHAMPAGNE: Hi, this is Jacob 11 Champagne with Entergy, I'll share my screen here in 12 a second. So, while this loads up I'll introduce a 13 couple people we have in the room. So, my name is 14 Jacob Champagne, I'm an Entergy project engineer. I'm 15 the responsible engineer for the core protection 16 calculator system here at Waterford. Also in the room 17 we have Dave Moody, who is the manager of project 18 engineering.
19 And we have William Truss, who is another 20 I&C engineer on the project. My Teams system is 21 loading, Entergy's system doesn't love Teams, so it's 22 loading, it's thinking right now, so just give me a 23 second.
24 MR. DARBALI: Hi Jacob, this is Samir, I 25 have your slides up, do you --
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
227 1 MR. CHAMPAGNE: Samir, I was going to ask 2 if you could share it, I think the system just 3 completely locked up.
4 MR. DARBALI: All right, will do.
5 MR. CHAMPAGNE: Appreciate it. Okay, I 6 can't even see it right now, but I'll continue on.
7 So, like I said, this is for the Waterford core 8 protection calculator project, and again, I'm Jacob 9 Champagne, and you can go to slide two. It's a pretty 10 brief presentation, but what I tried to do is break it 11 up into a project background, and this is to give you 12 an idea of the system, and a rough idea of the time 13 line of events.
14 So, this is the replacement of the 15 existing Interdata 7/16 digital computer system, and 16 we're replacing it with the Westinghouse Common Q 17 platform. If you have any familiarity with the 18 Interdata 7/16 system, it is a 1960s, or 70s era 19 system, so quite outdated, and obsolete. Our 20 engineering change, if you're interested, it is 21 documented in CMSBP, and it's 83843, and that was 22 approved in April of 2021.
23 One big piece of this whole design is 24 there's no impact to the existing safety related 25 algorithms. The hardware, and those related changes NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
228 1 are the primary changes. The LAR was submitted in 2 mid-2020, and there was some revisions, and follow up 3 meetings. I also glossed over, we had plenty of 4 presubmittal meetings since we filed a project in this 5 process, where we met with the NRC prior to COVID, and 6 after virtually to discuss how our submittal was 7 going.
8 And the approval of the amendment, and our 9 safety evaluation was received in August of 2021. We 10 conducted our factory acceptance test at the 11 Westinghouse testing facility, that went from late 12 July to early August 2021. The NRC was involved, they 13 observed us, and Westinghouse, and basically our 14 vendor oversight of Westinghouse. We had our site 15 acceptance testing at the Waterford location.
16 We had a testing location in one of our 17 training buildings that we made a CBA extension of our 18 warehouse. This was also observed over a number of 19 days by the NRC. Our implementation is planned for 20 refuel 24, which starts on April 22nd, so it's very 21 soon. If you can go to the next slide for me Samir.
22 So, this summarizes some lessons learned. We tried to 23 keep it high level, and we can discuss further.
24 But basically I'll cover the Entergy 25 lessons, and then I believe Warren Odess-Gillett will NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
229 1 talk with the Westinghouse. So, first of all is the 2 vendor oversight plan, and the summary. So, our 3 project had a formal vendor oversight plan for the 4 process, and the summary was what we actually 5 docketed. This was meant to be a shorter version of 6 the vendor oversight plan covering high level details, 7 but very specific details that were important to the 8 oversight process.
9 One of the lessons learned was early on 10 defining what would be in each, and identifying what 11 was requested in each. Because it's a vendor 12 oversight plan, I don't want to call it nebulous, but 13 it's kind of a document that doesn't have a direct 14 template, it can differ from project, to project. If 15 you have a discussion early on, and identify what's 16 needed in each document, I think that can save some 17 revisions of those documents.
18 The next bullet is an involvement of 19 regional inspectors early in the process. So, 20 obviously early on in the design phase, you're dealing 21 with the NRR as you work through the license 22 submittal, and all those details. And then as you 23 transition to the FAT, and the SAT, the reason we 24 became involved, and we did finally have to redo some 25 things, and fill in some gaps that we had provided in NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
230 1 previous submittals, and things.
2 And it wasn't a huge burden, or anything, 3 it just I think some efficiency could be gained, and 4 some consistency in the process if we did involve 5 region inspectors earlier in the different audits, 6 inspections, presubmittal meetings, just so they're 7 aware of what's been discussed, and even some of the 8 technical details behind the system.
9 The next one is a little more generic, but 10 clarifying terminology. So, obviously each fleet is 11 going to have its own terminology with regards to 12 engineering changes. So, in the standard design 13 process, you have a conceptual, and you have a 14 detailed design, and a final design, but a lot of the 15 terminology can be based on the actual product you're 16 getting.
17 And the conceptual design for the product 18 is different from the conceptual design in your 19 engineering change package. So, sometimes when we 20 would get questions for a particular life cycle phase, 21 some of those documents may not have been available, 22 some of the products weren't there yet that may have 23 been expected. So, there was quite a few calls that 24 I can remember that we had to clarify when things 25 would happen, and how they fit in our process.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
231 1 So, I think maybe early on in the process 2 lining up the different life cycles with the fleet, 3 the supplier, and the NRC, and making sure that 4 everyone's on the same page with what's going to be 5 available when, I think that would probably be 6 efficient. And the last one is really from an Entergy 7 side of things, the side river seat warehouse, and 8 custody control, for Waterford this was really the 9 first major digital upgrade of this kind of scale.
10 So, it just was learning to use our 11 existing fleet processes to that scale of things. So, 12 receipt inspection at the warehouse, storing it in our 13 testing location, making sure that we have proper 14 custody control per our procedures, and documenting 15 that appropriately. And we took an action from, 16 basically our factory acceptance test to our site 17 acceptance test, and we created a cyber-receipt 18 custody control document.
19 That's a project specific document that 20 just lays out as we move it into the plant, how we're 21 going to meet all those procedures that we have that 22 are oftentimes spread out all over the place. And I 23 think that'll help the NRC see exactly what we're 24 doing, but it definitely was something that we had to 25 learn to do on the fly, and apply those specific NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
232 1 procedures to this scale of a project. Warren, did 2 you want to pick up from here for a Westinghouse 3 lessons learned?
4 MR. ODESS-GILLETT: Thank you Jacob very 5 much. So, contrary to the Entergy experience with the 6 region inspectors, we were not expecting to engage 7 with the vendor QA branch until somewhat toward the 8 end of the detail design. But in fact though, we 9 started engagement with the vendor QA branch nearly 10 immediately after the LAR was accepted. It's not a 11 bad thing. I mean early engagement's a good thing.
12 But we weren't prepared for the 13 engagement, and so now we know that when a LAR is 14 submitted, and accepted, that the vendor should start 15 expecting engagement with the vendor QA branch. And 16 we did have a struggle understanding the swim lanes 17 between what the vendor QA branch was going to review, 18 and what the LAR review was.
19 We were a little confused there, because 20 we were the first out of the box. So, it was a little 21 confusing. So, as described at the last February 22 meeting in 2021, that was given to us early on to help 23 us understand what's a vendor QA inspection role, and 24 what is a law review role? Because they actually were 25 starting to occur concurrently during the law review.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
233 1 And then as I think Shiattin had mentioned, the FAT 2 was really quite a logistical challenge.
3 We hosted the NRC vendor QA branch. We 4 hosted the regional inspection branch, and we hosted 5 the licensee. And the licensee was overseeing the 6 vendor. The region branch was overseeing the 7 licensee, and the vendor QA branch was inspecting the 8 vendor. And so there was a lot of interplay between 9 the organizations.
10 And it was a challenge because at a vendor 11 site, unless you have unescorted access, you've got to 12 be prepared to have a number of resources to escort, 13 I would say -- I don't know how many people were there 14 at that time, maybe 15, or something like that. Maybe 15 even more, because now I'm thinking the Entergy party 16 was quite significant.
17 So, we needed to really make sure we had 18 the resources not only to answer all the questions 19 from all of the different organizations, but also from 20 a logistical perspective, making sure we had the 21 escorts we needed to have all the various parties go 22 from one location to another at the vendor site. So, 23 that's all I have Jacob.
24 MR. CHAMPAGNE: Thanks Warren, appreciate 25 that. Samir, you can go to the last slide, which is NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
234 1 really just questions, and comments. But I'll just 2 follow it up by saying, obviously this being a pilot 3 process, there was going to be lots of room for us to 4 find things that could be better. But I give credit 5 to both Westinghouse, and the NRC. It was always 6 flexible, we were always willing to work with each 7 other to make sure we got the documents, the 8 information that needed to be given.
9 I will mention that we used Certrec to 10 track all the questions, and provide those responses, 11 and I think that works well. Especially if you're 12 working virtually, which we were a lot of times. It 13 can be harder to make sure that you have those 14 questions documented. So, making sure that all those 15 things are documented well, and you can provide your 16 response, I think that was definitely something that 17 benefitted us all. So, with that I'll leave it for 18 any questions, or comment.
19 MR. JAIN: Are there any questions for 20 Entergy?
21 MR. WATERS: Yes.
22 MR. JAIN: Yes Mike, go ahead.
23 MR. WATERS: This is Mike Waters, this is 24 a great lessons learned presentation, thanks. And you 25 actually mentioned Certrec, and I recollect we were NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
235 1 confused -- confused might not be the word, but at 2 first because there were two separate rooms, or 3 portals for an inspection versus the licensing, and I 4 was running -- but any additional insights on the use 5 of open item process?
6 And the way we communicated open items, 7 and the frequency? We had bi-weekly meetings, and 8 sometimes I think both parties felt those were too 9 frequent sometimes, or not enough, anything to add 10 weight on there?
11 MR. CHAMPAGNE: Yeah, so I know especially 12 going into the FAT, and the SAT, we definitely had 13 multiple inspections open in Certrec, which can be 14 confusing. So, it does take some communication 15 between the site, and the NRC just to make sure that 16 we're in the same process. Because I know for the FAT 17 specifically, we loaded in a bunch of details, and I 18 don't think the right permissions were in Certrec, or 19 something administrative like that.
20 So, it definitely can be confusing, it can 21 become a burden, but I think with the proper 22 communication, it can definitely be a valuable tool.
23 For the meetings, I think the meetings were usually 24 very valuable, because we got to see what information 25 we provided, where the gaps were, or where the NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
236 1 information we provided, maybe we missed the 2 interpretation of the original question.
3 And it broke down some of those barriers, 4 that sometimes you have to wait awhile to get that, 5 and you end up being kind of inefficient, because you 6 could have answered that question earlier if you knew 7 the question better, or understood it better. But 8 it's one of those things where you setup a bi-weekly 9 meeting, and there's going to be parts of the project 10 where they're not going to require that bi-weekly 11 meeting.
12 But it still was useful, because you can 13 still, ahead of time kind of judge if you want to 14 cancel those, or if people are unavailable, or 15 something. I do think it was valuable, and I think 16 Entergy gained a lot of insight from those meetings.
17 MR. WATERS: And I just wanted to follow 18 up with one other comment here, and this is where I'm 19 the broken record in all the application meetings, for 20 future applicants, to inform us as much as you know, 21 when you're in long cycle phases, when you're doing 22 activities, it's more so a point in itself, and what 23 was said is right on, it was very challenging to 24 choreograph, and I know it was confusing to Entergy.
25 We see it in hindsight, but I think that NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
237 1 one of the issues is that the true RP -- I think there 2 was assumption that NRC's design approval would 3 happen, and then all of these, the implementation, 4 testing would happen after that. And we're already a 5 given fact of life those would happen in parallel with 6 the licensing, it was an extra challenge to coordinate 7 between the licensing review as one mode that we 8 developed on, and develop swim lanes on what was 9 licensee, and what was inspection. So, it was truly 10 a lesson learned.
11 MR. JAIN: Thank you Mike. Are there any 12 more questions, or comments? Yes, Richard?
13 MR. STATTEL: I think I heard you mentions 14 that one of the challenges had to do with figuring out 15 what information to provide at what point in the 16 development life cycle, I think. And this kind of 17 relates back to something that Pareez said during her 18 presentation. I'm not sure I really understand that, 19 because in ISG6, for the alternate review process, we 20 expect all the information that's needed for a safety 21 evaluation to be provided as part of the initial LAR.
22 And what Pareez had mentioned earlier was 23 that somehow you're saying the alternate review 24 process, the licensee is expected to submit design 25 information earlier in the project life cycle. But NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
238 1 I've been looking at ISG6, and it really doesn't --
2 it's pretty silent on what part of the project life 3 cycle the developers are at. Really what ISG6 4 identifies is what information is to be put into the 5 LAR.
6 And the other point I would like to make 7 is even with the tier one process, the exact same 8 information for design information is required at 9 exactly the same time, at the time of the LAR 10 submittal. That's not phase two information. So, I'm 11 trying to understand what the point was during 12 Pareez's presentation, and I'm not quite sure I 13 understand what your lesson learned was with regard to 14 that.
15 MR. CHAMPAGNE: Yeah Rich, thanks, this is 16 Jacob again. I'll speak to ours. So, ours was 17 specifically -- there's a difference in life cycle 18 when it comes to, for instance Westinghouse developing 19 the TCP system, their conceptual design, and detailed 20 design is a different part of the process than when we 21 get into our engineering change, which documents the 22 design basis, and the impacts of that.
23 Which as a conceptual design, we're really 24 implementing, and reviewing their design, and 25 incorporating it, making sure it meets our NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
239 1 requirements, making sure what design basis documents, 2 and requirements, licensing basis documents, and 3 there's a difference between those two.
4 MR. STATTEL: Okay, so you --
5 MR. CHAMPAGNE: Go ahead.
6 MR. STATTEL: I'm sorry, so you're 7 referring to the interactions between your design 8 processes, and Westinghouse's processes?
9 MR. CHAMPAGNE: Yes Rich. I think the 10 reason we put it on here is because sometimes when we 11 would get questions from you guys on a particular life 12 cycle phase, it fit where Westinghouse was in their 13 design more than it did in our engineering change 14 process. We didn't have those documents in our system 15 yet, we were still processing in Westinghouse, but it 16 was just a gap in time frame between those two 17 processes.
18 MR. STATTEL: Okay, thank you, that's 19 helpful.
20 MS. GOLUB: Richard, it's Pareez. I think 21 your point is well taken. I guess the only thing I 22 meant, and I really shouldn't have said earlier, 23 because it makes it sounds like it's earlier, as 24 compared to other processes, which is not the case.
25 What I really meant was just --
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
240 1 MR. STATTEL: I mean it's your choice, 2 right? You can --
3 MS. GOLUB: Yes, that's exactly right.
4 Because it's not -- matter of fact it's not that as 5 soon as one phase ends, you just submit everything, 6 right? You really have to pick the right time to 7 submit when you feel confident that the material 8 you're submitting won't change, that it is founding.
9 MR. STATTEL: Right, okay.
10 MS. GOLUB: And so that, I think was just 11 -- that's what I intended to say, however it came out, 12 it may not have been clear.
13 MR. STATTEL: Okay, thank you.
14 MR. JAIN: Thank you, any more questions 15 for Entergy on this topic? If not then we will move 16 on to, I'll request NextEra to make its presentation 17 on (telephonic interference) experiences.
18 MR. FREWIN: Good afternoon, this is West 19 Frewin, I'm just doing a microphone check first.
20 MR. JAIN: We can hear you.
21 MR. FREWIN: Okay, thank you. Samir, 22 could you also put my slides up as well please?
23 MR. DARBALI: Yes, give me a minute.
24 MR. FREWIN: Thank you. And while he's 25 putting those up, my name is Wes Frewin, I work for NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
241 1 NextEra Energy at the Jupiter West Corporate Office.
2 We're presently working on a safety related digital 3 upgrade for Turkey Point units three, and four. The 4 scope involves reactor protection, system engineering, 5 safety features, actuation systems, and nuclear 6 instrumentation system replacement.
7 So, it is a sizeable budget, there are a 8 lot of aspects to it, and just getting our arms around 9 it has been a challenge. What I want to go through 10 with my presentation is more of a focus on ISG6 from 11 a process standpoint. I'm not going to get into any 12 technical, or regulatory areas specific to a license 13 amendment request, it's still in development. And 14 we're still working with the NRC in the area of 15 presubmittal meetings.
16 That is not an area that I will be going 17 into at all. On the Teams call with me is Warren 18 Busch, he is the lead project engineer at NextEra 19 Energy, and we do have individuals from our 20 independent third party review team, as well as our 21 vendor. Our vendor is Framatome, and they're on the 22 Teams call as well. Samir, I don't see the 23 presentation up yet.
24 MR. DARBALI: I apologize, I thought I was 25 sharing.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
242 1 MR. FREWIN: No, that's okay. You can go 2 to slide two please. So, the approach I'm taking is 3 just to share some plus deltas challenges, improvement 4 opportunities as we see specific to ISG6, and any 5 comments, and questions at the end. Go to the next 6 slide please. So, the first area I want to get into 7 are the deltas. And so what we've been doing is 8 working with the NRC, and presubmittal meetings 9 without face to face engagement with the staff.
10 I do mention this as a delta because I 11 know I've worked with the NRC face to face before, I 12 was part of an NEI working group working on 50.59, and 13 changes to 9601 Rev 1. And I know working face to 14 face is far more efficient than via conference call.
15 So, that by itself does create a challenge. We have 16 obviously made it work because we've had to, but it is 17 something that I think -- face to face is a far better 18 way to engage, and get work done.
19 Scheduling has been a moving target, a lot 20 of it is because of the scope. Scope, as I mentioned 21 is sizeable, and unfortunately when it comes to a 22 schedule that becomes a moving target, that in turn 23 turns into challenges from an engineering standpoint, 24 as well as regulatory uncertainty. And so we've been 25 working very closely with the NRC, keeping them NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
243 1 informed of how things are going, and also seeking any 2 clarification on any regulatory issues that we need to 3 be aware of as we go along.
4 One of the other delta areas is a large 5 time gap between presubmittal meetings. We did see 6 this in one particular area between a presubmittal 7 meeting that occurred in August of 2021, and then the 8 next meeting was in March of 2022. The problem with 9 that is we end up with different plays that come into 10 the phone call on the second call that we're not 11 familiar with, and we start to hear expectations that 12 at least sound different than what we've understood 13 previously.
14 And so, that creates a challenge I think, 15 not only for us, but it creates a challenge as we try 16 to understand those perceived changes, or expectations 17 from the regulator side. So, there's kind of a 18 lessons learned as well, is to make sure your 19 presubmittal meetings, you keep them scheduled close 20 together, obviously at reasonable times, but keep them 21 close together, so that you do have continuity from 22 one presubmittal meeting to the next.
23 Next topic is screening between the 24 ultimate review process, and the tiered process.
25 Although it is straight forward, we understand it, it NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
244 1 is narrow in the way that it's defined within ISG6.
2 Some of the project specific considerations -- it 3 doesn't seem to give us a whole lot of flexibility 4 when it comes to perhaps moving to a hybrid approach.
5 So, this has been challenging.
6 And again, we've decided to work very 7 closely with the NRC, and in our presubmittal 8 meetings, keeping them informed, of what our LAR 9 submittal is going to look like, and how we will 10 accomplish making sure that they get the documents 11 they need as part of the LAR submittal, and any 12 subsequent documents that they would need in order to 13 ensure the approval of a license amendment request.
14 So, that's just one area that -- from an 15 ISG6 standpoint, it just gives us a little bit of 16 difficulty in working through that process. And the 17 background of ISG6 being an infrequently performed 18 process. It's not a procedure, or a regulatory 19 document that you pick up every day, and exercise as 20 you would other procedures that you have in house.
21 The ARP process is -- the way it was, combined with 22 the tiered process, it is very difficult to follow, 23 there are sections within sections.
24 And it made it very difficult to determine 25 okay, is this part of an ARP discussion, or is this NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
245 1 just part of a tiered approach discussion? And you 2 combine that with enclosure bravo, and some of the 3 topics that are discussed in the text are different 4 than those discussed in enclosure bravo. Some of 5 those are semantics, in other words, the words are 6 different, or the titles are different. But it still 7 required us to do some translation work with ISG6 in 8 order to make sure we understood what documents were 9 being referred to, and ensured that was going to be 10 part of our LAR submittal.
11 Then last lastly, human factors, I know 12 this has been brought up before, where it's brought up 13 as an outside of scope. It just seems out of place, 14 I guess is the best way to describe it. It seems out 15 of place within ISG6, given that human factors is 16 mentioned throughout ISG6 quite frequently actually.
17 And Rich Stattel had just brought up about where it 18 says in section C.2 that include necessary information 19 at time of the LAR submittal.
20 So, obviously necessary information is 21 going to include human factors documentation, so we 22 know that that has to be part of it. So, it just 23 seemed an odd example, I guess is the best way to say 24 it. Next slide please. If you could back up one 25 please. Or -- okay, something's out of order here.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
246 1 I'm on the slide that talks about pluses. Either 2 that, or I guess -- I'm sorry, my slide deck's out of 3 order, sorry about that.
4 MR. DARBALI: Is this one it?
5 MR. FREWIN: Yeah, that's it, I'm sorry 6 about that, I had slipped one side out of order. So 7 the pluses, the presubmittal meetings are a plus, gave 8 us an opportunity to work very closely with the NRC, 9 and continues to be an area that we can meet the NRC 10 teams, and get to know them, and understand what their 11 expectations are for ISG6, and the LAR submittal.
12 The NRC has been very open in the 13 discussion, which has been very helpful in 14 understanding ISG6, and the expectations of the 15 regulator. And it also gives us an opportunity to 16 hear the NRC perspective firsthand, as opposed to 17 either translating them through some guidance, or 18 through email, or some other way. Obviously hearing 19 it firsthand has been very helpful to us.
20 Although ISG6 does not talk about 21 independent third party review teams, I do have to 22 mention that is a plus. Not necessary for ISG6, but 23 order to navigate through ISG6, we have an independent 24 third party review team that is knowledgeable. There 25 are members that have been part of the development of NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
247 1 ISG6 Rev 2.
2 Those have been very helpful in order to 3 help us translate what he requirements are, what's 4 behind the words, the discussion that occurred during 5 the working teams with NEI, and the NRC on the 6 development. So, that gave us really a firsthand 7 perspective of understanding the direction we're going 8 in, and make sure that we're getting it right the 9 first time. Also their involvement in industry groups 10 such as NEI, EPRI, and INL.
11 That in combination has just been a 12 tremendous asset to the project team. And also the 13 abundance of technical reports, there's a boatload of 14 documents out there from various organizations such as 15 EPRI, IEEE, and the regulator themselves. One thing 16 ISG6 does is it helps us navigate through those 17 documents, and points out documents that are important 18 to putting together the LAR submittal, and the 19 approaches that are recommended by the NRC for those.
20 If you go to the next side please. So, 21 this one I just wanted to show the audience the 22 various topics that we've discussed over the eight 23 presubmittal meetings that we have had to date. The 24 top two sub bullets are actually the most popular, 25 which is vendor oversight plan, and the ISG6 Rev 2 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
248 1 process, and LAR content. So, that shouldn't be a 2 surprised, based upon what we heard this morning from 3 the NRC.
4 And a close second of course is human 5 factors, and there are a couple others such as D3, and 6 the life cycle phase process. So, that just gives you 7 at least kind of a look at the topics that we've 8 brought up during the presubmittal meetings. Next 9 slide please. Can you go to page six? Thanks. So, 10 the ISG6 challenges, just a couple things, these kind 11 of echo what I've presented before, is the criteria 12 between alternate review process, and the tiered 13 process is trying to navigate our way through that.
14 And determine whether, or not a hybrid is 15 more of where we are, and then defining what that is, 16 making sure that that's crystal clear to the 17 regulator, so that they understand that path that 18 we're working through, and can support from a 19 regulatory standpoint, the alternate review process 20 from their standpoint. So, that is a challenge, it 21 continues to be presently.
22 But we are working through that actively 23 right now. NRC representatives, primarily during the 24 presubmittal meetings have provided background on the 25 various ISG06 statements, and often times what we NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
249 1 perceive is that there is a -- it's favored -- in 2 other words the regulator tends to favor using the 3 tiered process over the alternate review process.
4 That just seems to be the impression that we get.
5 We still think we are in the alternate 6 review process, we think it's appropriate, we think we 7 screen into it. So, we are working with the 8 regulator, obviously very closely, to make sure that 9 we have all the right things in place for that, and to 10 support it. And then what I brought up earlier with 11 regard to the alternate review process, and the tier 12 process, is the way editorial it's put into ISG6 that 13 does offer some challenge there in understanding it.
14 So, improvement opportunities for ISG6, 15 again, does not define necessarily the limitations of 16 the alternate review process. We're kind of learning 17 that as we go. We're learning that from the 18 presubmittal meetings, our independent third party 19 review team, and workshops like this, where we can 20 hear firsthand from the NRC their position on certain 21 things. But we understand that there are other 22 attributes, such as complexity.
23 And what information is desired for 24 docketing, and also as we've been discussing for the 25 afternoon, human factors. And that concludes my NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
250 1 presentation, pending any comments, or questions.
2 MR. JAIN: Are there questions on 3 NextEra's presentation?
4 MR. JAIN: I see a hand, Justin. Justin, 5 do you have a question?
6 MR. VAZQUEZ: Hey yes, this is Justin 7 Vasquez, I'm the lead technical reviewer for the 8 Turkey Point application, the human factors team. I 9 work with the Brian's group. I did want to speak to 10 just one point really quick,, I think it was slide 11 five with the deltas listed. Yeah, that's the one.
12 So, just looking at the third bullet here. We did 13 want to speak to just one really quick point, to that 14 gap between that August meeting, and the March 15 meeting.
16 We do recognize that that is a significant 17 gap between I guess what we're technically classified 18 as the presubmittal period, it was some engagement. In 19 August of 2021, we learned some new information about 20 the scheduling, and expected timing, and that's kind 21 of what triggered us into having the internal 22 discussions about how to consider the human factors 23 engineering, and validation review, which we've been 24 discussing quite a bit today.
25 And during that time period that we see NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
251 1 discussed in this bullet, we did have some engagement 2 leading up to the end of the year period with the 3 licensee, just kind of talking to them about some of 4 our initial thoughts. We also had an additional 5 meeting in early February to talk initially about the 6 multi stage validation concept, and also hear some 7 additional feedback from the licensee.
8 And we appreciate this discussion, so we 9 just want to recommit that. We are committed to -- I 10 mean engaging as efficiently, and as consistently as 11 possible, and there also was mentioned the fact that 12 we had some staff changes on the assignment. And I 13 just wanted to clarify that a certain amount of that 14 can be expected with these long term projects.
15 Because with the NRC, I mean we do have our own staff 16 movement currently, so sometimes we will have 17 reassignments.
18 But we do maintain communication within 19 our teams, and we try to keep things as consistent as 20 possible throughout the process. So, we just wanted 21 to say -- I mean we appreciate the point that's made 22 here, and we also appreciate the frequent engagement 23 with the licensee, and then keeping us up to speed on 24 developments, but I just wanted to say that we have 25 been making an effort to stay as engaged as we could, NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
252 1 even through those gaps between meetings.
2 Making sure that we have as common an 3 understanding about expectations as possible. So, 4 thank you.
5 MR. JAIN: Thank you Justin. Are there 6 any more questions on NextEra presentation? If not, 7 then I'll request Constellation to make its 8 presentation on Limerick.
9 MR. DARBALI: We have a couple of 10 questions.
11 MR. JAIN: Please, go ahead.
12 MR. WATERS: Hey, this is Mike. First, 13 great presentation again, and to compliment NextEra, 14 I think the pre-application meetings, although 15 extensive, were beneficial, and again, I think NextEra 16 was great about being open about what they were doing, 17 were considering, and willing to engage us when we 18 said we need the following information on this, or 19 suggested topics for the next submittal of 20 information, so thanks for that.
21 Yes, good comments on the ARP versus 22 tiered process, that's something we need to chew on, 23 and think how we could communicate better. I think 24 one of the challenges here, and we can't solve the 25 problem here, is alternate review process, and tiered NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
253 1 process, it's both what is the content, but also what 2 is provided to us, right? So, there's differences, 3 there's at least two axes here.
4 And sometimes we may be focused one, the 5 timing versus the content, or vice versa. So, that's 6 something that we may need to think about how to 7 communicate it better down the road. Rich, did you 8 have something to add?
9 MR. STATTEL: Yeah, I just wanted to 10 respond to the process, the decision of which process 11 to use. The object of the process, the goal of each 12 process is the same. So, it's really just a matter of 13 which is more applicable. The tiered process is 14 oriented towards a product based evaluation. Whereas 15 the alternate review process is more based on a 16 process based evaluation.
17 So, the safety conclusions are the same, 18 it's just what do we base our safety conclusion on?
19 What information is provided to base your safety 20 conclusion on? So, it's not our intent to express a 21 preference for one, or the other, but we want to use 22 the process that's most appropriate for the 23 application, and the stage that you're at at the time 24 of the submittal.
25 That's what we want to really get out NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
254 1 there. The alternate review process is not 2 streamlined, it's not shorter, it's not less 3 resources. It's the same, it's just a different 4 process that uses different information as a basis for 5 the safety conclusions. So, neither one is beneficial 6 in terms of schedule, or commitments, or things like 7 that. They're just different in that way. I just 8 want to mention that.
9 MR. JAIN: any other comments from the 10 staff? If not, then I will request Constellation to 11 make its presentation on the Limerick.
12 MR. CONNELLY: Okay, Samir, can I ask you 13 to present? Got it, thank you. Well, good afternoon 14 everyone, I'm John Connelly, I'm the engineering 15 manager for the Limerick Modernization Project. As 16 has been stated multiple times during this meeting, we 17 really do appreciate the NRC providing the opportunity 18 to exchange information. This forum is particularly 19 valuable as we're putting new processes into motion.
20 ISG6 alternate review, standardized 21 digital engineering process, or NSBN04. Digital 22 engineering guide, and then obviously inspection 23 procedure 52003 as was discussed earlier. For those 24 who are not familiar with the Limerick Modernization 25 Project, we are digitizing the reactor protection NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
255 1 system, the nuclear steam supply shut off system, 2 emergency core cooling system.
3 All of which will be replaced with the 4 Westinghouse Common Q platform, and the redundant 5 reactivity control system, or ATWS will be replaced 6 with the Ovation platform. I'll be sharing some 7 licensee learnings and observations. We've covered a 8 lot of ground today, so I apologize in advance if 9 there's any overlap with presentation materials. So, 10 with that I'm going to touch very briefly on ISV 11 versus MSV.
12 There's been a lot of good discussion 13 today on that topic. I'll move forward into 14 presubmittal meetings, learnings from that, and the 15 NRC review process, and presubmittal discussion. So, 16 if you could proceed to the next slide. Okay, again 17 I'm going to cover this in very brief form. It's very 18 clear that this is a work in progress, and that the 19 industry, and the NRC are working to find a viable 20 solution.
21 I did want to share a couple of thoughts 22 that are pertinent to the subject. The multi stage 23 validation introduces a degree of what we perceive to 24 be some regulatory uncertainty in the form of neither 25 any IEEE 2411, or the NEA 7466 having been endorsed by NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
256 1 the NRC. I know there is some degree of adaptation 2 for that, but neither is an approved document. The 3 second sub bullet here, the potential applicability of 4 NuScale MSV is an interesting thing.
5 And may change the content for that second 6 sub bullet, but it's going to take a bit of research 7 to correlate two very different designs. And then to 8 circle back on a point that Brian Green made earlier, 9 from Chapter 18, that the ISV has to be complete 10 before the implementation. We do recognize that this 11 does put a timing constraint on both the licensee, and 12 the NRC, so I thought it worth pointing out here.
13 One of the key takeaways from this is the 14 development, and execution of a standalone MSV to be 15 mated up with an ISV implementation plan, we're trying 16 to figure out if that's more burden, or value. And 17 one of the things that's kind of a chief concern for 18 us, there's been a lot of discussion today around the 19 need to do research, and negotiate a process, and 20 create a path forward, and there's a lot of that kind 21 of discussion going on.
22 But for purposes of the Limerick 23 Modernization Project, our LAR submittal schedule has 24 very limited margins. So, delays may emerge from the 25 execution of a first of a kind activity does create NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
257 1 somewhat a degree of urgency to getting resolution 2 behind this. With that I'll move on to the next 3 slide. Okay, there's some pluses, and deltas. So, 4 using the presubmittal meeting to determine the LAR 5 content has been very helpful.
6 I think that was mentioned by the previous 7 presentation. This is particularly important, because 8 there's no single document from the NRC that lists 9 exactly what content has to go in for complex visual 10 modifications as the Limerick project does. Again, 11 the interactions with the NRC staff during the 12 presubmittal meetings is highly beneficial. But one 13 of the things that's been a bit of a struggle for us 14 is determining when the topic has been covered in 15 sufficient depth as to take it off the to do list so 16 to speak.
17 So, we want to make sure that we're 18 covering topics to the staff's satisfaction without 19 over performing on a given topic. The next 20 observation is the NRC post-meeting summaries are 21 sometimes at a level that's high enough that it's hard 22 to determine whether, or not we've fully addressed NRC 23 questions, comments, and concerns. So, deeper detail 24 would be advantageous for both us, and the NRC, the 25 licensees, and the NRC.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
258 1 Three way communications is a good thing.
2 Strictly following the alternate review process 3 becomes a bit more complicated as we're introducing 4 tangent processes to it. Human factors engineering is 5 a really good example, and we've talked about that at 6 length today. One thing I'll say that's been very 7 advantageous for us is getting early staff comments on 8 the presentation material in advance so that -- in 9 advance the presubmittal meeting.
10 So that we have the capability to do 11 whatever research is necessary, and formulate 12 responses to the staff's questions in very timely 13 fashion, so that's been very advantageous for us. And 14 with that, I'll move on to the next slide. Okay, one 15 of the other concerns we have, now we do understand 16 that the license submittal review is a complex, and 17 time consuming undertaking.
18 There's no way to minimalize that. But 19 there may be some opportunities worth exploring that 20 could reduce the total time require for LAR review, 21 and this in turn would reduce, or could reduce project 22 risk. As Warren Odess-Gillett, and Mike Waters 23 mentioned earlier this morning, providing a licensed 24 amendment, and draft form in advance of submittal does 25 provide an opportunity to give early staff feedback.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
259 1 This would be similar to the approach for 2 I believe Waterford, and Vogtle 3, and 4 for their 3 tech spec reductions. If there are areas that do 4 require additional content, or detail, it's better to 5 find them in advance, so that they can be addressed in 6 the LAR, rather than being addressed through avoidable 7 RAI. And we can leverage the presubmittal form as a 8 good place for that kind of closed loop 9 communications.
10 Second thing to consider, while every 11 licensed amendment is going to be unique, there may be 12 some opportunities to structure the presubmittal 13 meetings in a way that provides more efficiency, or 14 maximizes efficiency to ensure that specific topics 15 are addressed to staff satisfaction. This would 16 reduce the likelihood of avoidable RAIs. And the 17 thinking would be, the structure could be framed 18 around the scope of the project.
19 Not every project is similarly scoped, 20 some of them require HFE considerations, others do 21 not. So, there's going to be -- it's not intended to 22 be a one size fits all, but there may be some value in 23 coming up with somewhat of a framework that licensees, 24 and the NRC can operate to. So, that's the first 25 major bullet. The second major bullet I wanted to NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
260 1 talk about is something I just wanted to kind of put 2 on everybody's radar.
3 As this audience knows, the industry has 4 adopted the standardized digital engineering process, 5 or NSB104, and the EPRI Digital Engineering Guide 6 under the auspices of delivering the nuclear promise.
7 It goes without saying the price of admission is that 8 the design is at a level of maturity that is ready for 9 staff review at the time the LAR is submitted. One 10 thing to be aware of however, the complexity to be 11 aware of.
12 It's that the DEG is an iterative systems 13 engineering based process. This may create situations 14 where design artifacts may have minor evolutions over 15 time, and during the license amendment review. These 16 would be refinements, they would not be functional 17 changes. But this could create a degree of 18 discontinuity between the design process, and the 19 review process that the licensee, and the staff just 20 need to be aware of, and communications are going to 21 be essential as we progress here.
22 With that, that concludes my presentation.
23 And I'll move forward to any questions, comments?
24 MR. JAIN: Yes, Mike? Mike, you have to 25 unmute.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
261 1 MR. WATERS: Thanks, did you get that 2 John? I'm joking. Great presentation, interesting 3 comment on approving the pre-application process, 4 that's something we should explore further. It's a 5 good point, I think -- I personally view it as the 6 ramp up, or ramping up to a LAR, and of course we want 7 a LAR, no surprise to anyone except in the pre-8 application process as the way we're doing it, it 9 helps us ramp up to that, and gives us that stage.
10 I think you're suggesting something maybe 11 more structured, or formatted, or can you maybe 12 elaborate a little more on what you're thinking there?
13 MR. CONNELLY: Yeah, so to a degree, the 14 presubmittal meetings are -- I won't call them ad-hoc, 15 but they're flexible, and there's a lot of benefit to 16 that to be honest. We can canvas the NRC for topics 17 that they would be interested in hearing at the next 18 meeting. There's -- it's a flexible way to exchange 19 information. But if you -- and here's just one 20 consideration, the human factors was basically off 21 everybody's radar early on in the process.
22 And it would have been beneficial both for 23 the NRC, and the licensees to bring that forward early 24 on. So, if there was somewhat of a structure for 25 things like that that may come into play, that could NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
262 1 be advantageous. That was my thinking.
2 MR. WATERS: Thanks.
3 MR. JAIN: Yes Richard, you can go ahead.
4 MR. STATTEL: Yeah, I was just going to 5 mention the process that we use for pre-application 6 meetings, we essentially create an account. So, we 7 have something we charge our time to, but it's very 8 limited resources. We don't have a lot of resources, 9 so I think there's a little bit of difficulty involved 10 with spending more time, and effort, and resources on 11 pre-application activities.
12 And I think that's probably part of the 13 reason why our meeting summaries are at the high level 14 they are. And this also creates a problem when you 15 talk about free draft submittals. We really don't 16 have a process for that, and we don't have a real 17 means of allocating resources for that kind of 18 activity. So, we hear you, and we kind of went 19 through a pseudo review of a draft version.
20 We did that for Waterford, but again, 21 there's some opposition to doing this kind of out of 22 process activity for which we don't have the real 23 ability to allocate the necessary resources for it.
24 So, I'm not saying it can't change, but it's somewhat 25 problematic in our current environment.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
263 1 MR. CONNELLY: Thanks Rich, that's exactly 2 the kind of feedback that we're looking for.
3 MR. WATERS: Yeah, just to add on, 4 resources will be getting tighter for NRC when we 5 actually have these in house for future applications, 6 but the ultimate challenge, you always hear the 7 regular disclaimer, no decision will be made, and for 8 pre-application meetings, we don't make final 9 decisions that what you're going to submit is going to 10 be okay until we actually see it.
11 So, there's a limit to the feedback we can 12 give, and what we can document in the meeting 13 summaries in terms of if there's an intent to look 14 for, I guess closure to certain issues. And 15 obviously, I think on most topics we leave the 16 meetings with a greater understanding of what's going 17 to be submitted, and I think there's an understanding 18 based on NRC insights, and questions of whether you 19 all will hit the mark on certain topics. But it's 20 certainly something to think about, and improve upon 21 going forward.
22 MR. CONNELLY: Yeah Mike, I agree. We're 23 not looking for a regulatory commitment in the 24 presubmittal meetings, that's not the right forum for 25 that. Decidedly not asking for that, but the more NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
264 1 information that we can exchange, and getting closure 2 to a particular item, it could be something as simple 3 as okay, you've addressed this issue, we understand 4 your position, and we move on from there. It could be 5 something that simple.
6 MR. WATERS: Okay, absolutely.
7 MR. STATTEL: Yeah, part of the problem --
8 this is Richard again. Part of the problem is it's 9 difficult for us to provide formal feedback because 10 that can be used back against us, right? So, it's all 11 pre-decisional, it's pre-application, it's pre-12 decisional. So, if we were to provide formal feedback 13 to you, I think it's a legalistic problem, because 14 that can be construed as being some kind of approval.
15 Or some preempting decision that is made 16 during the license review itself. And so I think 17 there's some legal ramifications that would have to be 18 considered before we're able to do that.
19 MR. CONNELLY: Okay, thank you Rich.
20 MR. WATERS: This is Mike, and I want to 21 leave on a positive. So, I think they've been very 22 beneficial, and I think, I know Turkey Point, and 23 Limerick, Maine. I have great views here, and the 24 real test is when we get the LAR for acceptance 25 review, right? That's the real test, and that's when NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
265 1 we'll know. And I would say, I believe Waterford was 2 really successful.
3 I know looking backwards, after we issued 4 the license, I think that the biggest surprise, and 5 where we needed to align afterwards on the LAR was the 6 DOP for example. That was brand new, but I felt for 7 all the topics for example, Waterford the D3 that 8 required more specifications, and software 9 development, we had a great understanding of what was 10 going to be submitted through pre-application, and got 11 generally what we expected.
12 There were some open items, and RAIs, a 13 little more work for VOP, but in that regard I thought 14 it was pretty successful. I'm hoping that it'll be 15 true for these NextEra, and Constellation, to the 16 extent that we can on all these topics. But it's 17 something that we can continually focus on right up to 18 the LAR.
19 MR. CONNELLY: We share that hope.
20 MR. JAIN: We are pretty close to our 21 workshop schedule, so now the floor is open for any 22 question on any topic anybody might have. Would like 23 to discuss. If not, then I'd like to ask if any of the 24 participants would like to provide closing remarks.
25 MR. WATERS: Well BP, are you asking me, NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
266 1 or someone else?
2 MR. JAIN: Well, first we give a chance to 3 the stakeholders, if anyone of them want to make --
4 MR. WATERS: Yes, please, if there are any 5 members of the stakeholders, or the public that want 6 to make comments.
7 MR. JAIN: So, I see a hand raised. Yes 8 Alan, please go ahead.
9 MR. CAMPBELL: BP, I just wanted to again 10 thank the NRC for the opportunity to have this forum.
11 I think very helpful for many as prospective, and our 12 members. Just note that we had representatives from 13 I&C, cyber, HFE, vendor QA, safety analysis, the 14 regions, this is a very integrated process, and I 15 think that the ability to get us all together, and 16 share information is just invaluable, and so 17 appreciate that.
18 We do look forward to further discussion, 19 specifically on human factors, there's definitely some 20 interest there. Regarding furthering the discussion 21 at the industry level to ensure that we can continue 22 to provide the regulatory confidence using this 23 alternate review process. So, again, thank you all 24 very much for the engagement, and we appreciate the 25 time.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
267 1 MR. JAIN: Thank you Alan. Mike, can you 2 provide the closing remarks for the NRC?
3 MR. WATERS: Well, in a similar manner, a 4 few things, these are my perspectives. Yes, we do 5 need to engage more. And my view on human factors, 6 I'm not a supervisor of human factors, so I'll have to 7 defer to them, but I think a couple things in that 8 area, we recognize human factors is a topic, and I 9 think there's two tracks, and our PM can communicate, 10 well if the licensee is in the hopper on these 11 questions, you know who you are, and we can engage on 12 that in the pre-application submittal conversations 13 that we have ongoing, so we should do that.
14 And yes, if we need to engage further on 15 a joint, more generic basis on human factors, or 16 anything else related to I&C, let's do that. Second, 17 I thought it was a great meeting today. Awesome 18 participation from industry, and as you noted Alan, 19 all the NRC staff, so thanks to them. I did 20 appreciate the presentations, personally from NextEra, 21 and Constellation, Entergy on their perspectives.
22 I honestly wish we had more time to talk 23 about it in a more expanded discussion, so we could 24 really hone down on some improvements here, and 25 perhaps we can do that. Third, please in the next few NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com
268 1 days, weeks, provide BP any feedback on how the next 2 workshop can be better. What can we talk about, can 3 the format be different? And what can we do to 4 improve a future workshop?
5 The what, where, when, why, and how is 6 something we look for feedback as well. So, please 7 feel free to email BP, or even me, or Jeanne Johnston 8 on those topics. But that's great, I would offer 9 Brian, or Lauren, because HFE was a significant topic, 10 do you have any other closing remarks?
11 MR. JAIN: Brian?
12 MS. NIST: Hi, this is Lauren Nist, sorry.
13 I am the Operator Licensing and Human Factors Branch 14 Chief, and unfortunately Brian had to leave the 15 meeting. But I think I'll just echo your comments 16 Mike, and thank you everybody for your discussion 17 today.
18 MR. JAIN: Okay, I'd like to thank 19 everyone for their time, and if you have any comments, 20 or feedback on any aspect of this workshop, please 21 contact me, or Michael Marshall. We will provide you 22 the necessary forms. With that, the meeting is 23 adjourned, thank you everyone.
24 (Whereupon, the above-entitled matter went 25 off the record at 4:37 p.m.)
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1716 14th STREET, N.W., SUITE 200 (202) 234-4433 WASHINGTON, D.C. 20009-4309 www.nealrgross.com