ML19085A447
| ML19085A447 | |
| Person / Time | |
|---|---|
| Issue date: | 03/27/2019 |
| From: | Joe Golla NRC/NRR/DLP/PLPB |
| To: | |
| Golla J A, NRR/DLP, 415-1002 | |
| References | |
| NRC-3986 | |
| Download: ML19085A447 (201) | |
Text
Official Transcript of Proceedings NUCLEAR REGULATORY COMMISSIONTitle:Inspection of Digital Instrumentation and Control Installations Approved under the
Guidance of ISG-06Docket Number:(n/a)
Location:Rockville, Maryland Date:Thursday, November 16, 2018Work Order No.:NRC-3986Pages 1-200 NEAL R. GROSS AND CO., INC.
Court Reporters and Transcribers 1323 Rhode Island Avenue, N.W.
Washington, D.C. 20005 (202) 234-4433 1 UNITED STATES OF AMERICA 1 NUCLEAR REGULATORY COMMISSION 2+ + + + +3 INSPECTION WORKSHOP 4+ + + + +5 INSPECTION OF DIGITAL INSTRUMENTATION & CONTROL 6 INSTALLATIONS APPROVED UNDER THE GUIDANCE OF ISG-06 7+ + + + +8 THURSDAY 9 NOVEMBER 15, 2018 10+ + + + +11 ROCKVILLE, MARYLAND 12+ + + + +13 The workshop convened at the Nuclear 14 Regulatory Commission, One White Flint North, Room 15 10B04, 11555 Rockville Pike, at 1:00 p.m., Joe Golla, 16 NRR/DLP/PLBP, presiding.
17 18 NRC STAFF PRESENT:
19 JOE GOLLA, NRR/DLP/PLPB 20 AARON ARMSTRONG, NRO/DCIP/QVIB1 21 ERIC BENNER, NRR/DE 22 DOUG BOLLOCK, NRR/DIRS/IRIB 23 CALVIN CHEUNG, NRR/DE/EICB 24 SAMIR DARBALI, NRR/DE/EICB 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 2 THEO FANELLI, R-II/DRS/EB1 1 GREG GALLETTI, NRO/DCIP/QVIB1 2 PHIL McKENNA, NRR/DIRS/IRGB 3 RICHARD STATTEL, NRR/DE/EICB 4 MIKE WATERS, NRR/DE/EICB 5 DEANNA ZHANG, NRR/DE/EICA 6 7 8 ALSO PRESENT:
9 NEIL ARCHAMBO, Duke Energy 10 MARK BURZYNSKI, Rolls-Royce 11 STEVE DRAGOVICH, Exelon 12 PAREEZ GOLUB, EXCEL Services 13 RAY HERB, SNC 14 DAVE HERRELL, MPR Associates 15 FRANK NOVAK, GE Hitachi 16 WARREN ODESS-GILLETT, NEI 17 JOHN SCHRAYE, Entergy 18 STEPHEN VAUGHN, NEI 19 20 21 22 23 24 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 3 CONTENTS 1 Opening Remarks.................4 2 ISG-06 Alternate Review Process Summary 3 ISG-06 Update...............9 4 Industry Expectations on Alternate Review 5 Process Inspections............58 6 NRC Activities Related to DI&C Modifications 7 (General Roles and Responsibilities)
....8 Example of Audits and Inspection Activities 9 for a Modification Performed Under ISG-06 10 Rev. 2 Alternate Review Proces s.......11 Vendor Oversight Plan Discussion.......149 12Workshop Closing Comments/Follow-Up Actions..185 13 Public Comments................200 14 Adjourn....................200 15 16 17 18 19 20 21 22 23 24 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 4 P-R-O-C-E-E-D-I-N-G-S 1 1:00 p.m.2 MR. GOLLA: Okay, let's go ahead and get 3started. I'm Joe Golla. I'm the ISG-06 project 4 manager in the NRR Division of Licensing Programs.
5 The purpose of the workshop today is to 6 address audits and inspection of digital 7 instrumentation and control installations approved 8 under the guidance of the ISG-06 Rev. 2 licensing 9 process, particularly the Alternate Review Process 10 therein.
11 For this category two public meeting, all 12 portions are open. Comments from the public will be 13 taken before adjournment of the meeting.
14 We do have an attendance list going around 15 and we invite you to enter your information on that.
16 That will become part of the public record along with 17 the meeting summary.
18 And for folks on the phone, if you would 19 like your participation in the meeting reflected in 20 the attendance list, please send me an email at 21 joe.golla@nrc.gov.
22 For folks in the room, there's a little 23bit of housekeeping we always have to mention. In the 24 event of a fire alarm, listen for the announcement.
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 5 If instructed to do so, make an orderly exit using the 1 stairs, not the elevator.
2 The stairs are opposite the restrooms over 3 here, and turn right out of the building and make the 4 first right on Marinelli, and the mustering point is 5down the road one block. Turn right again and that's 6 that.7 Okay, so with that, let's have 8introductions. Again, I'm Joe Golla, the project 9 manager, NRC.
10 MR. WATERS: I'm Mike Waters. I'm Chief 11 of the Instrumentation Control Branch B at NRR.
12MR. STATTEL: Richard Stattel, I&C 13 technical reviewer.
14MR. DARBALI: Samir Darbali, I&C technical 15 reviewer.16MS. GOLUB: Pareez Golub, EXCEL Services.
17 I lead the industry team.
18MR. ODESS-GILLETT: Warren Odess-Gillett, 19 NEI. I co-lead the industry team.
20 MR. HERB: Ray Herb, Southern Nuclear.
21MR. DRAGOVICH: Steve Dragovich, corporate 22 engineering, EXELON.
23 MR. VAUGHN: Steve Vaughn, NEI.
24 MR. FANELLI: Theo Fanelli, Region II.
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 6MR. GALLETTI: Greg Galletti, vendor 1 inspection group.
2MR. BOLLOCK: Doug Bollock, I'm in the 3 Division of Inspection and Regional Support.
4MR. McKENNA: Phil McKenna, I'm in the 5 Division of Inspection and Regional Support.
6MR. CHEUNG: Calvin Cheung, I&C technical 7 reviewer, NRR.
8MS. ZHANG: Deanna Zhang, NRR, I&C 9 technical reviewer.
10MR. SCHRAYE: John Schraye, Entergy 11 corporate licensing.
12MR. BURZYNSKI: Mark Burzynski, Rolls 13 Royce.14MR. HERRELL: Dave Herrell, MPR 15 Associates.
16 MR. NOVAK: Frank Novak, GE Hitachi.
17MR. ARCHAMBO: Neil Archambo with Duke 18 Energy.19MR. GOLLA: Okay, thank you. How about 20folks on the phone? Would you care to introduce 21 yourself?22MS. ALVARADO: This is Rossnyev Alvarado.
23 I am the reviewer for NRR.
24MS. VENKATARAMAN: This is Booma 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 7 Venkataraman at DORL, the Division of Operating 1 Reactor Licensing.
2MS. SALGADO: Nancy Salgado, NRR. I'm 3 with the Alpha Branch.
4 MR. GOLLA: Okay, do we have any --
5MS. BERGMAN: Jana Bergman, Curtiss-6 Wright.7MR. GOLLA: Oh, Jana Bergman, yes. Jana's 8with Curtiss-Wright, she said, a writer. Anyone else 9 on the phone?
10MR. STERLING: Webb Sterling, NextEra 11 Energy.12MR. GOLLA: Okay, that sounds like it.
13 Mike, would you want to make any opening remarks?
14MR. WATERS: Yeah, I mean, I'll let Samir 15 lead the meeting, Samir, but I just want to say 16 welcome. I am happy to see a full room of people on 17a day like today. I am actually surprised to. I 18heard someone say they were across the room. We have 19 been looking forward to this workshop.
20 I know some of the goals are to give an 21 overview of NRC's inspection process from a 22 programmatic standpoint and talk about how it may be 23 streamlined or adjusted in the future as part of the 24 ISG-06 alternative review process. It sounds like a 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 8 lot of that will be centered around the licensee's 1vendor oversight plan. That's a hard thing for me to 2 say, so I look forward to dialogue.
3 I want to thank just similarly my team of 4 experts in technical review, licensing, and 5inspection, to get this dialogue started. With that, 6 I'll, I guess, I don't know if you have any remarks, 7 Pareez?8MS. GOLUB: First up, thank you very much 9 to all of you. It's wonderful to see some new faces 10 here, so we really appreciate you taking the time and 11coming out to join us today. We've been looking 12 forward to this workshop.
13And I know I've said this before, but 14 thank you very much to the NRC team that's been 15working on this for a long time, especially I know 16 over the last six months, you guys have done a lot to 17 keep this, to keep ISG-06 moving through the process.
18 We really appreciate everything you guys 19 have been doing to keep it on schedule and moving 20forward. The industry is eagerly anticipating getting 21 a chance to use this document, so thank you.
22MR. DARBALI: Thank you, Pareez. So, all 23 right, so we'll get started. I guess the first item 24 in the agenda is we're going to do a quick 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 9 presentation of ISG-06, kind of like a summary of 1that. Joe, if you can open the one, the general roles 2 and responsibility?
3MS. GOLUB: Samir, could I just ask a 4 quick question?
5 MR. DARBALI: Sure.
6MS. GOLUB: So if we have questions, 7 should we ask them along the way?
8 MR. DARBALI: Yeah, yeah, go ahead.
9 MS. GOLUB: All right.
10MR. DARBALI: All right, so the outline of 11 this presentation will take about 10 minutes to just 12 do a summary of what the Alternate Review Process is 13 and we'll provide an update on where the ISG is, and 14 then we have a separate presentation on, it's kind of 15 like an overview of what the different NRC staff 16 activities are when the DI&C modification is involved, 17 so we'll get to those. Next slide.
18 Okay, so the purpose and scope of ISG06, 19 and again, the overall purpose did not change from 20Revision 1 to Revision 2. We just made a change to a 21 new process, but ISG-06 defines the licensing process 22 used to support LAR reviews associated with DI&C, 23 equipment modifications, new operating plans, any new 24 plans once they become operational.
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 10 So that means that if a Part 52 plant 1 being under construction is going to do an LAR on the 2DI&C, this process would not be used, but once the 3 plant is in operation and they want to do a mod, then 4 the ISG-06 can be used.
5 The ISG provides guidance for activities 6 performed before the LAR such as the preapplication 7 coordination meetings and doing the LAR review. The 8 NRC staff uses the process described in the ISG to 9 evaluate compliance with NRC regulation, and the ISG 10 makes reference to and is supposed to be used in 11conjunction with SRP Chapter 7. Some people in the 12 past have asked is this meant to replace the SRP and 13we have clarified that. It's meant to be used 14 together. Next slide.
15 So you have seen this slide before. The 16 blue area in the middle covers what we in the past 17 typically identified as the licensee and vendor 18 development activities for the application.
19 I understand that under the detailed 20 engineering guide, those particular activities will 21 have different not mains, so because the staff has not 22 yet so familiarized with the DEG, we didn't change it 23 for this slide, but definitely in the future as we get 24 more familiarized with the DEG, definitely the 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 11 licensee can put a name, the name that they want to 1 the life cycle case and we can introduce that, but 2 it's meant to cover those activities.
3 The top portion of this slide shows what 4 review under the Tier 1, 2, and 3 review process would 5 look like where the LAR is submitted with the Phase 1 6 information before detailed hardware and software 7 design is provided.
8 And then during the NRC staff's review, 9 the licensee keeps producing and submitting the Phase 10 2 information, and then the license amendment is 11 issued after that, and then we have during site 12 acceptance testing and installation, then we have some 13regional inspections. This is the process we've been 14 working under currently and it's maintained under 15 Revision 2 of the ISG.
16 The new process that's been introduced, 17 which is the Alternate Review Process, is described at 18 the bottom where the LAR with all of the information 19needed to reach a safety determination is provided 20 during the detailed design review, but before 21 completion of the detailed design.
22 The staff would do their LAR review and 23 regulatory audits, and the plan would be to do 24 license, issued license amendment before factor 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 12 acceptance testing, and then that opens up this new 1 area of regional and vendor inspections of those 2 activities that we used to review before, and that's 3 the focus of this workshop, so next slide.
4MR. ODESS-GILLETT: Samir, may I make some 5 clarifications -
6 MR. DARBALI: Sure.
7MR. ODESS-GILLETT: - and comments on this 8slide here? And I'll speak loud so people on the 9 phone can hear me. But this here represents a scope 10of review. Would you agree that these bars here 11 describe, well, this is what the NRC would review as 12 part of the traditional tiered review?
13 MR. DARBALI: Correct.
14MR. ODESS-GILLETT: However, I don't think 15 that's the case down in this diagram because although 16you have the bars here, really it's here. The LAR 17 review content would be -
18 MR. DARBALI: Right, so -
19MR. ODESS-GILLETT: So this red line 20 really is not an expression of scope of review, and 21 this red line could almost be anywhere in here because 22 the scope of the review is here. Is that -
23 MR. DARBALI: That's correct, yes.
24 MR. ODESS-GILLETT: Okay.
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 13 MR. DARBALI: So it's a timeline. There 1 are some audit activities that would take place.
2MR. ODESS-GILLETT: Audit, yeah, audit, 3 but that, I mean, this is a time period.
4 MR. DARBALI: Right.
5MR. ODESS-GILLETT: But I would imagine 6 the audit activities would comprise this?
7MR. DARBALI: That would be the material -
8MR. ODESS-GILLETT: That would be audited?
9 MR. DARBALI: - in the LAR, correct, and 10 whatever audit would support the LAR.
11MR. ODESS-GILLETT: Whatever audit they 12 need to do.
13 MR. DARBALI: Correct, Rich?
14MR. STATTEL: So I'd like to add some 15clarification. So the green lines actually represent 16the time of submittal. So because the Alternate 17 Review Process is a one submittal process, that's all 18we get is what's at the green line.
So pretty much 19 everything we're looking, it took place and was 20 developed prior to that time.
21 That's not the case in the upper part of 22 this chart because the upper part is a two-part 23 submittal and we actually have purview and we will 24 review test activities, test reports, V&V summary 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 14reports which don't become available until later in 1 the process.
2 So the Phase 2 submittals actually are 3 basically material that's developed while we are in 4progress of developing the safety evaluation. So it's 5 basically we're doing our evaluation in parallel with 6 the development activities.
7 MR. DARBALI: Right.
8MR. STATTEL: We're going to be reviewing 9 products of those development activities during that 10 evaluation. That's what we do.
11 MR. DARBALI: So the review for the tier 12 process is comprised of the green line -
13 MR. ODESS-GILLETT: Right.
14 MR. DARBALI: - and the white box.
15 MR. ODESS-GILLETT: Correct.
16 MR. DARBALI: So Phase 1 and Phase 2.
17MR. ODESS-GILLETT: But Rich, were you 18 describing the Alternate Review Process?
19 MR. STATTEL: No, no.
20 MR. DARBALI: The tier process.
21 MR. ODESS-GILLETT: Got it.
22MR. STATTEL: The Tier 1, 2, and 3 review 23 process.24MR. ODESS-GILLETT: Okay, so just for 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 15 clarification, this red line doesn't mean that the 1 license amendment occurs necessarily with the detailed 2 hardware and software design fabrication ending? It 3 could. It could.
4 MR. DARBALI: It could, correct.
5MR. ODESS-GILLETT: It could be here. It 6 could be here and who knows where?
7MR. DARBALI: And in the other set of 8 slides that we talk about, the Oconee example, you'll 9 see we kind of put, like below the dash lines, we put 10like a more expanded area which describes that. It 11 could be before or after, right. So -
12MR. ARCHAMBO: Would there be an option if 13 a licensee wanted to get the license amendment before 14they even started doing their detailed design? I 15mean, I think that's what we're kind of asking. Based 16 on the first two boxes, you issue that and you get 17 your license amendment, and then you might decide to 18 do your detailed design. Is that an option?
19 MR. DARBALI: Well, I mean, every design 20 is different and the idea of what the slide is trying 21 to show is as the design is progressing, the licensee 22 and the vendor are doing, following their own 23schedule. That would be about the same time it would 24 take us to do a license amendment.
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 16 For the license amendment request, there 1 is going to be an amount of detailed hardware and 2 software information that's going to be needed to be 3included in the LAR. So I don't think that you could 4even submit the LAR before you started FAT.
5 Definitely you could submit it before you've completed 6 the design.
7MR. WATERS: This is Mike. I mean, 8 1:15:31 general understanding of how things would go.
9 I think the opposite side of the coin is under the 10 paradigm review process, you have a single submittal 11 with the acceptance review.
12 There might be a mindset to say well, they 13can supplement later. They can do this later, but 14 we'll do the acceptance review and either accept or 15 deny or whatnot under this new paradigm.
16 So I heard someone a few weeks ago who 17 said, "Well, we could always supplement if needed." 18 Well, that's not the case under this new paradigm.
19 It's a single submittal and alterations needed will be 20 in part at the time of the submittal where the red 21 line is shown.
22MR. HERB: Can I clarify? I think what 23 Neil was asking was there are certain, typically most 24 plants have certain resourcing schedule constraints, 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 17 and we would like the option to be able to submit LAR 1 once we have enough information for that LAR, to 2 populate that LAR, and we may not be purchasing 3 hardware yet.
4 And so we may wait until the LAR approval 5 before we even start to purchase hardware and they 6 start to actually stack and rack components, and so, 7 but we don't anticipate it to happen, but we just want 8 to make sure that you're not anticipating that you're 9 going to be able to see some of that stuff just 10 because of the timing issues and they all stack up and 11 that you need some of that rack and stack data to 12 approve the LAR.
13 MR. DARBALI: Right.
14MR. HERB: Because that's not our 15understanding. It's that the approval is on the 16 requirements -
17 MR. DARBALI: Correct.
18MR. HERB: - and not on the actual 19equipment. Now, the actual equipment is in the 20 inspection space, of course, and has to be confirmed.
21MR. DARBALI: And that detailed discussion 22 would occur during preapplication.
23 MR. HERB: That's right, so, yeah.
24MR. DARBALI: So we would identify what 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 18 parts of the ISG-06 you can provide the information 1 for.2MR. HERB: Yeah, what we don't want is 3 like to say, "Well, we can't finish our review until 4you buy equipment." You know, that kind of like, you 5 know, and we're just kind of slowing it down because 6 we know you're going to buy some stuff and we're going 7 to want to look at some of that stuff.
8MR. DARBALI: When you say "buy 9 equipment," you mean -
10 MS. GOLUB: That's the application part.
11MR. DARBALI: Right, you already 12 determined what the platform -
13 (Simultaneous speaking.)
14MR. HERB: Oh, yeah, correct, we would 15 never submit a LAR on an open platform -
16 MR. DARBALI: Okay.
17MR. HERB: - not specified. I mean, that 18 would - how would you even do that?
19 MR. DARBALI: Right, right, so you would 20have engaged the vendor. You would have had that 21 stuff on contract -
22 MR. HERB: Oh, yes.
23 MR. DARBALI: - but not purchased.
24 MR. HERB: The PO would be in place. We 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 19 would already have identified the scope of the mod -
1 MR. DARBALI: Right.
2MR. HERB: - the equipment and all that 3stuff, but we may not actually, the vendor may not 4 actually -
5MR. DARBALI: Starting working on the 6 actual programming.
7 MR. HERB: Sure.
8MR. WATERS: So I recollect on the last 9 public phone call, there was offered the idea that 10 maybe the blue line representation may be changed with 11your current process and then provided to us. Do you 12 have something to share, I mean, to help maybe 13 illustrate this better for future meetings?
14MS. GOLUB: Well, so the digital 15 engineering guide is the process, and on Monday of 16 this week, the court -
17MR. HERB: Actually, we're going to go 18 into that in the basic presentation.
19MS. GOLUB: Oh, that's right. That's 20right. Okay, so, yeah, so, but we'll get to that 21 soon.22MS. ZHANG: So I do have a question in 23regards to it. We have talked about using ISG-06 not 24 only for LAR reviews, but also for platform reviews 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 20too. Do you see any changes? Do you see using the 1 Alternate Review Process for platform review or would 2 you only use it -
3 MS. GOLUB: No.
4 MR. HERB: We don't see -
5 MS. GOLUB: Right.
6MR. DARBALI: Did you ask specifically for 7 the Alternate Review Process? Yeah, I don't -
8 MS. GOLUB: Yes.
9MR. DARBALI: It would be like on the same 10 page here.
11MR. STATTEL: Well, we, on the onset of 12 this - 13MR. BURZYNSKI: I think some vendors might 14 be interested in using the descriptions in there of 15 what kind of technical information to provide.
16 MR. DARBALI: But I think -
17MR. BURZYNSKI: - but was not actually 18 quoted under the alternate review.
19 MR. DARBALI: Right, right, okay.
20MS. GOLUB: Yes, yes, that's a really good 21 point.22MR. STATTEL: And we discussed this at the 23 beginning of this when we were developi ng this 24 revision and topical reports were not within the scope 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 21 of the ISG-06 revision.
1 MR. DARBALI: So I don't know if we want 2 to maintain - I don't think we've had the discussion 3 that we want to maintain topical report reviews under 4 the Rev. 1 or to keep them under -
5 MR. STATTEL: Well, it's just guidance.
6 MR. DARBALI: Right.
7MR. STATTEL: So in reality, every 8application is unique. Every - there is not one model 9 that's going to fit all applications. In reality, a 10 topical report review is just another type of 11application. Some of it applies, some of it doesn't, 12 so, and we've been dealing with this for several years 13 now with the original revision of ISG-06.
14 Now, we've applied ISG-06 as best we can 15 with the understanding that a lot of the regulatory 16 requirements won't be met until an application is 17 developed, so, you know, we have a process for dealing 18 with that.
19 So I think in truth, we can apply, even 20 though it's not with really, it wasn't developed with 21 this in mind, topical report review, we can apply the 22 criteria in some topical areas to topical report, and 23 that's what we intended to do that.
24 MR. DARBALI: Okay, next slide. So some 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 22 characteristics of the LAR used in the Alternate 1Review Process, the LAR would provide the necessary 2 instrumentation and design information to demonstrate 3 regulatory compliance. We've talked to that.
4 The LAR would describe the licensee's 5 Vendor Oversight Plan that ensures that the vendor 6executes the project. Actually, we haven't updated 7 this based on the language that the ISG was updated 8 to. 9 So the ISG would say that the LAR 10 describes the licensee's Vendor Oversight Plan which, 11 let's see, which once executed, can help ensure that 12 the vendor executes the product consistent with the 13 LAR and also once executed, can help ensure that the 14 vendor uses as a reference the 2015 version of NQA 1.
15MR. ODESS-GILLETT: So we had some 16 internal discussion on that. I know this is wording 17 and the purpose of this workshop is not to affect -
18 MR. DARBALI: Right.
19MR. ODESS-GILLETT: - the ISG-06 wording, 20 but we would like a clarification or interpretation of 21 the NRC staff expectation that the vendor's Appendix 22 B program is going to actually reference 2015 of NQA 23 1.24MR. DARBALI: So I don't - the language in 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 23 the ISG does not ask for the vendor's QA program to 1 reference the 2015 version. So the only "should" or 2 requirement here, and I'm using the word loosely -
3 MS. GOLUB: This is the language just in 4 case you want to see it.
5MR. DARBALI: Okay, is that the - it talks 6 about the VOP, but it doesn't talk about modifying the 7vendor's QA program. So the VOP should make sure that 8 somehow the NQA 1 is used as a reference, but it 9 doesn't mean it has to be implemented.
10MR. ODESS-GILLETT: So as a reference 11 refers to the VOP versus as a reference to the 12 vendor's QA program?
13 MR. DARBALI: Could you repeat that?
14 MR. ODESS-GILLETT: You should have as a 15 reference -
16 MR. DARBALI: Yes.
17 MR. ODESS-GILLETT: - the 2015.
18 MR. DARBALI: Yes.
19MR. GALLETTI: Just to clarify, the 20 purpose of using it as a reference is we don't want to 21 formally dictate that any vendor has to apply that 22 version of NQA 1 or any version of NQA 1.
23 If, however, you licensees, as part of 24 this oversight process, decide that you want them to 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 24 apply that to the purchase orders, you specify that.
1 Originally as this was written, it required them to 2 follow the 2015 version of NQA 1 and we would never 3 specify that.
4MR. DARBALI: So the original language was 5 consistent with the NQA 1 version and I think that 6 caused some problems, so we didn't -
7 MR. GALLETTI: We didn't want to dictate 8 that.9 MR. DARBALI: Right.
10MR. GALLETTI: You licensees need to 11 determine what quality program you want them to work 12under or approve the one they have and be satisfied 13 with something.
14MS. GOLUB: Could we possibly capture that 15in the notes? Because I know at least on our side, we 16 were, there was a lot of confusion in that discussion 17 that we had earlier today because we were kind of 18 worried when we saw the word "reference" that it meant 19 we would literally in the vendor's program a reference 20 to the 2015 version.
21MR. GALLETTI: No, no, that wouldn't make 22 sure it's necessary.
23 MS. GOLUB: Right, I agree with you, and 24 I know that was not the intent here.
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 25 MR. DARBALI: No.
1 MS. GOLUB: But the way that the wording 2 is written, we think, because, you know, we don't have 3 confirmation of that final document, we're just 4 concerned that people may misinterpret that who 5 weren't part of these initial discussions, so if it's 6 possible to at least capture that in the notes, 7 because I know it's too late now to change the ISG.
8MR. DARBALI: This is the latest language, 9 you guys.10MR. GALLETTI: It would be too late for a 11 wording change there too?
12 MR. DARBALI: Do you have the latest?
13 (Simultaneous speaking.)
14MS. GOLUB: This is - what we're 15struggling with is the "as a reference." We're 16 worried that somebody is going to pick it up and think 17 that the vendor's QA program has to reference this 18 2015 version, but when you say "as a reference," you 19 mean "consistent with," right, not so much that it's -
20MR. DARBALI: Well, that the NQA 1 is 21 considered.
22MS. GOLUB: Right, so the problem is this, 23 we're worried that somebody is going to see that as a 24 reference. If this wording can't be changed because 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 26 it's really far down the pike now -
1 MR. DARBALI: Right.
2MS. GOLUB: - then what we were hoping is 3 that as part of the notes for this meeting, we could 4 at least capture -
5 MR. DARBALI: Right, clarify that.
6 MR. GALLETTI: The interpretation.
7MS. GOLUB: Right, this - what's your 8 name?9 MR. DARBALI: Greg.
10 MS. GOLUB: Yeah, I think what Greg said 11was spot on. I think it's been all of our, you know, 12 all of our interpretation of that language, but today 13 we kind of looked at it, you know, and thought about 14it with fresh eyes. We were worried that in the 15 future, somebody may interpret that differently.
16 MR. DARBALI: So if -
17MS. GOLUB: Especially in a couple of 18 years, you know, when there's a new version -
19 MR. DARBALI: Right.
20 MS. GOLUB: - of NQA 1.
21MR. DARBALI: And I know we're not 22 rewriting it, but if it said, and I'm going to read 23from the latest language, let's see, right, "The 24 Vendor Oversight Plan, when execute, can also be used 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 27 to ensure that the vendor uses as a reference the 2015 1version." If it said, "can also be used to ensure 2 that the vendor references the 2015 version," -
3 MS. GOLUB: So the problem is, you know, 4 there's a list of references, right?
5 MR. DARBALI: Right.
6 MS. GOLUB: So like I'm a little worried 7 that people are going to say, "Hey, you know, where's 8 that reference to the 2015 version?" 9 MR. DARBALI: Right.
10MS. GOLUB: And so, and I hear Greg saying 11 that, you know, there is no intention here of NRC 12 imposing a certain -
13 MR. DARBALI: Right.
14MR. FANELLI: But shouldn't it say 15 something along the lines of the reference is whatever 16the licensee commits to in their license amendment 17 request?18MR. WATERS: Can I interject something?
19 I thought in previous meetings, it was the industry 20 use of this 2015 version.
21 MR. DARBALI: That's right.
22 MR. WATERS: So I'm trying to understand 23 how it evolved from that -
24MS. GOLUB: It's a framework because we -
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 28MR. ODESS-GILLETT: If we could just apply 1 it to the Vendor Oversight Plan and not to the QA 2 program, I think NEI will be satisfied.
3MR. DARBALI: Well, and we're not tying it 4to the vendor QA program. You had language there 5 before that said, "as implemented by the licensee's -
6 by the vendor," so we removed -
7MR. HERB: It's supposed to be the Vendor 8 Oversight Plan.
9 MR. DARBALI: Yes.
10 MR. HERB: So maybe there's a way we can 11 change the wording to make that more -
12PARTICIPANT: Should it be "licensee uses" 13 versus "vendor uses?" 14MR. ODESS-GILLETT: Yeah, yeah, that would 15 work.16MR. GALLETTI: I mean, because a lot of 17 the licensees are currently licensed to N452 18 standards.
19 MR. ODESS-GILLETT: I know. This is -
20MR. GALLETTI: So we're not trying to 21 impose -22 MS. GOLUB: No, but this is -
23MR. ODESS-GILLETT: I understand, Greg, 24 but this was sort of a step up that the staff wanted 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 29 licensees because, or the NEI team thought those 1 licensees that really don't have an NQA 1 QA program, 2 that some kind of step up was needed for the licensee 3 oversight, and that's why that language is there.
4 MR. GALLETTI: I understand, because sub 5 part two and part seven specifically talks about -
6 MR. ODESS-GILLETT: Yes.
7 MR. GALLETTI: - software quality -
8MR. ODESS-GILLETT: Exactly correct.
9 Exactly correct.
10 MR. GALLETTI: - where other QA programs 11 would go into that level of detail.
12MR. ODESS-GILLETT: Exactly, exactly 13 correct.14MS. GOLUB: But that's why the "consistent 15 with" language is used because the idea was not to 16 impose it, but -
17 MR. DARBALI: Right.
18MS. GOLUB: - just give a common framework 19 so we all, you know, kind of had the same mindset.
20 MR. DARBALI: So -
21MR. GALLETTI: And we took it just the 22 opposite.23 MR. DARBALI: Right.
24MR. GALLETTI: When you said, "consistent 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 30 with," it beared a sense that it was going to be 1 required of them, where this "as a reference" says, 2"Hey, just consider what this says." It has some 3 current language, current thinking of the industry 4 when you apply -
5 MS. GOLUB: Yes.
6MR. GALLETTI: - whatever you're doing to 7-8 MS. GOLUB: Yes.
9MR. GALLETTI: - the vendor, but not to 10 force a commitment in any way.
11MS. GOLUB: Yes, that's the intent. Maybe 12 that word "considered with" or, you know -
13 MR. VAUGHN: It sounds like we all agree 14 on what it should say, but we're interpreting the 15 complete opposite than you are, so maybe we should 16 just take an action to clarify it, and read it, and 17 make sure all of us agree that it says what we want it 18 to say?19 PARTICIPANT: So we'll -
20MR. WATERS: I can't commit to revising 21the ISG at this point. We'll look at that. Our legal 22staff, rightly so, will give me a hard time. They 23 didn't want to crush their holidays with changes, so, 24 or those other processes, but what's the best way to -
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 31MR. ODESS-GILLETT: We were just thinking 1 maybe documenting -
2 MS. GOLUB: In the notes.
3 MR. ODESS-GILLETT: - the interpretation 4 in the leading summary would suffice industry.
5 MR. WATERS: I don't an issue with that, 6 and we can see if we can make any last second changes.
7MS. GOLUB: Right, we had kind of made 8 that assumption, so we were hoping as long it's 9 documented here, we have something that we can, you 10 know, we can point to that captures the intent.
11 MR. DARBALI: Okay.
12MR. VAUGHN: Or even during the review, 13 maybe just specifically ask, "Hey, what does this mean 14 to you?" 15 MR. DARBALI: Right.
16MR. VAUGHN: And if that lawyer says it 17 means - I'd be interested to see what their 18 interpretation would be.
19MR. DARBALI: Okay, no, that's good 20feedback. Thank you. Okay, and then the LAR includes 21 appropriate commitments to complete plan specific 22 actions that are included in the reference topical 23 report, and the LAR includes appropriate commitments 24 to complete life cycle activities under the licensee's 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 32 QA program.
1 So this particular workshop is not on 2 commitments or conditions, but one aspect of the 3 Alternate Review Process is that it relies on the LAR 4 containing license information and additional 5 regulatory commitments to implement those remaining 6 development phases, activities by the licensee's QA 7 program after the license amendment is issued, and the 8 staff may likely translate some of those commitments 9 into license conditions. Next slide.
10 Okay, so a quick update on the ISG, we 11 have completed the concurrence process -
12 MS. GOLUB: Okay, great.
13MR. DARBALI: - and incorporated those 14concurrence comments. It's currently with OGC, so 15 they're doing their no legal objection review, and 16 we've had some interactions with the ACRS and we 17 believe that their concerns have been addressed.
18 MS. GOLUB: Great.
19MR. DARBALI: We are - so they provided us 20 with a letter when we presented to them back in July, 21and that letter asked us for a series of things, 22 including issuance for public comments.
23 They had us address some licensee 24 oversight for ownership and hardware configuration 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 33 management, and we incorporated that in the version 1 that went out for public comments, and then they said, 2"Well, after you've addressed public comments, send it 3 back to us," and we did that.
4 But they also had a concern in the body of 5 the letter that said - it addressed control of access, 6 so we're in the process of providing a second response 7 letter to them telling them how the ISG and other NRC 8 activities address control of access.
9 So although we haven't heard a final 10 message from the ACRS saying there is no need for 11 another presentation, we believe we're on that path 12 that we don't need to present, so it looks like we 13 will be able to issue the ISG, the final ISG by the 14 end of the year.
15 MS. GOLUB: That's great. That's great.
16 Thank you.
17MR. WATERS: And again, this looks like 18 that's our best understanding -
19 MR. DARBALI: Right.
20MR. WATERS: - of what they're looking 21 for.22MS. GOLUB: Is there like a point at which 23 if the ACRS does not require or request a 24 representation, you kind of say, "Okay, clearly that's 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 34 done with"?
1MR. DARBALI: I think there is, but I 2 think by the time that happens -
3 MS. GOLUB: Okay.
4 MR. DARBALI: - it might be maybe a week 5 or two before the -
6 MS. GOLUB: Okay, all right.
7 MR. DARBALI: - ISG is issued.
8 MS. GOLUB: Okay, okay.
9 MR. DARBALI: Just because there's a lot 10 of things that are going on right now.
11 MS. GOLUB: Okay, understood.
12MR. DARBALI: And there's formal processes 13 for engaging with the ACRS, so it takes some time.
14MS. GOLUB: Okay, understood. Now, after 15OGC, does it still need to go to the CRB and those 16 other -17 PARTICIPANT: OMB.
18 MS. GOLUB: - OMB and all these other -
19 MR. GOLLA: It's there now.
20MS. GOLUB: Oh, it's there now? Okay, 21 great, so it's concurrent.
22MR. DARBALI: Yes, we shored up a previous 23 version that doesn't change much of what they would 24 review, so.
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 35MS. GOLUB: What they review, okay, great.
1 MR. DARBALI: So we're trying to -
2 MR. GOLLA: We asked them to give it the 3 highest priority for our agency at least.
4 MS. GOLUB: Right.
5MR. GOLLA: They have a lot of other 6 things there that they're reviewing, but as far as I 7 know, we're at the top of the food chain for the NRC 8 right now.
9 MS. GOLUB: Okay, great.
10MR. DARBALI: I'm sure it's the same on 11 your side, but, you know, the people you see here are 12 just a fraction of the people -
13 MS. GOLUB: Oh, yes.
14 MR. DARBALI: - who are helping -
15 MS. GOLUB: Yes.
16 MR. DARBALI: - push this through.
17 MS. GOLUB: Yes.
18 MR. DARBALI: So, and we're grateful for 19 their support.
20 MS. GOLUB: Absolutely, thank you.
21MR. DARBALI: Okay, so this set of slides 22 covers a high level overview of what the staff 23 typically does in regards to a digital modification.
24So we do technical reviews. You'll see 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 36another slide on inspections, but the I&C staff 1 technical review with regards to a Digital I&C 2 modification, we get involved with topical report 3 reviews.
4 We do the LAR safety evaluations. We do 5regulatory audits. We support inspections, vendor 6 inspections, and also we also support the region, so 7 it might not be related to an inspection, but we do 8 provide support.
9 And some of the applicable guidance that 10 we use is ISG-06 Revision 2 once it becomes, once 11 that's a draft, so once it becomes final, it will be 12 the final version. We always use SRP Chapter 7, and 13 then there's a bunch of NRR office instructions.
14 We have LIC-101 on acceptance review, on 15 the LAR reviews, LIC-109 on acceptance reviews. For 16 regulatory audits, we have LIC-111, and for Topical 17 Report Reviews, we use LIC-500.
18 And then the next slide talks about just 19 regulatory audits because that's a portion that we'll 20 probably be talking more today, but it's what the 21 technical staff focuses on.
22 So this comes from LIC-111 on audits, so 23 regulatory's planned license or regulation-related 24activity. That includes the examination and 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 37 evaluation of primarily non-docketed information that 1 could occur either at the licensee's site, or at the 2 vendor's site, or remotely on an electric audit.
3 So regulatory is conducted with the intent 4 to gain understanding, to verify information, and to 5 identify information that will require docketing to 6 support the basis of the licensing or regulatory 7 decision.8 What we've done in the past, and most of 9 you have seen this before, when we do our LAR review, 10 we have an open items table or list and some of those 11 items will refer to an audit activity, to the closure 12 of the open item based on the audit may be we reviewed 13 it during the audit and the item is closed or we 14 reviewed it during the audit and "Please submit the 15 full document," or something like that on the docket.
16 Next slide.
17 So now I'll let Doug talk about DIRS.
18MR. BOLLOCK: All right, so this slide 19 just gives a general overall of what the Division of 20 Inspection and Regional Support does.
21 It provides specialized management of 22 reactor oversight programs, supports regional 23 residence inspection and base inspection activities 24 consistent with our mission, responsible for leading, 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 38 guiding, and coordinating Agency activities, and 1 coordinated implementation of these activities with 2 all its recommendation program changes.
3 So essentially if there are any changes 4 needed in especially licensees, we would be involved 5 working with industry and working with the regions to, 6 you know, if we had to make changes to any inspection 7 procedures or what we do in our oversight or 8 inspections.
9 Right now, and I think we'll get into a 10 little bit later, but there is an inspection procedure 11 for Device E52 Tier 03 and that's in our inspection 12 manual Chapter 25-15 Appendix C, so infrequent 13inspection. So right now, there is a process in 14 place. If there are any changes needed, it would be 15 coordinated through DIRS.
16 MR. GALLETTI: Okay, Greg Galletti here.
17 I'll speak to the vendor inspections, so the typical 18visualizing modifications. We have some 19 responsibilities for performing inspections at the 20 vendors' facilities.
21 Essentially, as you know, the inspections 22 will look to see how the vendor has implemented their 23 quality assurance plan to fulfill the purchase order 24 requirements that are levied on them by the licensee, 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 39 and we focus specifically on their implementation of 1 the Appendix B quality assurance program and also the 2 implementation of their 10 CFR Part 21 reporting of 3 defects or nonconformance program.
4With respect to those two areas, if we 5were to find some sort of deficiency at the vendor 6 facility, we may issue a notice of nonconformance 7 against one of the Appendix B criteria, and at the 8 vendor, in the case of any Part 21 issues or findings, 9 we would issue a notice of violation directly against 10 the vendor for the Part 21 failure.
11 The vendor inspection program verifies 12 that reactor applicants and licensees are fulfilling 13their regulatory obligations. Again, this is, the 14 primary focus of performing these inspections is to 15 ensure independently that the licensee has done due 16 diligence in performing oversight of the vendor 17 activities that they've procured.
18 Again, it's accomplished through a limited 19scope targeted inspection. We typically are very 20 performance based and we focus as much as possible on 21 the technical aspects of whatever the design is or 22 whatever is being procured.
23 We try to stay away from just doing a 24simple programmatic review of a QA program. That's 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 40 typically done by the licensees themselves or through 1 NUPIC of those vendors.
2 So we will focus ourselves on more of a 3 technical review looking at how that vendor has 4 implemented the various criteria of Appendix B to 5 whatever application they're applying it. So one of 6 the areas you'll see on the next slide, please -
7MR. ODESS-GILLETT: Before you do that, 8 Greg -9 MR. GALLETTI: Yes, yes?
10MR. ODESS-GILLETT: The last bullet, from 11 my experience with Diablo Canyon and with Part II 12 reviews, that I've seen regional inspectors support 13 technical audits because the region needs to be, like, 14 familiar with the system they're going to get and 15 ultimately inspect.
16 MR. GALLETTI: Right.
17MR. ODESS-GILLETT: So I was surprised 18 that vendor inspection got involved in supporting 19 these technical reviews because it's not been my OE.
20MR. GALLETTI: Okay, in specific cases, we 21 do actually support the audits that are done by the 22technical staff. If they're going out to a vendor 23 facility and they're doing some sort of auditing of 24 vendor activities, often they'll leverage our 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 41 expertise and experience in that area to support those 1 audits that they're doing.
2 MR. ODESS-GILLETT: Right.
3MR. STATTEL: It's not - we discussed this 4 this morning a little bit. It's not unprecedented.
5 MR. ODESS-GILLETT: Okay.
6 MR. STATTEL: So even with Diablo Canyon 7 you mentioned, on the ALS audit we did in Scottsdale, 8 we did have a vendor -
9 MR. ODESS-GILLETT: Oh, you did?
10 MR. STATTEL: - assessment -
11 MR. ODESS-GILLETT: Okay.
12 MR. STATTEL: - person with us.
13 MR. ODESS-GILLETT: Okay.
14MR. HERB: I have a question on that 15second bullet. It says that, "Verifies that the 16 reactor applicant licensee," I assume licensee here, 17"are fulfilling their regulatory obligation with 18 respect to providing effective oversight of the supply 19chain." Would that inspection be of our process?
20That wouldn't necessarily be of the vendor. I mean -
21 MR. GALLETTI: No, well the way -
22MR. HERB: So, I mean, I don't understand 23 how you can, how - I guess I don't understand how the 24 licensee's fulfilling their obligation by a parallel.
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 42 Are you going to inspect the vendor and compare 1 results or use it with our program that we've done and 2 see how effective it is?
3MR. GALLETTI: We would do both actually.
4So we would look at the licensee's program. So we are 5going to be doing certain audits or surveillance 6activities at that vendor location. So we would look 7 to see what sort of issues you've identified through 8 those processes to see how well you're engaging with 9 the vendor in identifying issues of concern.
10 In parallel, by studying that, we would be 11 looking at how the vendor themselves are implementing 12 the purchase order requirements that you've levied 13upon them to fulfill whatever the contract states. So 14 if you've applied certain regulatory guides, or 15 certain standards, certain, like NQA 1, we would be 16 looking to see how they've implemented those 17 requirements and how they've translated those 18 requirements into the actual design.
19MR. HERB: Great, so a follow on question, 20 since I'm a utility and I'm paying for the review time 21 for technical staff, that additional piece, it seems 22like more of a vendor inspection program piece. Am I 23 paying for that too as part of my license review or 24 are you paying for that out of your normal overhead?
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 43MR. GALLETTI: To date, we do it in our 1 overhead.2 MR. HERB: Yes, so if that's -
3 MR. GALLETTI: I don't know if that's -
4MR. HERB: If that seems like an 5 additional burden on my vendor and he's probably doing 6 work for me and he has to support that, I think that 7like - I don't know. Can you demonstrate how that 8 really supports my application versus you're just kind 9 of verifying that the vendor's quality is good?
10 MR. GALLETTI: Well -
11MR. FANELLI: I was going to say the 12 region will also support the vendor group on these 13 activities too, and we'd be inspecting your licensing 14 basis and how you are fulfilling that criteria and 15 obligation, so it would be a mutual, it could be a 16 mutual inspection.
17MR. McKENNA: That's not the question he's 18 asking.19 (Simultaneous speaking.)
20MR. BOLLOCK: Practically speaking, the 21 vendor inspection group inspects the vendors, right, 22the inspection report. So, you know, anybody from 23Greg's group goes. When they're going out with the 24 auditors, they write an inspection to the vendor.
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 44MR. GALLETTI: Right, and it's a separate 1 docket. It's not your docket.
2MR. BOLLOCK: Right, yeah, if we were 3 conducting, and this is where we're be involved and 4 the region would be involved, if we were inspecting 5 your oversight, we would let you know we'd be 6 inspecting the licensee's oversight.
7MR. HERB: I understand, but just the way 8 that bullet is read, it says, "verifies that applicant 9 are fulfilling their regulatory obligations," and so 10 I'm assuming you're coming to me for that, not to 11 bother my vendor or -
12 But, I mean, you have to do that as part 13 of your oversight of the vendors in general, I agree, 14 but are you planning to go every time somebody submits 15 a licensing to the vendor over, and over, and over 16 again, or just once in a normal process?
17 MR. GALLETTI: I don't know that we have 18 a firm answer to that.
19 MS. GOLUB: Yes, I think, and that's the 20 reason that we're having this workshop is at least on 21 our side of it, we didn't really understand how the 22 different groups within the NRC interacted to do sort 23 of, you know, the spectrum of audits and oversight.
24 And so I know at least I had a perception 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 45 that the vendor oversight group or the vendor branch 1 of the NRC did more of a, say, a generic inspection of 2 various vendors and not so much project specific.
3So that was our perception on this side 4 was that the project specific inspection was done 5 either through the audit process by the NRR technical 6 reviewers and then by the region after the license 7 amendment was issued.
8 So that was our perception of how, you 9 know, how that was carried out, and so we weren't - we 10 just assumed that the vendor branch, that was more of 11 a generic, you know, periodic inspection of the 12 vendors.13MR. GALLETTI: Understood, and you're 14correct. However, we also do much more focused 15 limited inspections, Westinghouse -
16 PARTICIPANT: Part 52.
17MR. GALLETTI: Right, things like that for 18their PMS system for example. Those inspections were 19 primarily led by the vendor organization focusing 20 directly on that one system.
21 Now, our same organization has gone to 22 Westinghouse and done just what you're saying, those 23 generic quality assurance, Appendix B verification 24 inspections if you will.
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 46MR. GOLLA: Excuse me for a moment.
1 Folks, on the phone, could I ask that you -
2PARTICIPANT: Oh, thank you for asking 3 that.4MS. GOLUB: - mute your phone, star six to 5mute, please? We're getting some occasional 6intermittent noises, a static sound. I don't know if 7 that's from - if muting will kill that or not, but -
8 PARTICIPANT: We can try.
9 MS. GOLUB: Yeah.
10MR. GALLETTI: We tend to try to not to 11duplicate industry efforts. So if you have seen a 12 NUPIC audit for example, they'll go out to the vendor 13 and do a very programmatic review of the Appendix B 14program. We tend not to do that. We tend to focus 15 more on the specific projects or systems, things of 16 that nature.
17MR. BOLLOCK: And we tend to avoid when 18 the licensee is doing their audit with their vendor, 19 and we tend to avoid -
20 MR. GALLETTI: Right, and we tend to try 21to avoid if NUPIC is coming in. We schedule and work 22 with NUPIC to try not to overburden the vendor.
23MR. HERB: Right, and so I guess 24 hypothetically in a follow up to that is that you said 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 47 you recently did, went to Westinghouse for a PMS type 1audit or review. If a vendor like myself said, "Well, 2 I'm going with the PMS league common platform," you 3 wouldn't necessarily have to repeat that just because 4 I've decided to -
5MR. GALLETTI: Not necessarily. As a 6matter of fact, I can tell you from our personal 7 experience, when we did those PMS inspections, we had 8 licensees at those inspections at the request of 9 Westinghouse.
10 MR. HERB: That's Part 52 though.
11 MS. GOLUB: That's the other part of it, 12 you know, is that we understand that was done under 13 Part 52 maybe in association with an ITAAC closure, 14 and so in this situation under Part 50 for the 15 operating fleet, I guess we're not sure we see that 16same, see where that applies. You know, it seems like 17 that's done in Part 52 space, but not in Part 50 18 space.19MR. BOLLOCK: It can be done in Part 50 20space. I mean, the vendor inspection group, like I 21 said, they are inspecting the vendor, but to help 22 focus the group, they've got a vendor inspection 23 program, a plan, yeah, where they plan and they 24 prioritize where they go, and that's why we'll include 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 48 some specific work, but they're inspecting the vendor.
1MR. GALLETTI: Right, and historically, 2the vendor program was Part 50. It existed well 3 before Part 52 was in existence.
4 MS. GOLUB: Okay, yeah, I think Warren's 5right. You know, we have a presentation, and our 6 presentation kind of shows our perspective and maybe, 7 you know, we'd be happy to hear your thoughts on that.
8MR. GALLETTI: Okay, next slide. I'll 9just go through this quickly. So about a year ago, we 10 released this new inspection procedure, 35710, which 11 was specifically looking at some of the quality 12 assurance inspection software, again we reference in 13QA 1, 2.7. There's a lot of similarity between what's 14 in 2.7 and what's in this guideline.
15 So for an application like this, we would 16want to go and verify safety-related software. We 17 would be looking at the QA program, ensuring it 18complies with Appendix B. So here at some of the 19main, I won't go through each one, but the main 20 criteria of Appendix B that may apply to one of these 21 modification inspections that we would do.
22 I'll point to criterion three for design 23control. Obviously the types of things we look for is 24 that the requirements that you've levied upon the 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 49 vendor are being translated properly. They are, you 1 know, implementing those requirements throughout the 2 life cycle of the design and development, and there is 3 traceability and things of that nature.
4 For criterion 10, test control, obviously 5 for like V&V activities or in-process testing, we want 6 to ensure that it's being controlled in a manner 7 consistent with Appendix B requirements and your QA 8 limitation, inspection procedures and plans, that sort 9 of thing.
10MR. VAUGHN: So I had a quick question on 11this. So this is a new procedure and the three types 12 of vendor procedures you have are routine, reactive, 13 and the new ones you mentioned. So it's going to be 14 a routine one, but it's an as-needed basis as opposed 15 to on this routine periodicity.
16MR. GALLETTI: Right, this would be as 17 needed. If we were doing a digital I&C modification 18 inspection, we would be referencing this, but we do a 19 lot of other inspections that have nothing to do with 20 that, so we would be referencing this.
21MR. VAUGHN: And I guess to just follow up 22 on Ray's question, you know, just going forward, if, 23 you know, many utilities had LARs following ISG-06, 24 would this as-needed IP be done every single time for 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 50 each one or -
1MR. GALLETTI: We don't know, not 2 necessarily.
3 PARTICIPANT: I would hope not.
4MR. VAUGHN: Yeah, we were just talking 5 about it at our internal meeting of how this would 6 unfold in the future.
7MR. WATERS: Right, so let me on that.
8I'm not sure if that's our view. I think what we can 9 all recognize is there are multiple inspection 10 procedures and programmatically what we're going to do 11 is look at that.
12It's a good question. It's a question we 13have too. What makes sense going forward? But I 14 appreciate the question. So in asking the question, 15 we'll take it into consideration, but we can't give an 16 answer -17 (Simultaneous speaking.)
18MR. HERB: And if I can frame why I asked 19 the question in just a short statement, and please -
20MR. STATTEL: We do have experience in 21 oversight.
22 MR. HERB: Let me frame why I asked that 23 question because I have a fleet-wide single platform 24 that we're going to choose through our regular 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 51 process, and so I'll be submitting three LARs for my 1 three separate units, usually the exact same platform.
2 I'm just saying I would hope that the cost to review 3 would not be that first one times the cycle and times 4 the third one.
5 It would get smaller because you wouldn't 6 have to go back and re-review those pieces over, and 7 over, and over, and over again, and that's - so I'm 8 interesting in deriving the costs of actually doing 9 this because otherwise we're not doing any of them 10 because the cost is too much.
11MR. WATERS: The answer is yes as long as 12 you get the first one right.
13 MR. HERB: I understand.
14 PARTICIPANT: Well said.
15 MR. HERB: Well said, yes, thank you.
16 PARTICIPANT: Right, I think -
17MR. ARMSTRONG: And also if it's the same 18 scope.19 PARTICIPANT: Right.
20 MR. ARMSTRONG: I mean, if it's the same 21 platform for the same use, if you're going to use the 22 platform for a total different use, I mean, we're 23 going to have to change the way we review all of that 24 and maybe accommodate that.
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 52MR. HERB: On the technical piece, yes, 1 but the programmatic piece, they're usually the same 2 part of the application, so I would have to say that 3 those programmatic pieces are probably the same.
4MR. STATTEL: So talking to recent 5 experience, so we've approved several license 6 amendments for digital systems in the last couple of 7years. Hope Creek installed a PRMS system. We did 8 not perform inspections on that system, right, and 9 that's an example of where we had several of those 10 applications that are in operation at the plant.
11 There are some that we have performed 12 inspections. We performed inspections at the Oconee 13 sites for the reactor protection system upgrades, 14 again, a first of a kind type of system.
15 So now, that's kind of an indicator that, 16 you know, it's not going to be 100 percent and it's 17 not going to be zero, right, but you also have to 18acknowledge the fact that by issuing this license 19amendment earlier in the process like we're saying, 20 we're really opening up a bigger part of the 21 development process to inspection, right?
22So we're moving the line over, so the 23 inspections are going to be different, a different 24 scope, and I really can't speak to the frequency of 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 53 the inspections because, you know, we'll see once we 1 get into the process.
2MR. FANELLI: So this is regional 3inspections. I'm going to be talking about this.
4These bullets up here are what the region does. We 5 ensure compliance with commitments and rights on the 6 basis that will be submitted with ours. Pretty much 7 it's just enforcement.
8 Whatever you've committed to, whatever 9 your license basis is, that's what we'll go inspect.
10 So all of these bullets for ensuring the licensee's 11 ensured development, these come right out of 12 inspection procedures that we may develop later on for 13 some of this.
14 The fourth bullet down is the one I would 15jump to. We're going to, we more than like would 16 confirm by inspection that the licensee has 17 sufficiently performed oversight for all of these life 18 cycle functions down here just below that.
19 We also provide audit support for 20technical reviewers. As you said, you've seen it 21before. We also provide support for vendor 22 inspections also sometimes. Next slide.
23 These are two inspection procedures that 24 we know of now. They haven't been updated in a long 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 54 time, so they're not ready for use now, but we may 1 develop into an inspection procedure later on with 2 DIRS.3 There's one that's not mentioned up here 4is DOT 23, 6501 DOT 23. It's for human factors' 5inspections. That might play a role in this too.
6 That's why I had to mention that. That's really all 7 there is to that. There's not much to it.
8MS. GOLUB: So it sounds like as we're 9 looking at, your introductory sentence there that 10says, "may be developed," so it sounds like you are 11 considering developing new inspection procedures or 12 maybe revising these procedures?
13MR. FANELLI: I would say that's a correct 14 statement, but that would be working with DIRS, but 15 whether we would just update 52.03 and maybe combine 16 all of these or make a new one, I couldn't tell you 17 right now.
18MR. DARBALI: Right, and we understand 19 some of these procedures are specific to Part 52 ITAAC 20inspections. So whether those procedures get updated 21 to also apply to this new process, or we develop a new 22 inspection procedure just for the Alternate Review 23 Process, we have been talking about that being a 24possibility in the near future. But definitely these 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 55 procedures -
1MR. FANELLI: It's just what we've used in 2 the past.3 MR. DARBALI: Right.
4MS. GOLUB: Right, and no, and I think it, 5 yeah, and I appreciate it says, "may be developed in 6consideration." So do you know when you'll be making 7 that decision on -
8MR. WATERS: That is with members of DIRS 9 and we've had this discussion on needing to update, if 10we need a new one or do something else. We've been 11 recently impressed with the ITAAC. We know it's not 12 ITAAC, but just a lot of parallels, and I think we 13used the later one. Part of my thing is what's the 14priority update and when do we do it? We want to look 15 into 2019.
16I'll be honest. One of our continued 17 frustrations is that it's nice to have a little more 18 specific idea about what it look like and timing so we 19 don't have to work in a vacuum, but I think the answer 20 is yes, we're going to look at these. We don't have 21 a schedule yet for that, but I do see it happening in 22 2019 unless there are other visual I&C activities that 23 are required.
24 MS. GOLUB: Okay, so 2019.
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 56 PARTICIPANT: It's something later.
1 PARTICIPANT: Yeah, but it's like -
2 MR. STATTEL: These -
3 PARTICIPANT: I'm sorry. Go ahead.
4 MR. STATTEL: These are relevant though.
5 I want to say that because if you look at the 2011 6 inspection procedure, essentially the ITAACs are 7 covering all aspects of the development process 8 including planning, implementation, and review of 9 design outputs, okay?
10And I want to mention that because the 11 planning part of that procedure, we probably would not 12 need to apply for a system that's developed under the 13 alternate tier process because we're, that's part of 14our licensing, right? We have that included, but the 15 implementation and design output side would apply, 16okay, so I just want to mention that. I would expect 17 a future inspection procedure to be similar in those 18 areas.19MS. GOLUB: And so in 2019, so, I mean, 20I'll just throw this out there. I don't know how the 21 process works, but industry would be happy to provide 22 input if we could or an ything as you develop those 23 processes.
24 MR. WATERS: Yeah, and I don't know what 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 57 the process is for inspection review and potential 1 updates, but, yeah, part of the goal is to do this.
2 I figured this is one of the first means to get that.
3MS. GOLUB: Absolutely, absolutely, yeah, 4 okay, thank you very much.
5MR. VAUGHN: I feel like for routine 6 changes like the feedback forms and the inspection 7 procedures we just updated, but certain things like 8 when the assessment, you know, chapter was updated, it 9 had a lot of industry, a lot of public involvement.
10 But certain ones, the staff can decide 11 it's not for comment. It's just shared through like 12the RP working group or whatever venue you want. You 13 share it at a public meeting, take it back a month 14later and provide comments. So it's done. It's just 15an informal way to do it through a public meeting.
16There's no official way. It's not like a Federal 17 Register type of thing.
18MS. GOLUB: But is that staff discretion?
19 Is that what you're saying, Steve?
20 MR. VAUGHN: Yeah, yeah.
21 MS. GOLUB: Okay, all right, so I didn't 22 know there was actually a process out there that could 23 be followed even at discretion, so that's good to 24 know.25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 58MR. DARBALI: Okay, so I think that is the 1 last slide in that presentation, so now we'll turn it 2 over to Pareez and Warren for your presentation.
3MS. GOLUB: Yes, so we have a 4presentation. Warren and Ray are actually going to be 5 doing the presenting.
6MR. ODESS-GILLETT: So, the purpose of 7 this presentation is now that we are moving toward an 8 alternate review process or digital INC LARs, that we 9 have -- we wanted to let the staff know what industry 10 is doing in the area of design process.
11 And also to relay our expectations as a 12 result of the alternate review process, requiring a 13licensee to step up the game in vendor oversight. And 14 documenting what that is in a plan as part of a LAR.
15So, we'll cover both of these things. Ray 16 is going to review those activities that the industry 17 has developed for the digital engineering guide, the 18 digital design guide, and other standard industry 19 processes for the design that integrates vendor 20 oversight.
21And Ray will talk about that. And then I 22 will talk about how we see how inspection would play 23into it from our perspective. And see how that 24 matches up with the -- with our perspective.
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 59 So with that, I'll turn it over to Ray and 1 the next slide.
2MR. HERB: And then the next slide. Okay.
3 Thank you. Thank you Warren.
4 So this is a little bit of an overview of 5 the vendor oversight process and is new that came up.
6 As part of our design process, we've always done an 7 acceptance review of most of the design of documents.
8 But, there has been a gap that was noted 9 in the Oconee reviews and in potentially in some of 10 the earlier Diablo Canyon that the vendor oversight 11 piece, especially of the design processes, was a 12 little lacking.
13 And we've always had a vendor oversight 14 process in place for engineered components. And for 15 actual hard components like pumps and pelters, pump 16 casings, motors, like that, it's very easy to go and 17 take measurements and measure tolerances and look at 18 drawings and verify at the vendor's facility what you 19 actually have.
20It's a little more fuzzy when it comes to 21 software. And so we committed as part of -- earlier 22 working with the staff on the alternate review 23 process, to further define what we mean by the vendor 24 oversight of those quality processes for software 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 60 development.
1 And so, as part of the industry standard 2 design process initiative, we have a procedure, IP-3ENG-001. And it is implemented across the board at 4 all U.S. nuclear power plants for the design process.
5 And just recently developed is the NISP 6 procedure, NISP-EN-04. It is the -- it's really the 7 digital, how do you apply the standard design process 8 in digital? Okay.
9 It's currently under consideration for 10 making it mandatory across the U.S. nuclear fleet.
11 But, most of the large fleets have already committed, 12 including Southern, I think Duke, TVA, and I've got 13Dominion and Exelon have all committed to implement 14 that in our fleet anyway.
15 Whether it comes out mandatory across the 16 industry or not. And so --
17MS. GOLUB: And Ray, you have a pretty 18 significant hurdle to work with on Monday.
19MR. HERB: Yes. It is. Yes, so on Monday 20 it went to the oversight committee for IMPO and they 21 basically said yes to this.
22 We believe -- they have not decided the 23 degree of when people have to make it. Whether it's 24 100 percent mandatory, or it's highly recommended.
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 61 And so that procedure has already been 1 approved. It was developed in conjunction with EPRI 2 when they developed the revision to the digital design 3 guides. And it's now leaning to digital engineering 4 guides.5 So, if you think of it, the process, the 6 forms, are standard in the ENG-001, the NIST is the 7 digital specific things that you must do to ensure 8 design quality and of a digital design. And it is a 9 fully graded approach with the highest gradation would 10 be of course, certainly a LAR.
11 And then the EPRI document, 3002-0118-16, 12the digital engineer, is the how to. The how to 13 piece. Okay.
14 And then each of the licensees have their 15own processes for vendor oversight. And so I'll talk 16a little bit about what Southern's are at the very 17 end.18 But, what I wanted to say is that 19 industry's design process, they required vendor 20 oversight planning long before the LAR is submitted.
21 It's really part of the supply chain purchase 22 agreement part, so.
23 Because you have certain requirements that 24you -- like I say, we don't pay and pray. We like to 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 62 trust but verify in our supply chain. Because you 1 spend a lot of money on these systems, and you want to 2 make sure you get exactly what you paid for.
3 MR. DRAGOVICH: I just wanted to mention 4 too that we adopted a lot of the vendor oversight in 5 the BOP that you reviewed from the digital engineering 6 guide.7 So, it's something that's we're using 8 that's already been established. We're not creating 9 new criteria.
10 But, one of the things is that we -- and 11 some of the comments we were given, we were led to 12 wonder whether the NRC considers vendor oversight to 13 include all the technical document reviews.
14 And we do that as a separate process as 15part of our EC. So, when we go into our vendor 16 oversight, you'll see it's more process driving and 17 not technical drive.
18 Because we already cover the technical as 19 part of our document reviews, our owner's acceptance 20reviews. So, that's something that we've always kept 21 separate.22 The exception would be, is if we do a 23 review and we find a lot of gaps in the quality of the 24 vendor documents. Then we'll actually add that as a 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 63 line item for a vendor oversight visit to discuss 1 quality issues and things like that on the technical 2 side.3 But otherwise, it's process driven 4 primarily. So, that's kind of why you'll see in our 5 VOP that a lot of the configuration control, things 6 like that are just checking the process.
7MR. HERB: Neil, do you have anything you 8 want to add?
9MR. ARCHAMBO: Oh, no. That was well 10 said. I have to agree with what's being considered.
11 MS. GOLUB: So it's kind of a revolution 12to us, it's really great. And we thought today, maybe 13 there's just a divergence there.
14 It's not clear that the technical side is 15being considered. It's just being considered, do you 16 need to do it now.
17MR. WATERS: Yeah. My apologies. Are you 18 expecting a real time inspection to do the digital 19 inspection?
20MR. HERB: The digital design process, 21 which is just standard procedure, is nuclear industry 22 standard procedure. That's what NIST stands for.
23 And EN-04 is the digital design process.
24 It actually it nests inside the IP-ENG-01, which is 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 64 the overall framework that governs designs.
1 You've got the conceptual design, how 2 meetings have to take place, how you gather inputs, 3 how you verify --
4 MR. WATERS: It's not just INC?
5MR. HERB: It's not INC. It's the entire 6 standard design process, how designs are done in the 7 industry.8 And the NIST is
-- adds that digital 9flavor. Because digital, you know, we already say 10it's special. And in some ways, it just requires a 11 little additional extra effort and some additional 12 hazards that have to be addressed in the design 13 process.14 And in some cases, specialized software, 15there are additional things that are done. So, the 16 NIST is written to basically add those additional 17 assurances to the standard design process.
18So, it's an additional procedure. And it 19 really -- it's about the -- the procedure size, I 20think is about 25 pages long. And it has this big 21 long attachment to the back, which allows you to use 22 that as a grading process it says.
23 If it's a really complex system, and it's 24 highly risk significant, you need to do these things.
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 65 If it's not so complex, and it's not so risky, you 1 need to do a lot smaller amount of stuff.
2 And so --3MR. WATERS: So, I mean - and I'm sorry 4again. So, I understand what it is. It is just amped 5up. I think you all can also go standardizing the 6 process into this.
7 MR. HERB: Yes.
8MR. WATERS: And this is a really good 9 thing. Encourage that. And we are just saying less 10 NRC oversights, need it based on this.
11I hope you understand that comment. I 12 think it challenges really nothing, you can benchmark 13 against it for lack of a better word.
14 MR. HERB: I understand.
15MR. WATERS: My question is, are you going 16 to use this for 59.59 upgrades as well?
17MR. HERB: Oh, this is used for all of 18 those changes.
19 MR. WATERS: All changes. All right.
20MR. HERB: All engineering changes. He 21 could use a EC and everything.
22MR. WATERS: It's not just for breaker.
23 Okay.24 MR. HERB: Yeah. All design changes. We 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 66call them design changes. You all call them 1 engineering changes.
2 I think we all have a different name for 3 them. But anything that we're -- everything that we 4do that changes the plant requires 50.59. And 5 everything that's screened as a 50.59 most certainly 6 requires a modification at some level that impacts the 7 plan.8 And again, there's several different --
9 you know, the standard design process has like a 10commercial change. It has a limited changed. It has 11 a detailed change. And so it's graded as well 12 But this provides that additional digital 13grading to that as well. And all digital 14 requirements.
15MR. WATERS: Out of my ignorance, can you 16give a run time of this? Whether they're going to do 17 a 50.59 or not, is there a feedback mechanism to 18 include this?
19 MR. HERB: Oh, yes. The standard design 20process, the EN -- IP-ENG-01 is in Rev Zero. And most 21of the industries are Rev Zero. But Rev One is out on 22 the streets.
23 And in the next three months we're 24 supposed to evaluate and implement it into our sites.
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 67MR. WATERS: I was thinking about the 1 lower ones. But yeah, that's right.
2MR. HERB: And even the lower ones.
3That's brand new. It's out on the street. So, we 4 haven't -- we haven't. You're right.
5 We don't have a lot of run time with it.
6 But, it essentially takes a proven standard design 7 price list and just adds an additional texture, 8 additional requirements for each one.
9MS. GOLUB: And Mike, I'm just going to 10 mention, but I know this is of interest to the 11 industry. And some of this has already happened.
12 Where EPRI through their MOU with research 13 has already done, and I'm calling it training, but 14 it's mostly just showing the NRC what's in that 15digital engineering guide. And how the industry is 16 going to be working.
17 And so we're hoping to do more of that.
18MR. WATERS: I just wanted to -- still, 19we're not -- we've endorsed it. And I don't really 20 call it training, but we understand what's in there.
21MS. GOLUB: It's really just sharing 22 information.
23MR. WATERS: So, we understand what 24similar systems have an effect. That's sure of it.
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 68 I appreciate though. I endorse it.
1MR. HERB: Right. I think the message --
2 the message I'm trying to present right here is that 3we have a standard process. And that standard process 4 includes a vendor oversight piece.
5 And so everybody who comes to the NRC with 6a LAR will be working from the standard process. You 7 should expect to see something very similar from 8 everybody. Okay.
9MR. DRAGOVICH: But also just to note too 10 that this digital engineering guide is a new EPRI 11report number. But there was an additional design 12 guide.13 And that's the feedback that came back to 14 make this a new and improved engineering guide.
15MR. HERB: That's right. And that's been 16 around since 2014.
17 MR. DRAGOVICH: Yes.
18MR. HERB: And we've always -- we've 19 always implemented that in our -- to inform our 20 digital designs at the plants.
21 Not just safety related, but all designs 22 that are related to digital, came out of it, that 23original DEG it was called. So this is just a 24 revision of the DEG.
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 69MR. WATERS: Um-hum. Would your -- well, 1last question here. Would your vendor oversight plan 2 in itself that you submit on the backup commit to 3 these?4MR. HERB: The vendor oversight plan would 5-- usually that's --
6MR. WATERS: Not unless you've already 7 started.8 MR. HERB: There is guidance in the EPRI 9 digital engineering guide that says you -- it says 10this is when -- well then this does. This is when you 11 should develop your vendor oversight plan.
12 And it says -- and then it goes to the DEG 13to say, here's the guidance. And the DEG says, use 14commercial grade dedication. Use critical digital 15reviews. Those are the elements that you would go 16 into the actual things that you should be looking for 17 and inspecting.
18 And it references some -- also some actual 19IEEE standards and what not. And so those things all 20 inform what is looked at and what is required to be 21 looked at.
22 But mostly, the vendor oversight planning 23 for the LAR, is going to be informed by their LAR 24itself. By Section D.4, by the system requirements 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 70 that we submit as well.
1MS. GOLUB: But the BOP section of the 2 digital engineering guide, I mean that -- there was a 3 lot more on it too.
4MR. HERB: It's very -- it is very broad.
5MS. GOLUB: And this inner level was added 6 at this last revision.
7MR. HERB: That's right. It's 8significantly improved. And like I said, vendor 9oversight has always been a part of this process. Not 10 just digital, but for regular design too.
11 And it just says, that you need to 12consider all of these things. And one of those now is 13 licensing commitments and NRC commitment questions.
14MR. GALLETTI: Can you -- you mentioned 15something that I thought I heard. But, new licensees 16 for your vendor oversight, which you've been doing for 17 a long time, that's generally programmatic.
18 And then for technical review, you have a 19 separate independent process.
20MR. HERB: Right. Design acceptance of 21output documents only. So, when we purchase a 22 component or services from a vendor, and that's what 23 this will be, we're purchasing their engineering 24 services to develop our application.
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 71 We get certain output documents from them.
1 We have an acceptance process and review. It's part 2 of the design process that we go through.
3 And that maybe, you know, a matrices, 4 requirements, you know, checking it to our actual 5 requirement documents.
6 MR. GALLETTI: Okay. But --
7MR. HERB: That's a technical piece rather 8 than a process. Look, the vendor oversight's always 9 been like almost like a NIST -- I mean, a NUPIC type 10 of audit.11 But it will be super-sized for software, 12you know, in our case. Because oftentimes NUPIC, you 13 know, they look at a lot of the QA processes and 14 they're not specifically looking at 2.70 inquiry type 15 stuff.16 So, we're adding those things as well.
17MR. GALLETTI: So, the technical stuff 18 will be added in you said?
19MR. HERB: The technical stuff is done.
20But it's done not necessarily as part. It may be 21 under the umbrella of the vendor oversight.
22But it really is already part of our 23 process with the acceptance of those documents, those 24 reviews.25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 72MR. DRAGOVICH: Yeah. So like if a vendor 1 prepares a calculation, you know, we'll have that in 2 our schedule to review that performing only on 3acceptance. We don't wait until a vendor oversight 4visit to do that review. It's just part of our design 5 process.6 The vendor oversight is strictly just 7 going through -- for example, let's say they're 8 building a cabinet and we might decide to do that 9 vendor oversight at a time where we want to look at, 10 inspect the quality of the cabinet being constructed.
11But it just is more of a process. Just 12 making sure that they're following the process. The 13 technical is driven from the engineering change in the 14 schedule.15 And it's not necessarily going to coincide 16 with the vendor oversight visit, unless there's a 17 problem where we have to go and inspect because of 18 that, you know, degraded condition, whether it's a 19 caliper or a drawing or whatever.
20 So, but that's just kind of independent of 21 the vendor oversight that we have in our VOP here.
22 MR. FANELLI: So you don't set it up for 23 vendor oversight?
24MR. DRAGOVICH: No. We have that -- I 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 73 mean --1MR. FANELLI: I mean, I'm not talking VOP.
2 MR. DRAGOVICH: Right.
3MR. FANELLI: I'm talking vendor oversight 4 in general. The criteria is set already.
5MR. DRAGOVICH: Right. No. No, that's 6 just part of what we would normally do as part of our 7 standard design process.
8MS. GOLUB: I don't know if it's a 9 terminology thing, but --
10MR. DRAGOVICH: And that's why we sense 11 there was a little bit of misunderstanding with the 12 comments we got.
13 MS. GOLUB: Yes. Yes.
14MR. DRAGOVICH: That we thought maybe the 15NRC considers that vendor oversight. And you're 16 asking, why don't we discuss any of those?
17Well, that's because we have it in our 18 design process already that -- where we address that.
19 It's part of --
20 MR. BURZYNSKI: It's more of a criterion 21for when we design the inner-space attributes that 22 have been for oversight.
23MR. DARBALI: So, I guess the question is 24--25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 74MS. GOLUB: Does that answer your 1 question?2MR. HERB: Well, we are doing all those 3 things.4MR. GALLETTI: I understand. I think that 5 we have maybe fundamentally a disconnect in what we 6 considered to be, you know, oversight of the vendor.
7 Because certainly in these technical 8 output documents produced by the vendor, you know, to 9 be reviewed and approved, we see that as a vendor 10 oversight activity.
11 MS. GOLUB: Oversight. Yes.
12MR. HERB: Right. We're just trying to 13 let you know that we are doing that too. It just 14maybe not underneath the umbrella of the vendor 15 oversight guidance.
16MR. BOLLOCK: Actually, it's part of your 17engineering changes. It's the engineering design 18 change process. Okay.
19MR. HERB: It's part of our engineering 20 design process going forward.
21 MR. BOLLOCK: Yeah.
22 MR. HERB: We are still doing both.
23 MR. VAUGHN: It's just a label. Because 24 it --25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 75 MR. HERB: We may not be -- you may have 1 to review all the licenses.
2MR. BOLLOCK: Yeah. So when we did the 3 pilots, the first engineering pilot at Oconee, that's 4what we ran into. The technical parts were in the 5 engineering change DC.
6 And, you know, we had a couple of vendor 7 inspectors that were looking at the procurement 8documents. And didn't realize that it was just how 9 you package it, where you kept it was in your EC that 10 had that.11 Yeah. They're looking at all the gap --
12 they had to look at that.
13 (Simultaneous speaking.)
14 MR. HERB: And it's all part of the big 15 picture. If you notice, --
16MR. BOLLOCK: We just printed it 17 separately.
18MR. HERB: Yeah. If it -- if deficiencies 19 are noted, and it's --
20MR. BOLLOCK: It was big. And yeah, they 21 did.22MR. HERB: Then you write a corrective 23 action report. And it comes out --
24 (Simultaneous speaking.)
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 76MR. BOLLOCK: Because that's part of.
1 That all is part of --
2MS. GOLUB: Right. Is part of that whole.
3 Yes.4MR. BOLLOCK: And so we asked you for, you 5 know, we want to look at your procuring set of 6documents. That -- we would expect that as well as 7 being treated.
8MR. HERB: And so, you know, with that 9 thought, you know, the actual -- you know, because we 10-- there was a question about, how big is this plan 11 compared to like what the review is.
12 If you included all that piece it would be 13 massive.14 MR. BOLLOCK: Yeah. Then it would be.
15MR. HERB: It more than that. It would be 16 thousands of pages.
17 (Simultaneous speaking.)
18MR. STATTEL: But put this in perspective 19now. Because that's -- when we started this, on SD6, 20 when we started this, right, we said we're going to 21 issue the license amendment prior to what we 22 traditionally perfom -- these activities traditionally 23 performed.
24 And a lot of those are related to 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 77reviewing specific artifacts of the design. Right.
1 Design output artifacts. Right.
2And so, that is document review. Now, we 3were told that -- so, since they were creating this 4 big hole, because we're no longer performing those 5 activities, and we were told that that hole would be 6 filled by these vendor oversight activities that would 7 be discussed in the vendor oversight plan.
8 MR. HERB: Yep.
9MR. STATTEL: And so kind of what we're 10 seeing is this doesn't entirely fill that hole. And 11 you're telling me that too.
12MR. ODESS-GILLETT: It does. Because the 13 vendor oversight plan summary, you'll notice there is 14 a statement about reviewing -- how did it get worded 15 Steve?16 The owners --
17 (Simultaneous speaking.)
18MR. DRAGOVICH: Oh, yeah, yeah. But it's 19 this.20MR. ODESS-GILLETT: It is in there. It's 21 just that it's not elaborated because it's part of 22 their standard engineering process.
23 So it is referenced in the vendor 24 oversight plan summary.
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 78MR. FANELLI: So what you're saying is the 1 process is there. But these are the outputs to that 2 process.3MR. HERB: We're not going to duplicate 4 this, you know, this compliance matrix to our purchase 5 order in the documents for vendor oversight documents.
6 So, our vendor oversight documentation 7 would consist of our audits of the vendor. Are they 8following the process? Maybe that's right, it's just 9 some of the things that you do.
10 But those -- the actual output documents 11 would be part of the engineering change package. It 12 would be part of the design change package and in our 13 vendor documentation, we just put it into our 14 documenting whole system.
15 And so those pieces are inspectible by the 16NRC. And you come and do mods inspections all the 17 time and you look at those components at that time.
18 And so, we're not saying you're not 19allowed to look at them. We're just saying is that 20 that's not going to be part of our vendor oversight 21 report.22 Our vendor oversight report is going to 23 be, we went to the vendor for these five different 24trips. We looked at their process. Is this what they 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 79 said they were going to -- how they were going to 1 develop our process?
2We inspected them to it. We believe 3they're following it. And we're -- and as of the 4 final verification, we're looking at the output 5 documents and we're seeing those.
6 But, that output --
7 (Simultaneous speaking.)
8MR. HERB: That owner acceptance of those 9documents is part of a different team. The technical 10 team is doing that piece.
11 And if they notice problems, they just 12 flow back into the vendor oversight process to say you 13 know, what? Something's happening.
14 They said they were following their 15 process. We're getting -- we're getting substandard 16 documents out. So, something is wrong.
17 So we would go back. We would escalate.
18 So we may actually put a person at the vendor's site 19 to make sure that they were doing what they said they 20 were going to do.
21 So it has escalation involved as well.
22MR. FANELLI: One question. By leaving 23 this out of your VOP, is what I'm hearing you say, 24 except for that statement you were talking about.
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 80 Then would you not be committing to doing 1these things in your license amendment? Or would you?
2MR. HERB: Well yeah. I mean, you've got 3 to put the design in this --
4 MS. GOLUB: And it's a formal regulatory 5commitment. So it's not -- it's not just sort of 6mentioned by chance. I mean, it's a formal commitment 7 which the ISG-06 process guides you to make in order 8 to use that.
9 MR. SCHRAYE: But part of our procedures 10 that we're developing and stopping of course because 11the vendor assessment is part of that. I mean, that's 12 one of our licensing basis.
13MS. GOLUB: And we're formally committing 14 to it. So, it's --
15MR. STATTEL: When you say formal 16 commitment, what specifically are you talking? What 17are you formally committing here? We need to 18 understand.
19MR. HERB: When we get our LAR, our 20 license amendment, that becomes part of our licensing 21basis. And any changes that we make to the plant get 22 compared against our licensing basis through the 50.59 23 process.24 Now, I'm not suggesting that we would 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 81 change the direction of the component we haven't even 1installed yet. But, there may be minor things that 2 come up.3 And if they fit within that 50.59 process 4review, then there's two outputs. We have to resubmit 5 the LAR because we've suddenly broken the boundaries 6 of what was accepted and was approved.
7 And -- but if it doesn't, then that goes 8 into our process --
9 MR. STATTEL: I think that's a different 10 topic then what I was asking about.
11MR. HERB: And so -- and so that again, 12 those final design documents, those final 13 implementation pieces, those are all reviewable 14 underneath the Mod and the inspection process.
15MR. DARBALI: Yeah. But Rich is asking 16 about the commitments that you would be making in the 17 LAR.18MS. GOLUB: Yeah. I'm just trying to open 19 up that section.
20 (Simultaneous speaking.)
21 MR. STATTEL: But this is outside of the 22scope of vendor oversight. So, in a sense, when I 23 look at what we do after basically implementation and 24 design efforts. I look at the activities we do.
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 82 Now, you can put them under the category 1of vendor oversights or not. Right. It doesn't 2 matter.3 I'm looking for an equivalent activity 4 that -- I understand, a licensee is taking 5 responsibility for performing those activities.
6 I'm looking for equivalent guidance, 7 equivalent activities that are going to be performed.
8And a commitment to perform those. And that's what 9 I'm not seeing right now.
10 I need to get to a place where I'm 11 confident that you're accomplishing the same thing 12 that we would have accomplished if we were performing 13 those activities in accordance with 50.52.
14MS. GOLUB: Yeah. So, that -- I mean, 15 this is our meeting from the -- I'm reading from the 16-- I'm reading from the last version of the ISG that 17I have access to. Which was the public, you know, the 18 public version from July.
19 And under C.2.2, which is licensee 20prerequisites, Item 3B. And I don't know how it's 21 currently worded, Samir.
22 But, the LAR should include appropriate 23 regulatory commitments to complete life cycle 24activities under the licensee's QA program Note that 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 83 these activities would be included in a Tier One, Two, 1 or Three licensing review.
2 Which is what you're talking about Rich.
3 But are not covered in the alternate review process 4 because they take place after the NRC staff decides 5 whether to issue or deny the license amendment.
6 But the point is, those are the activities 7 that the licensee is committing --
8MR. DARBALI: And from related equipment.
9 MS. GOLUB: Right.
10MR. STATTEL: I'm still missing the 11 guidance, right. So we have 50 pages of guidance on 12 how to perform those activities.
13 And we will no longer be performing those 14activities. Where is the equivalent of that guidance 15 that we have for the licensees to perform the same or 16 similar or equivalent activities?
17 It's not in the vendor oversight plan what 18 I'm seeing here.
19 MR. HERB: It is -- the vendor oversight 20 plan is to -- they have a -- we have signed up through 21 a purchase order, these set of requirements to be 22 implemented through this process that was defined in 23 Section D4.
24 Now, we will, through normal audits and 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 84inspections of the vendor oversight, check to make 1sure that they are following their process. And we 2 will confirm the outputs of those design activities, 3 to make sure that they're complete and, you know, 4 appropriate for what, and they match the requirements 5 of what we asked them to do.
6 Which we also submitted to alter for 7approval. And so, yes, we have to make sure that the 8-- that the application is developed and designed in 9 accordance with our highly detailed specifications.
10Do we have a procedure for doing that?
11What's currently design change procedure. And you 12know, defined what those are. And you accepted. And 13 you verified that you got what you designed.
14 And so, I don't know if there's anything 15-- I mean, you're not going to go there.
16 MS. GOLUB: Steve?
17MR. DRAGOVICH: So, I was going to say, so 18 when I prepared this, I went through and brought what 19our understandings of the VOP were included. And then 20 we got some comments saying no, we should include some 21 of these other reviews that Rich was mentioning.
22 So, we can take can action to talk about 23that. And determine if we want to enhance that to 24 include all that detail if we don't feel it's present 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 85 somewhere else in the LAR.
1 You know, in other words, if we talk about 2it somewhere else then maybe we don't need to. But 3otherwise, maybe it would be worth it to have that 4 technical component in this.
5MS. GOLUB: Yeah. Or at least describe 6how that work is being done. Even if we don't, you 7know, if we can show where it's been done elsewhere 8 within the licensee's programs.
9MR. HERRELL: And document that reference 10 material in the VOP.
11MR. STATTEL: We're going to talk specific 12-- more specifically about this VOP that we've -- some 13 of us later on?
14 MS. GOLUB: Um-hum. Yes.
15 MR. STATTEL: Later on?
16 MS. GOLUB: Yes.
17MR. STATTEL: Okay. Well, I'll defer 18 until then.
19MS. GOLUB: So maybe we should let Warren 20-- or Ray --
21 MR. HERB: Yeah. Let me just finish the 22document. Because this -- really because this is just 23 pointers to this to how the VOP is developed.
24And it really covers -- it covers the 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 86 process piece and the design output piece is covered 1by the design. But we didn't cover that here. We 2 don't talk about the design output piece.
3 But, again, the standard design part, 4sorry, I kind of already talked about all of these 5 slides.6 MS. GOLUB: Maybe we should do this more 7 quickly.8 MR. HERB: You can flip through this.
9MR. ODESS-GILLETT: Well, let's talk about 10 where the VOP falls into each other on this. If you 11 can go back one.
12 MR. HERB: Go back one then.
13MR. ODESS-GILLETT: Okay. It says the STP 14 vendor oversight plan implement --
15MR. HERB: Right. It just -- so, the 16 standard design process turns on vendor oversight 17 planning, is partly determined in the early conceptual 18 design phase.
19 There are contractual agreements and, you 20 know, we set actual times and negotiate option access 21to their processes at that time. And do all the 22 select what we believe to be appropriate, based on the 23 LAR submittal, those practices.
24 Now, when the LAR gets submitted, these 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 87things will already be done. So, we're going to -- as 1 part of our bid proposal and the acceptance, these 2 things are all going to be negotiated.
3 But when we submit that bid, it's on it.
4 It's really a lot of the requirements that we already 5 going to have in the LAR.
6 So, it's not like we're thinking about it 7 later after all this other stuff. You know what I'm 8saying. So, it's baked into the standard design 9 process.10And it's not just digital. We -- these 11 are commercial terms and agreements between us and the 12vendor. This protects both us and the vendor going 13forward. And so we have to do due diligence. So, 14 next slide.
15 And now they get very specific in the 16NIST. This is what I'm saying. If we're talking 17 about specific things we're looking at in process and 18 specific things we're looking at in how we accept a 19 document, the NIST goes through that and talks about 20 those.21And so the ones we believe that are 22 primarily process oriented, we need to go and to 23verify the quality or the quality design process. We 24 will call those out.
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 88 And we use that guidance there from the 1 DEG. And again, EPRI has discussed the DEG with the 2research division. And I believe there was like on 3 November 8 they had a training session.
4 And I have --
5MS. GOLUB: I think it was an information 6 exchange.7 MR. HERB: But when this -- when this --
8 when the IMPO and the design oversight working group 9 make this either an optional or a mandatory, there 10 will be a roll out in the change management process 11 and training.
12 So, I would think that like NRC is welcome 13to come to those. And just kind of like in the RIS 14attachment. I was moving a lot for you all to come to 15 see what we're doing.
16 And I would get there from the digital 17 design piece.
18MS. GOLUB: Especially because we, you 19 know, I think we're using different terms or, you 20 know, --21MR. HERB: That's right. Because clearly 22 we're not talking on the same words when we say 23 things.24 MS. GOLUB: Right.
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 89MR. HERB: But that's available. It's the 1 next slide. In the industry standard guidance. And 2 basically a lot of it comes out of commercial grade 3 dedication of commercial items.
4 And so a lot of that assessment is there.
5 There's also the critical digital review guidance for 6 things we look at.
7 And then there is, as Pareez said, from 8 the DDG to the DEG, the size for vendor oversight grew 9 by about three times the size of what it was before.
10 So, it became a lot more detailed. So next slide.
11 At Southern Nuclear we've always had a 12vendor oversight planning process for a long time.
13And as part of our site procedures, or our fleet 14 procedures, vendor oversight starts during the 15 negotiation, part of the purchase order.
16 And it's usually the plan is negotiated 17pre-award. And then post-award it's finalized with 18 dates and visits and scope of what's going on.
19 And this would certainly be a big piece 20going forward for a LAR. Because a lot of it falls on 21 us to demonstrate to the NRC that we're doing our due 22 diligence on ensuring quality software.
23 And again, it's -- all the outputs from 24 the vendor oversight plan, those audits, and those 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 90 inspections, and those activities.
1 You know, and those would include 2 possibly, you know, reviews of different testing plans 3 for the -- part of a QA record. And it certainly is 4 auditable or inspectible by the NRC.
5 And when we go on some of these vendor 6 oversight visits, I would think that, you know, if the 7 NRC was interested, we would love if you all went at 8the same time we went. That way, in a kind of an 9 observer mode, so you can see what we are doing and 10 get comfortable with how we are doing our inspections.
11 And then if you thought that there was a 12gap there, you could, you know, let us know. Or maybe 13 you'd have to do your own as well.
14 But, so then it's -- the vendor oversight 15 is part of the procurement process. It supports the 16 design by insuring quality at the procured 17 performance.
18 I mean, that's -- this is a not a digital 19 versus a non-digital thing. It's really part of our 20 process.21 And this was -- this came out of the non-22 digital side. And we have actually incorporated all 23 of that on our site to then incorporate those 24 learnings.
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 91 And accepting through IMPO and through 1 other OE type organizations of the industry, we feed 2 upon each other. We learn.
3 We -- we're required as part of our design 4change process to look at OE. And to answer where 5those thought failings were. So, we were just part of 6 this process.
7 If we were not the first one, you know, we 8 would look at all the OE from the Diablo Canyon, from 9Oconee, and some of these things. And all of those 10 things that were maybe bumps in the road for those 11 earlier ones, we would feed that into this process.
12 So, we're not creating this out of whole 13 cloth. We're building upon each other. And so some 14of these earlier ones might be a concern. They might 15 be a learning thing.
16 But the next ones we believe start getting 17better and better. So the next slide. So, that's it 18 for me.19MR. ODESS-GILLETT: So any questions on 20 the industry's standard design process digital 21 engineering guide and digital engineering process 22 before we proceed?
23MR. DARBALI: No. I think we're -- we see 24 these as moving in the right direction. And we, you 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 92 know, we're in the learning process and so future 1 engagements would help us better understand it.
2MR. HERB: And sitting in on some of that 3 training, you know, might help you get a little better 4 feel about where we're doing some of those technical 5 reviews that we're not going to like and that were not 6 part of this vendor assessment.
7 And if we have to tweak our vendor 8 assessment, to be --
9 MS. GOLUB: To be -- right. To describe 10 some of that in more detail.
11 MR. HERB: We can do that.
12MS. GOLUB: Then we should. Right. To 13 make them -- to make sure that the NRC is comfortable 14 that those activities are happening.
15 Even if, you know, the utility's, you 16 know, programs, they don't call them vendor oversight.
17 But we could describe that more fully.
18MR. HERB: You could still describe it 19 around this piece.
20 MS. GOLUB: Yeah.
21MR. HERB: Because I think part of our 22vendor oversight plan we also say we were going to 23 like verify PSIAs are implemented appropriately and 24 inspected.
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 93 And that's typically not part of a vendor 1 assessment.
2MS. GOLUB: But I think Rich is, you know, 3 Rich's comment on those design outputs and how those 4 were reviewed, I think that's something we could 5 enhance.6 MR. HERB: Yeah. We would enhance that.
7 MR. STATTEL: Yeah. So, I'll just leave 8with a -- leave this with a comment. And we'll get 9 into more discussion later.
10 But, my experience, and I've been involved 11 with a lot of digital modifications. The activities 12 that we have gone out to the vendor and performed, I 13 do not -- I have not seen licensees doing those type 14 of things.
15 So it maybe because they see that we're 16 doing them. So they don't do them. Right?
17 MR. HERB: That could be.
18 MR. STATTEL: That could be. I do agree 19that they should be. They should be doing them. I do 20 agree with that.
21 I think it would be a good thing. But I 22 don't see anything in the processes or the vendor 23 oversight plan that gives me confidence that they will 24 do those things.
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 94 MR. HERB: Right.
1MR. STATTEL: I do have several examples.
2 But, we can talk about those later.
3 MR. HERB: Okay.
4MR. ODESS-GILLETT: Yeah. Let me sort of 5 address that in the introduction to this. Because I 6 think, the industry concurs that on some of these past 7 projects or upgrades that the licensee has not met NRC 8 expectations under oversight.
9 But, we're seeing the convergence of two 10different initiatives happening at the same time. The 11 one side is this standard design process that 12integrates vendor oversight. It's never been done 13 before across the industry.
14 And the other is this alternate review 15 process that is basically placing the expectation on 16 the licensee to step up their game and do their vendor 17oversight. And we're giving the licensee the 18 opportunity to step up to that game.
19MR. STATTEL: Um-hum. And I'm not saying 20 that the licensees have been negligent or anything 21 like that.
22 MR. ODESS-GILLETT: I understand.
23MR. STATTEL: In a lot of cases, they just 24 don't know the processes. They don't know to go and 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 95 look for these things.
1MR. ODESS-GILLETT: Right. And that's --
2MR. HERB: And the learnings that we're 3 seeing now. And we are folding that into a standard 4 process with the whole industry.
5So it's not like you have to go back to 6each one of those. Those learnings are now 7distributed across everybody. So, you just share 8 design process.
9 MR. ODESS-GILLETT: Okay Joe, can you go 10to the next slide? Uh-oh. And that concludes my 11 presentation.
12 (Laughter.)
13MR. ODESS-GILLETT: All right. So, these 14 are the two initiatives that are occurring each from 15 the NRC side with their revised ISG that has the NRC 16 view of the LAR concluding before the detail design 17 implication test activities are completed.
18 And that the licensee is taking over that 19 role of vendor oversight to the equivalence of what 20 the technical reviewer from the NRC staff performed.
21 That's industry's expectation of the vendor oversight.
22And so we agree. We're stepping up the 23 games with the licensee's expectation of vendor 24oversight. So that what was done by the technical 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 96 reviewer will now be done by the licensee.
1 That's the expectation. On the industry 2side is where it's explained. The industry is 3 standardizing their design process, even digital mods 4 and the design process for digital mods that 5 incorporates vendor oversight.
6 So these are the two initiatives that are 7 converging today. Next slide.
8 So, with this convergence from the 9 industry side, we see maybe a new paradigm in looking 10at inspections. So, with the new draft ISG-06, and 11 the LAR review, the NRC is going to review the vendor 12 oversight plan.
13 And the NRC is going to make sure that 14that vendor oversight plan is -- meets the 15 expectations of the NRC staff for adequate oversight.
16 So, before we even complete the LAR 17 review, we'll have a vendor oversight plan that meets 18 NRC expectations of what is adequate for the licensee 19 to perform that vendor oversight.
20 As I said, the industry is standardizing 21their process. It incorporates vendor oversight. And 22 with the LAR technical review, the audits are now 23going to focus on those things that we think are 24 really important.
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 97 The system, the architecture, the plant 1 interfaces, the human faces, and the plant safely 2impacts. We see those are really the safety 3 significant aspects of a LAR review for our digital 4 safety-related mods. Okay.
5 NRC inspections of digital INC license 6 amendments in our opinion, can adjust too this new 7(phone noise) industry prepared them. And I'll in the 8 next slide show you what we think that paradigm would 9 look like.
10MR. GALLETTI: Warren, before you go.
11 Could I just ask you to go back to that last slide.
12 MR. ODESS-GILLETT: Yes.
13MR. GALLETTI: So the bullet that says NRC 14 audits new focus.
15 MR. ODESS-GILLETT: Right.
16MR. GALLETTI: So from the industry 17 perspective, what do you think the current audit focus 18 is?19MR. ODESS-GILLETT: Well, they only had so 20 many resources and so much time in order to inspect or 21 to audit not only these important things, but all of 22 the process aspects through detailed design, 23 implementation, and test.
24 So, this by focusing more on the -- what 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 98 we consider the safety significant aspects of the mod, 1 we make sure that we have a safe implementation of the 2 mod.3 It was interesting, at the last hearing, 4 Commission hearing on digital INC mods, did you -- I 5 don't know if anybody listened to that MIT professor.
6 But he basically said that industry at this point 7 knows how to implement requirements and software.
8 Okay?9 The biggest safety concerns are unintended 10consequences as a result of bad requirements. That 11 that's the biggest danger to our industry.
12 So, to us that should be the focus of the 13regulatory review. Because at this point, after 40 14 years of doing digital INC, we feel that it's not just 15 safety significant to be able to transfer requirements 16 into a design in software.
17 And so that's --
18 MS. GOLUB: I'm sorry. The hard part is 19 developing.
20MR. GALLETTI: The finding of the 21 requirements.
22MR. ODESS-GILLETT: The finding of the 23requirements. So that's -- that's right. That's what 24 we're referring to.
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 99 MR. VAUGHN: But was your question, what 1 was NRC's audits before? Because you said new. So, 2 you're saying that they were changed?
3MR. GALLETTI: Well, I think Warren 4 answered the question.
5 MR. VAUGHN: Okay.
6MR. GALLETTI: I was looking for the 7 industry perspective on what we have currently been 8 doing that's different from this.
9MR. STATTEL: In all honestly, I mean, 10 I've been trying to think about what it would -- what 11 a license review in the future would look like.
12 I don't even know that we would need to do 13audits. At all. Because pretty much everything will 14 be -- because most of what -- 99 percent of what we do 15 on a typical audit for a digital INC system, is now 16 being transferred to the licensee.
17MR. GALLETTI: Well, you've planned.
18 There's the requirements' pieces.
19MR. STATTEL: In the case of Diablo 20 Canyon, we did some early audit, early stage audits.
21 And our -- what we learned was, they were premature.
22 Because they hadn't progressed far enough 23 through the impleme ntation for the audits to be 24useful. So, we ended up doing a second set of audits 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 100 at the end of the process.
1 But, clearly, we wouldn't have that 2opportunity in the future. And for an alternate 3 review.4 MR. WATERS: When you use the word audit 5 on a stability, are you talking about NRC license 6reviews, an audit process or inspections? What are 7 you talking about?
8MR. ODESS-GILLETT: Audit would be during 9the technical review of the LAR. It would be the 10 technical review staff's audit of the system 11 requirements specification, the plans for digital 12 development as described in D4 and ISG-06.
13 So these are, you know, the LAR will 14describe these things. But, the NRC staff has its 15 prerogative to say, well, let me look at those plans 16 that you described in D4 and the ISG-06.
17 Let me see your system requirement 18specification. Because it should be developed as part 19of the LAR submission. And we consider those audit 20 material.21MR. WATERS: How do I judge plant CD 22 impacts based on what those things are?
23MR. ODESS-GILLETT: Your regulatory 24requirements for review. IEEE 603, your ISGs on data 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 101 communication, your GDCs on separation independence.
1 MR. WATERS: Is that the requirements?
2 MR. ODESS-GILLETT: Yeah.
3 MR. WATERS: Okay.
4 MR. ODESS-GILLETT: Yeah.
5MS. GOLUB: Well, and the conceptual 6 design. Your architectural.
7 MR. ODESS-GILLETT: Yeah. Architecture.
8 MS. GOLUB: There's more material in the 9 LAR then just the requirements.
10MR. WATERS: So it hinges on your 11 requirements then.
12MR. ODESS-GILLETT: It hinges on the 13 requirements, but Pareez is right, the LAR is going to 14 present the architecture.
15MR. FANELLI: How complete would the 16 requirements be by then?
17 MR. HERB: Complete.
18 MS. GOLUB: Complete.
19 MR. ODESS-GILLETT: Complete.
20 (Simultaneous speaking.)
21MR. HERB: When we write a PO, it's going 22 to be based on a complete set of requirements. Now, 23 there's a negotiation and normalization when you put 24 out a bid specification.
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 102You select the vendor. You select the 1 platform. You normalize that requirement. And when 2 you write that PO, that last normalized step, that's 3 what will go out in our LAR.
4That's what we've purchased. And that 5 will go in the LAR.
6MR. ODESS-GILLETT: I'd have to 7 respectfully --
8 MR. HERB: With some of the steps.
9MR. ODESS-GILLETT: Yeah. I'd have to 10 respectfully augment what my licensee colleague is 11 saying that once we get a purchase order, the vendor 12 would have to write a functional specification.
13 MR. HERB: Okay. Yeah.
14 MR. ODESS-GILLETT: A system requirement 15 specification, a system design specification.
16 MR. HERB: But those are all done before 17 the LAR is submitted.
18MR. ODESS-GILLETT: They're all done 19before the LAR submittal, correct. But after a 20 procurement contract.
21MR. HERB: Yeah. You're right. You're 22 right. Yeah.
23MR. DARBALI: So the scope of the audits.
24That depends a lot on each application. And there is 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 103 a chance that no audit is needed, or only an 1 electronic audit is needed.
2 But you could have the case where the 3 licensee submits the LAR, but by the time the LAR is 4 expect -- the review is expected to be completed, you 5haven't gone to testing, but you've gone through 6 fabrication.
7 So, those are some activities that the 8 staff could audit if you transferred.
9 MS. GOLUB: Yeah. No, I think --
10 (Simultaneous speaking.)
11MS. GOLUB: That could be inspected.
12 Yeah.13MR. ODESS-GILLETT: That would be -- yeah, 14 those would be considered post-amendment inspections.
15 MR. DARBALI: So, okay, not disagreeing.
16But let me clarify what I said. I said, before the 17 license amendment is issued, some development 18 activities have taken place.
19 MR. ODESS-GILLETT: Correct.
20MR. DARBALI: There's no Phase II 21 submittal.
22 MR. ODESS-GILLETT: Right.
23MR. DARBALI: So they're not related to 24Phase II. So, I guess my question is, if those are 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 104 activities being done before the license amendment is 1issued, so we cannot audit them. Can we do an 2 inspection before the license amendment is issued?
3Because we don't want to leave an area 4 where we can neither inspect nor audit.
5MR. ODESS-GILLETT: Well, it's not 6 excluding inspection.
7MR. DARBALI: But can we do an inspection 8 before the license amendment is issued?
9MR. FANELLI: The reason for this let's 10 say literal, you would have to have a license basis to 11--12 (Simultaneous speaking.)
13 MR. ODESS-GILLETT: Yeah, it falls under 14the licensee, the licensee's oversight. And works it 15 against the -- they have the plan that the NRC 16 approves.17 MS. GOLUB: Right. Yes.
18MR. DARBALI: Can we audit the oversight?
19 MR. ODESS-GILLETT: Yes. Sure.
20 (Simultaneous speaking.)
21MR. DARBALI: I mean, we can't -- it 22 cannot be an inspection before a license knowing 23 procedure. But those --
24 MR. ODESS-GILLETT: The licensee has the 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 105--1 (Simultaneous speaking.)
2MR. ODESS-GILLETT: Has the option to 3invite the NRC to observe their licensee oversight 4activities. But the audit, it's not in the purview of 5 the LAR.6 Because the LAR submittal --
7 MS. GOLUB: Is complete.
8MR. ODESS-GILLETT: Is based on the 9 detailed system design.
10MR. DARBALI: But the audit has to support 11 the safety violations.
12MS. GOLUB: The LAR. That's correct.
13 That's correct.
14 MR. DARBALI: Correct.
15MR. WATERS: I think what you're trying 16 out is, I mean, what if there's a critical development 17 process or test that happens during that license view 18period. It's not part of the audit, because we're not 19 basing that.
20 And how do you inspect it, because it's 21not a good license. So, what are we expected to do 22 under this?
23 MR. ODESS-GILLETT: So the lic -- that's 24where we're stepping up the game. Where the licensee 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 106 is responsible for those oversight activities.
1 Now, once we're --
2 MR. WATERS: Well, I don't think so.
3MR. ODESS-GILLETT: No, let me finish.
4Let me finish. When the amendment is issued, the 5 inspection staff can look at what was -- and we'll get 6 into this, what was performed by the licensee's 7 oversight activities in the past tense, to see did 8 they -- did they do what they said they were going to 9 do in their vendor oversight plan?
10 Are there any gaps between what they 11 reviewed and what they documented versus what's in the 12vendor oversight plan? That can be done. It's not 13 forgotten or left out.
14 But --15 MS. GOLUB: Or undocumented.
16 MR. ODESS-GILLETT: Yeah.
17MR. STATTEL: I also want to explain, the 18audit is more than just a document review. And we're 19 doing more than just going through a process of, you 20 know, looking at artifacts.
21 And there's a human element to it too.
22And there's a training element. So, half of the time 23 we spend on an audit is figuring out how these 24 processes really work.
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 107 Because when you read a plan, you kind of 1don't know exactly how that's used. What actually 2 takes place.
3 And that's why the audit has to take place 4at the vendor premises. It's because we need to talk 5 to the people who are doing the deeds.
6 And we need to find out from them, we 7 interview these people. We run them through certain 8exercises. You know, we have kind of a -- kind of an 9 undocumented cookbook we use for this, right.
10And we perform these interviews and we 11ask these standard questions to them. We talk to a 12software librarian. We talk about configuration 13 management, how that's done.
14 We talk about things like how do you make 15 sure -- we ask them like these esoteric questions 16 like, how do you make sure no software, undocumented 17 software gets introduced?
18 What process is -- what in this plan, what 19 in this procedure that you're using, prevents that 20 from happening?
21 And I know some of you have been on these 22audits. You've heard us ask these questions. And we 23listen to the answers. And we have these -- these 24 interactions with the people who are doing the deeds.
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 108 Right?1 And it gives us a whole lot better insight 2 then we would have through just a document review.
3 Because otherwise, you know, there -- we could all --
4 we could do it right here in the office here, you 5 know.6 And you know, if you're just looking at 7 during the requirements threads, you know, that's --
8you know, that's mechanical. I can just -- I don't 9 need to go to the vendor's site to do that 10 necessarily.
11 But to really understand their processes 12 and how they're being used, that's where the audit 13 comes into play. So, I know it's hard.
14 That's kind of the guidance for doing that 15that I'm looking for. If we're going to transfer that 16 activity over to the licensees, which is really what 17 this comes down to.
18 MR. ODESS-GILLETT: That's correct.
19MR. STATTEL: I want to have some 20 confidence that they're going to do kind what we've 21 come up with.
22 MR. ODESS-GILLETT: And we welcome -- we 23 welcome that feedback and that incorporation into the 24 vendor oversight plan.
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 109 MR. STATTEL: Um-hum. And the documents 1 don't necessarily reflect everything that takes place 2 during an audit.
3 MR. ODESS-GILLETT: Um-hum.
4MR. STATTEL: It's really -- it goes 5 beyond just a document review. It's an actual human 6 interaction.
7 MR. ODESS-GILLETT: Um-hum.
8MR. STATTEL: We're talking to each other.
9We're leaning your processes. We really need to have 10 those interviews.
11MR. ODESS-GILLETT: Right. But the 12 licensee really needs to have those interviews. And 13--14 MR. STATTEL: Right. Exactly.
15MR. ODESS-GILLETT: Understand the 16process. Because the licensees' ultimately 17responsible. So that's really what we're trying to 18 convey here.
19MS. GOLUB: Could I jump in and just 20mention? I know Steve, maybe you can talk a little 21 bit about this.
22 When you put together that vendor 23 oversight plan, I know you go through in detail a 24series of NRC audit plans and reports. Can you speak 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 110 just a little bit to that? Because I'd like to give 1 Rich a little bit of assurance that we did look at 2 what the NRC did.
3 Now, I understand you have an undocumented 4process, and undocumented questions. Obviously we 5 don't have access to your undocumented stuff.
6 But, we did go through several audit plans 7 and reports so that we could try to get a sense of 8 what the NRC did, so we could try to replicate that.
9 And I know Steve took some time to do that in detail.
10MR. DRAGOVICH: Yeah, so that's true.
11 But, so what we did was under project specific 12 performance measures. So we had general performance 13 measures and project specific performance measurers.
14 Project specific just came from audit 15plans. So we tried to pick -- I looked at several of 16them. And tried -- because there's a lot of overlap.
17 There's a lot of the same categories that 18 get -- so I tried to pick some of the more common ones 19 and listed them here to try to like Ray said, 20 replicate what the NRC had in the audit plans.
21 And now naturally, this is going to get 22more specific for projects. I mean, we're going to 23 enhance that.
24 But that's kind of where this section came 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 111 from. Because the general performance measures came 1 from the EPRI commercial grade dedications.
2MR. STATTEL: But you're referring to the 3 VOP, right?
4MR. DRAGOVICH: Yeah. Exactly. And we're 5 going to talk about that more later too. But that's 6 kind of what that --
7MS. GOLUB: I just wanted to because, you 8 know, I just want to -- just so Rich knows that we did 9 look at some of the material that the NRC -- that we 10 had in the public sphere.
11 So that we could try to imitate some of 12that work that the NRC has done. Look at the same 13 items.14 MR. SCHRAYE: Can I ask a question? You 15have an undocumented cookbook? So we don't know what 16 that is.17 MR. STATTEL: Well, I mean --
18 MS. GOLUB: Well, I mean it --
19 MR. STATTEL: Let me explain.
20 MR. SCHRAYE: Okay.
21MR. STATTEL: I know that came out wrong.
22We do -- we do document our audit plans. Right? Now 23 the audit plans also include performing interviews 24 with certain -- in certain key positions. People in 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 112 key positions.
1 And during those interviews, we kind of 2 run them through scenarios. So, an example would be 3 the software librarian, we would say, okay, I'm --
4 let's pretend I'm an engineer, a software development 5 engineer, and I want to change Part A of the code.
6 And how do I go about checking it out?
7 And he'll show me the form that they -- he fills out.
8 And the approvals he needs to do that.
9Okay. Now it's checked out. I'm working 10on it. Now engineer B walks -- comes along, and he 11 wants to check out the same code. Right?
12 So, what would prevent him from being able 13to do that? And what would prevent engineer B from 14 overriding engineer A's code? Right?
15So, you're not going to see a procedure 16 for going through that thought exercise in the audit 17plan. But the audit plan will say, we conducted an 18 interview with the software librarian. Right?
19And we will document, you know, we will 20 basically write that scenario up in our audit report.
21 So, it is all documented.
22 But, it's not to the level of detail, you 23 know, that we basically developed over the years.
24MR. ODESS-GILLETT: And I guess then it 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 113 would vary from auditor to auditor?
1 MR. STATTEL: Not so much. We try --
2 MR. ODESS-GILLETT: Why is that?
3 MR. STATTEL: Well, you know, we're kind 4of working with the same people. You know. So Samir 5 and I have done a lot of these audits. We've done 6 audits at General Electric. And we go to Reddy 7 (phonetic), and we go to different sites.
8 And we kind of have it down. I mean, we 9 do specialize in, at least in the last ten years.
10 MR. DARBALI: Right. When we --
11MR. STATTEL: We've kind of gotten it down 12 to a science.
13 MR. HERB: So basically there's only two 14 is what you're saying.
15MR. STATTEL: No, no. It's not two. It's 16 like --17 (Simultaneous speaking.)
18MR. DARBALI: Well, like Rich said, I 19 mean, what we're going to do is document that in the 20audit plan. And when we do this we document the audit 21 report before we do an audit.
22 And you know, we got more vendors then 23license amendment reviews. So, we're going to do a 24 topical report audit, we'll go back to the other ones 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 114 we've performed and we see one thing we look at, what 1 did we report?
2 You know, was there value in what we 3 looked at? And we use that as a base.
4MR. SCHRAYE: I guess my point wasn't 5trying accuse you of anything. It just seemed like 6 you have a lot of good information and approaches.
7 And that would be helpful.
8MR. STATTEL: I know and I don't know how 9 to transfer that.
10 (Laughter.)
11MR. STATTEL: Well no, I can't -- there's 12 not a document I can hand you.
13MR. DARBALI: Well, the audit reports and 14 the audit plans, that's --
15 MS. GOLUB: Which we looked at.
16 MR. DARBALI: Right.
17 MR. DRAGOVICH: That's all we had.
18MS. GOLUB: We did look at that. Yeah.
19 We did look at that to develop what we put together.
20MR. DRAGOVICH: Right. And what you said, 21 like we have a category called change control of 22 management. Right? It sounds like we would just be 23 able to enhance that to include maybe one of those.
24 MR. DARBALI: Right.
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 115MR. HERB: But, I mean, but most of us 1have supported NUPIC type audits. And it's a similar 2 process. You have the processes and procedures, but 3 you go talk to the people and you ask them to walk you 4 through it.
5 So, I mean, that's -- I don't think that's 6 unique to the industry.
7MR. STATTEL: And part of the process that 8 doesn't show up in the procedures and the documents is 9learning. Because every vendor has a -- their 10 requirements traceability is done very differently 11 from one vendor to another.
12And it's not -- it's not intuitive. I 13 can't just pick up the document, oh, now I know how 14it's done. Sometimes I have to have the vendor 15 explain to me. What's my document numbering system?
16 What's -- how are those requirements' 17traceability established? Is it a table in the 18 document itself? Is it a separate doors report?
19 Is it -- you know, and it's all over the 20map. Right? So, you know, when I come into a new 21 vendor and I'm looking at a process for the first 22 time, I kind of need him to teach me.
23 So, that instructional part of the audit 24 is also one of those things.
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 116 MS. GOLUB: But it's --
1MR. STATTEL: Now, we do document that.
2 When we put our report in, we describe those 3 processes. Because now that we know that.
4 But the audit itself was a learning 5 process so that we could -- it enabled us to be able 6 to write those reports.
7MS. ZHANG: So, in my opposite view in 8 terms of, I don't disagree that, you know, we look at 9 those processes in audits. But, from a new reactor, 10 when we did ours, right, we don't have all those 11 processes, you know, the outputs yet.
12 But we did focus a lot on the design and 13how we designed it. And because we're looking at the 14 overall plan, so we focused in on the integration of 15 the overall plan and the functions a lot.
16 But, through the ITAAC process, I would 17imagine for -- the result for looking at the design 18 outputs and the processes that we generate those 19design outputs. And do the same thing that Rich 20 described.
21 And I'm saying that, can we do the same 22thing here as part of inspection? Whether it's vendor 23 oversight inspection or --
24MR. FANELLI: We actually do. We actually 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 117 verify that. We do, very similar.
1MR. ODESS-GILLET: Yeah. Yeah. So, I 2 mean, that's inspection space.
3 MS. ZHANG: Yeah. And I would also --
4MR. STATTEL: The same interview is the 5 same questions.
6 MS. ZHANG: And then I would also expect 7 that the licensees, no matter whether we do 8 inspections or not, you will be taking over all those 9 responsibilities.
10 Well, you are already taking all those 11 responsibilities.
12MS. GOLUB: That's exactly right. Right.
13MR. ODESS-GILLETT: It's so much more 14 important in our opinion that the licensee step up --
15 MS. GOLUB: Absolutely right. Um-hum.
16MR. ODESS-GILLETT: And do what Rich 17described. Because it's his system that's going in 18the plant. It's his sys -- it's he that will be 19 answering those questions to the inspection staff.
20MS. GOLUB: Yes. So your point is well 21 taken.22MR. HERB: And then the reason why the 23 DEG, that's the amount of information for a vendor 24 outside tripled from the original one too now, is 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 118 because we have learned that we were acting in the 1 industry to trust those vendors.
2 And so what you saw in 2012 is not what 3 you're going to see in 2018 or 2019. So I believe.
4MS. ZHANG: So I think, you know, from us 5 we do have our past experiences that kind of adds a 6lot of burden. We do hope that, you know, things have 7 changed and there's, you know, a lot of improvements 8 made.9 But I think the first few LARS that we 10 will see, --
11 MR. HERB: Of course. Yes.
12MS. ZHANG: Expect that, you know, we will 13 do a lot of, you know, initial inspections to verify 14 that has been done.
15MR. HERB: We have to establish a track 16 record. And that's -- we're going to do that.
17MR. SCHRAYE: Our concern is that we're 18 going to be the first out of the block. So, what 19 color block should we do? Provide?
20MR. DARBALI: And like I said, I mean, 21 we'll continue these discussions during the 22 application meetings when you come back in.
23MR. STATTEL: I will say it's common sense 24 and we've had experiences with licensees where we, you 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 119 know, like I said they weren't performing with the 1 oversight activities that we were doing.
2PARTICIPANT: Yes, you're absolutely 3 right.4MR. STATTEL: But we did talk and say you 5 really need to be concerned about this. So we would 6 take the licensee engineer and we would show them, 7 look, this is what we found on the vendor and here is 8 where our concerns are, and guess what, once they 9 understood that they had the same concerns.
10 In a couple of instances they came back 11 and performed their own audits independent of us on 12 those vendors and they made course corrections, okay.
13 PARTICIPANT: Right.
14MR. STATTEL: So it is common sense stuff 15 and once engineers get the understanding of where the 16 concerns are and where we need to keep the vendors in 17 line and head in the right direction we do come to 18 agreement, okay. I hate to say it is common sense.
19MS. GOLUB: But, Rich, to your point, you 20 know, on the licensee side we are not leaving it to 21people using common sense. I mean that's why the DEG 22 has an enhanced section, a lot more information on 23 vendor oversight, a lot more guidance, because the 24 licensees recognized that they were lacking in that 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 120 area and are addressing this.
1 So you're not kind of waiting to be told 2 or to kind of pass it on by word of mouth or tribal 3knowledge or any of that. I mean it's a far more 4 detailed, more documented process.
5The one thing I will say is if you have 6 specific --
7PARTICIPANT: Greg wanted to say 8 something.
9 MS. GOLUB: Oh, I'm sorry.
10MR. GALLETTI: I just wanted to ask so 11 what I am hearing sounds very interesting but as far 12 as the VOP is concerned, so the details of the 13 licensee's oversight activities at the vendor would 14 that be incorporated in some sort of scheduling 15 mechanism in the VOP so the reviewer would know, hey, 16 they are going to do this at approximately this point 17 in the development process at that level of detail --
18 MS. GOLUB: I know what you are saying.
19MR. GALLETTI: -- so if the NRC wanted to 20 engage with your oversight activities it would be 21 obvious when it's going to occur, yes?
22 PARTICIPANT: Oh, yes.
23MR. DRAGOVICH: Yes. Well, as Ray pointed 24 out like when we negotiate the original procurement 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 121 spec we define when we want to have those oversights 1 and at which stages, but then again if they have gaps 2 then we might increase that oversight, too --
3 (Simultaneous speaking.)
4MR. GALLETTI: Okay. But there will be 5 some as part of the LAR submittal that at least a 6 baseline engagement would be understood.
7MR. HERB: Right. And it may be 8 milestone-based rather than actual-based, but that 9schedule is a live schedule and we maintain on a 10 SharePoint and we would agree, we would discuss that 11 in a Phase 0 meeting and some of those dates are 12 already known at the Phase 0 meeting that we can --
13 MS. GOLUB: Right.
14MR. HERB: And, again, we would welcome 15 your insights, you know, to go along with this, you 16 know, and observe to make sure that we are doing our 17 job properly and then provide input if you felt it was 18 necessary to adjust course corrections at that time.
19 I mean this is a new process for us and we 20 want you all to be comfortable with what we are doing.
21 MR. WATERS: So I think a common comment 22 is understanding the timing of your process --
23 MR. HERB: Yes, right.
24MR. WATERS: -- so we don't duplicate 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 122 efforts we don't particularly need to duplicate, 1 right.2MR. HERB: Right. Or in the Southern 3 procedures it says you will define these dates and it 4 actually has like --
5MR. WATERS: We were trying to understand 6 how we would be communicating in real time going after 7 LARs.8 MS. GOLUB: I think --
9 MR. HERRELL: So the one thing that I'll 10 push strongly for is the schedule itself doesn't go in 11 the VOP because of the LARs schedule it's going to be 12 changing far more often than I would desire because 13 referenced out of the VOP you put it someplace where 14 it can be maintained and where you guys can, where 15 everyone can see it.
16MR. WATERS: Yes, maybe because of 17 communication --
18 (Simultaneous speaking.)
19MR. WATERS: -- understand that but that's 20 one of the things we are talking about.
21 MR. ODESS-GILLETT: Right, but they have 22--23 (Simultaneous speaking.)
24MS. GOLUB: Oh, you're just looking for a 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 123 break.1 (Simultaneous speaking.)
2MR. ODESS-GILLETT: Okay, I'm almost done.
3 I'm almost done.
4MS. GOLUB: Oh, I didn't get it. The T 5 was for --
6 (Laughter.)
7MR. ODESS-GILLETT: So I think we've been 8 on this slide probably long enough, Joe, so we can go 9 to Slide 11. So what's this new inspection paradigm 10 from what we are seeing?
11 So in the NRC inspections from our point 12 of view relative to the AR process, a license 13amendment, we feel should start with the licensee 14 because the licensee is responsible for vendor 15 oversight.
16 So from our point of view the inspection 17 should focus on the licensee's fulfillment of its 18vendor oversight plan. NRC's inspections would focus 19 on the licensee's vendor oversight reports that 20 document those vendor audits and inspections of a 21 detailed design implementation and test processes.
22 Only if the NRC identifies gaps in the 23 vendor oversight process would it be in our opinion 24 appropriate for the NRC to augment licensee 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 124 inspections at the vendor site because what we don't 1 want to happen is the vendor inspection branch repeat 2 what the licensee did unless there is a perceived gap 3 or inadequacy in what the licensee did and then we can 4 see that the vendor inspection branch would want to 5 augment that.
6 MR. GALLETTI: Where in this paradigm do 7 you account for the notion of the NRC overseeing 8 licensee activities at the vendor facility as they are 9 actually occurring?
10MR. ODESS-GILLETT: Can I just -- That 11 first sub-bullet. You are going to look at what the 12 licensee's vendor oversight records document for those 13--14 (Simultaneous speaking.)
15 MR. GALLETTI: Yes, but that's after the 16 fact, you know. If you are going into a vendor shop 17to do certain oversight activities where would the 18 paradigm support the NRC reviewing your activities of 19 doing that oversight?
20 MR. ODESS-GILLETT: In real time?
21 MR. GALLETTI: Yes.
22MR. ODESS-GILLETT: Well what's the 23 requirement for doing it in real time?
24 MR. BOLLOCK: Well before it was part of 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 125 before the license was approved they --
1 (Simultaneous speaking.)
2MR. BOLLOCK: Yes, they were reviewing 3 things along the way, auditing, well now it's approved 4 earlier.5 (Simultaneous speaking.)
6MR. STATTEL: I've had these interactions.
7 So in the past we don't look at the vendor oversight 8 plan, for instance, so we don't know if they're --
9 MR. BOLLOCK: Right.
10MR. STATTEL: But I am there. I am at the 11 factory test, I am at the vendor facility, and every 12 time I go there, you know, I am working through the 13 licensee, right, I am working on a license amendment 14 so the licensee is there as well.
15 So, for instance, when we did our audit in 16 Wilmington we had several licensee representatives 17 present at those meetings and we did talk with the 18 licensees and we were asking well what's your role, 19 you know, and that's how we learned, right, but we 20 didn't see the vendor oversight plan.
21MR. WATERS: I mean is there a concern 22 with the -- There is two things here, a licensee 23 inspection versus a vendor inspection, or is it about 24 the end of a site because it's --
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 126MR. ODESS-GILLETT: It's about duplicating 1 work.2MR. WATERS: I understand that, but our 3 experience has been at the vendor site is if you can't 4 do a license inspection how are you overseeing it is 5where do we get the inspector processes. That's what 6 I am trying to --
7 (Simultaneous speaking.)
8 MR. ODESS-GILLETT: All right, so I mean 9 there is always the opportunity for a licensee to 10 invite the NRC inspection staff to observe their 11 vendor oversight activities, but the idea is to give 12 this responsibility to the licensee --
13 PARTICIPANT: Agree.
14MR. ODESS-GILLETT: -- and the NRC 15 oversight is did they do what they said they were 16 going to do.
17MR. GALLETTI: Well part of making that 18 determination would be understanding your VOP says 19 this is what you are going to do did you in fact 20 implement that as written.
21 MR. ODESS-GILLETT: Right.
22 MR. GALLETTI: The output report may not 23 tell me whether you did it according to how you said 24you were going to do it. My observation of you 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 127 actually performing that implementation would give me 1 that confidence, that information.
2 MR. ODESS-GILLETT: Well -- Go ahead.
3MR. HERB: I mean I think overall probably 4 it could be worked out in a Phase 0 meeting, you know, 5 if some of those activities have not been performed 6 yet, because, you know, some of these things by the 7 time Phase 0 some of those activities may have already 8 gone past.
9MR. GALLETTI: Let me give you an example, 10 you know, the bit about overlapping of duplication of 11effort. So one example would be typically if the NRC 12 was coming to do an inspection at a vendor facility 13doing some sort of modification we may look at that 14 IV&V and do a lot of discussions with their staff, 15 look at their phase summary reports, look at their 16 outputs, look at how they establish their testing and 17 all of that sort of thing.
18 Now we are saying that you licensees you 19have the ultimate responsibility for doing that so 20 we're not going to do that but we may come in and do 21an inspection of you doing that. And so where in this 22 paradigm would it allow us to do that or --
23MR. DRAGOVICH: Why would it have to be 24 real time versus just reviewing our report that we 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 128 generate after our visit, like he had a report that --
1 MR. ODESS-GILLETT: I think because Greg 2 is saying the report may not have enough detail and I 3 would like to address that.
4 MR. DRAGOVICH: Is that --
5 MR. GALLETTI: Again, the VOP as written 6 may not tell, should tell me what you plan to do, 7okay. How would I know you actually did that? It may 8 not be reflecting in the output report exactly what 9 you did and so for me to get confidence that you did 10 what you said you were going to do for that report I 11 think --12 (Simultaneous speaking.)
13 MR. ODESS-GILLETT: Well the other thing 14 you could do, Greg, is when you go to the licensee and 15 the report says well I looked at the V&V summary 16 report and I had no findings.
17 You could audit that V&V summary report by 18 asking for it. The vendor could provide that to the 19 licensee and say here is my V&V summary report that 20 you did your oversight on and you could then do an 21 inspection of that V&V summary report without 22 necessarily having to go to the vendor site.
23MR. FANELLI: Will you have access to all 24 of the documents --
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 129 (Simultaneous speaking.)
1 MR. ODESS-GILLETT: You will have access 2to all of the vendor documentation that they had if 3 you need to substantiate some aspect of the vendor 4 oversight program.
5 MR. GALLETTI: Okay.
6 MR. STATTEL: And a lot of it depends on 7 the quality of the report.
8 (Simultaneous speaking.)
9 MR. ODESS-GILLETT: That's what I wanted 10 to address, too.
11 (Simultaneous speaking.)
12 MR. STATTEL: -- V&V summary report it's 13 not just there was no issues with them, it's the plan 14 said we were going to do five activities, they did 15 five activities, right, they match, right, and what 16 were the results of those activities, and issues were 17 found, were they resolved --
18 PARTICIPANT: Right.
19MR. STATTEL: -- if they were not resolved 20 what was the justification for carrying them into the 21 next phase, you know, those are the types of things we 22 look in a V&V summary report.
23 PARTICIPANT: Right.
24MR. STATTEL: And the report if it's well 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 130 written it will be very clear, yes, you have executed 1that plan. I will say that if the report is well 2 written it would give that indication.
3MR. WATERS: Will a report discuss the 4 previous, the interview technique and interview 5 questions we talked about earlier --
6 (Simultaneous speaking.)
7 MR. ODESS-GILLETT: Well this is sort of 8 new material, new feedback to us, and so we are going 9 to take that back and to consider how that we can 10 incorporate that.
11 But we intend to put together industry 12 guidance for this ISG-06 alternate review process and 13one of the things that the licensees have agreed is 14 that there would be a standardized format and content 15 guidance for what would be in their oversight reports 16 so that there would be consistency across the industry 17 on what the inspection staff would see for vendor 18 oversight reports.
19 MR. STATTEL: Yes. Consistency is going 20to be difficult to achieve here. You know, we talked 21earlier about, you know, the staff. We have probably 22 ten people here that have done many, many digital I&C 23 reviews so we are able to be pretty consistent when we 24 perform our audits.
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 131 When we are talking about an industry of, 1 you know, all you guys I don't know how you can 2 achieve that same level of consistency.
3 (Simultaneous speaking.)
4 MR. STATTEL: It's a challenge. I'm not 5 saying it's impossible.
6MR. ODESS-GILLETT: Yes, okay. We got to 7 start somewhere, Rich.
8 MR. STATTEL: I know. I --
9MR. ODESS-GILLETT: And the starting at is 10 defining a format and content requirements --
11 PARTICIPANT: And level of detail.
12MR. ODESS-GILLETT: -- and level of detail 13 is the best we can do at this point.
14MR. FANELLI: So let me ask a question.
15When you go do your audits, like when you go to the 16 vendor and you place your reports, I am assuming you 17 are going to do sampling, sort of like we do, smart 18 sample, some kind of sampling technique, are you 19 expecting that when we come and review your oversight 20 that our understanding of those samples would be the 21 same, like those are the samples we'd pick or what if 22 we look at something you may not have looked at? I 23 mean we ask --
24MR. ODESS-GILLETT: What were the samples 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 132 that you picked?
1 MR. FANELLI: Hey, well, suppose we want 2 to look at something different than you looked at.
3MR. ODESS-GILLETT: This is the kind of 4 information we would need to document this having to 5 have to be documented in a vendor oversight report, 6 you know.7 MR. STATTEL: We've been --
8 (Simultaneous speaking.)
9MR. ODESS-GILLETT: What were the samples 10 taken, what were the requirements, you know, maybe 11 specify what those requirements are, how far did you 12 do the tracing to --
13 PARTICIPANT: Yes.
14MR. ODESS-GILLETT: -- did you go all the 15way down to test. I mean this is the kind of thing if 16 we want consistency in the industry we are going to 17 need to put this in the industry guidance.
18MR. STATTEL: And actually, you know, when 19 you talk about that, and we've been criticized for not 20 performing a statistically significant number of 21 samples for a requirement.
22 And, you know, when you got a project and 23 there is 10,000 requirements it's really hard to do, 24 right.25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 133 PARTICIPANT: It is.
1 MR. STATTEL: And we have really limited 2resources. The industry has a lot more resources than 3 we do. We have two people, right, and the industry, 4 your oversight process is taking place over the course 5 of years, right.
6 PARTICIPANT: Yes.
7MR. STATTEL: And you have more people 8 than we do so you are more -- I would expect actually 9your vendor oversight to be more comprehensive and 10 more effective than what we have been able to do.
11 You know, we feel somewhat limited because 12 we have so few people and we have to do --
13MR. ODESS-GILLETT: I mean you might have 14 10,000 --15 (Simultaneous speaking.)
16MR. STATTEL: We're doing more than one 17 job.18 (Simultaneous speaking.)
19MR. ODESS-GILLETT: You might have 10,000 20 requirements but what are the safety significant 21 requirements.
22 MR. STATTEL: Yes.
23 MR. ODESS-GILLETT: Painting the cabinet 24 blue is not necessarily something --
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 134 (Simultaneous speaking.)
1MR. STATTEL: And we do, we focus on what 2 is safety significant.
3 PARTICIPANT: Okay, all right.
4MR. WATERS: All right. So the first 5 bullet I think you generally understand and get that.
6 I think that's where we see how to do this.
7 It's the licensee, right, the licensee's 8oversight responsibility. I think we are not in line 9yet with the vendor site. The paradigm is suggesting 10 we understand that we consider that maybe that's where 11 we will be after the first few or six have been done, 12 I don't know, but we're not going to -- We don't see 13 that yet, not yet for what we are talking about here.
14MR. DRAGOVICH: But even if that's the 15 case can we do it as like an observer role so that the 16 licensee and the NRC can do it under one inspection 17 versus having to do the same inspection twice?
18MR. WATERS: That's a possibility I think.
19 I mean that's --
20 (Simultaneous speaking.)
21 MR. BOLLOCK: So, yes, I'm sorry --
22 MR. WATERS: Go ahead.
23MR. BOLLOCK: -- to interrupt. So do you 24 mean to essentially --
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 135MR. HERB: Shadow us on our own rather 1 than going in there separate.
2MR. BOLLOCK: Right. Well I mean because 3 that's, you know, Greg is saying they do their 4inspection of the vendor. So you are saying when you 5 do it instead of them or have them observe, so like a 6-- working through with the NRC feedback from NEI is 7 license self-assessments --
8 (Simultaneous speaking.)
9MR. BOLLOCK: She was talking about 10 something similar to that.
11 MR. WATERS: Yes, right.
12MR. BOLLOCK: Okay. I mean that is a 13 paradigm shift.
14 MR. WATERS: Yes.
15 MR. BOLLOCK: So you are talking that in 16 lieu of a vendor inspection, a vendor, you know, 17 observation in lieu of.
18MR. VAUGHN: We did talk about that the 19changing the design basis share of inspections are 20 some of the ones that the team, you know, inspections 21 they do leveraging licensee self-assessments.
22 We'll try and take that analogy and bring 23 it over to here from, to the vendor side, but see if 24 it's applicable.
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 136MR. BOLLOCK: Yes, because right now --
1 And Greg and Aaron can expand on this right now, I 2 mean the vendor inspectors go on, what 30 a year --
3 PARTICIPANT: Yes, 25 or so.
4MR. BOLLOCK: Twenty-five inspections a 5 year and they are picking the vendors to go based on 6the safety-significant work. So, you know, say there 7 is a vendor that is going to create some new digital 8 system they would probably be pretty high on the list 9 if it's something that's going to go to a number, 10 especially if it's going to go to a number of sites 11 and it's, you know, digital I&C for a safety-related 12 system, probably be high on the list.
13MR. HERB: But that's outside the process 14we're talking about. That's not a project specific 15 thing, that's just --
16 (Simultaneous speaking.)
17PARTICIPANT: That's just a routine 18 inspection.
19MR. BOLLOCK: And that's what they do 20right now. Now its tie, and Greg hit on this, you 21 know, they focus on where there is going to be testing 22 or there is going to be something like, you know, a 23 product being built that is going to be used because 24 that makes it more significant for them.
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 137 (Simultaneous speaking.)
1MR. HERB: Right. It's easier to do an 2 inspection on something they are actually building 3 instead of a theoretical process where they are not 4 actually doing something, right, so --
5MR. GALLETTI: Right. And, again, part of 6 the notion of not duplicating effort would be the 7possibility. Of course, you are going in and you are 8 engaging the vendor at the vendor location and you are 9 observing the work activities or reviewing the work 10 products and it would afford the NRC an opportunity to 11 see that in action rather than us doing that, 12 specifically you, we would be engaging in watching you 13 perform that oversight.
14 (Simultaneous speaking.)
15MR. ODESS-GILLETT: And that would be a 16 learning for the licensee, you know, because that 17 feedback from the NRC inspection staff would be 18 helpful to the licensee.
19 (Simultaneous speaking.)
20MR. BOLLOCK: Yes, but if you have a, I 21 mean if you have a license amendment and then are we 22 then watching your oversight --
23 MR. ODESS-GILLETT: Yes.
24MR. BOLLOCK: -- like we're inspecting 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 138 your Criterion 7 --
1 MS. GOLUB: Right, that's right.
2 (Simultaneous speaking.)
3MR. BOLLOCK: Yes, I mean so that is a 4 different paradigm.
5 MS. GOLUB: That's right.
6MR. BOLLOCK: Now the other side is we'll 7 try not to, you know, is this something that's, you 8 know, it's infrequent, right, not every plants hear 9 the modifications.
10 But we are -- Right, there is also, of 11 course, not to expand baseline inspections for actual 12inspections. So is this something you would want to, 13 you know, I guess on top of the Inspection Procedure 14 52-003 that infrequent procedure for a digital I&C mod 15 under 25.15 and you are saying along with that do 16 oversight, you are recommending that instead of having 17--18MR. ODESS-GILLETT: I am only bringing it 19 up if the license, if the inspection staff needs to 20 see something real time.
21PARTICIPANT: I think another thing to 22 come up, briefings --
23 (Simultaneous speaking.)
24MR. BOLLOCK: Some of them are because 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 139 that's going to be an inspection to the licensee and 1 that's going to be a regional report under the, you 2 know, they're going to be under the ROP, it's going to 3 go into either its own order --
4 (Simultaneous speaking.)
5MS. GOLUB: Steve, could you help us here 6 a little bit because I know --
7MR. BOLLOCK: Yes, and it's going to be --
8MS. GOLUB: -- you speak this language 9 better?10MR. BOLLOCK: -- feed-billed into the 11 licensee, right, completely different from what the 12 vendor group does right now, which is the overhead and 13 they look at the vendor.
14 PARTICIPANT: And the licensee performs.
15MR. VAUGHN: So this is one thing we 16 really didn't have a good understanding with was 17solely getting there. So when you go into the vendor 18 inspection and you have that procedure, it's the one 19 from January, what was it 35-710, I hope I got that 20 right, but it was new as needed.
21 So it was as needed and that kind of 22 caused well, it's as needed, so it's kind of a 23routine, it's not periodic. We didn't know if that 24"as needed" was every time a utility does a safety 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 140 significant, or an LAR, we're in this process, ISG-06, 1 and you do that inspection each time.
2 That's not the case because you said you 3 had 25 per year you pick. A vendor may get one --
4 PARTICIPANT: Right.
5MR. VAUGHN: -- and, okay, now we 6 understand that was Part 1. Part 2 is if the vendor 7 wanted to see a utility go through the whole VOP 8 verification and develop a report as sort of like an 9 observer to see how it happens because it used to be 10 done under licensing when Rich and others did it, they 11 could see it.
12 But it wouldn't be an inspection where you 13 bill, you know, you wouldn't bill Southern, it would 14just observe for confidence. And typically that is 15 sort of a paradigm shift because usually when the 16 staff does that it's, you know, owner's groups, PRA 17 peer review process where they observe stuff.
18 They may ask a couple questions but it's 19 really just observing and then they write a report on 20observations. It's just -- So this is a little bit 21 different because it's sort of tied to a licensing 22 action that is the paradigm shift, but that's the 23 second part.
24 The third part was when the regions go to 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 141 do their inspection that's I think sort of almost 1 status quo. That's going to happen.
2 (Simultaneous speaking.)
3 MS. GOLUB: Yes.
4MR. VAUGHN: That's going to happen. The 5 region is going to do those inspections, they will do 6 your sample selection and have their hours and, you 7 know, that's staying with --
8MR. FANELLI: How that's going to actually 9 look is still up for debate.
10 MR. VAUGHN: Yes.
11MS. GOLUB: Did that help? Does that 12 help? Will that stay the same?
13MR. BOLLOCK: Yes. No, I think I just 14 want to be clear what, like which of those, you know, 15 do you understand what we do now, especially the 16 vendor group --
17 (Multiple yeses.)
18 MR. BOLLOCK: -- because I know it's not 19--20 (Simultaneous speaking.)
21MR. BOLLOCK: -- not everybody understands 22 that.23MS. GOLUB: We do have a better 24 understanding.
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 142MR. HERB: It sounds a lot like our own 1 vendor inspection groups and they share and include 2 the NUPIC organizations but they do about 30 or 40 3 inspections a year, you know, of vendors and then 4 general quality inspections.
5 So it sounds like you all do a similar 6 thing that may be targeted and you pick based on maybe 7 projects that are going on because you're not looking 8 at --9 PARTICIPANT: Right.
10MR. HERB: We have to look at all of them 11 at a certain period where you are just doing a 12 sampling and you are going to the interesting ones, 13 but that's part of your normal every day.
14MR. GALLETTI: And as you pointed out, 15 yes, your typical NRC vendor inspection is we address 16the ancient criteria of Appendix B but we are much 17 more focused. It's almost like a limited scope --
18 (Simultaneous speaking.)
19MR. GALLETTI: -- limited scope 20inspection. So we are focusing in on certain system, 21 certain activities, and doing a deep dive of those.
22 It's not let's say a broad programmatic --
23 (Simultaneous speaking.)
24MR. HERB: We'd love to learn from you all 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 143 and watch you all do that.
1 MR. BOLLOCK: Yes, and it's exactly, you 2 know, we understand you all have to every three years 3 inspect every vendor. We don't --
4 (Simultaneous speaking.)
5MR. BOLLOCK: -- have the resources to do 6 that.7 (Simultaneous speaking.)
8MR. BOLLOCK: -- rely on you all to do it 9 and like NUPIC does it and, you know, this was all 10 worked out 40 some odd years ago, but we still spot 11 check essentially and --
12 (Simultaneous speaking.)
13 MR. BOLLOCK: Right. So that's --
14MR. STATTEL: Well I am just thinking, you 15 know, when we get an application, an alternate review 16 process application, I envision that I would want to, 17 and this is me talking, right, I would want to perform 18 an audit at the site with the licensee and I would 19 want to interview and talk to the people who are going 20 to be performing those vendor oversight activities and 21 just have a discussion with them and make sure I 22 understand how they interpret what is written in the 23 plan, right.
24 So I am thinking I would want to do that 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 144 because, you know, we have the advantage if we do --
1 This is what we do, right, whereas the licensee might 2 do one digital upgrade in ten years.
3 PARTICIPANT: Yes.
4MR. STATTEL: So that person has never 5done it before and probably never will do it again, 6 right, but --
7 (Simultaneous speaking.)
8MS. GOLUB: Well we do more digital 9upgrades than that. They may not all be safety 10 systems.11MR. HERB: We don't -- They're not all 12 safety systems and they are not all underneath the 13 LAR, but we do a lot of them.
14 MS. GOLUB: Yes, right.
15 MR. STATTEL: Yes, okay.
16MS. GOLUB: So this is not completely 17 unknown to licensees.
18 MR. STATTEL: Okay.
19 MR. BENNER: But I think -- Sorry I came 20 in late and after all the discussion, but I think some 21 of the reaction is the same reaction I am having is 22 what is the new NRC inspection paradigm.
23 And as an old inspector, both region-based 24 and resident, I would say the NRC's inspection 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 145 paradigm has always been there is an element of 1 document review and there is an element of direct 2 observation and you seem to be injecting the paradigm 3 that an inspection should be mainly document review.
4 So I think that's the disconnect is we are 5 not going to buy off on, oh, inspection can just be 6 document review and then do observation only if 7 necessary.
8 If you want to be document review it 9 should go back into a licensing process because that's 10 what, that's a process that ultimately does rely on, 11 you know, documents reviewed and REIs and back and 12 forth and all that, that's not inspection, right.
13 Document review is a component of 14 inspection but it will never, I don't think we'll ever 15 create an inspection paradigm that only relies on 16 document review.
17MR. HERRELL: And I think from my 18 impression a new paradigm is who you are actually 19 doing the discussions with, you're still doing 20 discussions with the vendor, you're doing discussions 21with the licensee to make sure that they are happy, 22 discussions you would have had or something similar to 23 them --24 (Simultaneous speaking.)
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 146MR. BENNER: And I go back to just like if 1 I am doing a maintenance inspection I am going to 2observe some maintenance activities. If I am going to 3 do an inspection of your vendor oversight I want to 4 directly observe some of those activities.
5 PARTICIPANT: Got it.
6MR. BENNER: Maybe we could get to a point 7 where like Mike said after we have confidence it could 8 be an after-the-fact document review. I got to tell 9you I am having the reaction that certainly on the 10 front end of doing these we're --
11 (Simultaneous speaking.)
12MS. GOLUB: Did you -- I don't know if you 13got to hear what Ray has been talking about having 14 inspection folks accompany them on some --
15 (Simultaneous speaking.)
16 MR. BENNER: No, I heard that part.
17 MS. GOLUB: Okay.
18MR. BENNER: But I also heard some 19 reactions of that would be strictly an observation --
20 (Simultaneous speaking.)
21MR. BENNER: -- and not an inspection and 22 I'm like, well, again, that's where we're going to 23 have --24 (Simultaneous speaking.)
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 147MR. VAUGHN: Well this might help because 1 I had a misunderstanding earlier, too, when we had 2 that overall plan up there, you know, the AR process, 3 the old process at the top.
4 It was ISG-06 and all the milestones and 5 then after it said, you know, vendor inspection. So 6 I interpret that as every time someone, every time a 7 utility goes to this AR you are going to get the 8 vendor inspection inspection.
9Yes, what if 50 utilities came in, the 10 same upgrade, there aren't even 50 utilities, but you 11 guys only have 25 per year so you couldn't, the vendor 12 inspection branch couldn't do all of them and you 13 probably wouldn't pick all of the same digital I&C you 14 would spread it out over --
15MR. BENNER: But we could. We would do 16 one because as we have talked about our protocol for 17 vendor inspection is spot checking vendor --
18 (Simultaneous speaking.)
19MR. VAUGHN: Yes, and so that completely, 20 that makes sense, and we're okay with that. The way 21 we interpret it as an inspection each time I think --
22 (Simultaneous speaking.)
23 MS. GOLUB: That's right.
24 (Simultaneous speaking.)
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 148 MR. VAUGHN: Yes, so that's -- Yes.
1 PARTICIPANT: Yes, we're okay. Right.
2MS. GOLUB: This has been a really good 3 learning I think for us as well to understand better, 4 you know, what each of the --
5 MR. BENNER: What the piece part is.
6 MS. GOLUB: Yes.
7MR. BENNER: What the purpose of the piece 8 parts are.
9MS. GOLUB: I am glad Greg was here 10 because he really provided us some good insights on --
11MR. HERB: Right, because that was our 12 concern --
13 (Simultaneous speaking.)
14MR. ODESS-GILLETT: So the bottom line is, 15 that last bullet is you just don't want to be having 16 the vendor inspection branch doing the same thing 17 again that the licensee activities did.
18 MS. GOLUB: On a project basis.
19MR. ODESS-GILLETT: We want some 20 efficiencies.
21 PARTICIPANT: Right.
22 MR. DARBALI: Right. So, Warren, we are 23 way past due for a break.
24 (Simultaneous speaking.)
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 149 MR. DARBALI: So I suggest we take --
1 MR. ODESS-GILLETT: I don't even need to 2do the conclusion because everybody knows what the 3 conclusion is.
4 (Simultaneous speaking.)
5MR. ODESS-GILLETT: So we can just stop 6 there and --
7 (Simultaneous speaking.)
8MR. DARBALI: Let's take a 5-minute break 9 and we'll come back at 3:35 p.m.
10 (Whereupon, the above-entitled matter went 11 off the record at 3:29 p.m. and resumed at 3:41 p.m.)
12MR. DARBALI: Okay, folks, we're going to 13 continue.14 I think we've stopped following the 15 agenda.16 (Laughter.)
17 But this is good because, I mean, our last 18 topic is the Vendor Oversight Plan, and we've talked 19about that throughout the whole afternoon. So, I 20think we're good timewise. So, this is going to be 21 the last presentation of the day, and I think we've 22covered a lot of those topics. So, we'll just go 23 through it.
24 The purpose of this example is to identify 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 150 types of LAR review and audit and post-license 1 amendment inspection activities that would be 2 performed in relation to an alternate review process 3review. And we kind of based it out of the Oconee 4 RPS/ESFAS modification.
5 Next slide.
6 So, here's some reference material, 7 although we're not specifically going through that 8today. I think it can help inform future 9interactions. So, that's just the license amendment, 10some audit reports. Some turned out to be 11proprietary. But that just gives you information, an 12 idea of the type of number of audits that were 13 performed.
14 Section 3.14 of the Safety Evaluation 15 includes the recommended inspection items, and we do 16 have March 0. Huh?
17 (Laughter.)
18 So, we do have the inspection reports, and 19this is just for the first unit that was installed.
20 I think that's the one that had more inspection 21 activity.22MR. STATTEL: I think it was a leap year.
23 (Laughter.)
24 MR. DARBALI: A leap year? Okay.
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 151 (Laughter.)
1 Nobody provided feedback to me.
2 (Laughter.)
3 And then, the Vendor Oversight Plan.
4 Okay. It's a new slide. It's an update 5 from the previous one we looked at. It just focuses 6on the alternate review process. So, for the license 7 amendment review submitted and license amendment 8 issuance date, timeframes, we kind of added like a 9 wider bar because it is going to depend on the 10 application when those activities are going to take 11 place.12 So, for the green box, the NRC review and 13 regulatory audits, because we're focusing on this 14 part, on the audits, we're envisioning we will be 15 doing electronic audits. And there's the option for 16 a vendor facility audit and/or a site audit. Again, 17 it's going to depend on the application.
18 And then, post-lice nse amendment issuance, 19 that orange box, is a combination of a Regional and 20 vendor inspection. We're envisioning one inspection 21 at the plant site and at least one inspection at the 22vendor facility. This is what we're working with 23 right now.
24 MR. HERB: Not every time?
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 152 MR. DARBALI: Not every time.
1 MR. HERB: Right, right.
2MS. GOLUB: But that was a good 3 clarification --
4 MR. DARBALI: Yes.
5MS. GOLUB: -- a really good clarification 6 for us.7MR. DARBALI: And it goes back to all the 8 conversations we've had.
9 And then, after the equipment has been 10 delivered to the site, then we're looking at a 11 Regional inspection at the site.
12 Then, that box at the bottom just notes 13 that, whether it's an audit or our inspection 14 activity, pretty much everybody you see here, all 15 these organizations are going to be involved one way 16 or another.
17 Next slide.
18 So, for the audits of the LAR, we mention 19 electronic audits, and those are typically audits of 20 non-docketed material and typically in response to an 21open item. And then, that's going to determine, 22 closure of the open item is going to determine whether 23 we issue a request for additional information or have 24 that addressed.
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 153 And then, an audit at the vendor facility 1or plant site. Again, it could be non-docketed 2material. Some vendors or licensees don't want to put 3that in an electronic portal. So, we can go there and 4see it. And we can do some form of configuration 5 management audit to support what we're reviewing.
6Secure development/operational environment. We could 7 do vendor quality and licensee vendor oversight.
8 So, like we've said before, the LAR is 9 going to contain a summary of the VOP, but it most 10likely will get audited. The full VOP will get 11 audited.12 Next slide.
13 And so, for post-license amendment 14 inspections, whether they be Regional or vendor or a 15 combination of both, the activities to be inspected 16 include the Vendor Oversight Plan or licensee vendor 17oversight. It could be requirements for stability, 18 test results, configuration management.
19 And then, site acceptance/installation, 20 that's mostly at the site. And again, this is going 21 to depend on the application.
22MR. ODESS-GILLETT: So, we had made a 23 comment to ourselves that actually all the dashes, 24 starting with "requirements, traceability," and ending 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 154 with "vendor quality," if you move those dashes to the 1 right under "Licensee Vendor Oversight," the licensee 2 will be doing those things as well.
3MR. DARBALI: And we had that discussion, 4 but, then, I think that acceptance testing, 5 installation and procedures would not fall under 6 vendor oversight.
7MR. ODESS-GILLETT: That's correct.
8 That's correct.
9MR. DARBALI: But that's a point well 10taken. Some of those would be covered by a licensee.
11 MR. STATTEL: So, the line can move from 12 one project to another?
13 MR. DARBALI: Right, right.
14 The next slide.
15 And so, what type of activities would be 16-- or what would be the idea of doing the inspections?
17 It would be to verify vendor oversight activities that 18 the licensee is performing, per the Vendor Oversight 19Plan. And that includes inspections of Vendor 20 Oversight Plan reports at the site.
21 Verify how the licensee and the vendor are 22 performing those same activities. And that includes 23 inspections at the vendor facility.
24 And then, verify site installation and 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 155 testing.1 Next slide.
2 And then, I'll turn it over to Rich.
3MR. STATTEL: Yes. So, I was asked to 4 give examples of what we would expect to see in the 5 Vendor Oversight Plan for guidance on how to perform 6 those activities.
7 So, go on to the next slide.
8 I mean, all I did was just look to BTP-14, 9 and I found -- I mean, you should recognize this.
10It's actually ISG-6, if you look at Section 9 of 11 ISG-6.12 These are really -- it just identifies the 13 evidence that implementation of the plans has been 14completed, and these are examples. They may vary from 15 project to project.
16 I picked the software safety analysis 17implementation. In that example, you can see what 18we're looking for, if you think about it. So, the 19 vendor performs a software safety analysis in 20accordance with their plans. Typically, that would 21 involve a list of tasks that they perform and a 22 description of how those tasks are going to be done --
23"going to be done," future tense.
24 When we look at the Safety Analysis 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 156Report, we see did they do those tasks. The 1 documentation shows that the analysis activities have 2been successfully accomplished. It basically 3summarizes the results of those activities. And 4that's it, you know. I mean, that's the guidance 5 that's in BTP-14.
6 Now I started an activity of basically 7 taking those sections, implementation and design 8 output, out of BTP-14 and just stripping it of all of 9the characteristics, right? Because there's a lot of 10 words in there that just talk about characteristics.
11 If you strip that out, you end up with sentences like 12this, which are common sense, right? I see a report.
13 It tells me they did everything they planned to do, 14and here are the results. I mean, fundamentally, 15 that's kind of what it comes down to.
16 So, when we're talking about probably 30 17 pages of guidance that's in BTP-14, if you really boil 18 it down to where the real guidance is, you know, what 19 are you going to look for, it comes down to these 20 types of statements.
21 And the next slide shows an example of --
22 MR. ODESS-GILLETT: Before you go to the 23 next slide --
24 MR. STATTEL: Sure.
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 157MR. ODESS-GILLETT: -- can I make a 1 couple of comments?
2 MR. STATTEL: Yes.
3MR. ODESS-GILLETT: One is I'm assuming 4 Safety Analysis Reports is not the USAR, but the 5 software safety analysis documentation in BTP-714, 6 which includes a software analysis --
7MR. STATTEL: So, I removed the word 8"software" from here.
9MR. ODESS-GILLETT: -- and software safety 10 plan.11MR. STATTEL: I removed the word 12"software" from here, but you're correct.
13 MS. GOLUB: You kept it in the example.
14MR. ODESS-GILLETT: Okay. And the second 15 thing is, if this is all described in D4 --
16MR. STATTEL: Yes, the plan is. It's not 17 in D4. This is --
18MR. ODESS-GILLETT: Okay, but the 19 acceptance criteria should be against the plan and not 20 necessarily what's in BTP-714, because the staff 21 approved --
22 MR. STATTEL: The plan.
23MR. ODESS-GILLETT: -- and the plan and 24 the criteria that would be for the output documents of 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 158 that --1MR. STATTEL: Okay. So, BTP-14 is divided 2 into three sections.
3 MR. ODESS-GILLETT: Yes.
4MR. STATTEL: Right? It has here is 5criteria for reviewing plans. Here's how to determine 6 whether you have an adequate plan.
7 MR. ODESS-GILLETT: Right.
8 MR. STATTEL: And it goes through all 12 9 plans.10 MR. ODESS-GILLETT: Right.
11MR. STATTEL: The second part is 12 implementation.
13 MR. ODESS-GILLETT: Right.
14MR. STATTEL: Here's how you make sure 15 those plans were correctly implemented, and it has a 16 doppelganger section for each plan.
17 MR. ODESS-GILLETT: Right.
18MR. STATTEL: And each one points back to 19 the plan, just like these words do in this example.
20It points back to the plan. The tasks that were 21 identified in the plan have been successfully 22 accomplished, right?
23 MR. ODESS-GILLETT: Okay.
24MR. STATTEL: So, it's a little subtle.
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 159 The guidance for the plan itself or the adequacy of 1 the plan is different than the guidance for verifying 2 that it's correctly implemented. Okay?
3 MR. ODESS-GILLETT: Okay.
4 MR. STATTEL: And it's really a separate 5 section, and there's 12 separate subsections in there, 6 in the implementation section, that covered -- well, 7 maybe not 12 -- but if they could cover this. Now 8 some of these things get multiplied.
9 For example, V&V report -- and a lot of 10this is project-specific. There are projects that 11 don't do a separate software safety analysis. Okay?
12 They call it something else or they accomplish it in 13some other way. And that's fine. But the idea is the 14 Vendor Oversight Plan would have some direction to the 15 person implementing that or verifying that the plan 16was implemented, and it would basically be the 17language would be specific to what's in the plan. So, 18 if there's four activities identified in the plan, we 19 would identify them by name, and there might be 20 special guidance for that.
21 But we're not looking, my real point is 22 we're not looking for a hundred-page document on how 23to verify design implementation. We're really just 24 looking for common-sense sentences, some kind of along 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 160this order. And this is common sense, you know.
1 You're telling me you've implemented the plan and you 2have a report. I would expect to see the report list 3 the tasks and tell me what the results of those tasks 4 are, and tell me how, if you've identified anomalies 5 or issues, tell me what they are, how have you 6resolved them. And those ones that are not resolved, 7how are you carrying those forward, right? How are 8 you addressing those?
9 MR. ODESS-GILLETT: Okay.
10MR. STATTEL: And really, that's 11 fundamentally what we want to see, and that's what we 12 do, right, and that's what we would expect the 13licensee. And these are things that a licensee would 14 want to do.
15 MR. ODESS-GILLETT: Right.
16 MR. STATTEL: Now, if you go to the next 17 section, it really refers to the third section -- the 18next slide -- refers to the third section, which is 19the design output. Now, again, it's document-20 oriented. That's just the way the BPT is written.
21 But, again, if we're going to verify 22 design outputs, what are the artifacts of having an 23implemented and completed design? And this is an 24 example list.
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 161 You know, I talked with Gary Johnson, who 1wrote a lot of this. Again, it's going to vary. Code 2 listings may not be at all applicable to a FPGA system 3 or something like that. But in a lot of cases there 4 are parallel or similar type documents that get 5 produced.6 And I pulled the example here for the SRS 7 because most systems have a software requirement 8 specification or something equivalent. This kind of 9 comes down to performing threat analysis. You know, 10 there may be guidance for performing that. And what 11 we're looking for is that the requirements are 12traceable forward and backwards. The requirements 13 traceability matrix and the requirements traceability 14 analysis reports, which are actually more important, 15 they show that you're not implementing code without 16 having it documented; you don't have any broken 17 traces, things like that.
18So, again, it's common-sense stuff. It's 19not a difficult activity. I'm not making this out to 20 be, you know, we're looking for a 300-page guidance 21document here, but just some words. And these are the 22 type of things I would expect to see in either a 23 Vendor Oversight Plan or in some other document.
24 This is the kind of thing I would be 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 162 looking for to make sure that these activities that 1 are post-license amendment issuance -- you know, you 2 have in your plans, it's the document review that's 3part of your engineering process. Fine, you have a 4software requirement specification. Somebody from the 5 licensee signs, "I've reviewed it." Well, what does 6that signature mean? What's behind that signature?
7What did he do? I want to understand that, right?
8 And so, that's kind of it.
9 Now, going to the Vendor Oversight Plan 10 that you provided us, I'm assuming this -- when I 11 first read this and, then, I gave you this feedback, 12it looks like an instruction for how to create a 13 Vendor Oversight Plan --
14 MR. HERB: It essentially is.
15MR. STATTEL: -- rather than a Vendor 16 Oversight Plan itself.
17MS. GOLUB: Right. It's the summary of 18 the Vendor Oversight Plan.
19MR. STATTEL: I just want to make sure we 20 have the right understanding because I don't want to 21 see, when I get a real application in for a plant, I 22 don't want to see an instruction for how to create a 23 plan.24 MS. GOLUB: Right, I agree. I agree.
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 163MR. STATTEL: I want to see an actual plan 1 that's the base for a summary of a plan --
2 MS. GOLUB: Right.
3MR. STATTEL: -- nothing real long, but a 4 summary of a plan that describes an existing plan, an 5 actual plan.
6 MS. GOLUB: Right, right, right.
7MR. STATTEL: And I want to have access to 8 the actual plan to see this type of guidance, right, 9 this level of guidance.
10 So, that's really all I had to say about 11 this. I just kind of wanted to give an idea of what 12 our expectations were.
13MR. HERB: So, the reason why it's in that 14 format is, when we put the summary -- the summary plan 15 is where you have a level of sweet talk. You know 16what? We put a little bit more detail about what 17 really goes into the plan --
18 MR. STATTEL: Yes.
19MR. HERB: -- and then, follow the 20summary. So, it gives you a little more idea about 21 the plan.22MR. STATTEL: I mean, without actually 23 reading it --
24MR. HERB: And we opted not to do an 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 164 actual plan.
1MS. GOLUB: But it isn't a plan yet. It's 2 just a summary.
3MR. STATTEL: Now we saw some markups that 4 you gave us, right --
5 MS. GOLUB: Right.
6MR. STATTEL: -- the changes? I have a 7couple of comments on that. Or you wanted me to hold 8 off on that?
9MR. DARBALI: No, I think -- okay, so this 10just feeds into it. I don't know if there's the need 11to put WOP on the screen. These are just some 12 questions that we have, and we've started those.
13 I think the last one was Mike's question 14 as to you're going to be performing these activities 15 and we need to coordinate with our inspectors when 16those inspections would take place. So, we do need to 17 establish a method of updating the staff on when those 18 would be taking place.
19 And then, we've talked about these 20throughout. So, we can leave these up here and, then, 21 look at the hard copies of the VOP. And we're on 22 time.23 So, Rich, you wanted to go through each 24 section?25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 165MR. STATTEL: I'll start. So, let's go to 1page 7. And this applies to more than one section.
2 I just picked this one to make a point.
3 So, you added a reference into ISG-6, D4, 4 24, I think it was. Okay? Everybody following me?
5 MR. ODESS-GILLETT: Yes, software V&V.
6MR. STATTEL: Right. So, to me, that 7seems like the wrong reference. So, D4 is essentially 8-- that's the alternate review process, and that 9 provides the guidance for the V&V plan, right? It's 10 not the guidance for evaluating implementation of that 11 plan, which is actually in ISG-6 in Section D9, right?
12 So, to me, a more appropriate reference, if you're 13 going to refer to ISG-6, it seems to me that D9 would 14 have been a more appropriate reference.
15MR. BURZYNSKI: Maybe you need both parts.
16 One tells you what you should do and the other one 17 tells you how to do it.
18MR. STATTEL: Well, right, and what you 19 actually need, you need to have a plan in hand to know 20 what's going to be done. And then, you need to have 21 the report, and, okay, did they do that plan, right?
22MR. ODESS-GILLETT: But isn't that what 23 this thing is saying? It's saying make sure, verify 24 that the application software, being the program, 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 166 complies with.
1MR. STATTEL: Well, it doesn't comply with 2the guidance for what's an acceptable plan. It 3 complies with the plan itself.
4MR. ODESS-GILLETT: See, this seems 5 duplicative because we've already evaluated the plan.
6 We've already approved the plan.
7MR. HERB: Right. I understand. So, I 8 think the disconnect on what Warren is saying in a way 9 is that, when we refer to D4, we're referring to the 10 D4 section of the LAR we submitted rather than what 11 the guidance -- what goes in the LAR.
12MS. GOLUB: No, I don't think that's what 13 Warren is saying.
14MR. ODESS-GILLETT: No, no, no. I'm 15 saying that it says a way to verify that the 16 application software/V&V program complies with the V&V 17program approved by the NRC. So, you're going to look 18at what was done. So, we're looking at what was done, 19 comparing it to the approved program in the LAR, as 20 described in ISG-6, D4.
21MR. STATTEL: But why do you even need to 22say "as described in ISG-6"? I think that's 23 unnecessary.
24MR. DARBALI: Well, right, this is your 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 167 plan. So, these should be referencing your internal 1 procedure documents for performing that oversight.
2MR. ODESS-GILLETT: I would reference the 3LAR section, but I couldn't. So, the best we could do 4 is reference the ISG-6 section that pertained to No.
5 B in the LAR.
6MR. STATTEL: I guess in a real Vendor 7 Oversight Plan --
8MR. ODESS-GILLETT: You would reference 9 the LAR.10 MR. STATTEL: -- you would reference the 11 LAR.12MR. HERB: Unless the plan itself is 13 actually created for our procedures, you would create 14 an actual plan, and it's per our procedures.
15MR. STATTEL: I guess we're trying to 16create links here that just don't make any -- it's 17 just they're not going to make any sense.
18MR. ODESS-GILLETT: I agree that it's 19 convoluted, only because we didn't have a real LAR 20 with a section number for the V&V --
21MR. STATTEL: And when I saw that, I'm 22 like, well, if you really want to look at guidance for 23implementation of a V&V plan, it exists. You can 24 either point to BTP-14, right, which, clearly, you 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 168 don't want to do, or you can point to Section D9, but 1 D9 also provides references to BTP-14. It basically 2 duplicates the BTP-14 guidance.
3 So, I'm not saying you need the 4references. I prefer not having the references. When 5 I see a real Vendor Oversight Plan, I just want you to 6 know my expectation is there would be some words in 7 there that explain, well, what is that; what does that 8 mean.9 If you're going to have a signature from 10 the licensee on an SRS document or a V&V summary 11 report -- so, the vendor is creating a V&V summary 12 report, and there's a signature by the licensee. He 13 acknowledges and accepts that report. What's behind 14 that signature? And if there is like a paragraph in 15 here that explains, well, here's what the licensee is 16 doing?17And the last part of that paragraph I 18agree with. It says, right, "The V&V program is 19 implemented in a manner that reliably verifies and 20 validates the design outputs at each stage of the 21 design process." Right?
22 So, in reality, and in my experience, we 23 typically review six or seven or eight of these 24things. It's not just one report, right? There's 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 169 multiple. It multiplies, right? Because each phase 1 has its own V&V summary report.
2MR. WATERS: One thing, in a real VOP 3 summary, you would not refer to ISG-6?
4 MR. STATTEL: That's right.
5MR. WATERS: Okay. It's just how it could 6 be used --
7 MR. STATTEL: Right.
8MR. WATERS: You can't do that for 9 multiple reasons.
10MR. HERB: No, we wouldn't refer to ISG-6.
11MS. GOLUB: But, Warren, I just want to 12 make sure that I'm clear. So, what you're saying is 13 that we wouldn't refer to ISG-6; we would refer to the 14 specific section of the LAR that describes that?
15 MR. ODESS-GILLETT: Yes.
16MS. GOLUB: Okay. But I think I also hear 17 Rich saying that you're looking for, in the actual 18 Vendor Oversight Plan, criteria as well?
19 MR. STATTEL: Criteria, yes.
20 MS. GOLUB: So, not just to point to it, 21 but actual criteria?
22MR. STATTEL: And I think you answered 23that earlier. Because when I first read it, I had the 24 impression that was no plan and you were planning on 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 170 creating a plan. But, in reality, when we get this, 1 it's going to be a summary of an existing --
2MS. GOLUB: That's right. Absolutely 3 right, yes.
4MR. HERB: Yes, it's not a promise to 5create a plan. We will have a plan when we submit our 6 LAR.7 MR. ODESS-GILLETT: Actually, it will be 8 approved by the NRC staff.
9 MR. HERB: Yes, yes.
10 MR. ODESS-GILLETT: It's part of the LAR 11 review.12MR. HERB: Yes. And we're just putting a 13summary in the LAR, but the plan itself will be 14 potentially docketable, but we would prefer not to, 15because it's a live plan. We would rather -- but you 16 would see a full plan.
17MR. DARBALI: So, I want to go -- did you 18 have --19MR. ODESS-GILLETT: Well, it's just Pareez 20 is saying that, and I was sort of reiterating that, 21 the plan is the criteria upon which you review the 22adequacy of the V&V program. But I think I'm hearing 23 that he would like to see some activities that sort of 24 describe that.
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 171MS. GOLUB: And that's why you have those 1examples? So, you want more than just a "point to 2 it"?3 MR. STATTEL: I'm not really tasking you 4with coming up with a pretend example today. I'm just 5 kind of explaining what my expectation is when what's 6 on the plan actually comes in.
7 MS. GOLUB: But, Rich, I mean, the whole 8 reason we're going through this exercise -- and, you 9 know, we put a lot of time into trying to create this 10 exercise -- is to make sure that we understand your 11expectations and that we have something captured. So 12 that when the first licensee tries to use this, they 13know what target they're shooting at. Because, as we 14 said, this is new.
15 MR. STATTEL: Okay.
16MS. GOLUB: And so, we would like to make 17 sure that we understand.
18MR. DARBALI: So, what we have explains 19 the what. You're saying we need to see "the have"?
20MR. BURZYNSKI: Or an example of what you 21 do for "the how" is in the second part of BTP-714.
22 MR. STATTEL: That's correct. I know we 23 get accused of it a lot, but we don't come up with 24this on our own. It's actually written down in there.
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 172 (Laughter.)
1MR. BURZYNSKI: I'm just trying to 2summarize. And "the how" for the design outputs is in 3 the third part of BTP-714?
4MR. STATTEL: That's right. That's 5correct. That's correct. And I know it's hard to 6 find because it's mixed in with all those attribute 7discussions and everything like that. But, honestly, 8 when you boil it down, and like I said before, those 9 words like I provide in those examples, those are 10 right out of the BTP-14.
11 So, I don't know what's going to become of 12 this. I don't know what happens with this.
13MS. GOLUB: It's going to be incorporated.
14 The idea was that all of these tabletop examples --
15 and we went through a whole bigger tabletop, as you 16 know, in June -- all those examples are supposed to 17 become part of the industry guidance document, so that 18people who are doing this work, they have a target 19they're shooting at. And we're hoping it will provide 20 more industry consistency in the documentation that's 21 created that you're eventually going to look at.
22 So, I understand this is facilitating our 23 discussions, but I don't want to -- is it important 24 that we document --
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 173 MR. STATTEL: So, it would be it becomes 1 some NEI --
2 MS. GOLUB: Yes, that's right.
3MR. STATTEL: That the licensee would take 4 up and --5MS. GOLUB: It would be an example. This 6 wouldn't be an exact template, but it would be an 7 example that the staff has provided input on.
8 And we'll have to kind of take this back 9 ourselves and contemplate this a little bit, about if 10 we try to build out one of these sections more with 11 more information, but I want to be careful. Because 12 if we include this in an NEI document, licensees are 13going to create something similar. And so, we want to 14 make sure that the target that we're providing is a 15 target that makes some sense.
16MR. STATTEL: If we don't put it right, 17 it's just going to be copied and pasted into 18 somebody's amendment, and that's not going to cut it.
19So, what I would suggest is something like this. I 20don't like the references to ISG or BTP-14. But what 21 I would suggest is, you know, I started an activity of 22kind of boiling down the BTP-14 guidance. I'll 23complete that, and I can provide you some words. And 24 maybe we can put them in brackets or something.
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 174 Essentially, something like this is what we would 1expect, and to tell the licensee: don't copy this, 2 but write what fits for you -- or the vendor, you 3 know.4MS. GOLUB: Your input would be welcomed.
5 MR. STATTEL: Do you see what I mean?
6 MS. GOLUB: Yes.
7MR. STATTEL: So, I'll take that task on.
8MS. GOLUB: And it doesn't have to be for 9all of it. I mean, you gave us a couple of examples.
10 I think that would be helpful.
11 MR. STATTEL: It's not that much really.
12MR. HERB: When you say that, on our side 13 we think you're asking, it almost feels like you're 14 asking us to replicate your platform acceptance if it 15has already been done. And then, part of that may be 16a software application manual and it plans it. So, 17 when we purchase that, we come with an assurance that 18 that's probably a very rigorous development process 19 that they've already done, and we're not going to 20 accept it at blind faith, but I don't know that we 21 have to go to the degree that you already have done, 22 to say that's an acceptable process, as per the LAR.
23MR. STATTEL: Well, I'm really not 24 offering that.
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 175 MR. HERB: Are you asking us to do that?
1MR. STATTEL: No. I'm really not talking 2 about platform.
3MR. HERB: Okay. Well, I'm talking about 4 the platform and the application manual.
5MR. ODESS-GILLETT: I think he meant, when 6 he said "platform," I think it's like the software 7 program manual.
8 MR. HERB: The software program manual.
9MR. STATTEL: Well, right, and that's 10true. There are -- what? -- three software program 11 manuals approved.
12 MR. HERB: Right, but if we had one that 13 wasn't an approved one, do we have to go back and 14 reconfirm that --
15MR. STATTEL: It's hard. It's going to be 16hard. If you have an approved software program 17 manual --18 MR. BURZYNSKI: No, but you just need to 19 address if you've implemented it.
20MR. ODESS-GILLETT: What Rich is saying is 21 that, okay, just list a few activities as it's kind of 22described in BTP-714. How would you do that to the 23 SPM, for instance?
24MR. STATTEL: So, I refer to these as the 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 176 Lozier words, because in the revision to BTP-14 in 1 2007, Paul wrote these words, right? He added those 2 into BTP-14. And I can recognize them. I key-in on 3 those.4MR. HERRELL: It's not hard -- it's harder 5 to find them and the other words that are there aren't 6 as useful.
7MR. STATTEL: That's right. That's right.
8 But, no, going back to the SPM, I think a 9 lot of credit can be t aken for an approved SPM.
10 There's really no need to re-review the plans unless 11we know of significant changes, right? But the 12 implementation is going to be every time you have a --
13MR. HERB: No, I understand that. So, 14 we're confirming that they're following they're 15 following the plans that were probably approved and 16the processes. And we want to get deep into that 17 process through questioning, through interviews, 18 through inspection of our products when we get back.
19And so, I understand. That's what I thought you were 20 asking, but when you started those other things you 21 were talking about, I was thinking, wait a second; are 22 we questioning the validity of the plan that was part 23 of the --24MS. GOLUB: Yes. No, I don't think that's 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 177 what Rich is saying.
1MR. HERB: Are we going to look to see 2 what the criteria of that plan and compare it to the 3 original BTP-714 criteria?
4 MS. GOLUB: Just the implementation.
5 MR. HERB: Or just the implementation?
6 MS. GOLUB: Right.
7MR. HERB: So, I didn't think it was just 8 the implementation.
9MS. GOLUB: But I think Rich is looking 10 for the Vendor Oversight Plan to contain some criteria 11 for how you plan to verify that implementation, that 12 it's not just, you know, "I did it in accordance with 13 the plan." I mean, it's something with a few --
14 MR. STATTEL: I don't think we can write 15 this in a generic way --
16 MS. GOLUB: Right.
17 MR. STATTEL: -- that it can just be cut 18 and pasted into --
19MS. GOLUB: Right. You cannot, yes. You 20 can provide a couple of examples, but, yes, it's got 21 to be specific.
22 MR. STATTEL: So, I think we're going to 23 have to just bracket it and just something like this.
24 Otherwise, we're just going to be transferring BTP-14 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 178 into here.
1MS. GOLUB: Right. That's why I was 2 thinking, you know, if we could get an actual example 3 of something that would be a criteria, we could pick 4 it from one of the vendors. I don't know.
5MR. ODESS-GILLETT: Or we could just --
6 you know, we've gotten feedback, and Mark has already 7 highlighted this for us, those sections of BTP-714.
8 We could come up with those little pieces of criteria 9 for each one of these.
10 MR. STATTEL: Well, I've already started 11 down that path of boiling it down. So, why don't we 12 finish that activity?
13MS. GOLUB: Yes, if you could give us 14that, that would be great. And then, we could kind of 15bounce it off of what we have. But Mark actually went 16 through and did the exercise of highlighting certain 17 sections. You actually boiled it down a little bit.
18 But what Mark had done --
19MR. HERB: But we actually got into the 20 plan rather than the actual summary of a plan.
21MS. GOLUB: But Mark went through and kind 22 of did something similar.
23 MR. STATTEL: And I took some liberties.
24 Even in these examples, I did change some words 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 179 around. So, it's not a quote.
1MS. GOLUB: Right, right. That's, I 2think, the difference. But we did kind of start down 3 that path, just to see if in general it made some 4 sense.5 MR. HERB: It's good feedback.
6MS. GOLUB: Uh-hum, good feedback, really 7 good feedback.
8MR. DARBALI: So, I need some 9clarification. Earlier, before the break, right 10 before the break, we were talking about the licensee 11 as the term vendor oversight for the process. All 12 right, but the technical -- you have your design 13 acceptance of documents --
14MR. HERB: That's called owner acceptance.
15 MR. DARBALI: Right.
16 MR. HERB: Yes.
17MR. DARBALI: So, we agree they're all 18 license activities of vendor activities -- licensee 19 oversight activities of vendor development activities.
20 MS. GOLUB: Right.
21MR. DARBALI: But what I want to be clear 22 is that the Vendor Oversight Plan and the summary is 23 going to include both.
24 MR. HERB: Okay. I understand.
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 180MS. GOLUB: Right. I think we got that 1 message, yes.
2 MR. DARBALI: Okay.
3 MR. HERB: I think we got that, yes.
4 MR. DARBALI: Okay.
5MR. BURZYNSKI: And some discussion in the 6vendor oversight template that married your part 3 7 expectations with what we do in our standard design 8 process, that that would illustrate what we're doing 9 and how we're doing it.
10 MR. DARBALI: Right. Because when we're 11 talking about inspection and vendor oversight, there 12was a disconnect. "Oh, we don't call that vendor 13 oversight." 14 MS. GOLUB: Yes.
15 MR. DARBALI: And regardless of what you 16 call it, we're looking at the vendor oversight.
17MS. GOLUB: That's right. That's exactly 18right, yes. No, good feedback. That's good feedback.
19MR. HERB: And we're saying that's part of 20owner acceptance reviews and that would be here. And 21 when you all came to inspect, you would know where to 22 go to get it.
23 MR. DARBALI: Right.
24 MR. HERB: Exactly.
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 181MR. FANELLI: They would be careful. This 1 would become less -- put in there and accepted, right?
2MR. STATTEL: Well, in the actual plan, 3 yes.4MR. ODESS-GILLETT: Yes, the summary of 5 the plan.6 MR. FANELLI: So, be very careful of the 7 words.8 MR. STATTEL: And you'll be using that.
9 (Laughter.)
10MS. GOLUB: It carries a little more 11 weight then, doesn't it?
12But your point is well taken. It's almost 13 like vendor oversight, the capital "VOP" versus like 14 the activity of vendor oversight.
15 Are you okay with that, Steve?
16 MR. DRAGOVICH: Yes, I think it's a good 17 plan.18MS. GOLUB: Putting a couple of 19 paragraphs?
20 MR. STATTEL: So, if it's okay with you, 21 I think we have a Word version of this. So, I'll go 22 ahead and edit right into it?
23 MS. GOLUB: That would be great.
24MR. WATERS: We've got to be careful when 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 182 we share information.
1 MR. STATTEL: Okay.
2 MR. WATERS: I want to --
3 MR. STATTEL: Either way.
4 MR. DARBALI: We can try to have another 5 meeting or it could be a call, but I would suggest 6 after next year, after the holidays.
7MS. GOLUB: Yes, this activity, the 8 inspection process, this whole meeting has been super-9 helpful, a lot of clarity, and I think we feel a 10 little bit better about the direction this whole thing 11 is going.12 MR. DARBALI: Okay.
13MS. GOLUB: But it's a really important 14 activity for the industry.
15MR. WATERS: Let me clarify. I have no 16 problem with sharing his insights on this, but I don't 17 want it coming across as we are developing a plan for 18you. The plan should be -- just to share this is all.
19MS. GOLUB: Yes, right, and we would take 20it in that context. But the point I'm trying to make 21 is that we're happy to support another meeting, you 22 know, another phone call, whatever it takes to make 23 sure that we're in a good place.
24 We just don't want the first licensee, who 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 183 happens to be sitting here, to have to wade through 1 these waters on their own.
2MR. SCHRAYE: That won't hold up the 3 issuance of an ISG, will it?
4MR. DARBALI: No. No, no, this is 5 separate.6 MR. ODESS-GILLETT: This is the industry 7 guidance --
8 MS. GOLUB: Yes.
9MR. ODESS-GILLETT: -- that follows the 10 ISG.11 MS. GOLUB: That's right. Well said.
12MR. DARBALI: So, unless there's any more 13 questions on the topics today, could you expand a 14 little bit on that industry guidance document? When 15 do you expect it to be complete?
16MS. GOLUB: Yes. So, right now, we're 17 targeting end of first quarter 2019.
18 MR. DARBALI: Okay.
19 MS. GOLUB: So, it's not too far off.
20 MR. DARBALI: Okay.
21MS. GOLUB: I'm writing it and, then, Dave 22Houghton is writing it. Dave went through some 23 health-related issues.
24 MR. DARBALI: Right.
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 184MS. GOLUB: And so, he's doing well. When 1 he comes back -- you know, that's why we're kind of 2shifting when we're writing it a little bit. But 3 we're sort of targeting that timeframe.
4MR. DARBALI: Okay. And will you be 5 working with the new, the first use of ISG-06 licensee 6 on this?7 MS. GOLUB: Yes.
8 MR. HERB: To the extent we can. To the 9 extent we can.
10 MR. DARBALI: Okay. Because it would be 11 helpful --
12 MS. GOLUB: Yes.
13MR. DARBALI: -- both the learning 14 experience, but also help improvement --
15 MS. GOLUB: I agree. I agree.
16 MR. DARBALI: -- and for them to use the 17 document.18 MS. GOLUB: Yes, agreed. Absolutely.
19MR. DARBALI: Okay. And are there any 20 plans for the staff to get to see that?
21 MS. GOLUB: Yes. So, we've talked about 22that before. I mean, the industry would welcome your 23thoughts, your feedback. Obviously, we're not giving 24 it to you for review per se.
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 185 MR. DARBALI: Right, right.
1MS. GOLUB: But, yes, we would 2 absolutely -- because, again, this is going to be 3 what's guiding licensees as they go down this path.
4 And it's an expensive path. It's an important path, 5 a path to better safety, you know, plant longevity, 6asset management, all of that good stuff. So, we 7 would welcome input, so that the guidance is really 8 good guidance; it's guidance that makes sense.
9 And hopefully, it will be a living 10document. So, as the first licensee, the second 11 licensee goes through, we can amend that guidance with 12 those lessons learned.
13MR. DARBALI: Well, the next topic on the 14 agenda is the closing comments and any follow-up 15 actions. So, before the comments, we have an action 16 to take a look at that ISG-06 language that says use 17NQA-1 as a reference. Like we said, the ISG is 18 currently with OGC. So, we will internally see what 19 language we can agree with that cannot be 20 misunderstood, and see if we can incorporate that into 21 the ISG.22MR. ODESS-GILLETT: Or the alternative is 23 to document the interpretation in meetings --
24 MR. DARBALI: Right, right.
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 186MS. GOLUB: We prefer the former, though.
1MR. ODESS-GILLETT: Yes, we prefer the 2 former, but if that's --
3 MS. GOLUB: Exactly. If we don't have to --
4 MR. DARBALI: Right.
5 MS. GOLUB: So, everybody can see it.
6 MR. DARBALI: Mike, I'm going to put you 7on the spot. Do you think we can take five minutes 8 here to try to agree on the language or?
9 MR. WATERS: Sure.
10MR. ODESS-GILLETT: Well, actually, it was 11 either you or Mike who suggested changing "vendor" to 12"licensee".
13MR. DARBALI: Well, yes, I did write that 14 down. I think it was Rich or Mike that said that.
15MR. ODESS-GILLETT: Maybe we could put it 16 up and take a look.
17 MR. DARBALI: So, I'll read it.
18MS. GOLUB: Yes, the wording is different 19 in the actual document.
20MR. DARBALI: It is, yes. I mean, you can 21 type it.22MR. STATTEL: This isn't a prerequisite to 23 using the alternate review process?
24 MR. DARBALI: It's repeated in the 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 187prerequisite, yes. Yes, it's also on the page before 1 it.2 MR. STATTEL: Okay.
3MR. DARBALI: Joe, can we type? Can we 4 change? Can we go out and edit the slide?
5MR. GALLETTI: Hey, Warren, could you 6repeat? I mean, I'm trying to get my hands around why 7 originally the industry proposed having any reference 8 to NQA-115, Subpart 2.7.
9MR. ODESS-GILLETT: Gosh, the team, can 10 you help me remember?
11MS. GOLUB: Yes. At the time, we were 12 thinking about -- and I don't actually have it here.
13 We wanted a common framework. You know, we wanted 14 sort of a common base that we were both working from.
15 So, we weren't just kind of committing to something 16 arbitrary, but we were committing to, the licensees 17 were committing to examine their vendor to a certain 18 process, to a certain level, right.
19 So, the idea was to have a common 20 framework, and that's why we ended up settling on this 21 Section 2.7 of this particular version. And we kind 22of went back and forth on that, too. Some people 23said, "Oh, we should change it to Reg Guide 1128." 24 Right?25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 188MR. HERB: Right. Yes, and we thought 1 about just taking the year off of it completely, but, 2 then, they said --
3MS. GOLUB: But I think the idea was a 4 common framework, you know.
5MR. ODESS-GILLETT: Yes, as you had 6 mentioned, licensees all have different levels of --
7MR. GALLETTI: Right, but, I mean, if you 8 were to remove this completely from this document, the 9 way this would, in practicality, get implemented at 10 each vendor consistently was to have some sort of 11 understanding that in the PO that you're sending them 12you would specify this, whatever you want it to be.
13 But that's the formal mechanism to get that into the 14 requirements, if you will.
15 The problem, when we looked at this 16 originally, and it said something to the effect that 17 you'll ensure that it's consistent with this, it 18 implied that, hey, you, Licensee, you're an N45.2 19 plant. Well, you're not committed to this. So, all 20 of a sudden, now you're committing to something that 21 we don't want you to necessarily do, right?
22MS. GOLUB: Yes, your point is well taken.
23 And we went kind of round and round on our side with 24 those words because, I mean, some of the folks sitting 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 189at the table have N45.2 plants. And so, that 1"consistent with" was chosen carefully not to be 2"comply with" or --
3MR. BURZYNSKI: Yes, we thought it was not 4 saying "conforms with".
5 MS. GOLUB: Right.
6 MR. FANELLI: So, you're just saying you 7 really want some software-specific quality assurance?
8 MS. GOLUB: Yes, that's exactly right.
9 MR. FANELLI: So, why don't you say that 10 instead of --
11MS. GOLUB: Well, because we weren't sure 12 how that would be interpreted. So, having like kind 13 of a common framework we thought would make it easier 14 for licensees to understand what they were doing and 15 vendors to understand what that framework was, and NRC 16 to understand what was going on in licensee/vendor 17space. So, right, wrong, or indifferent, that was the 18 idea.19MR. GALLETTI: But the purpose of doing 20 that was it was giving you, the licensee, who has 21 responsibility for oversight of the vendor, right --
22 basically, you're putting them on record as saying, 23 hey, Vendor, I'm going to come out and I'm going to 24evaluate your program, and I'm going to use this as 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 190guidance. Right? Is that sort of what you're saying?
1 MR. HERB: Yes.
2 MR. GALLETTI: Okay.
3MR. HERB: Yes. It says the N45 plants 4 shouldn't have any --
5MR. HERRELL: The reason we chose the 2015 6version was (a) it was the most kind and (b) it was 7 the one place where we actually managed to put most of 8 the software requirements in one section.
9 MR. STATTEL: Right, and we endorsed it, 10 which is even better.
11 MS. GOLUB: Yes. Yes, that's right.
12MR. ODESS-GILLETT: Right. And we wanted 13 something endorsed, but we didn't want to like say 14 IEEE 730 because that's not an endorsable standard.
15MR. DRAGOVICH: I agree, though, couldn't 16 it be handled as part of the procurement specs? For 17 example, we send out a spec and we say you have to 18 comply with 2015, and they come back and say, well, we 19 have 2012. So, they write a conform spec. We agree 20 with that, and then, that's what we end up using.
21 Would that solve that problem, if we rely on that, or 22 would we have to?
23MR. HERRELL: We really want to establish 24 a base --25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 191 MS. GOLUB: Right.
1 MR. DRAGOVICH: And why can't we use the 2 procurement spec to do that?
3MR. HERRELL: You start with this, and 4they come back and give you an alternative. You 5 accept the alternative.
6 MR. VAUGHN: What about a vendor uses --
7 you heard that referenced -- the appropriate software 8standards and list as many examples as you want? I'm 9 not saying you should do that, but you can list more 10than this one. Isn't that what you want, some sort of 11 software quality assurance standard and you want some 12base level? Say that, and then, if you have a 13 different one -- and I don't even know what it was --
14 do that as an example. If this is the best example, 15 have this one first.
16MS. GOLUB: Yes, so this was the best 17 example right now.
18 MR. VAUGHN: But, then, you make it -- I 19 mean, when the lawyers look at this, they're going to 20think that it has to be this, I think. Don't make it 21like that. Make it appropriate software quality 22assurance standard. That's an "e.g.". "Such as" 23 this; "such as" this. Yes, "such as," even this.
24MS. GOLUB: You're right. So, we still 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 192 get the function of having some kind of a base level, 1 but it softens the language to make -- yes, so that 2somebody reading this five years from now doesn't 3 inadvertently --
4 MR. STATTEL: I would make the example a 5separate sentence, right? So, in the middle bullet 6 you would say, "The vendor uses" an appropriate 7 reference QA program, whatever, something like that.
8 And then, in the final bullet you would say, "As an 9 example," NQA-1 could be the reference or something 10like that. That way, you don't even put it in the 11 same sentence. There's no chance anyone is going to 12 interpret that as being the required criteria.
13MR. HERRELL: Rich, as a vendor, I am 14 required to use the most current version of NQA-a, 15 unless I've got an SPM that's based on an earlier 16 version.17MR. GALLETTI: Well, you're required to 18 use whatever the licensee puts in their PO, right?
19MR. HERRELL: It requires my program to be 20 based on the most current guidance.
21 MR. FANELLI: As a general rule.
22 MR. HERRELL: As a general rule.
23 MR. STATTEL: Well, that first statement 24you made is not true for all vendors. I can guarantee 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 193 that.1MR. GALLETTI: Yes, I mean, we've got 2 vendors that are approved in 1994.
3MR. STATTEL: Or foreign vendors that 4 don't even --
5 MR. GALLETTI: But, I mean, you may want 6 to put something in here, an appropriate QA software 7program. You may want to consider something that's 8 been endorsed by the NRC, and that covers several 9 versions of NQA-1.
10MS. GOLUB: I mean, I like this language.
11What do you guys think? Dave? Mark? Frank? I want 12 to make sure I follow the -- yes, Frank, I know you've 13 had some energy about this and had some good comments 14 on this in the past.
15MR. GALLETTI: What Rich said, I mean, end 16 the sentence there, and then, have another bullet "for 17example," like another bullet. "For example," NRC-18 approved versions of NQA-1. And you can put in 19 parentheses "(2008, 2009, 2014-15 addenda)," et 20 cetera.21MR. ODESS-GILLETT: We didn't really like 22"NRC-approved" earlier in the document.
23 MR. DARBALI: NRC-endorsed, right?
24MS. GOLUB: I think even if we keep the 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 194 year on there in the subpart, like that S ubpart 2, 1 just because --
2MR. GALLETTI: Well, that is the subpart.
3 That's the one that's the current --
4MS. GOLUB: No, but other versions have it 5 spread out in a different place. Right? So, that's 6 the reason that we picked 2 015, was because it was 7 condensed.
8MR. GALLETTI: I think we know we have 9 2.7, and '14-15 addenda are 2.7.
10 MR. HERB: '98 is like 2.7-plus.
11 MR. GALLETTI: Yes, but I don't think we 12 endorsed --
13MS. GOLUB: I think if we have one 14 example, one example is sufficient.
15MR. HERB: So, all the ones you've 16 endorsed, it's Subpart 2.7?
17MR. GALLETTI: Well, at least for 2008 --
18MS. GOLUB: For example, NRC endorsed the 192015 version of blah, blah, blah. I think that should 20 be sufficient.
21MR. GALLETTI: We endorsed '94 and, then, 22 there was a long hiatus because the industry had done 23 a bunch of things to it that we didn't approve of.
24Well, you basically have 2008-2009 addenda, and you 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 195 have 2014 to --
1MS. GOLUB: Yes, I think that makes sense 2in there. So, it's an example. We only need one 3 example.4 MR. GALLETTI: It is an example.
5MS. GOLUB: Right, yes. And it kind of 6 captures that NRC endorsed --
7 MR. HERB: So, you don't need the 2015?
8 MS. GOLUB: No, I think we do.
9MS. ZHANG: I think, for example, the NRC 10 endorsed 2015, I think it's good.
11 MS. GOLUB: I do, too, yes.
12MR. DRAGOVICH: So, how about if you 13 delete "can also be used to" and making sure plural --
14 because you have "used" twice in that first sentence.
15 Is it saying the same thing?
16MR. DARBALI: Well, what we don't want to 17 say is that the VOP ensures something.
18 MS. GOLUB: Yes, that's right. That's 19 right.20MR. DARBALI: Yes, we're not striving for 21 perfect.22 (Laughter.)
23 So, it seems that everybody is in 24 agreement with this?
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 196 MS. GOLUB: Yes.
1 MR. DARBALI: I mean, we're not going to 2 reconcur, but we need to see if that language changes 3the concurrence. Fortunately, the folks that are 4interested in that part are here. So, we'll take that 5 as the base and we'll do that after --
6MS. GOLUB: We appreciate that. Thank 7 you. Thank you for your consideration.
8MR. VAUGHN: If OGC has a comment on this, 9it will be easy. If they don't, then you have to say, 10 well -- and they may or may not want to --
11 MS. GOLUB: But this was the intent.
12 MR. DARBALI: Yes.
13MS. GOLUB: You know, this was the intent.
14 It may not have been worded quite as well, but it was 15 the intent.
16 MR. DARBALI: Okay.
17MR. WATERS: Well, at a minimum, in the 18 meeting summary you can say we discussed this and --
19MR. GALLETTI: You just say it was a 20 clarification.
21MS. GOLUB: That's right, it is a 22 clarification. It's not changing the intent.
23 MR. DARBALI: All right.
24 MS. GOLUB: Thank you for doing that.
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 197MR. DARBALI: So, I guess we were doing 1the follow-up actions. So, this is some of our 2 follow-up actions.
3 I believe, Rich, you took the action to 4 look at those specific activities and update the 5 Vendor Oversight Plan?
6 MR. STATTEL: Yes, yes.
7MR. DARBALI: And I guess I had written 8 industry was to consider how to add technical aspects 9 into the VOP.
10 MS. GOLUB: Yes, that's right. That's 11 right.12MR. DARBALI: And again, the ISG is almost 13done. We're not going to be doing tracking of action 14 items.15 MS. GOLUB: That's right.
16 And then, maybe, Samir, we can kind of 17 contemplate if we want a phone call or something to 18 follow up on this.
19MR. DARBALI: Right, right. So, follow-up 20 just to schedule follow-up on it.
21 MS. GOLUB: Right.
22MR. DARBALI: Okay. And with that, I 23 think the other item was just to do closing arguments.
24 So, I'll turn it to Mike.
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 198 (Laughter.)
1 Well, Eric's not here.
2MR. WATERS: No, I have nothing to say.
3I think, as you know, it was a really beneficial 4 meeting, and I think we learned a lot from both sides 5 and come to a clearer understanding.
6 My real question -- and I don't have an 7 answer -- is a little clarification, and we'll look at 8inspection programmatic issues to make sure. The ISG 9 is going to be issued in December. That's our plan.
10What's next? I don't want to leave on this. I've 11 heard rumor of a licensee coming in next year, which 12may push a test of this. But the question to think 13 about is, what do we do next? What is our next? We 14don't need to interact anymore on this or not? So, 15 something to think about.
16 MS. GOLUB: Yes, let's think about that.
17 And, Mike, I know you've asked this several times, and 18 we kind of keep dodging it a little bit, about what 19the schedule is. But I know licensees aren't 20 providing you a schedule, but I think I can speak to 21 just, on Tuesday-Wednesday of this week, we had a big 22meeting at EPRI, a lot of licensees present. There's 23 a huge amount of interest in moving forward with these 24modernization plans. And at this point, it's just 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 199 figuring out the "how" and the "when". But it's not 1 a possibility. I mean, it sounds --
2MR. HERB: No, these are big plans that 3take a lot of planning. We've changed the way the 4 industry, I think, is viewing this process, largely 5through this effort. And so, these things take a 6while to spin up. They're large capital projects.
7 Capital projects and budgets are out 5 to 10 years.
8 And so, to do these projects, you have to move other 9capital projects out of the way. So, that takes time 10 and effort. And a large part of that assurance that 11 we have a good process goes a long way to --
12 MS. GOLUB: Yes.
13MR. WATERS: And I hear you, and NRC's 14 business is to tell you when it's in the application 15 and not be critical --
16 MS. GOLUB: No, but --
17 MR. HERB: We want it as soon as we can.
18 MS. GOLUB: That's right. That's right, 19 but this effort is going a huge way toward providing 20 licensees that assurance, though.
21MR. HERB: I would like to submit more, if 22 I had a magic ball to be able to go back five years 23 and be prepared.
24 (Laughter.)
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433 200 But I don't.
1MR. WATERS: We have inspection folks 2here. Do you have any comments to make to his remarks 3 or any further observations?
4MR. GALLETTI: I thought it was a very 5 fruitful meeting. I appreciate the candor and their 6 discussion.
7MS. GOLUB: Yes, we thank you very much 8 for your time. We appreciate all of you guys coming 9 in and participating.
10And, Samir, thank you. I know this was a 11lot to put together. We really, really appreciate all 12 of your time on this. It was really helpful for us.
13 So, thank you.
14MR. GOLLA: Okay. Are there are any 15 comments from members of the public on the phone?
16 (No response.)
17 All right. Hearing none, anyone else on 18 the phone, if you would like to make a comment?
19 (No response.)
20 Okay. Thanks for a very good meeting.
21 MS. GOLUB: Thank you very much.
22 MR. GOLLA: We really enjoyed it.
23 (Whereupon, at 4:38 p.m., the meeting was 24 adjourned.)
25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.(202) 234-4433WASHINGTON, D.C. 20005-3701(202) 234-4433