ML19085A447
| ML19085A447 | |
| Person / Time | |
|---|---|
| Issue date: | 03/27/2019 |
| From: | Joe Golla NRC/NRR/DLP/PLPB |
| To: | |
| Golla J, NRR/DLP, 415-1002 | |
| References | |
| NRC-3986 | |
| Download: ML19085A447 (201) | |
Text
Official Transcript of Proceedings NUCLEAR REGULATORY COMMISSION
Title:
Inspection of Digital Instrumentation and Control Installations Approved under the Guidance of ISG-06 Docket Number: (n/a)
Location: Rockville, Maryland Date: Thursday, November 16, 2018 Work Order No.: NRC-3986 Pages 1-200 NEAL R. GROSS AND CO., INC.
Court Reporters and Transcribers 1323 Rhode Island Avenue, N.W.
Washington, D.C. 20005 (202) 234-4433
1 1 UNITED STATES OF AMERICA 2 NUCLEAR REGULATORY COMMISSION 3 + + + + +
4 INSPECTION WORKSHOP 5 + + + + +
6 INSPECTION OF DIGITAL INSTRUMENTATION & CONTROL 7 INSTALLATIONS APPROVED UNDER THE GUIDANCE OF ISG-06 8 + + + + +
9 THURSDAY 10 NOVEMBER 15, 2018 11 + + + + +
12 ROCKVILLE, MARYLAND 13 + + + + +
14 The workshop convened at the Nuclear 15 Regulatory Commission, One White Flint North, Room 16 10B04, 11555 Rockville Pike, at 1:00 p.m., Joe Golla, 17 NRR/DLP/PLBP, presiding.
18 19 NRC STAFF PRESENT:
20 JOE GOLLA, NRR/DLP/PLPB 21 AARON ARMSTRONG, NRO/DCIP/QVIB1 22 ERIC BENNER, NRR/DE 23 DOUG BOLLOCK, NRR/DIRS/IRIB 24 CALVIN CHEUNG, NRR/DE/EICB 25 SAMIR DARBALI, NRR/DE/EICB NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
2 1 THEO FANELLI, R-II/DRS/EB1 2 GREG GALLETTI, NRO/DCIP/QVIB1 3 PHIL McKENNA, NRR/DIRS/IRGB 4 RICHARD STATTEL, NRR/DE/EICB 5 MIKE WATERS, NRR/DE/EICB 6 DEANNA ZHANG, NRR/DE/EICA 7
8 9 ALSO PRESENT:
10 NEIL ARCHAMBO, Duke Energy 11 MARK BURZYNSKI, Rolls-Royce 12 STEVE DRAGOVICH, Exelon 13 PAREEZ GOLUB, EXCEL Services 14 RAY HERB, SNC 15 DAVE HERRELL, MPR Associates 16 FRANK NOVAK, GE Hitachi 17 WARREN ODESS-GILLETT, NEI 18 JOHN SCHRAYE, Entergy 19 STEPHEN VAUGHN, NEI 20 21 22 23 24 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
3 1 CONTENTS 2 Opening Remarks . . . . . . . . . . . . . . . . . 4 3 ISG-06 Alternate Review Process Summary 4 ISG-06 Update . . . . . . . . . . . . . . . 9 5 Industry Expectations on Alternate Review 6 Process Inspections . . . . . . . . . . . . 58 7 NRC Activities Related to DI&C Modifications 8 (General Roles and Responsibilities) . . . .
9 Example of Audits and Inspection Activities 10 for a Modification Performed Under ISG-06 11 Rev. 2 Alternate Review Process . . . . . . .
12 Vendor Oversight Plan Discussion . . . . . . . 149 13 Workshop Closing Comments/Follow-Up Actions . . 185 14 Public Comments . . . . . . . . . . . . . . . . 200 15 Adjourn . . . . . . . . . . . . . . . . . . . . 200 16 17 18 19 20 21 22 23 24 25 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
4 1 P-R-O-C-E-E-D-I-N-G-S 2 1:00 p.m.
3 MR. GOLLA: Okay, let's go ahead and get 4 started. I'm Joe Golla. I'm the ISG-06 project 5 manager in the NRR Division of Licensing Programs.
6 The purpose of the workshop today is to 7 address audits and inspection of digital 8 instrumentation and control installations approved 9 under the guidance of the ISG-06 Rev. 2 licensing 10 process, particularly the Alternate Review Process 11 therein.
12 For this category two public meeting, all 13 portions are open. Comments from the public will be 14 taken before adjournment of the meeting.
15 We do have an attendance list going around 16 and we invite you to enter your information on that.
17 That will become part of the public record along with 18 the meeting summary.
19 And for folks on the phone, if you would 20 like your participation in the meeting reflected in 21 the attendance list, please send me an email at 22 joe.golla@nrc.gov.
23 For folks in the room, there's a little 24 bit of housekeeping we always have to mention. In the 25 event of a fire alarm, listen for the announcement.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
5 1 If instructed to do so, make an orderly exit using the 2 stairs, not the elevator.
3 The stairs are opposite the restrooms over 4 here, and turn right out of the building and make the 5 first right on Marinelli, and the mustering point is 6 down the road one block. Turn right again and that's 7 that.
8 Okay, so with that, let's have 9 introductions. Again, I'm Joe Golla, the project 10 manager, NRC.
11 MR. WATERS: I'm Mike Waters. I'm Chief 12 of the Instrumentation Control Branch B at NRR.
13 MR. STATTEL: Richard Stattel, I&C 14 technical reviewer.
15 MR. DARBALI: Samir Darbali, I&C technical 16 reviewer.
17 MS. GOLUB: Pareez Golub, EXCEL Services.
18 I lead the industry team.
19 MR. ODESS-GILLETT: Warren Odess-Gillett, 20 NEI. I co-lead the industry team.
21 MR. HERB: Ray Herb, Southern Nuclear.
22 MR. DRAGOVICH: Steve Dragovich, corporate 23 engineering, EXELON.
24 MR. VAUGHN: Steve Vaughn, NEI.
25 MR. FANELLI: Theo Fanelli, Region II.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
6 1 MR. GALLETTI: Greg Galletti, vendor 2 inspection group.
3 MR. BOLLOCK: Doug Bollock, I'm in the 4 Division of Inspection and Regional Support.
5 MR. McKENNA: Phil McKenna, I'm in the 6 Division of Inspection and Regional Support.
7 MR. CHEUNG: Calvin Cheung, I&C technical 8 reviewer, NRR.
9 MS. ZHANG: Deanna Zhang, NRR, I&C 10 technical reviewer.
11 MR. SCHRAYE: John Schraye, Entergy 12 corporate licensing.
13 MR. BURZYNSKI: Mark Burzynski, Rolls 14 Royce.
15 MR. HERRELL: Dave Herrell, MPR 16 Associates.
17 MR. NOVAK: Frank Novak, GE Hitachi.
18 MR. ARCHAMBO: Neil Archambo with Duke 19 Energy.
20 MR. GOLLA: Okay, thank you. How about 21 folks on the phone? Would you care to introduce 22 yourself?
23 MS. ALVARADO: This is Rossnyev Alvarado.
24 I am the reviewer for NRR.
25 MS. VENKATARAMAN: This is Booma NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
7 1 Venkataraman at DORL, the Division of Operating 2 Reactor Licensing.
3 MS. SALGADO: Nancy Salgado, NRR. I'm 4 with the Alpha Branch.
5 MR. GOLLA: Okay, do we have any --
6 MS. BERGMAN: Jana Bergman, Curtiss-7 Wright.
8 MR. GOLLA: Oh, Jana Bergman, yes. Jana's 9 with Curtiss-Wright, she said, a writer. Anyone else 10 on the phone?
11 MR. STERLING: Webb Sterling, NextEra 12 Energy.
13 MR. GOLLA: Okay, that sounds like it.
14 Mike, would you want to make any opening remarks?
15 MR. WATERS: Yeah, I mean, I'll let Samir 16 lead the meeting, Samir, but I just want to say 17 welcome. I am happy to see a full room of people on 18 a day like today. I am actually surprised to. I 19 heard someone say they were across the room. We have 20 been looking forward to this workshop.
21 I know some of the goals are to give an 22 overview of NRC's inspection process from a 23 programmatic standpoint and talk about how it may be 24 streamlined or adjusted in the future as part of the 25 ISG-06 alternative review process. It sounds like a NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
8 1 lot of that will be centered around the licensee's 2 vendor oversight plan. That's a hard thing for me to 3 say, so I look forward to dialogue.
4 I want to thank just similarly my team of 5 experts in technical review, licensing, and 6 inspection, to get this dialogue started. With that, 7 I'll, I guess, I don't know if you have any remarks, 8 Pareez?
9 MS. GOLUB: First up, thank you very much 10 to all of you. It's wonderful to see some new faces 11 here, so we really appreciate you taking the time and 12 coming out to join us today. We've been looking 13 forward to this workshop.
14 And I know I've said this before, but 15 thank you very much to the NRC team that's been 16 working on this for a long time, especially I know 17 over the last six months, you guys have done a lot to 18 keep this, to keep ISG-06 moving through the process.
19 We really appreciate everything you guys 20 have been doing to keep it on schedule and moving 21 forward. The industry is eagerly anticipating getting 22 a chance to use this document, so thank you.
23 MR. DARBALI: Thank you, Pareez. So, all 24 right, so we'll get started. I guess the first item 25 in the agenda is we're going to do a quick NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
9 1 presentation of ISG-06, kind of like a summary of 2 that. Joe, if you can open the one, the general roles 3 and responsibility?
4 MS. GOLUB: Samir, could I just ask a 5 quick question?
6 MR. DARBALI: Sure.
7 MS. GOLUB: So if we have questions, 8 should we ask them along the way?
9 MR. DARBALI: Yeah, yeah, go ahead.
10 MS. GOLUB: All right.
11 MR. DARBALI: All right, so the outline of 12 this presentation will take about 10 minutes to just 13 do a summary of what the Alternate Review Process is 14 and we'll provide an update on where the ISG is, and 15 then we have a separate presentation on, it's kind of 16 like an overview of what the different NRC staff 17 activities are when the DI&C modification is involved, 18 so we'll get to those. Next slide.
19 Okay, so the purpose and scope of ISG06, 20 and again, the overall purpose did not change from 21 Revision 1 to Revision 2. We just made a change to a 22 new process, but ISG-06 defines the licensing process 23 used to support LAR reviews associated with DI&C, 24 equipment modifications, new operating plans, any new 25 plans once they become operational.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
10 1 So that means that if a Part 52 plant 2 being under construction is going to do an LAR on the 3 DI&C, this process would not be used, but once the 4 plant is in operation and they want to do a mod, then 5 the ISG-06 can be used.
6 The ISG provides guidance for activities 7 performed before the LAR such as the preapplication 8 coordination meetings and doing the LAR review. The 9 NRC staff uses the process described in the ISG to 10 evaluate compliance with NRC regulation, and the ISG 11 makes reference to and is supposed to be used in 12 conjunction with SRP Chapter 7. Some people in the 13 past have asked is this meant to replace the SRP and 14 we have clarified that. It's meant to be used 15 together. Next slide.
16 So you have seen this slide before. The 17 blue area in the middle covers what we in the past 18 typically identified as the licensee and vendor 19 development activities for the application.
20 I understand that under the detailed 21 engineering guide, those particular activities will 22 have different not mains, so because the staff has not 23 yet so familiarized with the DEG, we didn't change it 24 for this slide, but definitely in the future as we get 25 more familiarized with the DEG, definitely the NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
11 1 licensee can put a name, the name that they want to 2 the life cycle case and we can introduce that, but 3 it's meant to cover those activities.
4 The top portion of this slide shows what 5 review under the Tier 1, 2, and 3 review process would 6 look like where the LAR is submitted with the Phase 1 7 information before detailed hardware and software 8 design is provided.
9 And then during the NRC staff's review, 10 the licensee keeps producing and submitting the Phase 11 2 information, and then the license amendment is 12 issued after that, and then we have during site 13 acceptance testing and installation, then we have some 14 regional inspections. This is the process we've been 15 working under currently and it's maintained under 16 Revision 2 of the ISG.
17 The new process that's been introduced, 18 which is the Alternate Review Process, is described at 19 the bottom where the LAR with all of the information 20 needed to reach a safety determination is provided 21 during the detailed design review, but before 22 completion of the detailed design.
23 The staff would do their LAR review and 24 regulatory audits, and the plan would be to do 25 license, issued license amendment before factor NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
12 1 acceptance testing, and then that opens up this new 2 area of regional and vendor inspections of those 3 activities that we used to review before, and that's 4 the focus of this workshop, so next slide.
5 MR. ODESS-GILLETT: Samir, may I make some 6 clarifications -
7 MR. DARBALI: Sure.
8 MR. ODESS-GILLETT: - and comments on this 9 slide here? And I'll speak loud so people on the 10 phone can hear me. But this here represents a scope 11 of review. Would you agree that these bars here 12 describe, well, this is what the NRC would review as 13 part of the traditional tiered review?
14 MR. DARBALI: Correct.
15 MR. ODESS-GILLETT: However, I don't think 16 that's the case down in this diagram because although 17 you have the bars here, really it's here. The LAR 18 review content would be -
19 MR. DARBALI: Right, so -
20 MR. ODESS-GILLETT: So this red line 21 really is not an expression of scope of review, and 22 this red line could almost be anywhere in here because 23 the scope of the review is here. Is that -
24 MR. DARBALI: That's correct, yes.
25 MR. ODESS-GILLETT: Okay.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
13 1 MR. DARBALI: So it's a timeline. There 2 are some audit activities that would take place.
3 MR. ODESS-GILLETT: Audit, yeah, audit, 4 but that, I mean, this is a time period.
5 MR. DARBALI: Right.
6 MR. ODESS-GILLETT: But I would imagine 7 the audit activities would comprise this?
8 MR. DARBALI: That would be the material -
9 MR. ODESS-GILLETT: That would be audited?
10 MR. DARBALI: - in the LAR, correct, and 11 whatever audit would support the LAR.
12 MR. ODESS-GILLETT: Whatever audit they 13 need to do.
14 MR. DARBALI: Correct, Rich?
15 MR. STATTEL: So I'd like to add some 16 clarification. So the green lines actually represent 17 the time of submittal. So because the Alternate 18 Review Process is a one submittal process, that's all 19 we get is what's at the green line. So pretty much 20 everything we're looking, it took place and was 21 developed prior to that time.
22 That's not the case in the upper part of 23 this chart because the upper part is a two-part 24 submittal and we actually have purview and we will 25 review test activities, test reports, V&V summary NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
14 1 reports which don't become available until later in 2 the process.
3 So the Phase 2 submittals actually are 4 basically material that's developed while we are in 5 progress of developing the safety evaluation. So it's 6 basically we're doing our evaluation in parallel with 7 the development activities.
8 MR. DARBALI: Right.
9 MR. STATTEL: We're going to be reviewing 10 products of those development activities during that 11 evaluation. That's what we do.
12 MR. DARBALI: So the review for the tier 13 process is comprised of the green line -
14 MR. ODESS-GILLETT: Right.
15 MR. DARBALI: - and the white box.
16 MR. ODESS-GILLETT: Correct.
17 MR. DARBALI: So Phase 1 and Phase 2.
18 MR. ODESS-GILLETT: But Rich, were you 19 describing the Alternate Review Process?
20 MR. STATTEL: No, no.
21 MR. DARBALI: The tier process.
22 MR. ODESS-GILLETT: Got it.
23 MR. STATTEL: The Tier 1, 2, and 3 review 24 process.
25 MR. ODESS-GILLETT: Okay, so just for NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
15 1 clarification, this red line doesn't mean that the 2 license amendment occurs necessarily with the detailed 3 hardware and software design fabrication ending? It 4 could. It could.
5 MR. DARBALI: It could, correct.
6 MR. ODESS-GILLETT: It could be here. It 7 could be here and who knows where?
8 MR. DARBALI: And in the other set of 9 slides that we talk about, the Oconee example, you'll 10 see we kind of put, like below the dash lines, we put 11 like a more expanded area which describes that. It 12 could be before or after, right. So -
13 MR. ARCHAMBO: Would there be an option if 14 a licensee wanted to get the license amendment before 15 they even started doing their detailed design? I 16 mean, I think that's what we're kind of asking. Based 17 on the first two boxes, you issue that and you get 18 your license amendment, and then you might decide to 19 do your detailed design. Is that an option?
20 MR. DARBALI: Well, I mean, every design 21 is different and the idea of what the slide is trying 22 to show is as the design is progressing, the licensee 23 and the vendor are doing, following their own 24 schedule. That would be about the same time it would 25 take us to do a license amendment.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
16 1 For the license amendment request, there 2 is going to be an amount of detailed hardware and 3 software information that's going to be needed to be 4 included in the LAR. So I don't think that you could 5 even submit the LAR before you started FAT.
6 Definitely you could submit it before you've completed 7 the design.
8 MR. WATERS: This is Mike. I mean, 9 1:15:31 general understanding of how things would go.
10 I think the opposite side of the coin is under the 11 paradigm review process, you have a single submittal 12 with the acceptance review.
13 There might be a mindset to say well, they 14 can supplement later. They can do this later, but 15 we'll do the acceptance review and either accept or 16 deny or whatnot under this new paradigm.
17 So I heard someone a few weeks ago who 18 said, "Well, we could always supplement if needed."
19 Well, that's not the case under this new paradigm.
20 It's a single submittal and alterations needed will be 21 in part at the time of the submittal where the red 22 line is shown.
23 MR. HERB: Can I clarify? I think what 24 Neil was asking was there are certain, typically most 25 plants have certain resourcing schedule constraints, NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
17 1 and we would like the option to be able to submit LAR 2 once we have enough information for that LAR, to 3 populate that LAR, and we may not be purchasing 4 hardware yet.
5 And so we may wait until the LAR approval 6 before we even start to purchase hardware and they 7 start to actually stack and rack components, and so, 8 but we don't anticipate it to happen, but we just want 9 to make sure that you're not anticipating that you're 10 going to be able to see some of that stuff just 11 because of the timing issues and they all stack up and 12 that you need some of that rack and stack data to 13 approve the LAR.
14 MR. DARBALI: Right.
15 MR. HERB: Because that's not our 16 understanding. It's that the approval is on the 17 requirements -
18 MR. DARBALI: Correct.
19 MR. HERB: - and not on the actual 20 equipment. Now, the actual equipment is in the 21 inspection space, of course, and has to be confirmed.
22 MR. DARBALI: And that detailed discussion 23 would occur during preapplication.
24 MR. HERB: That's right, so, yeah.
25 MR. DARBALI: So we would identify what NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
18 1 parts of the ISG-06 you can provide the information 2 for.
3 MR. HERB: Yeah, what we don't want is 4 like to say, "Well, we can't finish our review until 5 you buy equipment." You know, that kind of like, you 6 know, and we're just kind of slowing it down because 7 we know you're going to buy some stuff and we're going 8 to want to look at some of that stuff.
9 MR. DARBALI: When you say "buy 10 equipment," you mean -
11 MS. GOLUB: That's the application part.
12 MR. DARBALI: Right, you already 13 determined what the platform -
14 (Simultaneous speaking.)
15 MR. HERB: Oh, yeah, correct, we would 16 never submit a LAR on an open platform -
17 MR. DARBALI: Okay.
18 MR. HERB: - not specified. I mean, that 19 would - how would you even do that?
20 MR. DARBALI: Right, right, so you would 21 have engaged the vendor. You would have had that 22 stuff on contract -
23 MR. HERB: Oh, yes.
24 MR. DARBALI: - but not purchased.
25 MR. HERB: The PO would be in place. We NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
19 1 would already have identified the scope of the mod -
2 MR. DARBALI: Right.
3 MR. HERB: - the equipment and all that 4 stuff, but we may not actually, the vendor may not 5 actually -
6 MR. DARBALI: Starting working on the 7 actual programming.
8 MR. HERB: Sure.
9 MR. WATERS: So I recollect on the last 10 public phone call, there was offered the idea that 11 maybe the blue line representation may be changed with 12 your current process and then provided to us. Do you 13 have something to share, I mean, to help maybe 14 illustrate this better for future meetings?
15 MS. GOLUB: Well, so the digital 16 engineering guide is the process, and on Monday of 17 this week, the court -
18 MR. HERB: Actually, we're going to go 19 into that in the basic presentation.
20 MS. GOLUB: Oh, that's right. That's 21 right. Okay, so, yeah, so, but we'll get to that 22 soon.
23 MS. ZHANG: So I do have a question in 24 regards to it. We have talked about using ISG-06 not 25 only for LAR reviews, but also for platform reviews NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
20 1 too. Do you see any changes? Do you see using the 2 Alternate Review Process for platform review or would 3 you only use it -
4 MS. GOLUB: No.
5 MR. HERB: We don't see -
6 MS. GOLUB: Right.
7 MR. DARBALI: Did you ask specifically for 8 the Alternate Review Process? Yeah, I don't -
9 MS. GOLUB: Yes.
10 MR. DARBALI: It would be like on the same 11 page here.
12 MR. STATTEL: Well, we, on the onset of 13 this -
14 MR. BURZYNSKI: I think some vendors might 15 be interested in using the descriptions in there of 16 what kind of technical information to provide.
17 MR. DARBALI: But I think -
18 MR. BURZYNSKI: - but was not actually 19 quoted under the alternate review.
20 MR. DARBALI: Right, right, okay.
21 MS. GOLUB: Yes, yes, that's a really good 22 point.
23 MR. STATTEL: And we discussed this at the 24 beginning of this when we were developing this 25 revision and topical reports were not within the scope NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
21 1 of the ISG-06 revision.
2 MR. DARBALI: So I don't know if we want 3 to maintain - I don't think we've had the discussion 4 that we want to maintain topical report reviews under 5 the Rev. 1 or to keep them under -
6 MR. STATTEL: Well, it's just guidance.
7 MR. DARBALI: Right.
8 MR. STATTEL: So in reality, every 9 application is unique. Every - there is not one model 10 that's going to fit all applications. In reality, a 11 topical report review is just another type of 12 application. Some of it applies, some of it doesn't, 13 so, and we've been dealing with this for several years 14 now with the original revision of ISG-06.
15 Now, we've applied ISG-06 as best we can 16 with the understanding that a lot of the regulatory 17 requirements won't be met until an application is 18 developed, so, you know, we have a process for dealing 19 with that.
20 So I think in truth, we can apply, even 21 though it's not with really, it wasn't developed with 22 this in mind, topical report review, we can apply the 23 criteria in some topical areas to topical report, and 24 that's what we intended to do that.
25 MR. DARBALI: Okay, next slide. So some NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
22 1 characteristics of the LAR used in the Alternate 2 Review Process, the LAR would provide the necessary 3 instrumentation and design information to demonstrate 4 regulatory compliance. We've talked to that.
5 The LAR would describe the licensee's 6 Vendor Oversight Plan that ensures that the vendor 7 executes the project. Actually, we haven't updated 8 this based on the language that the ISG was updated 9 to.
10 So the ISG would say that the LAR 11 describes the licensee's Vendor Oversight Plan which, 12 let's see, which once executed, can help ensure that 13 the vendor executes the product consistent with the 14 LAR and also once executed, can help ensure that the 15 vendor uses as a reference the 2015 version of NQA 1.
16 MR. ODESS-GILLETT: So we had some 17 internal discussion on that. I know this is wording 18 and the purpose of this workshop is not to affect -
19 MR. DARBALI: Right.
20 MR. ODESS-GILLETT: - the ISG-06 wording, 21 but we would like a clarification or interpretation of 22 the NRC staff expectation that the vendor's Appendix 23 B program is going to actually reference 2015 of NQA 24 1.
25 MR. DARBALI: So I don't - the language in NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
23 1 the ISG does not ask for the vendor's QA program to 2 reference the 2015 version. So the only "should" or 3 requirement here, and I'm using the word loosely -
4 MS. GOLUB: This is the language just in 5 case you want to see it.
6 MR. DARBALI: Okay, is that the - it talks 7 about the VOP, but it doesn't talk about modifying the 8 vendor's QA program. So the VOP should make sure that 9 somehow the NQA 1 is used as a reference, but it 10 doesn't mean it has to be implemented.
11 MR. ODESS-GILLETT: So as a reference 12 refers to the VOP versus as a reference to the 13 vendor's QA program?
14 MR. DARBALI: Could you repeat that?
15 MR. ODESS-GILLETT: You should have as a 16 reference -
17 MR. DARBALI: Yes.
18 MR. ODESS-GILLETT: - the 2015.
19 MR. DARBALI: Yes.
20 MR. GALLETTI: Just to clarify, the 21 purpose of using it as a reference is we don't want to 22 formally dictate that any vendor has to apply that 23 version of NQA 1 or any version of NQA 1.
24 If, however, you licensees, as part of 25 this oversight process, decide that you want them to NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
24 1 apply that to the purchase orders, you specify that.
2 Originally as this was written, it required them to 3 follow the 2015 version of NQA 1 and we would never 4 specify that.
5 MR. DARBALI: So the original language was 6 consistent with the NQA 1 version and I think that 7 caused some problems, so we didn't -
8 MR. GALLETTI: We didn't want to dictate 9 that.
10 MR. DARBALI: Right.
11 MR. GALLETTI: You licensees need to 12 determine what quality program you want them to work 13 under or approve the one they have and be satisfied 14 with something.
15 MS. GOLUB: Could we possibly capture that 16 in the notes? Because I know at least on our side, we 17 were, there was a lot of confusion in that discussion 18 that we had earlier today because we were kind of 19 worried when we saw the word "reference" that it meant 20 we would literally in the vendor's program a reference 21 to the 2015 version.
22 MR. GALLETTI: No, no, that wouldn't make 23 sure it's necessary.
24 MS. GOLUB: Right, I agree with you, and 25 I know that was not the intent here.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
25 1 MR. DARBALI: No.
2 MS. GOLUB: But the way that the wording 3 is written, we think, because, you know, we don't have 4 confirmation of that final document, we're just 5 concerned that people may misinterpret that who 6 weren't part of these initial discussions, so if it's 7 possible to at least capture that in the notes, 8 because I know it's too late now to change the ISG.
9 MR. DARBALI: This is the latest language, 10 you guys.
11 MR. GALLETTI: It would be too late for a 12 wording change there too?
13 MR. DARBALI: Do you have the latest?
14 (Simultaneous speaking.)
15 MS. GOLUB: This is - what we're 16 struggling with is the "as a reference." We're 17 worried that somebody is going to pick it up and think 18 that the vendor's QA program has to reference this 19 2015 version, but when you say "as a reference," you 20 mean "consistent with," right, not so much that it's -
21 MR. DARBALI: Well, that the NQA 1 is 22 considered.
23 MS. GOLUB: Right, so the problem is this, 24 we're worried that somebody is going to see that as a 25 reference. If this wording can't be changed because NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
26 1 it's really far down the pike now -
2 MR. DARBALI: Right.
3 MS. GOLUB: - then what we were hoping is 4 that as part of the notes for this meeting, we could 5 at least capture -
6 MR. DARBALI: Right, clarify that.
7 MR. GALLETTI: The interpretation.
8 MS. GOLUB: Right, this - what's your 9 name?
10 MR. DARBALI: Greg.
11 MS. GOLUB: Yeah, I think what Greg said 12 was spot on. I think it's been all of our, you know, 13 all of our interpretation of that language, but today 14 we kind of looked at it, you know, and thought about 15 it with fresh eyes. We were worried that in the 16 future, somebody may interpret that differently.
17 MR. DARBALI: So if -
18 MS. GOLUB: Especially in a couple of 19 years, you know, when there's a new version -
20 MR. DARBALI: Right.
21 MS. GOLUB: - of NQA 1.
22 MR. DARBALI: And I know we're not 23 rewriting it, but if it said, and I'm going to read 24 from the latest language, let's see, right, "The 25 Vendor Oversight Plan, when execute, can also be used NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
27 1 to ensure that the vendor uses as a reference the 2015 2 version." If it said, "can also be used to ensure 3 that the vendor references the 2015 version," -
4 MS. GOLUB: So the problem is, you know, 5 there's a list of references, right?
6 MR. DARBALI: Right.
7 MS. GOLUB: So like I'm a little worried 8 that people are going to say, "Hey, you know, where's 9 that reference to the 2015 version?"
10 MR. DARBALI: Right.
11 MS. GOLUB: And so, and I hear Greg saying 12 that, you know, there is no intention here of NRC 13 imposing a certain -
14 MR. DARBALI: Right.
15 MR. FANELLI: But shouldn't it say 16 something along the lines of the reference is whatever 17 the licensee commits to in their license amendment 18 request?
19 MR. WATERS: Can I interject something?
20 I thought in previous meetings, it was the industry 21 use of this 2015 version.
22 MR. DARBALI: That's right.
23 MR. WATERS: So I'm trying to understand 24 how it evolved from that -
25 MS. GOLUB: It's a framework because we -
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
28 1 MR. ODESS-GILLETT: If we could just apply 2 it to the Vendor Oversight Plan and not to the QA 3 program, I think NEI will be satisfied.
4 MR. DARBALI: Well, and we're not tying it 5 to the vendor QA program. You had language there 6 before that said, "as implemented by the licensee's -
7 by the vendor," so we removed -
8 MR. HERB: It's supposed to be the Vendor 9 Oversight Plan.
10 MR. DARBALI: Yes.
11 MR. HERB: So maybe there's a way we can 12 change the wording to make that more -
13 PARTICIPANT: Should it be "licensee uses" 14 versus "vendor uses?"
15 MR. ODESS-GILLETT: Yeah, yeah, that would 16 work.
17 MR. GALLETTI: I mean, because a lot of 18 the licensees are currently licensed to N452 19 standards.
20 MR. ODESS-GILLETT: I know. This is -
21 MR. GALLETTI: So we're not trying to 22 impose -
23 MS. GOLUB: No, but this is -
24 MR. ODESS-GILLETT: I understand, Greg, 25 but this was sort of a step up that the staff wanted NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
29 1 licensees because, or the NEI team thought those 2 licensees that really don't have an NQA 1 QA program, 3 that some kind of step up was needed for the licensee 4 oversight, and that's why that language is there.
5 MR. GALLETTI: I understand, because sub 6 part two and part seven specifically talks about -
7 MR. ODESS-GILLETT: Yes.
8 MR. GALLETTI: - software quality -
9 MR. ODESS-GILLETT: Exactly correct.
10 Exactly correct.
11 MR. GALLETTI: - where other QA programs 12 would go into that level of detail.
13 MR. ODESS-GILLETT: Exactly, exactly 14 correct.
15 MS. GOLUB: But that's why the "consistent 16 with" language is used because the idea was not to 17 impose it, but -
18 MR. DARBALI: Right.
19 MS. GOLUB: - just give a common framework 20 so we all, you know, kind of had the same mindset.
21 MR. DARBALI: So -
22 MR. GALLETTI: And we took it just the 23 opposite.
24 MR. DARBALI: Right.
25 MR. GALLETTI: When you said, "consistent NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
30 1 with," it beared a sense that it was going to be 2 required of them, where this "as a reference" says, 3 "Hey, just consider what this says." It has some 4 current language, current thinking of the industry 5 when you apply -
6 MS. GOLUB: Yes.
7 MR. GALLETTI: - whatever you're doing to 8 -
9 MS. GOLUB: Yes.
10 MR. GALLETTI: - the vendor, but not to 11 force a commitment in any way.
12 MS. GOLUB: Yes, that's the intent. Maybe 13 that word "considered with" or, you know -
14 MR. VAUGHN: It sounds like we all agree 15 on what it should say, but we're interpreting the 16 complete opposite than you are, so maybe we should 17 just take an action to clarify it, and read it, and 18 make sure all of us agree that it says what we want it 19 to say?
20 PARTICIPANT: So we'll -
21 MR. WATERS: I can't commit to revising 22 the ISG at this point. We'll look at that. Our legal 23 staff, rightly so, will give me a hard time. They 24 didn't want to crush their holidays with changes, so, 25 or those other processes, but what's the best way to -
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
31 1 MR. ODESS-GILLETT: We were just thinking 2 maybe documenting -
3 MS. GOLUB: In the notes.
4 MR. ODESS-GILLETT: - the interpretation 5 in the leading summary would suffice industry.
6 MR. WATERS: I don't an issue with that, 7 and we can see if we can make any last second changes.
8 MS. GOLUB: Right, we had kind of made 9 that assumption, so we were hoping as long it's 10 documented here, we have something that we can, you 11 know, we can point to that captures the intent.
12 MR. DARBALI: Okay.
13 MR. VAUGHN: Or even during the review, 14 maybe just specifically ask, "Hey, what does this mean 15 to you?"
16 MR. DARBALI: Right.
17 MR. VAUGHN: And if that lawyer says it 18 means - I'd be interested to see what their 19 interpretation would be.
20 MR. DARBALI: Okay, no, that's good 21 feedback. Thank you. Okay, and then the LAR includes 22 appropriate commitments to complete plan specific 23 actions that are included in the reference topical 24 report, and the LAR includes appropriate commitments 25 to complete life cycle activities under the licensee's NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
32 1 QA program.
2 So this particular workshop is not on 3 commitments or conditions, but one aspect of the 4 Alternate Review Process is that it relies on the LAR 5 containing license information and additional 6 regulatory commitments to implement those remaining 7 development phases, activities by the licensee's QA 8 program after the license amendment is issued, and the 9 staff may likely translate some of those commitments 10 into license conditions. Next slide.
11 Okay, so a quick update on the ISG, we 12 have completed the concurrence process -
13 MS. GOLUB: Okay, great.
14 MR. DARBALI: - and incorporated those 15 concurrence comments. It's currently with OGC, so 16 they're doing their no legal objection review, and 17 we've had some interactions with the ACRS and we 18 believe that their concerns have been addressed.
19 MS. GOLUB: Great.
20 MR. DARBALI: We are - so they provided us 21 with a letter when we presented to them back in July, 22 and that letter asked us for a series of things, 23 including issuance for public comments.
24 They had us address some licensee 25 oversight for ownership and hardware configuration NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
33 1 management, and we incorporated that in the version 2 that went out for public comments, and then they said, 3 "Well, after you've addressed public comments, send it 4 back to us," and we did that.
5 But they also had a concern in the body of 6 the letter that said - it addressed control of access, 7 so we're in the process of providing a second response 8 letter to them telling them how the ISG and other NRC 9 activities address control of access.
10 So although we haven't heard a final 11 message from the ACRS saying there is no need for 12 another presentation, we believe we're on that path 13 that we don't need to present, so it looks like we 14 will be able to issue the ISG, the final ISG by the 15 end of the year.
16 MS. GOLUB: That's great. That's great.
17 Thank you.
18 MR. WATERS: And again, this looks like 19 that's our best understanding -
20 MR. DARBALI: Right.
21 MR. WATERS: - of what they're looking 22 for.
23 MS. GOLUB: Is there like a point at which 24 if the ACRS does not require or request a 25 representation, you kind of say, "Okay, clearly that's NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
34 1 done with"?
2 MR. DARBALI: I think there is, but I 3 think by the time that happens -
4 MS. GOLUB: Okay.
5 MR. DARBALI: - it might be maybe a week 6 or two before the -
7 MS. GOLUB: Okay, all right.
8 MR. DARBALI: - ISG is issued.
9 MS. GOLUB: Okay, okay.
10 MR. DARBALI: Just because there's a lot 11 of things that are going on right now.
12 MS. GOLUB: Okay, understood.
13 MR. DARBALI: And there's formal processes 14 for engaging with the ACRS, so it takes some time.
15 MS. GOLUB: Okay, understood. Now, after 16 OGC, does it still need to go to the CRB and those 17 other -
18 PARTICIPANT: OMB.
19 MS. GOLUB: - OMB and all these other -
20 MR. GOLLA: It's there now.
21 MS. GOLUB: Oh, it's there now? Okay, 22 great, so it's concurrent.
23 MR. DARBALI: Yes, we shored up a previous 24 version that doesn't change much of what they would 25 review, so.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
35 1 MS. GOLUB: What they review, okay, great.
2 MR. DARBALI: So we're trying to -
3 MR. GOLLA: We asked them to give it the 4 highest priority for our agency at least.
5 MS. GOLUB: Right.
6 MR. GOLLA: They have a lot of other 7 things there that they're reviewing, but as far as I 8 know, we're at the top of the food chain for the NRC 9 right now.
10 MS. GOLUB: Okay, great.
11 MR. DARBALI: I'm sure it's the same on 12 your side, but, you know, the people you see here are 13 just a fraction of the people -
14 MS. GOLUB: Oh, yes.
15 MR. DARBALI: - who are helping -
16 MS. GOLUB: Yes.
17 MR. DARBALI: - push this through.
18 MS. GOLUB: Yes.
19 MR. DARBALI: So, and we're grateful for 20 their support.
21 MS. GOLUB: Absolutely, thank you.
22 MR. DARBALI: Okay, so this set of slides 23 covers a high level overview of what the staff 24 typically does in regards to a digital modification.
25 So we do technical reviews. You'll see NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
36 1 another slide on inspections, but the I&C staff 2 technical review with regards to a Digital I&C 3 modification, we get involved with topical report 4 reviews.
5 We do the LAR safety evaluations. We do 6 regulatory audits. We support inspections, vendor 7 inspections, and also we also support the region, so 8 it might not be related to an inspection, but we do 9 provide support.
10 And some of the applicable guidance that 11 we use is ISG-06 Revision 2 once it becomes, once 12 that's a draft, so once it becomes final, it will be 13 the final version. We always use SRP Chapter 7, and 14 then there's a bunch of NRR office instructions.
15 We have LIC-101 on acceptance review, on 16 the LAR reviews, LIC-109 on acceptance reviews. For 17 regulatory audits, we have LIC-111, and for Topical 18 Report Reviews, we use LIC-500.
19 And then the next slide talks about just 20 regulatory audits because that's a portion that we'll 21 probably be talking more today, but it's what the 22 technical staff focuses on.
23 So this comes from LIC-111 on audits, so 24 regulatory's planned license or regulation-related 25 activity. That includes the examination and NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
37 1 evaluation of primarily non-docketed information that 2 could occur either at the licensee's site, or at the 3 vendor's site, or remotely on an electric audit.
4 So regulatory is conducted with the intent 5 to gain understanding, to verify information, and to 6 identify information that will require docketing to 7 support the basis of the licensing or regulatory 8 decision.
9 What we've done in the past, and most of 10 you have seen this before, when we do our LAR review, 11 we have an open items table or list and some of those 12 items will refer to an audit activity, to the closure 13 of the open item based on the audit may be we reviewed 14 it during the audit and the item is closed or we 15 reviewed it during the audit and "Please submit the 16 full document," or something like that on the docket.
17 Next slide.
18 So now I'll let Doug talk about DIRS.
19 MR. BOLLOCK: All right, so this slide 20 just gives a general overall of what the Division of 21 Inspection and Regional Support does.
22 It provides specialized management of 23 reactor oversight programs, supports regional 24 residence inspection and base inspection activities 25 consistent with our mission, responsible for leading, NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
38 1 guiding, and coordinating Agency activities, and 2 coordinated implementation of these activities with 3 all its recommendation program changes.
4 So essentially if there are any changes 5 needed in especially licensees, we would be involved 6 working with industry and working with the regions to, 7 you know, if we had to make changes to any inspection 8 procedures or what we do in our oversight or 9 inspections.
10 Right now, and I think we'll get into a 11 little bit later, but there is an inspection procedure 12 for Device E52 Tier 03 and that's in our inspection 13 manual Chapter 25-15 Appendix C, so infrequent 14 inspection. So right now, there is a process in 15 place. If there are any changes needed, it would be 16 coordinated through DIRS.
17 MR. GALLETTI: Okay, Greg Galletti here.
18 I'll speak to the vendor inspections, so the typical 19 visualizing modifications. We have some 20 responsibilities for performing inspections at the 21 vendors' facilities.
22 Essentially, as you know, the inspections 23 will look to see how the vendor has implemented their 24 quality assurance plan to fulfill the purchase order 25 requirements that are levied on them by the licensee, NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
39 1 and we focus specifically on their implementation of 2 the Appendix B quality assurance program and also the 3 implementation of their 10 CFR Part 21 reporting of 4 defects or nonconformance program.
5 With respect to those two areas, if we 6 were to find some sort of deficiency at the vendor 7 facility, we may issue a notice of nonconformance 8 against one of the Appendix B criteria, and at the 9 vendor, in the case of any Part 21 issues or findings, 10 we would issue a notice of violation directly against 11 the vendor for the Part 21 failure.
12 The vendor inspection program verifies 13 that reactor applicants and licensees are fulfilling 14 their regulatory obligations. Again, this is, the 15 primary focus of performing these inspections is to 16 ensure independently that the licensee has done due 17 diligence in performing oversight of the vendor 18 activities that they've procured.
19 Again, it's accomplished through a limited 20 scope targeted inspection. We typically are very 21 performance based and we focus as much as possible on 22 the technical aspects of whatever the design is or 23 whatever is being procured.
24 We try to stay away from just doing a 25 simple programmatic review of a QA program. That's NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
40 1 typically done by the licensees themselves or through 2 NUPIC of those vendors.
3 So we will focus ourselves on more of a 4 technical review looking at how that vendor has 5 implemented the various criteria of Appendix B to 6 whatever application they're applying it. So one of 7 the areas you'll see on the next slide, please -
8 MR. ODESS-GILLETT: Before you do that, 9 Greg -
10 MR. GALLETTI: Yes, yes?
11 MR. ODESS-GILLETT: The last bullet, from 12 my experience with Diablo Canyon and with Part II 13 reviews, that I've seen regional inspectors support 14 technical audits because the region needs to be, like, 15 familiar with the system they're going to get and 16 ultimately inspect.
17 MR. GALLETTI: Right.
18 MR. ODESS-GILLETT: So I was surprised 19 that vendor inspection got involved in supporting 20 these technical reviews because it's not been my OE.
21 MR. GALLETTI: Okay, in specific cases, we 22 do actually support the audits that are done by the 23 technical staff. If they're going out to a vendor 24 facility and they're doing some sort of auditing of 25 vendor activities, often they'll leverage our NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
41 1 expertise and experience in that area to support those 2 audits that they're doing.
3 MR. ODESS-GILLETT: Right.
4 MR. STATTEL: It's not - we discussed this 5 this morning a little bit. It's not unprecedented.
6 MR. ODESS-GILLETT: Okay.
7 MR. STATTEL: So even with Diablo Canyon 8 you mentioned, on the ALS audit we did in Scottsdale, 9 we did have a vendor -
10 MR. ODESS-GILLETT: Oh, you did?
11 MR. STATTEL: - assessment -
12 MR. ODESS-GILLETT: Okay.
13 MR. STATTEL: - person with us.
14 MR. ODESS-GILLETT: Okay.
15 MR. HERB: I have a question on that 16 second bullet. It says that, "Verifies that the 17 reactor applicant licensee," I assume licensee here, 18 "are fulfilling their regulatory obligation with 19 respect to providing effective oversight of the supply 20 chain." Would that inspection be of our process?
21 That wouldn't necessarily be of the vendor. I mean -
22 MR. GALLETTI: No, well the way -
23 MR. HERB: So, I mean, I don't understand 24 how you can, how - I guess I don't understand how the 25 licensee's fulfilling their obligation by a parallel.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
42 1 Are you going to inspect the vendor and compare 2 results or use it with our program that we've done and 3 see how effective it is?
4 MR. GALLETTI: We would do both actually.
5 So we would look at the licensee's program. So we are 6 going to be doing certain audits or surveillance 7 activities at that vendor location. So we would look 8 to see what sort of issues you've identified through 9 those processes to see how well you're engaging with 10 the vendor in identifying issues of concern.
11 In parallel, by studying that, we would be 12 looking at how the vendor themselves are implementing 13 the purchase order requirements that you've levied 14 upon them to fulfill whatever the contract states. So 15 if you've applied certain regulatory guides, or 16 certain standards, certain, like NQA 1, we would be 17 looking to see how they've implemented those 18 requirements and how they've translated those 19 requirements into the actual design.
20 MR. HERB: Great, so a follow on question, 21 since I'm a utility and I'm paying for the review time 22 for technical staff, that additional piece, it seems 23 like more of a vendor inspection program piece. Am I 24 paying for that too as part of my license review or 25 are you paying for that out of your normal overhead?
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
43 1 MR. GALLETTI: To date, we do it in our 2 overhead.
3 MR. HERB: Yes, so if that's -
4 MR. GALLETTI: I don't know if that's -
5 MR. HERB: If that seems like an 6 additional burden on my vendor and he's probably doing 7 work for me and he has to support that, I think that 8 like - I don't know. Can you demonstrate how that 9 really supports my application versus you're just kind 10 of verifying that the vendor's quality is good?
11 MR. GALLETTI: Well -
12 MR. FANELLI: I was going to say the 13 region will also support the vendor group on these 14 activities too, and we'd be inspecting your licensing 15 basis and how you are fulfilling that criteria and 16 obligation, so it would be a mutual, it could be a 17 mutual inspection.
18 MR. McKENNA: That's not the question he's 19 asking.
20 (Simultaneous speaking.)
21 MR. BOLLOCK: Practically speaking, the 22 vendor inspection group inspects the vendors, right, 23 the inspection report. So, you know, anybody from 24 Greg's group goes. When they're going out with the 25 auditors, they write an inspection to the vendor.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
44 1 MR. GALLETTI: Right, and it's a separate 2 docket. It's not your docket.
3 MR. BOLLOCK: Right, yeah, if we were 4 conducting, and this is where we're be involved and 5 the region would be involved, if we were inspecting 6 your oversight, we would let you know we'd be 7 inspecting the licensee's oversight.
8 MR. HERB: I understand, but just the way 9 that bullet is read, it says, "verifies that applicant 10 are fulfilling their regulatory obligations," and so 11 I'm assuming you're coming to me for that, not to 12 bother my vendor or -
13 But, I mean, you have to do that as part 14 of your oversight of the vendors in general, I agree, 15 but are you planning to go every time somebody submits 16 a licensing to the vendor over, and over, and over 17 again, or just once in a normal process?
18 MR. GALLETTI: I don't know that we have 19 a firm answer to that.
20 MS. GOLUB: Yes, I think, and that's the 21 reason that we're having this workshop is at least on 22 our side of it, we didn't really understand how the 23 different groups within the NRC interacted to do sort 24 of, you know, the spectrum of audits and oversight.
25 And so I know at least I had a perception NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
45 1 that the vendor oversight group or the vendor branch 2 of the NRC did more of a, say, a generic inspection of 3 various vendors and not so much project specific.
4 So that was our perception on this side 5 was that the project specific inspection was done 6 either through the audit process by the NRR technical 7 reviewers and then by the region after the license 8 amendment was issued.
9 So that was our perception of how, you 10 know, how that was carried out, and so we weren't - we 11 just assumed that the vendor branch, that was more of 12 a generic, you know, periodic inspection of the 13 vendors.
14 MR. GALLETTI: Understood, and you're 15 correct. However, we also do much more focused 16 limited inspections, Westinghouse -
17 PARTICIPANT: Part 52.
18 MR. GALLETTI: Right, things like that for 19 their PMS system for example. Those inspections were 20 primarily led by the vendor organization focusing 21 directly on that one system.
22 Now, our same organization has gone to 23 Westinghouse and done just what you're saying, those 24 generic quality assurance, Appendix B verification 25 inspections if you will.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
46 1 MR. GOLLA: Excuse me for a moment.
2 Folks, on the phone, could I ask that you -
3 PARTICIPANT: Oh, thank you for asking 4 that.
5 MS. GOLUB: - mute your phone, star six to 6 mute, please? We're getting some occasional 7 intermittent noises, a static sound. I don't know if 8 that's from - if muting will kill that or not, but -
9 PARTICIPANT: We can try.
10 MS. GOLUB: Yeah.
11 MR. GALLETTI: We tend to try to not to 12 duplicate industry efforts. So if you have seen a 13 NUPIC audit for example, they'll go out to the vendor 14 and do a very programmatic review of the Appendix B 15 program. We tend not to do that. We tend to focus 16 more on the specific projects or systems, things of 17 that nature.
18 MR. BOLLOCK: And we tend to avoid when 19 the licensee is doing their audit with their vendor, 20 and we tend to avoid -
21 MR. GALLETTI: Right, and we tend to try 22 to avoid if NUPIC is coming in. We schedule and work 23 with NUPIC to try not to overburden the vendor.
24 MR. HERB: Right, and so I guess 25 hypothetically in a follow up to that is that you said NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
47 1 you recently did, went to Westinghouse for a PMS type 2 audit or review. If a vendor like myself said, "Well, 3 I'm going with the PMS league common platform," you 4 wouldn't necessarily have to repeat that just because 5 I've decided to -
6 MR. GALLETTI: Not necessarily. As a 7 matter of fact, I can tell you from our personal 8 experience, when we did those PMS inspections, we had 9 licensees at those inspections at the request of 10 Westinghouse.
11 MR. HERB: That's Part 52 though.
12 MS. GOLUB: That's the other part of it, 13 you know, is that we understand that was done under 14 Part 52 maybe in association with an ITAAC closure, 15 and so in this situation under Part 50 for the 16 operating fleet, I guess we're not sure we see that 17 same, see where that applies. You know, it seems like 18 that's done in Part 52 space, but not in Part 50 19 space.
20 MR. BOLLOCK: It can be done in Part 50 21 space. I mean, the vendor inspection group, like I 22 said, they are inspecting the vendor, but to help 23 focus the group, they've got a vendor inspection 24 program, a plan, yeah, where they plan and they 25 prioritize where they go, and that's why we'll include NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
48 1 some specific work, but they're inspecting the vendor.
2 MR. GALLETTI: Right, and historically, 3 the vendor program was Part 50. It existed well 4 before Part 52 was in existence.
5 MS. GOLUB: Okay, yeah, I think Warren's 6 right. You know, we have a presentation, and our 7 presentation kind of shows our perspective and maybe, 8 you know, we'd be happy to hear your thoughts on that.
9 MR. GALLETTI: Okay, next slide. I'll 10 just go through this quickly. So about a year ago, we 11 released this new inspection procedure, 35710, which 12 was specifically looking at some of the quality 13 assurance inspection software, again we reference in 14 QA 1, 2.7. There's a lot of similarity between what's 15 in 2.7 and what's in this guideline.
16 So for an application like this, we would 17 want to go and verify safety-related software. We 18 would be looking at the QA program, ensuring it 19 complies with Appendix B. So here at some of the 20 main, I won't go through each one, but the main 21 criteria of Appendix B that may apply to one of these 22 modification inspections that we would do.
23 I'll point to criterion three for design 24 control. Obviously the types of things we look for is 25 that the requirements that you've levied upon the NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
49 1 vendor are being translated properly. They are, you 2 know, implementing those requirements throughout the 3 life cycle of the design and development, and there is 4 traceability and things of that nature.
5 For criterion 10, test control, obviously 6 for like V&V activities or in-process testing, we want 7 to ensure that it's being controlled in a manner 8 consistent with Appendix B requirements and your QA 9 limitation, inspection procedures and plans, that sort 10 of thing.
11 MR. VAUGHN: So I had a quick question on 12 this. So this is a new procedure and the three types 13 of vendor procedures you have are routine, reactive, 14 and the new ones you mentioned. So it's going to be 15 a routine one, but it's an as-needed basis as opposed 16 to on this routine periodicity.
17 MR. GALLETTI: Right, this would be as 18 needed. If we were doing a digital I&C modification 19 inspection, we would be referencing this, but we do a 20 lot of other inspections that have nothing to do with 21 that, so we would be referencing this.
22 MR. VAUGHN: And I guess to just follow up 23 on Ray's question, you know, just going forward, if, 24 you know, many utilities had LARs following ISG-06, 25 would this as-needed IP be done every single time for NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
50 1 each one or -
2 MR. GALLETTI: We don't know, not 3 necessarily.
4 PARTICIPANT: I would hope not.
5 MR. VAUGHN: Yeah, we were just talking 6 about it at our internal meeting of how this would 7 unfold in the future.
8 MR. WATERS: Right, so let me on that.
9 I'm not sure if that's our view. I think what we can 10 all recognize is there are multiple inspection 11 procedures and programmatically what we're going to do 12 is look at that.
13 It's a good question. It's a question we 14 have too. What makes sense going forward? But I 15 appreciate the question. So in asking the question, 16 we'll take it into consideration, but we can't give an 17 answer -
18 (Simultaneous speaking.)
19 MR. HERB: And if I can frame why I asked 20 the question in just a short statement, and please -
21 MR. STATTEL: We do have experience in 22 oversight.
23 MR. HERB: Let me frame why I asked that 24 question because I have a fleet-wide single platform 25 that we're going to choose through our regular NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
51 1 process, and so I'll be submitting three LARs for my 2 three separate units, usually the exact same platform.
3 I'm just saying I would hope that the cost to review 4 would not be that first one times the cycle and times 5 the third one.
6 It would get smaller because you wouldn't 7 have to go back and re-review those pieces over, and 8 over, and over, and over again, and that's - so I'm 9 interesting in deriving the costs of actually doing 10 this because otherwise we're not doing any of them 11 because the cost is too much.
12 MR. WATERS: The answer is yes as long as 13 you get the first one right.
14 MR. HERB: I understand.
15 PARTICIPANT: Well said.
16 MR. HERB: Well said, yes, thank you.
17 PARTICIPANT: Right, I think -
18 MR. ARMSTRONG: And also if it's the same 19 scope.
20 PARTICIPANT: Right.
21 MR. ARMSTRONG: I mean, if it's the same 22 platform for the same use, if you're going to use the 23 platform for a total different use, I mean, we're 24 going to have to change the way we review all of that 25 and maybe accommodate that.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
52 1 MR. HERB: On the technical piece, yes, 2 but the programmatic piece, they're usually the same 3 part of the application, so I would have to say that 4 those programmatic pieces are probably the same.
5 MR. STATTEL: So talking to recent 6 experience, so we've approved several license 7 amendments for digital systems in the last couple of 8 years. Hope Creek installed a PRMS system. We did 9 not perform inspections on that system, right, and 10 that's an example of where we had several of those 11 applications that are in operation at the plant.
12 There are some that we have performed 13 inspections. We performed inspections at the Oconee 14 sites for the reactor protection system upgrades, 15 again, a first of a kind type of system.
16 So now, that's kind of an indicator that, 17 you know, it's not going to be 100 percent and it's 18 not going to be zero, right, but you also have to 19 acknowledge the fact that by issuing this license 20 amendment earlier in the process like we're saying, 21 we're really opening up a bigger part of the 22 development process to inspection, right?
23 So we're moving the line over, so the 24 inspections are going to be different, a different 25 scope, and I really can't speak to the frequency of NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
53 1 the inspections because, you know, we'll see once we 2 get into the process.
3 MR. FANELLI: So this is regional 4 inspections. I'm going to be talking about this.
5 These bullets up here are what the region does. We 6 ensure compliance with commitments and rights on the 7 basis that will be submitted with ours. Pretty much 8 it's just enforcement.
9 Whatever you've committed to, whatever 10 your license basis is, that's what we'll go inspect.
11 So all of these bullets for ensuring the licensee's 12 ensured development, these come right out of 13 inspection procedures that we may develop later on for 14 some of this.
15 The fourth bullet down is the one I would 16 jump to. We're going to, we more than like would 17 confirm by inspection that the licensee has 18 sufficiently performed oversight for all of these life 19 cycle functions down here just below that.
20 We also provide audit support for 21 technical reviewers. As you said, you've seen it 22 before. We also provide support for vendor 23 inspections also sometimes. Next slide.
24 These are two inspection procedures that 25 we know of now. They haven't been updated in a long NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
54 1 time, so they're not ready for use now, but we may 2 develop into an inspection procedure later on with 3 DIRS.
4 There's one that's not mentioned up here 5 is DOT 23, 6501 DOT 23. It's for human factors' 6 inspections. That might play a role in this too.
7 That's why I had to mention that. That's really all 8 there is to that. There's not much to it.
9 MS. GOLUB: So it sounds like as we're 10 looking at, your introductory sentence there that 11 says, "may be developed," so it sounds like you are 12 considering developing new inspection procedures or 13 maybe revising these procedures?
14 MR. FANELLI: I would say that's a correct 15 statement, but that would be working with DIRS, but 16 whether we would just update 52.03 and maybe combine 17 all of these or make a new one, I couldn't tell you 18 right now.
19 MR. DARBALI: Right, and we understand 20 some of these procedures are specific to Part 52 ITAAC 21 inspections. So whether those procedures get updated 22 to also apply to this new process, or we develop a new 23 inspection procedure just for the Alternate Review 24 Process, we have been talking about that being a 25 possibility in the near future. But definitely these NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
55 1 procedures -
2 MR. FANELLI: It's just what we've used in 3 the past.
4 MR. DARBALI: Right.
5 MS. GOLUB: Right, and no, and I think it, 6 yeah, and I appreciate it says, "may be developed in 7 consideration." So do you know when you'll be making 8 that decision on -
9 MR. WATERS: That is with members of DIRS 10 and we've had this discussion on needing to update, if 11 we need a new one or do something else. We've been 12 recently impressed with the ITAAC. We know it's not 13 ITAAC, but just a lot of parallels, and I think we 14 used the later one. Part of my thing is what's the 15 priority update and when do we do it? We want to look 16 into 2019.
17 I'll be honest. One of our continued 18 frustrations is that it's nice to have a little more 19 specific idea about what it look like and timing so we 20 don't have to work in a vacuum, but I think the answer 21 is yes, we're going to look at these. We don't have 22 a schedule yet for that, but I do see it happening in 23 2019 unless there are other visual I&C activities that 24 are required.
25 MS. GOLUB: Okay, so 2019.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
56 1 PARTICIPANT: It's something later.
2 PARTICIPANT: Yeah, but it's like -
3 MR. STATTEL: These -
4 PARTICIPANT: I'm sorry. Go ahead.
5 MR. STATTEL: These are relevant though.
6 I want to say that because if you look at the 2011 7 inspection procedure, essentially the ITAACs are 8 covering all aspects of the development process 9 including planning, implementation, and review of 10 design outputs, okay?
11 And I want to mention that because the 12 planning part of that procedure, we probably would not 13 need to apply for a system that's developed under the 14 alternate tier process because we're, that's part of 15 our licensing, right? We have that included, but the 16 implementation and design output side would apply, 17 okay, so I just want to mention that. I would expect 18 a future inspection procedure to be similar in those 19 areas.
20 MS. GOLUB: And so in 2019, so, I mean, 21 I'll just throw this out there. I don't know how the 22 process works, but industry would be happy to provide 23 input if we could or anything as you develop those 24 processes.
25 MR. WATERS: Yeah, and I don't know what NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
57 1 the process is for inspection review and potential 2 updates, but, yeah, part of the goal is to do this.
3 I figured this is one of the first means to get that.
4 MS. GOLUB: Absolutely, absolutely, yeah, 5 okay, thank you very much.
6 MR. VAUGHN: I feel like for routine 7 changes like the feedback forms and the inspection 8 procedures we just updated, but certain things like 9 when the assessment, you know, chapter was updated, it 10 had a lot of industry, a lot of public involvement.
11 But certain ones, the staff can decide 12 it's not for comment. It's just shared through like 13 the RP working group or whatever venue you want. You 14 share it at a public meeting, take it back a month 15 later and provide comments. So it's done. It's just 16 an informal way to do it through a public meeting.
17 There's no official way. It's not like a Federal 18 Register type of thing.
19 MS. GOLUB: But is that staff discretion?
20 Is that what you're saying, Steve?
21 MR. VAUGHN: Yeah, yeah.
22 MS. GOLUB: Okay, all right, so I didn't 23 know there was actually a process out there that could 24 be followed even at discretion, so that's good to 25 know.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
58 1 MR. DARBALI: Okay, so I think that is the 2 last slide in that presentation, so now we'll turn it 3 over to Pareez and Warren for your presentation.
4 MS. GOLUB: Yes, so we have a 5 presentation. Warren and Ray are actually going to be 6 doing the presenting.
7 MR. ODESS-GILLETT: So, the purpose of 8 this presentation is now that we are moving toward an 9 alternate review process or digital INC LARs, that we 10 have -- we wanted to let the staff know what industry 11 is doing in the area of design process.
12 And also to relay our expectations as a 13 result of the alternate review process, requiring a 14 licensee to step up the game in vendor oversight. And 15 documenting what that is in a plan as part of a LAR.
16 So, we'll cover both of these things. Ray 17 is going to review those activities that the industry 18 has developed for the digital engineering guide, the 19 digital design guide, and other standard industry 20 processes for the design that integrates vendor 21 oversight.
22 And Ray will talk about that. And then I 23 will talk about how we see how inspection would play 24 into it from our perspective. And see how that 25 matches up with the -- with our perspective.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
59 1 So with that, I'll turn it over to Ray and 2 the next slide.
3 MR. HERB: And then the next slide. Okay.
4 Thank you. Thank you Warren.
5 So this is a little bit of an overview of 6 the vendor oversight process and is new that came up.
7 As part of our design process, we've always done an 8 acceptance review of most of the design of documents.
9 But, there has been a gap that was noted 10 in the Oconee reviews and in potentially in some of 11 the earlier Diablo Canyon that the vendor oversight 12 piece, especially of the design processes, was a 13 little lacking.
14 And we've always had a vendor oversight 15 process in place for engineered components. And for 16 actual hard components like pumps and pelters, pump 17 casings, motors, like that, it's very easy to go and 18 take measurements and measure tolerances and look at 19 drawings and verify at the vendor's facility what you 20 actually have.
21 It's a little more fuzzy when it comes to 22 software. And so we committed as part of -- earlier 23 working with the staff on the alternate review 24 process, to further define what we mean by the vendor 25 oversight of those quality processes for software NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
60 1 development.
2 And so, as part of the industry standard 3 design process initiative, we have a procedure, IP-4 ENG-001. And it is implemented across the board at 5 all U.S. nuclear power plants for the design process.
6 And just recently developed is the NISP 7 procedure, NISP-EN-04. It is the -- it's really the 8 digital, how do you apply the standard design process 9 in digital? Okay.
10 It's currently under consideration for 11 making it mandatory across the U.S. nuclear fleet.
12 But, most of the large fleets have already committed, 13 including Southern, I think Duke, TVA, and I've got 14 Dominion and Exelon have all committed to implement 15 that in our fleet anyway.
16 Whether it comes out mandatory across the 17 industry or not. And so --
18 MS. GOLUB: And Ray, you have a pretty 19 significant hurdle to work with on Monday.
20 MR. HERB: Yes. It is. Yes, so on Monday 21 it went to the oversight committee for IMPO and they 22 basically said yes to this.
23 We believe -- they have not decided the 24 degree of when people have to make it. Whether it's 25 100 percent mandatory, or it's highly recommended.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
61 1 And so that procedure has already been 2 approved. It was developed in conjunction with EPRI 3 when they developed the revision to the digital design 4 guides. And it's now leaning to digital engineering 5 guides.
6 So, if you think of it, the process, the 7 forms, are standard in the ENG-001, the NIST is the 8 digital specific things that you must do to ensure 9 design quality and of a digital design. And it is a 10 fully graded approach with the highest gradation would 11 be of course, certainly a LAR.
12 And then the EPRI document, 3002-0118-16, 13 the digital engineer, is the how to. The how to 14 piece. Okay.
15 And then each of the licensees have their 16 own processes for vendor oversight. And so I'll talk 17 a little bit about what Southern's are at the very 18 end.
19 But, what I wanted to say is that 20 industry's design process, they required vendor 21 oversight planning long before the LAR is submitted.
22 It's really part of the supply chain purchase 23 agreement part, so.
24 Because you have certain requirements that 25 you -- like I say, we don't pay and pray. We like to NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
62 1 trust but verify in our supply chain. Because you 2 spend a lot of money on these systems, and you want to 3 make sure you get exactly what you paid for.
4 MR. DRAGOVICH: I just wanted to mention 5 too that we adopted a lot of the vendor oversight in 6 the BOP that you reviewed from the digital engineering 7 guide.
8 So, it's something that's we're using 9 that's already been established. We're not creating 10 new criteria.
11 But, one of the things is that we -- and 12 some of the comments we were given, we were led to 13 wonder whether the NRC considers vendor oversight to 14 include all the technical document reviews.
15 And we do that as a separate process as 16 part of our EC. So, when we go into our vendor 17 oversight, you'll see it's more process driving and 18 not technical drive.
19 Because we already cover the technical as 20 part of our document reviews, our owner's acceptance 21 reviews. So, that's something that we've always kept 22 separate.
23 The exception would be, is if we do a 24 review and we find a lot of gaps in the quality of the 25 vendor documents. Then we'll actually add that as a NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
63 1 line item for a vendor oversight visit to discuss 2 quality issues and things like that on the technical 3 side.
4 But otherwise, it's process driven 5 primarily. So, that's kind of why you'll see in our 6 VOP that a lot of the configuration control, things 7 like that are just checking the process.
8 MR. HERB: Neil, do you have anything you 9 want to add?
10 MR. ARCHAMBO: Oh, no. That was well 11 said. I have to agree with what's being considered.
12 MS. GOLUB: So it's kind of a revolution 13 to us, it's really great. And we thought today, maybe 14 there's just a divergence there.
15 It's not clear that the technical side is 16 being considered. It's just being considered, do you 17 need to do it now.
18 MR. WATERS: Yeah. My apologies. Are you 19 expecting a real time inspection to do the digital 20 inspection?
21 MR. HERB: The digital design process, 22 which is just standard procedure, is nuclear industry 23 standard procedure. That's what NIST stands for.
24 And EN-04 is the digital design process.
25 It actually it nests inside the IP-ENG-01, which is NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
64 1 the overall framework that governs designs.
2 You've got the conceptual design, how 3 meetings have to take place, how you gather inputs, 4 how you verify --
5 MR. WATERS: It's not just INC?
6 MR. HERB: It's not INC. It's the entire 7 standard design process, how designs are done in the 8 industry.
9 And the NIST is -- adds that digital 10 flavor. Because digital, you know, we already say 11 it's special. And in some ways, it just requires a 12 little additional extra effort and some additional 13 hazards that have to be addressed in the design 14 process.
15 And in some cases, specialized software, 16 there are additional things that are done. So, the 17 NIST is written to basically add those additional 18 assurances to the standard design process.
19 So, it's an additional procedure. And it 20 really -- it's about the -- the procedure size, I 21 think is about 25 pages long. And it has this big 22 long attachment to the back, which allows you to use 23 that as a grading process it says.
24 If it's a really complex system, and it's 25 highly risk significant, you need to do these things.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
65 1 If it's not so complex, and it's not so risky, you 2 need to do a lot smaller amount of stuff.
3 And so --
4 MR. WATERS: So, I mean - and I'm sorry 5 again. So, I understand what it is. It is just amped 6 up. I think you all can also go standardizing the 7 process into this.
8 MR. HERB: Yes.
9 MR. WATERS: And this is a really good 10 thing. Encourage that. And we are just saying less 11 NRC oversights, need it based on this.
12 I hope you understand that comment. I 13 think it challenges really nothing, you can benchmark 14 against it for lack of a better word.
15 MR. HERB: I understand.
16 MR. WATERS: My question is, are you going 17 to use this for 59.59 upgrades as well?
18 MR. HERB: Oh, this is used for all of 19 those changes.
20 MR. WATERS: All changes. All right.
21 MR. HERB: All engineering changes. He 22 could use a EC and everything.
23 MR. WATERS: It's not just for breaker.
24 Okay.
25 MR. HERB: Yeah. All design changes. We NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
66 1 call them design changes. You all call them 2 engineering changes.
3 I think we all have a different name for 4 them. But anything that we're -- everything that we 5 do that changes the plant requires 50.59. And 6 everything that's screened as a 50.59 most certainly 7 requires a modification at some level that impacts the 8 plan.
9 And again, there's several different --
10 you know, the standard design process has like a 11 commercial change. It has a limited changed. It has 12 a detailed change. And so it's graded as well 13 But this provides that additional digital 14 grading to that as well. And all digital 15 requirements.
16 MR. WATERS: Out of my ignorance, can you 17 give a run time of this? Whether they're going to do 18 a 50.59 or not, is there a feedback mechanism to 19 include this?
20 MR. HERB: Oh, yes. The standard design 21 process, the EN -- IP-ENG-01 is in Rev Zero. And most 22 of the industries are Rev Zero. But Rev One is out on 23 the streets.
24 And in the next three months we're 25 supposed to evaluate and implement it into our sites.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
67 1 MR. WATERS: I was thinking about the 2 lower ones. But yeah, that's right.
3 MR. HERB: And even the lower ones.
4 That's brand new. It's out on the street. So, we 5 haven't -- we haven't. You're right.
6 We don't have a lot of run time with it.
7 But, it essentially takes a proven standard design 8 price list and just adds an additional texture, 9 additional requirements for each one.
10 MS. GOLUB: And Mike, I'm just going to 11 mention, but I know this is of interest to the 12 industry. And some of this has already happened.
13 Where EPRI through their MOU with research 14 has already done, and I'm calling it training, but 15 it's mostly just showing the NRC what's in that 16 digital engineering guide. And how the industry is 17 going to be working.
18 And so we're hoping to do more of that.
19 MR. WATERS: I just wanted to -- still, 20 we're not -- we've endorsed it. And I don't really 21 call it training, but we understand what's in there.
22 MS. GOLUB: It's really just sharing 23 information.
24 MR. WATERS: So, we understand what 25 similar systems have an effect. That's sure of it.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
68 1 I appreciate though. I endorse it.
2 MR. HERB: Right. I think the message --
3 the message I'm trying to present right here is that 4 we have a standard process. And that standard process 5 includes a vendor oversight piece.
6 And so everybody who comes to the NRC with 7 a LAR will be working from the standard process. You 8 should expect to see something very similar from 9 everybody. Okay.
10 MR. DRAGOVICH: But also just to note too 11 that this digital engineering guide is a new EPRI 12 report number. But there was an additional design 13 guide.
14 And that's the feedback that came back to 15 make this a new and improved engineering guide.
16 MR. HERB: That's right. And that's been 17 around since 2014.
18 MR. DRAGOVICH: Yes.
19 MR. HERB: And we've always -- we've 20 always implemented that in our -- to inform our 21 digital designs at the plants.
22 Not just safety related, but all designs 23 that are related to digital, came out of it, that 24 original DEG it was called. So this is just a 25 revision of the DEG.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
69 1 MR. WATERS: Um-hum. Would your -- well, 2 last question here. Would your vendor oversight plan 3 in itself that you submit on the backup commit to 4 these?
5 MR. HERB: The vendor oversight plan would 6 -- usually that's --
7 MR. WATERS: Not unless you've already 8 started.
9 MR. HERB: There is guidance in the EPRI 10 digital engineering guide that says you -- it says 11 this is when -- well then this does. This is when you 12 should develop your vendor oversight plan.
13 And it says -- and then it goes to the DEG 14 to say, here's the guidance. And the DEG says, use 15 commercial grade dedication. Use critical digital 16 reviews. Those are the elements that you would go 17 into the actual things that you should be looking for 18 and inspecting.
19 And it references some -- also some actual 20 IEEE standards and what not. And so those things all 21 inform what is looked at and what is required to be 22 looked at.
23 But mostly, the vendor oversight planning 24 for the LAR, is going to be informed by their LAR 25 itself. By Section D.4, by the system requirements NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
70 1 that we submit as well.
2 MS. GOLUB: But the BOP section of the 3 digital engineering guide, I mean that -- there was a 4 lot more on it too.
5 MR. HERB: It's very -- it is very broad.
6 MS. GOLUB: And this inner level was added 7 at this last revision.
8 MR. HERB: That's right. It's 9 significantly improved. And like I said, vendor 10 oversight has always been a part of this process. Not 11 just digital, but for regular design too.
12 And it just says, that you need to 13 consider all of these things. And one of those now is 14 licensing commitments and NRC commitment questions.
15 MR. GALLETTI: Can you -- you mentioned 16 something that I thought I heard. But, new licensees 17 for your vendor oversight, which you've been doing for 18 a long time, that's generally programmatic.
19 And then for technical review, you have a 20 separate independent process.
21 MR. HERB: Right. Design acceptance of 22 output documents only. So, when we purchase a 23 component or services from a vendor, and that's what 24 this will be, we're purchasing their engineering 25 services to develop our application.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
71 1 We get certain output documents from them.
2 We have an acceptance process and review. It's part 3 of the design process that we go through.
4 And that maybe, you know, a matrices, 5 requirements, you know, checking it to our actual 6 requirement documents.
7 MR. GALLETTI: Okay. But --
8 MR. HERB: That's a technical piece rather 9 than a process. Look, the vendor oversight's always 10 been like almost like a NIST -- I mean, a NUPIC type 11 of audit.
12 But it will be super-sized for software, 13 you know, in our case. Because oftentimes NUPIC, you 14 know, they look at a lot of the QA processes and 15 they're not specifically looking at 2.70 inquiry type 16 stuff.
17 So, we're adding those things as well.
18 MR. GALLETTI: So, the technical stuff 19 will be added in you said?
20 MR. HERB: The technical stuff is done.
21 But it's done not necessarily as part. It may be 22 under the umbrella of the vendor oversight.
23 But it really is already part of our 24 process with the acceptance of those documents, those 25 reviews.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
72 1 MR. DRAGOVICH: Yeah. So like if a vendor 2 prepares a calculation, you know, we'll have that in 3 our schedule to review that performing only on 4 acceptance. We don't wait until a vendor oversight 5 visit to do that review. It's just part of our design 6 process.
7 The vendor oversight is strictly just 8 going through -- for example, let's say they're 9 building a cabinet and we might decide to do that 10 vendor oversight at a time where we want to look at, 11 inspect the quality of the cabinet being constructed.
12 But it just is more of a process. Just 13 making sure that they're following the process. The 14 technical is driven from the engineering change in the 15 schedule.
16 And it's not necessarily going to coincide 17 with the vendor oversight visit, unless there's a 18 problem where we have to go and inspect because of 19 that, you know, degraded condition, whether it's a 20 caliper or a drawing or whatever.
21 So, but that's just kind of independent of 22 the vendor oversight that we have in our VOP here.
23 MR. FANELLI: So you don't set it up for 24 vendor oversight?
25 MR. DRAGOVICH: No. We have that -- I NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
73 1 mean --
2 MR. FANELLI: I mean, I'm not talking VOP.
3 MR. DRAGOVICH: Right.
4 MR. FANELLI: I'm talking vendor oversight 5 in general. The criteria is set already.
6 MR. DRAGOVICH: Right. No. No, that's 7 just part of what we would normally do as part of our 8 standard design process.
9 MS. GOLUB: I don't know if it's a 10 terminology thing, but --
11 MR. DRAGOVICH: And that's why we sense 12 there was a little bit of misunderstanding with the 13 comments we got.
14 MS. GOLUB: Yes. Yes.
15 MR. DRAGOVICH: That we thought maybe the 16 NRC considers that vendor oversight. And you're 17 asking, why don't we discuss any of those?
18 Well, that's because we have it in our 19 design process already that -- where we address that.
20 It's part of --
21 MR. BURZYNSKI: It's more of a criterion 22 for when we design the inner-space attributes that 23 have been for oversight.
24 MR. DARBALI: So, I guess the question is 25 --
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
74 1 MS. GOLUB: Does that answer your 2 question?
3 MR. HERB: Well, we are doing all those 4 things.
5 MR. GALLETTI: I understand. I think that 6 we have maybe fundamentally a disconnect in what we 7 considered to be, you know, oversight of the vendor.
8 Because certainly in these technical 9 output documents produced by the vendor, you know, to 10 be reviewed and approved, we see that as a vendor 11 oversight activity.
12 MS. GOLUB: Oversight. Yes.
13 MR. HERB: Right. We're just trying to 14 let you know that we are doing that too. It just 15 maybe not underneath the umbrella of the vendor 16 oversight guidance.
17 MR. BOLLOCK: Actually, it's part of your 18 engineering changes. It's the engineering design 19 change process. Okay.
20 MR. HERB: It's part of our engineering 21 design process going forward.
22 MR. BOLLOCK: Yeah.
23 MR. HERB: We are still doing both.
24 MR. VAUGHN: It's just a label. Because 25 it --
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
75 1 MR. HERB: We may not be -- you may have 2 to review all the licenses.
3 MR. BOLLOCK: Yeah. So when we did the 4 pilots, the first engineering pilot at Oconee, that's 5 what we ran into. The technical parts were in the 6 engineering change DC.
7 And, you know, we had a couple of vendor 8 inspectors that were looking at the procurement 9 documents. And didn't realize that it was just how 10 you package it, where you kept it was in your EC that 11 had that.
12 Yeah. They're looking at all the gap --
13 they had to look at that.
14 (Simultaneous speaking.)
15 MR. HERB: And it's all part of the big 16 picture. If you notice, --
17 MR. BOLLOCK: We just printed it 18 separately.
19 MR. HERB: Yeah. If it -- if deficiencies 20 are noted, and it's --
21 MR. BOLLOCK: It was big. And yeah, they 22 did.
23 MR. HERB: Then you write a corrective 24 action report. And it comes out --
25 (Simultaneous speaking.)
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
76 1 MR. BOLLOCK: Because that's part of.
2 That all is part of --
3 MS. GOLUB: Right. Is part of that whole.
4 Yes.
5 MR. BOLLOCK: And so we asked you for, you 6 know, we want to look at your procuring set of 7 documents. That -- we would expect that as well as 8 being treated.
9 MR. HERB: And so, you know, with that 10 thought, you know, the actual -- you know, because we 11 -- there was a question about, how big is this plan 12 compared to like what the review is.
13 If you included all that piece it would be 14 massive.
15 MR. BOLLOCK: Yeah. Then it would be.
16 MR. HERB: It more than that. It would be 17 thousands of pages.
18 (Simultaneous speaking.)
19 MR. STATTEL: But put this in perspective 20 now. Because that's -- when we started this, on SD6, 21 when we started this, right, we said we're going to 22 issue the license amendment prior to what we 23 traditionally perfom -- these activities traditionally 24 performed.
25 And a lot of those are related to NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
77 1 reviewing specific artifacts of the design. Right.
2 Design output artifacts. Right.
3 And so, that is document review. Now, we 4 were told that -- so, since they were creating this 5 big hole, because we're no longer performing those 6 activities, and we were told that that hole would be 7 filled by these vendor oversight activities that would 8 be discussed in the vendor oversight plan.
9 MR. HERB: Yep.
10 MR. STATTEL: And so kind of what we're 11 seeing is this doesn't entirely fill that hole. And 12 you're telling me that too.
13 MR. ODESS-GILLETT: It does. Because the 14 vendor oversight plan summary, you'll notice there is 15 a statement about reviewing -- how did it get worded 16 Steve?
17 The owners --
18 (Simultaneous speaking.)
19 MR. DRAGOVICH: Oh, yeah, yeah. But it's 20 this.
21 MR. ODESS-GILLETT: It is in there. It's 22 just that it's not elaborated because it's part of 23 their standard engineering process.
24 So it is referenced in the vendor 25 oversight plan summary.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
78 1 MR. FANELLI: So what you're saying is the 2 process is there. But these are the outputs to that 3 process.
4 MR. HERB: We're not going to duplicate 5 this, you know, this compliance matrix to our purchase 6 order in the documents for vendor oversight documents.
7 So, our vendor oversight documentation 8 would consist of our audits of the vendor. Are they 9 following the process? Maybe that's right, it's just 10 some of the things that you do.
11 But those -- the actual output documents 12 would be part of the engineering change package. It 13 would be part of the design change package and in our 14 vendor documentation, we just put it into our 15 documenting whole system.
16 And so those pieces are inspectible by the 17 NRC. And you come and do mods inspections all the 18 time and you look at those components at that time.
19 And so, we're not saying you're not 20 allowed to look at them. We're just saying is that 21 that's not going to be part of our vendor oversight 22 report.
23 Our vendor oversight report is going to 24 be, we went to the vendor for these five different 25 trips. We looked at their process. Is this what they NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
79 1 said they were going to -- how they were going to 2 develop our process?
3 We inspected them to it. We believe 4 they're following it. And we're -- and as of the 5 final verification, we're looking at the output 6 documents and we're seeing those.
7 But, that output --
8 (Simultaneous speaking.)
9 MR. HERB: That owner acceptance of those 10 documents is part of a different team. The technical 11 team is doing that piece.
12 And if they notice problems, they just 13 flow back into the vendor oversight process to say you 14 know, what? Something's happening.
15 They said they were following their 16 process. We're getting -- we're getting substandard 17 documents out. So, something is wrong.
18 So we would go back. We would escalate.
19 So we may actually put a person at the vendor's site 20 to make sure that they were doing what they said they 21 were going to do.
22 So it has escalation involved as well.
23 MR. FANELLI: One question. By leaving 24 this out of your VOP, is what I'm hearing you say, 25 except for that statement you were talking about.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
80 1 Then would you not be committing to doing 2 these things in your license amendment? Or would you?
3 MR. HERB: Well yeah. I mean, you've got 4 to put the design in this --
5 MS. GOLUB: And it's a formal regulatory 6 commitment. So it's not -- it's not just sort of 7 mentioned by chance. I mean, it's a formal commitment 8 which the ISG-06 process guides you to make in order 9 to use that.
10 MR. SCHRAYE: But part of our procedures 11 that we're developing and stopping of course because 12 the vendor assessment is part of that. I mean, that's 13 one of our licensing basis.
14 MS. GOLUB: And we're formally committing 15 to it. So, it's --
16 MR. STATTEL: When you say formal 17 commitment, what specifically are you talking? What 18 are you formally committing here? We need to 19 understand.
20 MR. HERB: When we get our LAR, our 21 license amendment, that becomes part of our licensing 22 basis. And any changes that we make to the plant get 23 compared against our licensing basis through the 50.59 24 process.
25 Now, I'm not suggesting that we would NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
81 1 change the direction of the component we haven't even 2 installed yet. But, there may be minor things that 3 come up.
4 And if they fit within that 50.59 process 5 review, then there's two outputs. We have to resubmit 6 the LAR because we've suddenly broken the boundaries 7 of what was accepted and was approved.
8 And -- but if it doesn't, then that goes 9 into our process --
10 MR. STATTEL: I think that's a different 11 topic then what I was asking about.
12 MR. HERB: And so -- and so that again, 13 those final design documents, those final 14 implementation pieces, those are all reviewable 15 underneath the Mod and the inspection process.
16 MR. DARBALI: Yeah. But Rich is asking 17 about the commitments that you would be making in the 18 LAR.
19 MS. GOLUB: Yeah. I'm just trying to open 20 up that section.
21 (Simultaneous speaking.)
22 MR. STATTEL: But this is outside of the 23 scope of vendor oversight. So, in a sense, when I 24 look at what we do after basically implementation and 25 design efforts. I look at the activities we do.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
82 1 Now, you can put them under the category 2 of vendor oversights or not. Right. It doesn't 3 matter.
4 I'm looking for an equivalent activity 5 that -- I understand, a licensee is taking 6 responsibility for performing those activities.
7 I'm looking for equivalent guidance, 8 equivalent activities that are going to be performed.
9 And a commitment to perform those. And that's what 10 I'm not seeing right now.
11 I need to get to a place where I'm 12 confident that you're accomplishing the same thing 13 that we would have accomplished if we were performing 14 those activities in accordance with 50.52.
15 MS. GOLUB: Yeah. So, that -- I mean, 16 this is our meeting from the -- I'm reading from the 17 -- I'm reading from the last version of the ISG that 18 I have access to. Which was the public, you know, the 19 public version from July.
20 And under C.2.2, which is licensee 21 prerequisites, Item 3B. And I don't know how it's 22 currently worded, Samir.
23 But, the LAR should include appropriate 24 regulatory commitments to complete life cycle 25 activities under the licensee's QA program Note that NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
83 1 these activities would be included in a Tier One, Two, 2 or Three licensing review.
3 Which is what you're talking about Rich.
4 But are not covered in the alternate review process 5 because they take place after the NRC staff decides 6 whether to issue or deny the license amendment.
7 But the point is, those are the activities 8 that the licensee is committing --
9 MR. DARBALI: And from related equipment.
10 MS. GOLUB: Right.
11 MR. STATTEL: I'm still missing the 12 guidance, right. So we have 50 pages of guidance on 13 how to perform those activities.
14 And we will no longer be performing those 15 activities. Where is the equivalent of that guidance 16 that we have for the licensees to perform the same or 17 similar or equivalent activities?
18 It's not in the vendor oversight plan what 19 I'm seeing here.
20 MR. HERB: It is -- the vendor oversight 21 plan is to -- they have a -- we have signed up through 22 a purchase order, these set of requirements to be 23 implemented through this process that was defined in 24 Section D4.
25 Now, we will, through normal audits and NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
84 1 inspections of the vendor oversight, check to make 2 sure that they are following their process. And we 3 will confirm the outputs of those design activities, 4 to make sure that they're complete and, you know, 5 appropriate for what, and they match the requirements 6 of what we asked them to do.
7 Which we also submitted to alter for 8 approval. And so, yes, we have to make sure that the 9 -- that the application is developed and designed in 10 accordance with our highly detailed specifications.
11 Do we have a procedure for doing that?
12 What's currently design change procedure. And you 13 know, defined what those are. And you accepted. And 14 you verified that you got what you designed.
15 And so, I don't know if there's anything 16 -- I mean, you're not going to go there.
17 MS. GOLUB: Steve?
18 MR. DRAGOVICH: So, I was going to say, so 19 when I prepared this, I went through and brought what 20 our understandings of the VOP were included. And then 21 we got some comments saying no, we should include some 22 of these other reviews that Rich was mentioning.
23 So, we can take can action to talk about 24 that. And determine if we want to enhance that to 25 include all that detail if we don't feel it's present NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
85 1 somewhere else in the LAR.
2 You know, in other words, if we talk about 3 it somewhere else then maybe we don't need to. But 4 otherwise, maybe it would be worth it to have that 5 technical component in this.
6 MS. GOLUB: Yeah. Or at least describe 7 how that work is being done. Even if we don't, you 8 know, if we can show where it's been done elsewhere 9 within the licensee's programs.
10 MR. HERRELL: And document that reference 11 material in the VOP.
12 MR. STATTEL: We're going to talk specific 13 -- more specifically about this VOP that we've -- some 14 of us later on?
15 MS. GOLUB: Um-hum. Yes.
16 MR. STATTEL: Later on?
17 MS. GOLUB: Yes.
18 MR. STATTEL: Okay. Well, I'll defer 19 until then.
20 MS. GOLUB: So maybe we should let Warren 21 -- or Ray --
22 MR. HERB: Yeah. Let me just finish the 23 document. Because this -- really because this is just 24 pointers to this to how the VOP is developed.
25 And it really covers -- it covers the NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
86 1 process piece and the design output piece is covered 2 by the design. But we didn't cover that here. We 3 don't talk about the design output piece.
4 But, again, the standard design part, 5 sorry, I kind of already talked about all of these 6 slides.
7 MS. GOLUB: Maybe we should do this more 8 quickly.
9 MR. HERB: You can flip through this.
10 MR. ODESS-GILLETT: Well, let's talk about 11 where the VOP falls into each other on this. If you 12 can go back one.
13 MR. HERB: Go back one then.
14 MR. ODESS-GILLETT: Okay. It says the STP 15 vendor oversight plan implement --
16 MR. HERB: Right. It just -- so, the 17 standard design process turns on vendor oversight 18 planning, is partly determined in the early conceptual 19 design phase.
20 There are contractual agreements and, you 21 know, we set actual times and negotiate option access 22 to their processes at that time. And do all the 23 select what we believe to be appropriate, based on the 24 LAR submittal, those practices.
25 Now, when the LAR gets submitted, these NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
87 1 things will already be done. So, we're going to -- as 2 part of our bid proposal and the acceptance, these 3 things are all going to be negotiated.
4 But when we submit that bid, it's on it.
5 It's really a lot of the requirements that we already 6 going to have in the LAR.
7 So, it's not like we're thinking about it 8 later after all this other stuff. You know what I'm 9 saying. So, it's baked into the standard design 10 process.
11 And it's not just digital. We -- these 12 are commercial terms and agreements between us and the 13 vendor. This protects both us and the vendor going 14 forward. And so we have to do due diligence. So, 15 next slide.
16 And now they get very specific in the 17 NIST. This is what I'm saying. If we're talking 18 about specific things we're looking at in process and 19 specific things we're looking at in how we accept a 20 document, the NIST goes through that and talks about 21 those.
22 And so the ones we believe that are 23 primarily process oriented, we need to go and to 24 verify the quality or the quality design process. We 25 will call those out.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
88 1 And we use that guidance there from the 2 DEG. And again, EPRI has discussed the DEG with the 3 research division. And I believe there was like on 4 November 8 they had a training session.
5 And I have --
6 MS. GOLUB: I think it was an information 7 exchange.
8 MR. HERB: But when this -- when this --
9 when the IMPO and the design oversight working group 10 make this either an optional or a mandatory, there 11 will be a roll out in the change management process 12 and training.
13 So, I would think that like NRC is welcome 14 to come to those. And just kind of like in the RIS 15 attachment. I was moving a lot for you all to come to 16 see what we're doing.
17 And I would get there from the digital 18 design piece.
19 MS. GOLUB: Especially because we, you 20 know, I think we're using different terms or, you 21 know, --
22 MR. HERB: That's right. Because clearly 23 we're not talking on the same words when we say 24 things.
25 MS. GOLUB: Right.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
89 1 MR. HERB: But that's available. It's the 2 next slide. In the industry standard guidance. And 3 basically a lot of it comes out of commercial grade 4 dedication of commercial items.
5 And so a lot of that assessment is there.
6 There's also the critical digital review guidance for 7 things we look at.
8 And then there is, as Pareez said, from 9 the DDG to the DEG, the size for vendor oversight grew 10 by about three times the size of what it was before.
11 So, it became a lot more detailed. So next slide.
12 At Southern Nuclear we've always had a 13 vendor oversight planning process for a long time.
14 And as part of our site procedures, or our fleet 15 procedures, vendor oversight starts during the 16 negotiation, part of the purchase order.
17 And it's usually the plan is negotiated 18 pre-award. And then post-award it's finalized with 19 dates and visits and scope of what's going on.
20 And this would certainly be a big piece 21 going forward for a LAR. Because a lot of it falls on 22 us to demonstrate to the NRC that we're doing our due 23 diligence on ensuring quality software.
24 And again, it's -- all the outputs from 25 the vendor oversight plan, those audits, and those NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
90 1 inspections, and those activities.
2 You know, and those would include 3 possibly, you know, reviews of different testing plans 4 for the -- part of a QA record. And it certainly is 5 auditable or inspectible by the NRC.
6 And when we go on some of these vendor 7 oversight visits, I would think that, you know, if the 8 NRC was interested, we would love if you all went at 9 the same time we went. That way, in a kind of an 10 observer mode, so you can see what we are doing and 11 get comfortable with how we are doing our inspections.
12 And then if you thought that there was a 13 gap there, you could, you know, let us know. Or maybe 14 you'd have to do your own as well.
15 But, so then it's -- the vendor oversight 16 is part of the procurement process. It supports the 17 design by insuring quality at the procured 18 performance.
19 I mean, that's -- this is a not a digital 20 versus a non-digital thing. It's really part of our 21 process.
22 And this was -- this came out of the non-23 digital side. And we have actually incorporated all 24 of that on our site to then incorporate those 25 learnings.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
91 1 And accepting through IMPO and through 2 other OE type organizations of the industry, we feed 3 upon each other. We learn.
4 We -- we're required as part of our design 5 change process to look at OE. And to answer where 6 those thought failings were. So, we were just part of 7 this process.
8 If we were not the first one, you know, we 9 would look at all the OE from the Diablo Canyon, from 10 Oconee, and some of these things. And all of those 11 things that were maybe bumps in the road for those 12 earlier ones, we would feed that into this process.
13 So, we're not creating this out of whole 14 cloth. We're building upon each other. And so some 15 of these earlier ones might be a concern. They might 16 be a learning thing.
17 But the next ones we believe start getting 18 better and better. So the next slide. So, that's it 19 for me.
20 MR. ODESS-GILLETT: So any questions on 21 the industry's standard design process digital 22 engineering guide and digital engineering process 23 before we proceed?
24 MR. DARBALI: No. I think we're -- we see 25 these as moving in the right direction. And we, you NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
92 1 know, we're in the learning process and so future 2 engagements would help us better understand it.
3 MR. HERB: And sitting in on some of that 4 training, you know, might help you get a little better 5 feel about where we're doing some of those technical 6 reviews that we're not going to like and that were not 7 part of this vendor assessment.
8 And if we have to tweak our vendor 9 assessment, to be --
10 MS. GOLUB: To be -- right. To describe 11 some of that in more detail.
12 MR. HERB: We can do that.
13 MS. GOLUB: Then we should. Right. To 14 make them -- to make sure that the NRC is comfortable 15 that those activities are happening.
16 Even if, you know, the utility's, you 17 know, programs, they don't call them vendor oversight.
18 But we could describe that more fully.
19 MR. HERB: You could still describe it 20 around this piece.
21 MS. GOLUB: Yeah.
22 MR. HERB: Because I think part of our 23 vendor oversight plan we also say we were going to 24 like verify PSIAs are implemented appropriately and 25 inspected.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
93 1 And that's typically not part of a vendor 2 assessment.
3 MS. GOLUB: But I think Rich is, you know, 4 Rich's comment on those design outputs and how those 5 were reviewed, I think that's something we could 6 enhance.
7 MR. HERB: Yeah. We would enhance that.
8 MR. STATTEL: Yeah. So, I'll just leave 9 with a -- leave this with a comment. And we'll get 10 into more discussion later.
11 But, my experience, and I've been involved 12 with a lot of digital modifications. The activities 13 that we have gone out to the vendor and performed, I 14 do not -- I have not seen licensees doing those type 15 of things.
16 So it maybe because they see that we're 17 doing them. So they don't do them. Right?
18 MR. HERB: That could be.
19 MR. STATTEL: That could be. I do agree 20 that they should be. They should be doing them. I do 21 agree with that.
22 I think it would be a good thing. But I 23 don't see anything in the processes or the vendor 24 oversight plan that gives me confidence that they will 25 do those things.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
94 1 MR. HERB: Right.
2 MR. STATTEL: I do have several examples.
3 But, we can talk about those later.
4 MR. HERB: Okay.
5 MR. ODESS-GILLETT: Yeah. Let me sort of 6 address that in the introduction to this. Because I 7 think, the industry concurs that on some of these past 8 projects or upgrades that the licensee has not met NRC 9 expectations under oversight.
10 But, we're seeing the convergence of two 11 different initiatives happening at the same time. The 12 one side is this standard design process that 13 integrates vendor oversight. It's never been done 14 before across the industry.
15 And the other is this alternate review 16 process that is basically placing the expectation on 17 the licensee to step up their game and do their vendor 18 oversight. And we're giving the licensee the 19 opportunity to step up to that game.
20 MR. STATTEL: Um-hum. And I'm not saying 21 that the licensees have been negligent or anything 22 like that.
23 MR. ODESS-GILLETT: I understand.
24 MR. STATTEL: In a lot of cases, they just 25 don't know the processes. They don't know to go and NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
95 1 look for these things.
2 MR. ODESS-GILLETT: Right. And that's --
3 MR. HERB: And the learnings that we're 4 seeing now. And we are folding that into a standard 5 process with the whole industry.
6 So it's not like you have to go back to 7 each one of those. Those learnings are now 8 distributed across everybody. So, you just share 9 design process.
10 MR. ODESS-GILLETT: Okay Joe, can you go 11 to the next slide? Uh-oh. And that concludes my 12 presentation.
13 (Laughter.)
14 MR. ODESS-GILLETT: All right. So, these 15 are the two initiatives that are occurring each from 16 the NRC side with their revised ISG that has the NRC 17 view of the LAR concluding before the detail design 18 implication test activities are completed.
19 And that the licensee is taking over that 20 role of vendor oversight to the equivalence of what 21 the technical reviewer from the NRC staff performed.
22 That's industry's expectation of the vendor oversight.
23 And so we agree. We're stepping up the 24 games with the licensee's expectation of vendor 25 oversight. So that what was done by the technical NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
96 1 reviewer will now be done by the licensee.
2 That's the expectation. On the industry 3 side is where it's explained. The industry is 4 standardizing their design process, even digital mods 5 and the design process for digital mods that 6 incorporates vendor oversight.
7 So these are the two initiatives that are 8 converging today. Next slide.
9 So, with this convergence from the 10 industry side, we see maybe a new paradigm in looking 11 at inspections. So, with the new draft ISG-06, and 12 the LAR review, the NRC is going to review the vendor 13 oversight plan.
14 And the NRC is going to make sure that 15 that vendor oversight plan is -- meets the 16 expectations of the NRC staff for adequate oversight.
17 So, before we even complete the LAR 18 review, we'll have a vendor oversight plan that meets 19 NRC expectations of what is adequate for the licensee 20 to perform that vendor oversight.
21 As I said, the industry is standardizing 22 their process. It incorporates vendor oversight. And 23 with the LAR technical review, the audits are now 24 going to focus on those things that we think are 25 really important.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
97 1 The system, the architecture, the plant 2 interfaces, the human faces, and the plant safely 3 impacts. We see those are really the safety 4 significant aspects of a LAR review for our digital 5 safety-related mods. Okay.
6 NRC inspections of digital INC license 7 amendments in our opinion, can adjust too this new 8 (phone noise) industry prepared them. And I'll in the 9 next slide show you what we think that paradigm would 10 look like.
11 MR. GALLETTI: Warren, before you go.
12 Could I just ask you to go back to that last slide.
13 MR. ODESS-GILLETT: Yes.
14 MR. GALLETTI: So the bullet that says NRC 15 audits new focus.
16 MR. ODESS-GILLETT: Right.
17 MR. GALLETTI: So from the industry 18 perspective, what do you think the current audit focus 19 is?
20 MR. ODESS-GILLETT: Well, they only had so 21 many resources and so much time in order to inspect or 22 to audit not only these important things, but all of 23 the process aspects through detailed design, 24 implementation, and test.
25 So, this by focusing more on the -- what NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
98 1 we consider the safety significant aspects of the mod, 2 we make sure that we have a safe implementation of the 3 mod.
4 It was interesting, at the last hearing, 5 Commission hearing on digital INC mods, did you -- I 6 don't know if anybody listened to that MIT professor.
7 But he basically said that industry at this point 8 knows how to implement requirements and software.
9 Okay?
10 The biggest safety concerns are unintended 11 consequences as a result of bad requirements. That 12 that's the biggest danger to our industry.
13 So, to us that should be the focus of the 14 regulatory review. Because at this point, after 40 15 years of doing digital INC, we feel that it's not just 16 safety significant to be able to transfer requirements 17 into a design in software.
18 And so that's --
19 MS. GOLUB: I'm sorry. The hard part is 20 developing.
21 MR. GALLETTI: The finding of the 22 requirements.
23 MR. ODESS-GILLETT: The finding of the 24 requirements. So that's -- that's right. That's what 25 we're referring to.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
99 1 MR. VAUGHN: But was your question, what 2 was NRC's audits before? Because you said new. So, 3 you're saying that they were changed?
4 MR. GALLETTI: Well, I think Warren 5 answered the question.
6 MR. VAUGHN: Okay.
7 MR. GALLETTI: I was looking for the 8 industry perspective on what we have currently been 9 doing that's different from this.
10 MR. STATTEL: In all honestly, I mean, 11 I've been trying to think about what it would -- what 12 a license review in the future would look like.
13 I don't even know that we would need to do 14 audits. At all. Because pretty much everything will 15 be -- because most of what -- 99 percent of what we do 16 on a typical audit for a digital INC system, is now 17 being transferred to the licensee.
18 MR. GALLETTI: Well, you've planned.
19 There's the requirements' pieces.
20 MR. STATTEL: In the case of Diablo 21 Canyon, we did some early audit, early stage audits.
22 And our -- what we learned was, they were premature.
23 Because they hadn't progressed far enough 24 through the implementation for the audits to be 25 useful. So, we ended up doing a second set of audits NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
100 1 at the end of the process.
2 But, clearly, we wouldn't have that 3 opportunity in the future. And for an alternate 4 review.
5 MR. WATERS: When you use the word audit 6 on a stability, are you talking about NRC license 7 reviews, an audit process or inspections? What are 8 you talking about?
9 MR. ODESS-GILLETT: Audit would be during 10 the technical review of the LAR. It would be the 11 technical review staff's audit of the system 12 requirements specification, the plans for digital 13 development as described in D4 and ISG-06.
14 So these are, you know, the LAR will 15 describe these things. But, the NRC staff has its 16 prerogative to say, well, let me look at those plans 17 that you described in D4 and the ISG-06.
18 Let me see your system requirement 19 specification. Because it should be developed as part 20 of the LAR submission. And we consider those audit 21 material.
22 MR. WATERS: How do I judge plant CD 23 impacts based on what those things are?
24 MR. ODESS-GILLETT: Your regulatory 25 requirements for review. IEEE 603, your ISGs on data NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
101 1 communication, your GDCs on separation independence.
2 MR. WATERS: Is that the requirements?
3 MR. ODESS-GILLETT: Yeah.
4 MR. WATERS: Okay.
5 MR. ODESS-GILLETT: Yeah.
6 MS. GOLUB: Well, and the conceptual 7 design. Your architectural.
8 MR. ODESS-GILLETT: Yeah. Architecture.
9 MS. GOLUB: There's more material in the 10 LAR then just the requirements.
11 MR. WATERS: So it hinges on your 12 requirements then.
13 MR. ODESS-GILLETT: It hinges on the 14 requirements, but Pareez is right, the LAR is going to 15 present the architecture.
16 MR. FANELLI: How complete would the 17 requirements be by then?
18 MR. HERB: Complete.
19 MS. GOLUB: Complete.
20 MR. ODESS-GILLETT: Complete.
21 (Simultaneous speaking.)
22 MR. HERB: When we write a PO, it's going 23 to be based on a complete set of requirements. Now, 24 there's a negotiation and normalization when you put 25 out a bid specification.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
102 1 You select the vendor. You select the 2 platform. You normalize that requirement. And when 3 you write that PO, that last normalized step, that's 4 what will go out in our LAR.
5 That's what we've purchased. And that 6 will go in the LAR.
7 MR. ODESS-GILLETT: I'd have to 8 respectfully --
9 MR. HERB: With some of the steps.
10 MR. ODESS-GILLETT: Yeah. I'd have to 11 respectfully augment what my licensee colleague is 12 saying that once we get a purchase order, the vendor 13 would have to write a functional specification.
14 MR. HERB: Okay. Yeah.
15 MR. ODESS-GILLETT: A system requirement 16 specification, a system design specification.
17 MR. HERB: But those are all done before 18 the LAR is submitted.
19 MR. ODESS-GILLETT: They're all done 20 before the LAR submittal, correct. But after a 21 procurement contract.
22 MR. HERB: Yeah. You're right. You're 23 right. Yeah.
24 MR. DARBALI: So the scope of the audits.
25 That depends a lot on each application. And there is NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
103 1 a chance that no audit is needed, or only an 2 electronic audit is needed.
3 But you could have the case where the 4 licensee submits the LAR, but by the time the LAR is 5 expect -- the review is expected to be completed, you 6 haven't gone to testing, but you've gone through 7 fabrication.
8 So, those are some activities that the 9 staff could audit if you transferred.
10 MS. GOLUB: Yeah. No, I think --
11 (Simultaneous speaking.)
12 MS. GOLUB: That could be inspected.
13 Yeah.
14 MR. ODESS-GILLETT: That would be -- yeah, 15 those would be considered post-amendment inspections.
16 MR. DARBALI: So, okay, not disagreeing.
17 But let me clarify what I said. I said, before the 18 license amendment is issued, some development 19 activities have taken place.
20 MR. ODESS-GILLETT: Correct.
21 MR. DARBALI: There's no Phase II 22 submittal.
23 MR. ODESS-GILLETT: Right.
24 MR. DARBALI: So they're not related to 25 Phase II. So, I guess my question is, if those are NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
104 1 activities being done before the license amendment is 2 issued, so we cannot audit them. Can we do an 3 inspection before the license amendment is issued?
4 Because we don't want to leave an area 5 where we can neither inspect nor audit.
6 MR. ODESS-GILLETT: Well, it's not 7 excluding inspection.
8 MR. DARBALI: But can we do an inspection 9 before the license amendment is issued?
10 MR. FANELLI: The reason for this let's 11 say literal, you would have to have a license basis to 12 --
13 (Simultaneous speaking.)
14 MR. ODESS-GILLETT: Yeah, it falls under 15 the licensee, the licensee's oversight. And works it 16 against the -- they have the plan that the NRC 17 approves.
18 MS. GOLUB: Right. Yes.
19 MR. DARBALI: Can we audit the oversight?
20 MR. ODESS-GILLETT: Yes. Sure.
21 (Simultaneous speaking.)
22 MR. DARBALI: I mean, we can't -- it 23 cannot be an inspection before a license knowing 24 procedure. But those --
25 MR. ODESS-GILLETT: The licensee has the NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
105 1 --
2 (Simultaneous speaking.)
3 MR. ODESS-GILLETT: Has the option to 4 invite the NRC to observe their licensee oversight 5 activities. But the audit, it's not in the purview of 6 the LAR.
7 Because the LAR submittal --
8 MS. GOLUB: Is complete.
9 MR. ODESS-GILLETT: Is based on the 10 detailed system design.
11 MR. DARBALI: But the audit has to support 12 the safety violations.
13 MS. GOLUB: The LAR. That's correct.
14 That's correct.
15 MR. DARBALI: Correct.
16 MR. WATERS: I think what you're trying 17 out is, I mean, what if there's a critical development 18 process or test that happens during that license view 19 period. It's not part of the audit, because we're not 20 basing that.
21 And how do you inspect it, because it's 22 not a good license. So, what are we expected to do 23 under this?
24 MR. ODESS-GILLETT: So the lic -- that's 25 where we're stepping up the game. Where the licensee NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
106 1 is responsible for those oversight activities.
2 Now, once we're --
3 MR. WATERS: Well, I don't think so.
4 MR. ODESS-GILLETT: No, let me finish.
5 Let me finish. When the amendment is issued, the 6 inspection staff can look at what was -- and we'll get 7 into this, what was performed by the licensee's 8 oversight activities in the past tense, to see did 9 they -- did they do what they said they were going to 10 do in their vendor oversight plan?
11 Are there any gaps between what they 12 reviewed and what they documented versus what's in the 13 vendor oversight plan? That can be done. It's not 14 forgotten or left out.
15 But --
16 MS. GOLUB: Or undocumented.
17 MR. ODESS-GILLETT: Yeah.
18 MR. STATTEL: I also want to explain, the 19 audit is more than just a document review. And we're 20 doing more than just going through a process of, you 21 know, looking at artifacts.
22 And there's a human element to it too.
23 And there's a training element. So, half of the time 24 we spend on an audit is figuring out how these 25 processes really work.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
107 1 Because when you read a plan, you kind of 2 don't know exactly how that's used. What actually 3 takes place.
4 And that's why the audit has to take place 5 at the vendor premises. It's because we need to talk 6 to the people who are doing the deeds.
7 And we need to find out from them, we 8 interview these people. We run them through certain 9 exercises. You know, we have kind of a -- kind of an 10 undocumented cookbook we use for this, right.
11 And we perform these interviews and we 12 ask these standard questions to them. We talk to a 13 software librarian. We talk about configuration 14 management, how that's done.
15 We talk about things like how do you make 16 sure -- we ask them like these esoteric questions 17 like, how do you make sure no software, undocumented 18 software gets introduced?
19 What process is -- what in this plan, what 20 in this procedure that you're using, prevents that 21 from happening?
22 And I know some of you have been on these 23 audits. You've heard us ask these questions. And we 24 listen to the answers. And we have these -- these 25 interactions with the people who are doing the deeds.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
108 1 Right?
2 And it gives us a whole lot better insight 3 then we would have through just a document review.
4 Because otherwise, you know, there -- we could all --
5 we could do it right here in the office here, you 6 know.
7 And you know, if you're just looking at 8 during the requirements threads, you know, that's --
9 you know, that's mechanical. I can just -- I don't 10 need to go to the vendor's site to do that 11 necessarily.
12 But to really understand their processes 13 and how they're being used, that's where the audit 14 comes into play. So, I know it's hard.
15 That's kind of the guidance for doing that 16 that I'm looking for. If we're going to transfer that 17 activity over to the licensees, which is really what 18 this comes down to.
19 MR. ODESS-GILLETT: That's correct.
20 MR. STATTEL: I want to have some 21 confidence that they're going to do kind what we've 22 come up with.
23 MR. ODESS-GILLETT: And we welcome -- we 24 welcome that feedback and that incorporation into the 25 vendor oversight plan.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
109 1 MR. STATTEL: Um-hum. And the documents 2 don't necessarily reflect everything that takes place 3 during an audit.
4 MR. ODESS-GILLETT: Um-hum.
5 MR. STATTEL: It's really -- it goes 6 beyond just a document review. It's an actual human 7 interaction.
8 MR. ODESS-GILLETT: Um-hum.
9 MR. STATTEL: We're talking to each other.
10 We're leaning your processes. We really need to have 11 those interviews.
12 MR. ODESS-GILLETT: Right. But the 13 licensee really needs to have those interviews. And 14 --
15 MR. STATTEL: Right. Exactly.
16 MR. ODESS-GILLETT: Understand the 17 process. Because the licensees' ultimately 18 responsible. So that's really what we're trying to 19 convey here.
20 MS. GOLUB: Could I jump in and just 21 mention? I know Steve, maybe you can talk a little 22 bit about this.
23 When you put together that vendor 24 oversight plan, I know you go through in detail a 25 series of NRC audit plans and reports. Can you speak NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
110 1 just a little bit to that? Because I'd like to give 2 Rich a little bit of assurance that we did look at 3 what the NRC did.
4 Now, I understand you have an undocumented 5 process, and undocumented questions. Obviously we 6 don't have access to your undocumented stuff.
7 But, we did go through several audit plans 8 and reports so that we could try to get a sense of 9 what the NRC did, so we could try to replicate that.
10 And I know Steve took some time to do that in detail.
11 MR. DRAGOVICH: Yeah, so that's true.
12 But, so what we did was under project specific 13 performance measures. So we had general performance 14 measures and project specific performance measurers.
15 Project specific just came from audit 16 plans. So we tried to pick -- I looked at several of 17 them. And tried -- because there's a lot of overlap.
18 There's a lot of the same categories that 19 get -- so I tried to pick some of the more common ones 20 and listed them here to try to like Ray said, 21 replicate what the NRC had in the audit plans.
22 And now naturally, this is going to get 23 more specific for projects. I mean, we're going to 24 enhance that.
25 But that's kind of where this section came NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
111 1 from. Because the general performance measures came 2 from the EPRI commercial grade dedications.
3 MR. STATTEL: But you're referring to the 4 VOP, right?
5 MR. DRAGOVICH: Yeah. Exactly. And we're 6 going to talk about that more later too. But that's 7 kind of what that --
8 MS. GOLUB: I just wanted to because, you 9 know, I just want to -- just so Rich knows that we did 10 look at some of the material that the NRC -- that we 11 had in the public sphere.
12 So that we could try to imitate some of 13 that work that the NRC has done. Look at the same 14 items.
15 MR. SCHRAYE: Can I ask a question? You 16 have an undocumented cookbook? So we don't know what 17 that is.
18 MR. STATTEL: Well, I mean --
19 MS. GOLUB: Well, I mean it --
20 MR. STATTEL: Let me explain.
21 MR. SCHRAYE: Okay.
22 MR. STATTEL: I know that came out wrong.
23 We do -- we do document our audit plans. Right? Now 24 the audit plans also include performing interviews 25 with certain -- in certain key positions. People in NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
112 1 key positions.
2 And during those interviews, we kind of 3 run them through scenarios. So, an example would be 4 the software librarian, we would say, okay, I'm --
5 let's pretend I'm an engineer, a software development 6 engineer, and I want to change Part A of the code.
7 And how do I go about checking it out?
8 And he'll show me the form that they -- he fills out.
9 And the approvals he needs to do that.
10 Okay. Now it's checked out. I'm working 11 on it. Now engineer B walks -- comes along, and he 12 wants to check out the same code. Right?
13 So, what would prevent him from being able 14 to do that? And what would prevent engineer B from 15 overriding engineer A's code? Right?
16 So, you're not going to see a procedure 17 for going through that thought exercise in the audit 18 plan. But the audit plan will say, we conducted an 19 interview with the software librarian. Right?
20 And we will document, you know, we will 21 basically write that scenario up in our audit report.
22 So, it is all documented.
23 But, it's not to the level of detail, you 24 know, that we basically developed over the years.
25 MR. ODESS-GILLETT: And I guess then it NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
113 1 would vary from auditor to auditor?
2 MR. STATTEL: Not so much. We try --
3 MR. ODESS-GILLETT: Why is that?
4 MR. STATTEL: Well, you know, we're kind 5 of working with the same people. You know. So Samir 6 and I have done a lot of these audits. We've done 7 audits at General Electric. And we go to Reddy 8 (phonetic), and we go to different sites.
9 And we kind of have it down. I mean, we 10 do specialize in, at least in the last ten years.
11 MR. DARBALI: Right. When we --
12 MR. STATTEL: We've kind of gotten it down 13 to a science.
14 MR. HERB: So basically there's only two 15 is what you're saying.
16 MR. STATTEL: No, no. It's not two. It's 17 like --
18 (Simultaneous speaking.)
19 MR. DARBALI: Well, like Rich said, I 20 mean, what we're going to do is document that in the 21 audit plan. And when we do this we document the audit 22 report before we do an audit.
23 And you know, we got more vendors then 24 license amendment reviews. So, we're going to do a 25 topical report audit, we'll go back to the other ones NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
114 1 we've performed and we see one thing we look at, what 2 did we report?
3 You know, was there value in what we 4 looked at? And we use that as a base.
5 MR. SCHRAYE: I guess my point wasn't 6 trying accuse you of anything. It just seemed like 7 you have a lot of good information and approaches.
8 And that would be helpful.
9 MR. STATTEL: I know and I don't know how 10 to transfer that.
11 (Laughter.)
12 MR. STATTEL: Well no, I can't -- there's 13 not a document I can hand you.
14 MR. DARBALI: Well, the audit reports and 15 the audit plans, that's --
16 MS. GOLUB: Which we looked at.
17 MR. DARBALI: Right.
18 MR. DRAGOVICH: That's all we had.
19 MS. GOLUB: We did look at that. Yeah.
20 We did look at that to develop what we put together.
21 MR. DRAGOVICH: Right. And what you said, 22 like we have a category called change control of 23 management. Right? It sounds like we would just be 24 able to enhance that to include maybe one of those.
25 MR. DARBALI: Right.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
115 1 MR. HERB: But, I mean, but most of us 2 have supported NUPIC type audits. And it's a similar 3 process. You have the processes and procedures, but 4 you go talk to the people and you ask them to walk you 5 through it.
6 So, I mean, that's -- I don't think that's 7 unique to the industry.
8 MR. STATTEL: And part of the process that 9 doesn't show up in the procedures and the documents is 10 learning. Because every vendor has a -- their 11 requirements traceability is done very differently 12 from one vendor to another.
13 And it's not -- it's not intuitive. I 14 can't just pick up the document, oh, now I know how 15 it's done. Sometimes I have to have the vendor 16 explain to me. What's my document numbering system?
17 What's -- how are those requirements' 18 traceability established? Is it a table in the 19 document itself? Is it a separate doors report?
20 Is it -- you know, and it's all over the 21 map. Right? So, you know, when I come into a new 22 vendor and I'm looking at a process for the first 23 time, I kind of need him to teach me.
24 So, that instructional part of the audit 25 is also one of those things.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
116 1 MS. GOLUB: But it's --
2 MR. STATTEL: Now, we do document that.
3 When we put our report in, we describe those 4 processes. Because now that we know that.
5 But the audit itself was a learning 6 process so that we could -- it enabled us to be able 7 to write those reports.
8 MS. ZHANG: So, in my opposite view in 9 terms of, I don't disagree that, you know, we look at 10 those processes in audits. But, from a new reactor, 11 when we did ours, right, we don't have all those 12 processes, you know, the outputs yet.
13 But we did focus a lot on the design and 14 how we designed it. And because we're looking at the 15 overall plan, so we focused in on the integration of 16 the overall plan and the functions a lot.
17 But, through the ITAAC process, I would 18 imagine for -- the result for looking at the design 19 outputs and the processes that we generate those 20 design outputs. And do the same thing that Rich 21 described.
22 And I'm saying that, can we do the same 23 thing here as part of inspection? Whether it's vendor 24 oversight inspection or --
25 MR. FANELLI: We actually do. We actually NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
117 1 verify that. We do, very similar.
2 MR. ODESS-GILLET: Yeah. Yeah. So, I 3 mean, that's inspection space.
4 MS. ZHANG: Yeah. And I would also --
5 MR. STATTEL: The same interview is the 6 same questions.
7 MS. ZHANG: And then I would also expect 8 that the licensees, no matter whether we do 9 inspections or not, you will be taking over all those 10 responsibilities.
11 Well, you are already taking all those 12 responsibilities.
13 MS. GOLUB: That's exactly right. Right.
14 MR. ODESS-GILLETT: It's so much more 15 important in our opinion that the licensee step up --
16 MS. GOLUB: Absolutely right. Um-hum.
17 MR. ODESS-GILLETT: And do what Rich 18 described. Because it's his system that's going in 19 the plant. It's his sys -- it's he that will be 20 answering those questions to the inspection staff.
21 MS. GOLUB: Yes. So your point is well 22 taken.
23 MR. HERB: And then the reason why the 24 DEG, that's the amount of information for a vendor 25 outside tripled from the original one too now, is NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
118 1 because we have learned that we were acting in the 2 industry to trust those vendors.
3 And so what you saw in 2012 is not what 4 you're going to see in 2018 or 2019. So I believe.
5 MS. ZHANG: So I think, you know, from us 6 we do have our past experiences that kind of adds a 7 lot of burden. We do hope that, you know, things have 8 changed and there's, you know, a lot of improvements 9 made.
10 But I think the first few LARS that we 11 will see, --
12 MR. HERB: Of course. Yes.
13 MS. ZHANG: Expect that, you know, we will 14 do a lot of, you know, initial inspections to verify 15 that has been done.
16 MR. HERB: We have to establish a track 17 record. And that's -- we're going to do that.
18 MR. SCHRAYE: Our concern is that we're 19 going to be the first out of the block. So, what 20 color block should we do? Provide?
21 MR. DARBALI: And like I said, I mean, 22 we'll continue these discussions during the 23 application meetings when you come back in.
24 MR. STATTEL: I will say it's common sense 25 and we've had experiences with licensees where we, you NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
119 1 know, like I said they weren't performing with the 2 oversight activities that we were doing.
3 PARTICIPANT: Yes, you're absolutely 4 right.
5 MR. STATTEL: But we did talk and say you 6 really need to be concerned about this. So we would 7 take the licensee engineer and we would show them, 8 look, this is what we found on the vendor and here is 9 where our concerns are, and guess what, once they 10 understood that they had the same concerns.
11 In a couple of instances they came back 12 and performed their own audits independent of us on 13 those vendors and they made course corrections, okay.
14 PARTICIPANT: Right.
15 MR. STATTEL: So it is common sense stuff 16 and once engineers get the understanding of where the 17 concerns are and where we need to keep the vendors in 18 line and head in the right direction we do come to 19 agreement, okay. I hate to say it is common sense.
20 MS. GOLUB: But, Rich, to your point, you 21 know, on the licensee side we are not leaving it to 22 people using common sense. I mean that's why the DEG 23 has an enhanced section, a lot more information on 24 vendor oversight, a lot more guidance, because the 25 licensees recognized that they were lacking in that NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
120 1 area and are addressing this.
2 So you're not kind of waiting to be told 3 or to kind of pass it on by word of mouth or tribal 4 knowledge or any of that. I mean it's a far more 5 detailed, more documented process.
6 The one thing I will say is if you have 7 specific --
8 PARTICIPANT: Greg wanted to say 9 something.
10 MS. GOLUB: Oh, I'm sorry.
11 MR. GALLETTI: I just wanted to ask so 12 what I am hearing sounds very interesting but as far 13 as the VOP is concerned, so the details of the 14 licensee's oversight activities at the vendor would 15 that be incorporated in some sort of scheduling 16 mechanism in the VOP so the reviewer would know, hey, 17 they are going to do this at approximately this point 18 in the development process at that level of detail --
19 MS. GOLUB: I know what you are saying.
20 MR. GALLETTI: -- so if the NRC wanted to 21 engage with your oversight activities it would be 22 obvious when it's going to occur, yes?
23 PARTICIPANT: Oh, yes.
24 MR. DRAGOVICH: Yes. Well, as Ray pointed 25 out like when we negotiate the original procurement NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
121 1 spec we define when we want to have those oversights 2 and at which stages, but then again if they have gaps 3 then we might increase that oversight, too --
4 (Simultaneous speaking.)
5 MR. GALLETTI: Okay. But there will be 6 some as part of the LAR submittal that at least a 7 baseline engagement would be understood.
8 MR. HERB: Right. And it may be 9 milestone-based rather than actual-based, but that 10 schedule is a live schedule and we maintain on a 11 SharePoint and we would agree, we would discuss that 12 in a Phase 0 meeting and some of those dates are 13 already known at the Phase 0 meeting that we can --
14 MS. GOLUB: Right.
15 MR. HERB: And, again, we would welcome 16 your insights, you know, to go along with this, you 17 know, and observe to make sure that we are doing our 18 job properly and then provide input if you felt it was 19 necessary to adjust course corrections at that time.
20 I mean this is a new process for us and we 21 want you all to be comfortable with what we are doing.
22 MR. WATERS: So I think a common comment 23 is understanding the timing of your process --
24 MR. HERB: Yes, right.
25 MR. WATERS: -- so we don't duplicate NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
122 1 efforts we don't particularly need to duplicate, 2 right.
3 MR. HERB: Right. Or in the Southern 4 procedures it says you will define these dates and it 5 actually has like --
6 MR. WATERS: We were trying to understand 7 how we would be communicating in real time going after 8 LARs.
9 MS. GOLUB: I think --
10 MR. HERRELL: So the one thing that I'll 11 push strongly for is the schedule itself doesn't go in 12 the VOP because of the LARs schedule it's going to be 13 changing far more often than I would desire because 14 referenced out of the VOP you put it someplace where 15 it can be maintained and where you guys can, where 16 everyone can see it.
17 MR. WATERS: Yes, maybe because of 18 communication --
19 (Simultaneous speaking.)
20 MR. WATERS: -- understand that but that's 21 one of the things we are talking about.
22 MR. ODESS-GILLETT: Right, but they have 23 --
24 (Simultaneous speaking.)
25 MS. GOLUB: Oh, you're just looking for a NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
123 1 break.
2 (Simultaneous speaking.)
3 MR. ODESS-GILLETT: Okay, I'm almost done.
4 I'm almost done.
5 MS. GOLUB: Oh, I didn't get it. The T 6 was for --
7 (Laughter.)
8 MR. ODESS-GILLETT: So I think we've been 9 on this slide probably long enough, Joe, so we can go 10 to Slide 11. So what's this new inspection paradigm 11 from what we are seeing?
12 So in the NRC inspections from our point 13 of view relative to the AR process, a license 14 amendment, we feel should start with the licensee 15 because the licensee is responsible for vendor 16 oversight.
17 So from our point of view the inspection 18 should focus on the licensee's fulfillment of its 19 vendor oversight plan. NRC's inspections would focus 20 on the licensee's vendor oversight reports that 21 document those vendor audits and inspections of a 22 detailed design implementation and test processes.
23 Only if the NRC identifies gaps in the 24 vendor oversight process would it be in our opinion 25 appropriate for the NRC to augment licensee NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
124 1 inspections at the vendor site because what we don't 2 want to happen is the vendor inspection branch repeat 3 what the licensee did unless there is a perceived gap 4 or inadequacy in what the licensee did and then we can 5 see that the vendor inspection branch would want to 6 augment that.
7 MR. GALLETTI: Where in this paradigm do 8 you account for the notion of the NRC overseeing 9 licensee activities at the vendor facility as they are 10 actually occurring?
11 MR. ODESS-GILLETT: Can I just -- That 12 first sub-bullet. You are going to look at what the 13 licensee's vendor oversight records document for those 14 --
15 (Simultaneous speaking.)
16 MR. GALLETTI: Yes, but that's after the 17 fact, you know. If you are going into a vendor shop 18 to do certain oversight activities where would the 19 paradigm support the NRC reviewing your activities of 20 doing that oversight?
21 MR. ODESS-GILLETT: In real time?
22 MR. GALLETTI: Yes.
23 MR. ODESS-GILLETT: Well what's the 24 requirement for doing it in real time?
25 MR. BOLLOCK: Well before it was part of NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
125 1 before the license was approved they --
2 (Simultaneous speaking.)
3 MR. BOLLOCK: Yes, they were reviewing 4 things along the way, auditing, well now it's approved 5 earlier.
6 (Simultaneous speaking.)
7 MR. STATTEL: I've had these interactions.
8 So in the past we don't look at the vendor oversight 9 plan, for instance, so we don't know if they're --
10 MR. BOLLOCK: Right.
11 MR. STATTEL: But I am there. I am at the 12 factory test, I am at the vendor facility, and every 13 time I go there, you know, I am working through the 14 licensee, right, I am working on a license amendment 15 so the licensee is there as well.
16 So, for instance, when we did our audit in 17 Wilmington we had several licensee representatives 18 present at those meetings and we did talk with the 19 licensees and we were asking well what's your role, 20 you know, and that's how we learned, right, but we 21 didn't see the vendor oversight plan.
22 MR. WATERS: I mean is there a concern 23 with the -- There is two things here, a licensee 24 inspection versus a vendor inspection, or is it about 25 the end of a site because it's --
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
126 1 MR. ODESS-GILLETT: It's about duplicating 2 work.
3 MR. WATERS: I understand that, but our 4 experience has been at the vendor site is if you can't 5 do a license inspection how are you overseeing it is 6 where do we get the inspector processes. That's what 7 I am trying to --
8 (Simultaneous speaking.)
9 MR. ODESS-GILLETT: All right, so I mean 10 there is always the opportunity for a licensee to 11 invite the NRC inspection staff to observe their 12 vendor oversight activities, but the idea is to give 13 this responsibility to the licensee --
14 PARTICIPANT: Agree.
15 MR. ODESS-GILLETT: -- and the NRC 16 oversight is did they do what they said they were 17 going to do.
18 MR. GALLETTI: Well part of making that 19 determination would be understanding your VOP says 20 this is what you are going to do did you in fact 21 implement that as written.
22 MR. ODESS-GILLETT: Right.
23 MR. GALLETTI: The output report may not 24 tell me whether you did it according to how you said 25 you were going to do it. My observation of you NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
127 1 actually performing that implementation would give me 2 that confidence, that information.
3 MR. ODESS-GILLETT: Well -- Go ahead.
4 MR. HERB: I mean I think overall probably 5 it could be worked out in a Phase 0 meeting, you know, 6 if some of those activities have not been performed 7 yet, because, you know, some of these things by the 8 time Phase 0 some of those activities may have already 9 gone past.
10 MR. GALLETTI: Let me give you an example, 11 you know, the bit about overlapping of duplication of 12 effort. So one example would be typically if the NRC 13 was coming to do an inspection at a vendor facility 14 doing some sort of modification we may look at that 15 IV&V and do a lot of discussions with their staff, 16 look at their phase summary reports, look at their 17 outputs, look at how they establish their testing and 18 all of that sort of thing.
19 Now we are saying that you licensees you 20 have the ultimate responsibility for doing that so 21 we're not going to do that but we may come in and do 22 an inspection of you doing that. And so where in this 23 paradigm would it allow us to do that or --
24 MR. DRAGOVICH: Why would it have to be 25 real time versus just reviewing our report that we NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
128 1 generate after our visit, like he had a report that --
2 MR. ODESS-GILLETT: I think because Greg 3 is saying the report may not have enough detail and I 4 would like to address that.
5 MR. DRAGOVICH: Is that --
6 MR. GALLETTI: Again, the VOP as written 7 may not tell, should tell me what you plan to do, 8 okay. How would I know you actually did that? It may 9 not be reflecting in the output report exactly what 10 you did and so for me to get confidence that you did 11 what you said you were going to do for that report I 12 think --
13 (Simultaneous speaking.)
14 MR. ODESS-GILLETT: Well the other thing 15 you could do, Greg, is when you go to the licensee and 16 the report says well I looked at the V&V summary 17 report and I had no findings.
18 You could audit that V&V summary report by 19 asking for it. The vendor could provide that to the 20 licensee and say here is my V&V summary report that 21 you did your oversight on and you could then do an 22 inspection of that V&V summary report without 23 necessarily having to go to the vendor site.
24 MR. FANELLI: Will you have access to all 25 of the documents --
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
129 1 (Simultaneous speaking.)
2 MR. ODESS-GILLETT: You will have access 3 to all of the vendor documentation that they had if 4 you need to substantiate some aspect of the vendor 5 oversight program.
6 MR. GALLETTI: Okay.
7 MR. STATTEL: And a lot of it depends on 8 the quality of the report.
9 (Simultaneous speaking.)
10 MR. ODESS-GILLETT: That's what I wanted 11 to address, too.
12 (Simultaneous speaking.)
13 MR. STATTEL: -- V&V summary report it's 14 not just there was no issues with them, it's the plan 15 said we were going to do five activities, they did 16 five activities, right, they match, right, and what 17 were the results of those activities, and issues were 18 found, were they resolved --
19 PARTICIPANT: Right.
20 MR. STATTEL: -- if they were not resolved 21 what was the justification for carrying them into the 22 next phase, you know, those are the types of things we 23 look in a V&V summary report.
24 PARTICIPANT: Right.
25 MR. STATTEL: And the report if it's well NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
130 1 written it will be very clear, yes, you have executed 2 that plan. I will say that if the report is well 3 written it would give that indication.
4 MR. WATERS: Will a report discuss the 5 previous, the interview technique and interview 6 questions we talked about earlier --
7 (Simultaneous speaking.)
8 MR. ODESS-GILLETT: Well this is sort of 9 new material, new feedback to us, and so we are going 10 to take that back and to consider how that we can 11 incorporate that.
12 But we intend to put together industry 13 guidance for this ISG-06 alternate review process and 14 one of the things that the licensees have agreed is 15 that there would be a standardized format and content 16 guidance for what would be in their oversight reports 17 so that there would be consistency across the industry 18 on what the inspection staff would see for vendor 19 oversight reports.
20 MR. STATTEL: Yes. Consistency is going 21 to be difficult to achieve here. You know, we talked 22 earlier about, you know, the staff. We have probably 23 ten people here that have done many, many digital I&C 24 reviews so we are able to be pretty consistent when we 25 perform our audits.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
131 1 When we are talking about an industry of, 2 you know, all you guys I don't know how you can 3 achieve that same level of consistency.
4 (Simultaneous speaking.)
5 MR. STATTEL: It's a challenge. I'm not 6 saying it's impossible.
7 MR. ODESS-GILLETT: Yes, okay. We got to 8 start somewhere, Rich.
9 MR. STATTEL: I know. I --
10 MR. ODESS-GILLETT: And the starting at is 11 defining a format and content requirements --
12 PARTICIPANT: And level of detail.
13 MR. ODESS-GILLETT: -- and level of detail 14 is the best we can do at this point.
15 MR. FANELLI: So let me ask a question.
16 When you go do your audits, like when you go to the 17 vendor and you place your reports, I am assuming you 18 are going to do sampling, sort of like we do, smart 19 sample, some kind of sampling technique, are you 20 expecting that when we come and review your oversight 21 that our understanding of those samples would be the 22 same, like those are the samples we'd pick or what if 23 we look at something you may not have looked at? I 24 mean we ask --
25 MR. ODESS-GILLETT: What were the samples NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
132 1 that you picked?
2 MR. FANELLI: Hey, well, suppose we want 3 to look at something different than you looked at.
4 MR. ODESS-GILLETT: This is the kind of 5 information we would need to document this having to 6 have to be documented in a vendor oversight report, 7 you know.
8 MR. STATTEL: We've been --
9 (Simultaneous speaking.)
10 MR. ODESS-GILLETT: What were the samples 11 taken, what were the requirements, you know, maybe 12 specify what those requirements are, how far did you 13 do the tracing to --
14 PARTICIPANT: Yes.
15 MR. ODESS-GILLETT: -- did you go all the 16 way down to test. I mean this is the kind of thing if 17 we want consistency in the industry we are going to 18 need to put this in the industry guidance.
19 MR. STATTEL: And actually, you know, when 20 you talk about that, and we've been criticized for not 21 performing a statistically significant number of 22 samples for a requirement.
23 And, you know, when you got a project and 24 there is 10,000 requirements it's really hard to do, 25 right.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
133 1 PARTICIPANT: It is.
2 MR. STATTEL: And we have really limited 3 resources. The industry has a lot more resources than 4 we do. We have two people, right, and the industry, 5 your oversight process is taking place over the course 6 of years, right.
7 PARTICIPANT: Yes.
8 MR. STATTEL: And you have more people 9 than we do so you are more -- I would expect actually 10 your vendor oversight to be more comprehensive and 11 more effective than what we have been able to do.
12 You know, we feel somewhat limited because 13 we have so few people and we have to do --
14 MR. ODESS-GILLETT: I mean you might have 15 10,000 --
16 (Simultaneous speaking.)
17 MR. STATTEL: We're doing more than one 18 job.
19 (Simultaneous speaking.)
20 MR. ODESS-GILLETT: You might have 10,000 21 requirements but what are the safety significant 22 requirements.
23 MR. STATTEL: Yes.
24 MR. ODESS-GILLETT: Painting the cabinet 25 blue is not necessarily something --
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
134 1 (Simultaneous speaking.)
2 MR. STATTEL: And we do, we focus on what 3 is safety significant.
4 PARTICIPANT: Okay, all right.
5 MR. WATERS: All right. So the first 6 bullet I think you generally understand and get that.
7 I think that's where we see how to do this.
8 It's the licensee, right, the licensee's 9 oversight responsibility. I think we are not in line 10 yet with the vendor site. The paradigm is suggesting 11 we understand that we consider that maybe that's where 12 we will be after the first few or six have been done, 13 I don't know, but we're not going to -- We don't see 14 that yet, not yet for what we are talking about here.
15 MR. DRAGOVICH: But even if that's the 16 case can we do it as like an observer role so that the 17 licensee and the NRC can do it under one inspection 18 versus having to do the same inspection twice?
19 MR. WATERS: That's a possibility I think.
20 I mean that's --
21 (Simultaneous speaking.)
22 MR. BOLLOCK: So, yes, I'm sorry --
23 MR. WATERS: Go ahead.
24 MR. BOLLOCK: -- to interrupt. So do you 25 mean to essentially --
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
135 1 MR. HERB: Shadow us on our own rather 2 than going in there separate.
3 MR. BOLLOCK: Right. Well I mean because 4 that's, you know, Greg is saying they do their 5 inspection of the vendor. So you are saying when you 6 do it instead of them or have them observe, so like a 7 -- working through with the NRC feedback from NEI is 8 license self-assessments --
9 (Simultaneous speaking.)
10 MR. BOLLOCK: She was talking about 11 something similar to that.
12 MR. WATERS: Yes, right.
13 MR. BOLLOCK: Okay. I mean that is a 14 paradigm shift.
15 MR. WATERS: Yes.
16 MR. BOLLOCK: So you are talking that in 17 lieu of a vendor inspection, a vendor, you know, 18 observation in lieu of.
19 MR. VAUGHN: We did talk about that the 20 changing the design basis share of inspections are 21 some of the ones that the team, you know, inspections 22 they do leveraging licensee self-assessments.
23 We'll try and take that analogy and bring 24 it over to here from, to the vendor side, but see if 25 it's applicable.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
136 1 MR. BOLLOCK: Yes, because right now --
2 And Greg and Aaron can expand on this right now, I 3 mean the vendor inspectors go on, what 30 a year --
4 PARTICIPANT: Yes, 25 or so.
5 MR. BOLLOCK: Twenty-five inspections a 6 year and they are picking the vendors to go based on 7 the safety-significant work. So, you know, say there 8 is a vendor that is going to create some new digital 9 system they would probably be pretty high on the list 10 if it's something that's going to go to a number, 11 especially if it's going to go to a number of sites 12 and it's, you know, digital I&C for a safety-related 13 system, probably be high on the list.
14 MR. HERB: But that's outside the process 15 we're talking about. That's not a project specific 16 thing, that's just --
17 (Simultaneous speaking.)
18 PARTICIPANT: That's just a routine 19 inspection.
20 MR. BOLLOCK: And that's what they do 21 right now. Now its tie, and Greg hit on this, you 22 know, they focus on where there is going to be testing 23 or there is going to be something like, you know, a 24 product being built that is going to be used because 25 that makes it more significant for them.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
137 1 (Simultaneous speaking.)
2 MR. HERB: Right. It's easier to do an 3 inspection on something they are actually building 4 instead of a theoretical process where they are not 5 actually doing something, right, so --
6 MR. GALLETTI: Right. And, again, part of 7 the notion of not duplicating effort would be the 8 possibility. Of course, you are going in and you are 9 engaging the vendor at the vendor location and you are 10 observing the work activities or reviewing the work 11 products and it would afford the NRC an opportunity to 12 see that in action rather than us doing that, 13 specifically you, we would be engaging in watching you 14 perform that oversight.
15 (Simultaneous speaking.)
16 MR. ODESS-GILLETT: And that would be a 17 learning for the licensee, you know, because that 18 feedback from the NRC inspection staff would be 19 helpful to the licensee.
20 (Simultaneous speaking.)
21 MR. BOLLOCK: Yes, but if you have a, I 22 mean if you have a license amendment and then are we 23 then watching your oversight --
24 MR. ODESS-GILLETT: Yes.
25 MR. BOLLOCK: -- like we're inspecting NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
138 1 your Criterion 7 --
2 MS. GOLUB: Right, that's right.
3 (Simultaneous speaking.)
4 MR. BOLLOCK: Yes, I mean so that is a 5 different paradigm.
6 MS. GOLUB: That's right.
7 MR. BOLLOCK: Now the other side is we'll 8 try not to, you know, is this something that's, you 9 know, it's infrequent, right, not every plants hear 10 the modifications.
11 But we are -- Right, there is also, of 12 course, not to expand baseline inspections for actual 13 inspections. So is this something you would want to, 14 you know, I guess on top of the Inspection Procedure 15 52-003 that infrequent procedure for a digital I&C mod 16 under 25.15 and you are saying along with that do 17 oversight, you are recommending that instead of having 18 --
19 MR. ODESS-GILLETT: I am only bringing it 20 up if the license, if the inspection staff needs to 21 see something real time.
22 PARTICIPANT: I think another thing to 23 come up, briefings --
24 (Simultaneous speaking.)
25 MR. BOLLOCK: Some of them are because NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
139 1 that's going to be an inspection to the licensee and 2 that's going to be a regional report under the, you 3 know, they're going to be under the ROP, it's going to 4 go into either its own order --
5 (Simultaneous speaking.)
6 MS. GOLUB: Steve, could you help us here 7 a little bit because I know --
8 MR. BOLLOCK: Yes, and it's going to be --
9 MS. GOLUB: -- you speak this language 10 better?
11 MR. BOLLOCK: -- feed-billed into the 12 licensee, right, completely different from what the 13 vendor group does right now, which is the overhead and 14 they look at the vendor.
15 PARTICIPANT: And the licensee performs.
16 MR. VAUGHN: So this is one thing we 17 really didn't have a good understanding with was 18 solely getting there. So when you go into the vendor 19 inspection and you have that procedure, it's the one 20 from January, what was it 35-710, I hope I got that 21 right, but it was new as needed.
22 So it was as needed and that kind of 23 caused well, it's as needed, so it's kind of a 24 routine, it's not periodic. We didn't know if that 25 "as needed" was every time a utility does a safety NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
140 1 significant, or an LAR, we're in this process, ISG-06, 2 and you do that inspection each time.
3 That's not the case because you said you 4 had 25 per year you pick. A vendor may get one --
5 PARTICIPANT: Right.
6 MR. VAUGHN: -- and, okay, now we 7 understand that was Part 1. Part 2 is if the vendor 8 wanted to see a utility go through the whole VOP 9 verification and develop a report as sort of like an 10 observer to see how it happens because it used to be 11 done under licensing when Rich and others did it, they 12 could see it.
13 But it wouldn't be an inspection where you 14 bill, you know, you wouldn't bill Southern, it would 15 just observe for confidence. And typically that is 16 sort of a paradigm shift because usually when the 17 staff does that it's, you know, owner's groups, PRA 18 peer review process where they observe stuff.
19 They may ask a couple questions but it's 20 really just observing and then they write a report on 21 observations. It's just -- So this is a little bit 22 different because it's sort of tied to a licensing 23 action that is the paradigm shift, but that's the 24 second part.
25 The third part was when the regions go to NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
141 1 do their inspection that's I think sort of almost 2 status quo. That's going to happen.
3 (Simultaneous speaking.)
4 MS. GOLUB: Yes.
5 MR. VAUGHN: That's going to happen. The 6 region is going to do those inspections, they will do 7 your sample selection and have their hours and, you 8 know, that's staying with --
9 MR. FANELLI: How that's going to actually 10 look is still up for debate.
11 MR. VAUGHN: Yes.
12 MS. GOLUB: Did that help? Does that 13 help? Will that stay the same?
14 MR. BOLLOCK: Yes. No, I think I just 15 want to be clear what, like which of those, you know, 16 do you understand what we do now, especially the 17 vendor group --
18 (Multiple yeses.)
19 MR. BOLLOCK: -- because I know it's not 20 --
21 (Simultaneous speaking.)
22 MR. BOLLOCK: -- not everybody understands 23 that.
24 MS. GOLUB: We do have a better 25 understanding.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
142 1 MR. HERB: It sounds a lot like our own 2 vendor inspection groups and they share and include 3 the NUPIC organizations but they do about 30 or 40 4 inspections a year, you know, of vendors and then 5 general quality inspections.
6 So it sounds like you all do a similar 7 thing that may be targeted and you pick based on maybe 8 projects that are going on because you're not looking 9 at --
10 PARTICIPANT: Right.
11 MR. HERB: We have to look at all of them 12 at a certain period where you are just doing a 13 sampling and you are going to the interesting ones, 14 but that's part of your normal every day.
15 MR. GALLETTI: And as you pointed out, 16 yes, your typical NRC vendor inspection is we address 17 the ancient criteria of Appendix B but we are much 18 more focused. It's almost like a limited scope --
19 (Simultaneous speaking.)
20 MR. GALLETTI: -- limited scope 21 inspection. So we are focusing in on certain system, 22 certain activities, and doing a deep dive of those.
23 It's not let's say a broad programmatic --
24 (Simultaneous speaking.)
25 MR. HERB: We'd love to learn from you all NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
143 1 and watch you all do that.
2 MR. BOLLOCK: Yes, and it's exactly, you 3 know, we understand you all have to every three years 4 inspect every vendor. We don't --
5 (Simultaneous speaking.)
6 MR. BOLLOCK: -- have the resources to do 7 that.
8 (Simultaneous speaking.)
9 MR. BOLLOCK: -- rely on you all to do it 10 and like NUPIC does it and, you know, this was all 11 worked out 40 some odd years ago, but we still spot 12 check essentially and --
13 (Simultaneous speaking.)
14 MR. BOLLOCK: Right. So that's --
15 MR. STATTEL: Well I am just thinking, you 16 know, when we get an application, an alternate review 17 process application, I envision that I would want to, 18 and this is me talking, right, I would want to perform 19 an audit at the site with the licensee and I would 20 want to interview and talk to the people who are going 21 to be performing those vendor oversight activities and 22 just have a discussion with them and make sure I 23 understand how they interpret what is written in the 24 plan, right.
25 So I am thinking I would want to do that NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
144 1 because, you know, we have the advantage if we do --
2 This is what we do, right, whereas the licensee might 3 do one digital upgrade in ten years.
4 PARTICIPANT: Yes.
5 MR. STATTEL: So that person has never 6 done it before and probably never will do it again, 7 right, but --
8 (Simultaneous speaking.)
9 MS. GOLUB: Well we do more digital 10 upgrades than that. They may not all be safety 11 systems.
12 MR. HERB: We don't -- They're not all 13 safety systems and they are not all underneath the 14 LAR, but we do a lot of them.
15 MS. GOLUB: Yes, right.
16 MR. STATTEL: Yes, okay.
17 MS. GOLUB: So this is not completely 18 unknown to licensees.
19 MR. STATTEL: Okay.
20 MR. BENNER: But I think -- Sorry I came 21 in late and after all the discussion, but I think some 22 of the reaction is the same reaction I am having is 23 what is the new NRC inspection paradigm.
24 And as an old inspector, both region-based 25 and resident, I would say the NRC's inspection NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
145 1 paradigm has always been there is an element of 2 document review and there is an element of direct 3 observation and you seem to be injecting the paradigm 4 that an inspection should be mainly document review.
5 So I think that's the disconnect is we are 6 not going to buy off on, oh, inspection can just be 7 document review and then do observation only if 8 necessary.
9 If you want to be document review it 10 should go back into a licensing process because that's 11 what, that's a process that ultimately does rely on, 12 you know, documents reviewed and REIs and back and 13 forth and all that, that's not inspection, right.
14 Document review is a component of 15 inspection but it will never, I don't think we'll ever 16 create an inspection paradigm that only relies on 17 document review.
18 MR. HERRELL: And I think from my 19 impression a new paradigm is who you are actually 20 doing the discussions with, you're still doing 21 discussions with the vendor, you're doing discussions 22 with the licensee to make sure that they are happy, 23 discussions you would have had or something similar to 24 them --
25 (Simultaneous speaking.)
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
146 1 MR. BENNER: And I go back to just like if 2 I am doing a maintenance inspection I am going to 3 observe some maintenance activities. If I am going to 4 do an inspection of your vendor oversight I want to 5 directly observe some of those activities.
6 PARTICIPANT: Got it.
7 MR. BENNER: Maybe we could get to a point 8 where like Mike said after we have confidence it could 9 be an after-the-fact document review. I got to tell 10 you I am having the reaction that certainly on the 11 front end of doing these we're --
12 (Simultaneous speaking.)
13 MS. GOLUB: Did you -- I don't know if you 14 got to hear what Ray has been talking about having 15 inspection folks accompany them on some --
16 (Simultaneous speaking.)
17 MR. BENNER: No, I heard that part.
18 MS. GOLUB: Okay.
19 MR. BENNER: But I also heard some 20 reactions of that would be strictly an observation --
21 (Simultaneous speaking.)
22 MR. BENNER: -- and not an inspection and 23 I'm like, well, again, that's where we're going to 24 have --
25 (Simultaneous speaking.)
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
147 1 MR. VAUGHN: Well this might help because 2 I had a misunderstanding earlier, too, when we had 3 that overall plan up there, you know, the AR process, 4 the old process at the top.
5 It was ISG-06 and all the milestones and 6 then after it said, you know, vendor inspection. So 7 I interpret that as every time someone, every time a 8 utility goes to this AR you are going to get the 9 vendor inspection inspection.
10 Yes, what if 50 utilities came in, the 11 same upgrade, there aren't even 50 utilities, but you 12 guys only have 25 per year so you couldn't, the vendor 13 inspection branch couldn't do all of them and you 14 probably wouldn't pick all of the same digital I&C you 15 would spread it out over --
16 MR. BENNER: But we could. We would do 17 one because as we have talked about our protocol for 18 vendor inspection is spot checking vendor --
19 (Simultaneous speaking.)
20 MR. VAUGHN: Yes, and so that completely, 21 that makes sense, and we're okay with that. The way 22 we interpret it as an inspection each time I think --
23 (Simultaneous speaking.)
24 MS. GOLUB: That's right.
25 (Simultaneous speaking.)
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
148 1 MR. VAUGHN: Yes, so that's -- Yes.
2 PARTICIPANT: Yes, we're okay. Right.
3 MS. GOLUB: This has been a really good 4 learning I think for us as well to understand better, 5 you know, what each of the --
6 MR. BENNER: What the piece part is.
7 MS. GOLUB: Yes.
8 MR. BENNER: What the purpose of the piece 9 parts are.
10 MS. GOLUB: I am glad Greg was here 11 because he really provided us some good insights on --
12 MR. HERB: Right, because that was our 13 concern --
14 (Simultaneous speaking.)
15 MR. ODESS-GILLETT: So the bottom line is, 16 that last bullet is you just don't want to be having 17 the vendor inspection branch doing the same thing 18 again that the licensee activities did.
19 MS. GOLUB: On a project basis.
20 MR. ODESS-GILLETT: We want some 21 efficiencies.
22 PARTICIPANT: Right.
23 MR. DARBALI: Right. So, Warren, we are 24 way past due for a break.
25 (Simultaneous speaking.)
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
149 1 MR. DARBALI: So I suggest we take --
2 MR. ODESS-GILLETT: I don't even need to 3 do the conclusion because everybody knows what the 4 conclusion is.
5 (Simultaneous speaking.)
6 MR. ODESS-GILLETT: So we can just stop 7 there and --
8 (Simultaneous speaking.)
9 MR. DARBALI: Let's take a 5-minute break 10 and we'll come back at 3:35 p.m.
11 (Whereupon, the above-entitled matter went 12 off the record at 3:29 p.m. and resumed at 3:41 p.m.)
13 MR. DARBALI: Okay, folks, we're going to 14 continue.
15 I think we've stopped following the 16 agenda.
17 (Laughter.)
18 But this is good because, I mean, our last 19 topic is the Vendor Oversight Plan, and we've talked 20 about that throughout the whole afternoon. So, I 21 think we're good timewise. So, this is going to be 22 the last presentation of the day, and I think we've 23 covered a lot of those topics. So, we'll just go 24 through it.
25 The purpose of this example is to identify NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
150 1 types of LAR review and audit and post-license 2 amendment inspection activities that would be 3 performed in relation to an alternate review process 4 review. And we kind of based it out of the Oconee 5 RPS/ESFAS modification.
6 Next slide.
7 So, here's some reference material, 8 although we're not specifically going through that 9 today. I think it can help inform future 10 interactions. So, that's just the license amendment, 11 some audit reports. Some turned out to be 12 proprietary. But that just gives you information, an 13 idea of the type of number of audits that were 14 performed.
15 Section 3.14 of the Safety Evaluation 16 includes the recommended inspection items, and we do 17 have March 0. Huh?
18 (Laughter.)
19 So, we do have the inspection reports, and 20 this is just for the first unit that was installed.
21 I think that's the one that had more inspection 22 activity.
23 MR. STATTEL: I think it was a leap year.
24 (Laughter.)
25 MR. DARBALI: A leap year? Okay.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
151 1 (Laughter.)
2 Nobody provided feedback to me.
3 (Laughter.)
4 And then, the Vendor Oversight Plan.
5 Okay. It's a new slide. It's an update 6 from the previous one we looked at. It just focuses 7 on the alternate review process. So, for the license 8 amendment review submitted and license amendment 9 issuance date, timeframes, we kind of added like a 10 wider bar because it is going to depend on the 11 application when those activities are going to take 12 place.
13 So, for the green box, the NRC review and 14 regulatory audits, because we're focusing on this 15 part, on the audits, we're envisioning we will be 16 doing electronic audits. And there's the option for 17 a vendor facility audit and/or a site audit. Again, 18 it's going to depend on the application.
19 And then, post-license amendment issuance, 20 that orange box, is a combination of a Regional and 21 vendor inspection. We're envisioning one inspection 22 at the plant site and at least one inspection at the 23 vendor facility. This is what we're working with 24 right now.
25 MR. HERB: Not every time?
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
152 1 MR. DARBALI: Not every time.
2 MR. HERB: Right, right.
3 MS. GOLUB: But that was a good 4 clarification --
5 MR. DARBALI: Yes.
6 MS. GOLUB: -- a really good clarification 7 for us.
8 MR. DARBALI: And it goes back to all the 9 conversations we've had.
10 And then, after the equipment has been 11 delivered to the site, then we're looking at a 12 Regional inspection at the site.
13 Then, that box at the bottom just notes 14 that, whether it's an audit or our inspection 15 activity, pretty much everybody you see here, all 16 these organizations are going to be involved one way 17 or another.
18 Next slide.
19 So, for the audits of the LAR, we mention 20 electronic audits, and those are typically audits of 21 non-docketed material and typically in response to an 22 open item. And then, that's going to determine, 23 closure of the open item is going to determine whether 24 we issue a request for additional information or have 25 that addressed.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
153 1 And then, an audit at the vendor facility 2 or plant site. Again, it could be non-docketed 3 material. Some vendors or licensees don't want to put 4 that in an electronic portal. So, we can go there and 5 see it. And we can do some form of configuration 6 management audit to support what we're reviewing.
7 Secure development/operational environment. We could 8 do vendor quality and licensee vendor oversight.
9 So, like we've said before, the LAR is 10 going to contain a summary of the VOP, but it most 11 likely will get audited. The full VOP will get 12 audited.
13 Next slide.
14 And so, for post-license amendment 15 inspections, whether they be Regional or vendor or a 16 combination of both, the activities to be inspected 17 include the Vendor Oversight Plan or licensee vendor 18 oversight. It could be requirements for stability, 19 test results, configuration management.
20 And then, site acceptance/installation, 21 that's mostly at the site. And again, this is going 22 to depend on the application.
23 MR. ODESS-GILLETT: So, we had made a 24 comment to ourselves that actually all the dashes, 25 starting with "requirements, traceability," and ending NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
154 1 with "vendor quality," if you move those dashes to the 2 right under "Licensee Vendor Oversight," the licensee 3 will be doing those things as well.
4 MR. DARBALI: And we had that discussion, 5 but, then, I think that acceptance testing, 6 installation and procedures would not fall under 7 vendor oversight.
8 MR. ODESS-GILLETT: That's correct.
9 That's correct.
10 MR. DARBALI: But that's a point well 11 taken. Some of those would be covered by a licensee.
12 MR. STATTEL: So, the line can move from 13 one project to another?
14 MR. DARBALI: Right, right.
15 The next slide.
16 And so, what type of activities would be 17 -- or what would be the idea of doing the inspections?
18 It would be to verify vendor oversight activities that 19 the licensee is performing, per the Vendor Oversight 20 Plan. And that includes inspections of Vendor 21 Oversight Plan reports at the site.
22 Verify how the licensee and the vendor are 23 performing those same activities. And that includes 24 inspections at the vendor facility.
25 And then, verify site installation and NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
155 1 testing.
2 Next slide.
3 And then, I'll turn it over to Rich.
4 MR. STATTEL: Yes. So, I was asked to 5 give examples of what we would expect to see in the 6 Vendor Oversight Plan for guidance on how to perform 7 those activities.
8 So, go on to the next slide.
9 I mean, all I did was just look to BTP-14, 10 and I found -- I mean, you should recognize this.
11 It's actually ISG-6, if you look at Section 9 of 12 ISG-6.
13 These are really -- it just identifies the 14 evidence that implementation of the plans has been 15 completed, and these are examples. They may vary from 16 project to project.
17 I picked the software safety analysis 18 implementation. In that example, you can see what 19 we're looking for, if you think about it. So, the 20 vendor performs a software safety analysis in 21 accordance with their plans. Typically, that would 22 involve a list of tasks that they perform and a 23 description of how those tasks are going to be done --
24 "going to be done," future tense.
25 When we look at the Safety Analysis NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
156 1 Report, we see did they do those tasks. The 2 documentation shows that the analysis activities have 3 been successfully accomplished. It basically 4 summarizes the results of those activities. And 5 that's it, you know. I mean, that's the guidance 6 that's in BTP-14.
7 Now I started an activity of basically 8 taking those sections, implementation and design 9 output, out of BTP-14 and just stripping it of all of 10 the characteristics, right? Because there's a lot of 11 words in there that just talk about characteristics.
12 If you strip that out, you end up with sentences like 13 this, which are common sense, right? I see a report.
14 It tells me they did everything they planned to do, 15 and here are the results. I mean, fundamentally, 16 that's kind of what it comes down to.
17 So, when we're talking about probably 30 18 pages of guidance that's in BTP-14, if you really boil 19 it down to where the real guidance is, you know, what 20 are you going to look for, it comes down to these 21 types of statements.
22 And the next slide shows an example of --
23 MR. ODESS-GILLETT: Before you go to the 24 next slide --
25 MR. STATTEL: Sure.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
157 1 MR. ODESS-GILLETT: -- can I make a 2 couple of comments?
3 MR. STATTEL: Yes.
4 MR. ODESS-GILLETT: One is I'm assuming 5 Safety Analysis Reports is not the USAR, but the 6 software safety analysis documentation in BTP-714, 7 which includes a software analysis --
8 MR. STATTEL: So, I removed the word 9 "software" from here.
10 MR. ODESS-GILLETT: -- and software safety 11 plan.
12 MR. STATTEL: I removed the word 13 "software" from here, but you're correct.
14 MS. GOLUB: You kept it in the example.
15 MR. ODESS-GILLETT: Okay. And the second 16 thing is, if this is all described in D4 --
17 MR. STATTEL: Yes, the plan is. It's not 18 in D4. This is --
19 MR. ODESS-GILLETT: Okay, but the 20 acceptance criteria should be against the plan and not 21 necessarily what's in BTP-714, because the staff 22 approved --
23 MR. STATTEL: The plan.
24 MR. ODESS-GILLETT: -- and the plan and 25 the criteria that would be for the output documents of NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
158 1 that --
2 MR. STATTEL: Okay. So, BTP-14 is divided 3 into three sections.
4 MR. ODESS-GILLETT: Yes.
5 MR. STATTEL: Right? It has here is 6 criteria for reviewing plans. Here's how to determine 7 whether you have an adequate plan.
8 MR. ODESS-GILLETT: Right.
9 MR. STATTEL: And it goes through all 12 10 plans.
11 MR. ODESS-GILLETT: Right.
12 MR. STATTEL: The second part is 13 implementation.
14 MR. ODESS-GILLETT: Right.
15 MR. STATTEL: Here's how you make sure 16 those plans were correctly implemented, and it has a 17 doppelganger section for each plan.
18 MR. ODESS-GILLETT: Right.
19 MR. STATTEL: And each one points back to 20 the plan, just like these words do in this example.
21 It points back to the plan. The tasks that were 22 identified in the plan have been successfully 23 accomplished, right?
24 MR. ODESS-GILLETT: Okay.
25 MR. STATTEL: So, it's a little subtle.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
159 1 The guidance for the plan itself or the adequacy of 2 the plan is different than the guidance for verifying 3 that it's correctly implemented. Okay?
4 MR. ODESS-GILLETT: Okay.
5 MR. STATTEL: And it's really a separate 6 section, and there's 12 separate subsections in there, 7 in the implementation section, that covered -- well, 8 maybe not 12 -- but if they could cover this. Now 9 some of these things get multiplied.
10 For example, V&V report -- and a lot of 11 this is project-specific. There are projects that 12 don't do a separate software safety analysis. Okay?
13 They call it something else or they accomplish it in 14 some other way. And that's fine. But the idea is the 15 Vendor Oversight Plan would have some direction to the 16 person implementing that or verifying that the plan 17 was implemented, and it would basically be the 18 language would be specific to what's in the plan. So, 19 if there's four activities identified in the plan, we 20 would identify them by name, and there might be 21 special guidance for that.
22 But we're not looking, my real point is 23 we're not looking for a hundred-page document on how 24 to verify design implementation. We're really just 25 looking for common-sense sentences, some kind of along NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
160 1 this order. And this is common sense, you know.
2 You're telling me you've implemented the plan and you 3 have a report. I would expect to see the report list 4 the tasks and tell me what the results of those tasks 5 are, and tell me how, if you've identified anomalies 6 or issues, tell me what they are, how have you 7 resolved them. And those ones that are not resolved, 8 how are you carrying those forward, right? How are 9 you addressing those?
10 MR. ODESS-GILLETT: Okay.
11 MR. STATTEL: And really, that's 12 fundamentally what we want to see, and that's what we 13 do, right, and that's what we would expect the 14 licensee. And these are things that a licensee would 15 want to do.
16 MR. ODESS-GILLETT: Right.
17 MR. STATTEL: Now, if you go to the next 18 section, it really refers to the third section -- the 19 next slide -- refers to the third section, which is 20 the design output. Now, again, it's document-21 oriented. That's just the way the BPT is written.
22 But, again, if we're going to verify 23 design outputs, what are the artifacts of having an 24 implemented and completed design? And this is an 25 example list.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
161 1 You know, I talked with Gary Johnson, who 2 wrote a lot of this. Again, it's going to vary. Code 3 listings may not be at all applicable to a FPGA system 4 or something like that. But in a lot of cases there 5 are parallel or similar type documents that get 6 produced.
7 And I pulled the example here for the SRS 8 because most systems have a software requirement 9 specification or something equivalent. This kind of 10 comes down to performing threat analysis. You know, 11 there may be guidance for performing that. And what 12 we're looking for is that the requirements are 13 traceable forward and backwards. The requirements 14 traceability matrix and the requirements traceability 15 analysis reports, which are actually more important, 16 they show that you're not implementing code without 17 having it documented; you don't have any broken 18 traces, things like that.
19 So, again, it's common-sense stuff. It's 20 not a difficult activity. I'm not making this out to 21 be, you know, we're looking for a 300-page guidance 22 document here, but just some words. And these are the 23 type of things I would expect to see in either a 24 Vendor Oversight Plan or in some other document.
25 This is the kind of thing I would be NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
162 1 looking for to make sure that these activities that 2 are post-license amendment issuance -- you know, you 3 have in your plans, it's the document review that's 4 part of your engineering process. Fine, you have a 5 software requirement specification. Somebody from the 6 licensee signs, "I've reviewed it." Well, what does 7 that signature mean? What's behind that signature?
8 What did he do? I want to understand that, right?
9 And so, that's kind of it.
10 Now, going to the Vendor Oversight Plan 11 that you provided us, I'm assuming this -- when I 12 first read this and, then, I gave you this feedback, 13 it looks like an instruction for how to create a 14 Vendor Oversight Plan --
15 MR. HERB: It essentially is.
16 MR. STATTEL: -- rather than a Vendor 17 Oversight Plan itself.
18 MS. GOLUB: Right. It's the summary of 19 the Vendor Oversight Plan.
20 MR. STATTEL: I just want to make sure we 21 have the right understanding because I don't want to 22 see, when I get a real application in for a plant, I 23 don't want to see an instruction for how to create a 24 plan.
25 MS. GOLUB: Right, I agree. I agree.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
163 1 MR. STATTEL: I want to see an actual plan 2 that's the base for a summary of a plan --
3 MS. GOLUB: Right.
4 MR. STATTEL: -- nothing real long, but a 5 summary of a plan that describes an existing plan, an 6 actual plan.
7 MS. GOLUB: Right, right, right.
8 MR. STATTEL: And I want to have access to 9 the actual plan to see this type of guidance, right, 10 this level of guidance.
11 So, that's really all I had to say about 12 this. I just kind of wanted to give an idea of what 13 our expectations were.
14 MR. HERB: So, the reason why it's in that 15 format is, when we put the summary -- the summary plan 16 is where you have a level of sweet talk. You know 17 what? We put a little bit more detail about what 18 really goes into the plan --
19 MR. STATTEL: Yes.
20 MR. HERB: -- and then, follow the 21 summary. So, it gives you a little more idea about 22 the plan.
23 MR. STATTEL: I mean, without actually 24 reading it --
25 MR. HERB: And we opted not to do an NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
164 1 actual plan.
2 MS. GOLUB: But it isn't a plan yet. It's 3 just a summary.
4 MR. STATTEL: Now we saw some markups that 5 you gave us, right --
6 MS. GOLUB: Right.
7 MR. STATTEL: -- the changes? I have a 8 couple of comments on that. Or you wanted me to hold 9 off on that?
10 MR. DARBALI: No, I think -- okay, so this 11 just feeds into it. I don't know if there's the need 12 to put WOP on the screen. These are just some 13 questions that we have, and we've started those.
14 I think the last one was Mike's question 15 as to you're going to be performing these activities 16 and we need to coordinate with our inspectors when 17 those inspections would take place. So, we do need to 18 establish a method of updating the staff on when those 19 would be taking place.
20 And then, we've talked about these 21 throughout. So, we can leave these up here and, then, 22 look at the hard copies of the VOP. And we're on 23 time.
24 So, Rich, you wanted to go through each 25 section?
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
165 1 MR. STATTEL: I'll start. So, let's go to 2 page 7. And this applies to more than one section.
3 I just picked this one to make a point.
4 So, you added a reference into ISG-6, D4, 5 24, I think it was. Okay? Everybody following me?
6 MR. ODESS-GILLETT: Yes, software V&V.
7 MR. STATTEL: Right. So, to me, that 8 seems like the wrong reference. So, D4 is essentially 9 -- that's the alternate review process, and that 10 provides the guidance for the V&V plan, right? It's 11 not the guidance for evaluating implementation of that 12 plan, which is actually in ISG-6 in Section D9, right?
13 So, to me, a more appropriate reference, if you're 14 going to refer to ISG-6, it seems to me that D9 would 15 have been a more appropriate reference.
16 MR. BURZYNSKI: Maybe you need both parts.
17 One tells you what you should do and the other one 18 tells you how to do it.
19 MR. STATTEL: Well, right, and what you 20 actually need, you need to have a plan in hand to know 21 what's going to be done. And then, you need to have 22 the report, and, okay, did they do that plan, right?
23 MR. ODESS-GILLETT: But isn't that what 24 this thing is saying? It's saying make sure, verify 25 that the application software, being the program, NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
166 1 complies with.
2 MR. STATTEL: Well, it doesn't comply with 3 the guidance for what's an acceptable plan. It 4 complies with the plan itself.
5 MR. ODESS-GILLETT: See, this seems 6 duplicative because we've already evaluated the plan.
7 We've already approved the plan.
8 MR. HERB: Right. I understand. So, I 9 think the disconnect on what Warren is saying in a way 10 is that, when we refer to D4, we're referring to the 11 D4 section of the LAR we submitted rather than what 12 the guidance -- what goes in the LAR.
13 MS. GOLUB: No, I don't think that's what 14 Warren is saying.
15 MR. ODESS-GILLETT: No, no, no. I'm 16 saying that it says a way to verify that the 17 application software/V&V program complies with the V&V 18 program approved by the NRC. So, you're going to look 19 at what was done. So, we're looking at what was done, 20 comparing it to the approved program in the LAR, as 21 described in ISG-6, D4.
22 MR. STATTEL: But why do you even need to 23 say "as described in ISG-6"? I think that's 24 unnecessary.
25 MR. DARBALI: Well, right, this is your NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
167 1 plan. So, these should be referencing your internal 2 procedure documents for performing that oversight.
3 MR. ODESS-GILLETT: I would reference the 4 LAR section, but I couldn't. So, the best we could do 5 is reference the ISG-6 section that pertained to No.
6 B in the LAR.
7 MR. STATTEL: I guess in a real Vendor 8 Oversight Plan --
9 MR. ODESS-GILLETT: You would reference 10 the LAR.
11 MR. STATTEL: -- you would reference the 12 LAR.
13 MR. HERB: Unless the plan itself is 14 actually created for our procedures, you would create 15 an actual plan, and it's per our procedures.
16 MR. STATTEL: I guess we're trying to 17 create links here that just don't make any -- it's 18 just they're not going to make any sense.
19 MR. ODESS-GILLETT: I agree that it's 20 convoluted, only because we didn't have a real LAR 21 with a section number for the V&V --
22 MR. STATTEL: And when I saw that, I'm 23 like, well, if you really want to look at guidance for 24 implementation of a V&V plan, it exists. You can 25 either point to BTP-14, right, which, clearly, you NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
168 1 don't want to do, or you can point to Section D9, but 2 D9 also provides references to BTP-14. It basically 3 duplicates the BTP-14 guidance.
4 So, I'm not saying you need the 5 references. I prefer not having the references. When 6 I see a real Vendor Oversight Plan, I just want you to 7 know my expectation is there would be some words in 8 there that explain, well, what is that; what does that 9 mean.
10 If you're going to have a signature from 11 the licensee on an SRS document or a V&V summary 12 report -- so, the vendor is creating a V&V summary 13 report, and there's a signature by the licensee. He 14 acknowledges and accepts that report. What's behind 15 that signature? And if there is like a paragraph in 16 here that explains, well, here's what the licensee is 17 doing?
18 And the last part of that paragraph I 19 agree with. It says, right, "The V&V program is 20 implemented in a manner that reliably verifies and 21 validates the design outputs at each stage of the 22 design process." Right?
23 So, in reality, and in my experience, we 24 typically review six or seven or eight of these 25 things. It's not just one report, right? There's NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
169 1 multiple. It multiplies, right? Because each phase 2 has its own V&V summary report.
3 MR. WATERS: One thing, in a real VOP 4 summary, you would not refer to ISG-6?
5 MR. STATTEL: That's right.
6 MR. WATERS: Okay. It's just how it could 7 be used --
8 MR. STATTEL: Right.
9 MR. WATERS: You can't do that for 10 multiple reasons.
11 MR. HERB: No, we wouldn't refer to ISG-6.
12 MS. GOLUB: But, Warren, I just want to 13 make sure that I'm clear. So, what you're saying is 14 that we wouldn't refer to ISG-6; we would refer to the 15 specific section of the LAR that describes that?
16 MR. ODESS-GILLETT: Yes.
17 MS. GOLUB: Okay. But I think I also hear 18 Rich saying that you're looking for, in the actual 19 Vendor Oversight Plan, criteria as well?
20 MR. STATTEL: Criteria, yes.
21 MS. GOLUB: So, not just to point to it, 22 but actual criteria?
23 MR. STATTEL: And I think you answered 24 that earlier. Because when I first read it, I had the 25 impression that was no plan and you were planning on NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
170 1 creating a plan. But, in reality, when we get this, 2 it's going to be a summary of an existing --
3 MS. GOLUB: That's right. Absolutely 4 right, yes.
5 MR. HERB: Yes, it's not a promise to 6 create a plan. We will have a plan when we submit our 7 LAR.
8 MR. ODESS-GILLETT: Actually, it will be 9 approved by the NRC staff.
10 MR. HERB: Yes, yes.
11 MR. ODESS-GILLETT: It's part of the LAR 12 review.
13 MR. HERB: Yes. And we're just putting a 14 summary in the LAR, but the plan itself will be 15 potentially docketable, but we would prefer not to, 16 because it's a live plan. We would rather -- but you 17 would see a full plan.
18 MR. DARBALI: So, I want to go -- did you 19 have --
20 MR. ODESS-GILLETT: Well, it's just Pareez 21 is saying that, and I was sort of reiterating that, 22 the plan is the criteria upon which you review the 23 adequacy of the V&V program. But I think I'm hearing 24 that he would like to see some activities that sort of 25 describe that.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
171 1 MS. GOLUB: And that's why you have those 2 examples? So, you want more than just a "point to 3 it"?
4 MR. STATTEL: I'm not really tasking you 5 with coming up with a pretend example today. I'm just 6 kind of explaining what my expectation is when what's 7 on the plan actually comes in.
8 MS. GOLUB: But, Rich, I mean, the whole 9 reason we're going through this exercise -- and, you 10 know, we put a lot of time into trying to create this 11 exercise -- is to make sure that we understand your 12 expectations and that we have something captured. So 13 that when the first licensee tries to use this, they 14 know what target they're shooting at. Because, as we 15 said, this is new.
16 MR. STATTEL: Okay.
17 MS. GOLUB: And so, we would like to make 18 sure that we understand.
19 MR. DARBALI: So, what we have explains 20 the what. You're saying we need to see "the have"?
21 MR. BURZYNSKI: Or an example of what you 22 do for "the how" is in the second part of BTP-714.
23 MR. STATTEL: That's correct. I know we 24 get accused of it a lot, but we don't come up with 25 this on our own. It's actually written down in there.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
172 1 (Laughter.)
2 MR. BURZYNSKI: I'm just trying to 3 summarize. And "the how" for the design outputs is in 4 the third part of BTP-714?
5 MR. STATTEL: That's right. That's 6 correct. That's correct. And I know it's hard to 7 find because it's mixed in with all those attribute 8 discussions and everything like that. But, honestly, 9 when you boil it down, and like I said before, those 10 words like I provide in those examples, those are 11 right out of the BTP-14.
12 So, I don't know what's going to become of 13 this. I don't know what happens with this.
14 MS. GOLUB: It's going to be incorporated.
15 The idea was that all of these tabletop examples --
16 and we went through a whole bigger tabletop, as you 17 know, in June -- all those examples are supposed to 18 become part of the industry guidance document, so that 19 people who are doing this work, they have a target 20 they're shooting at. And we're hoping it will provide 21 more industry consistency in the documentation that's 22 created that you're eventually going to look at.
23 So, I understand this is facilitating our 24 discussions, but I don't want to -- is it important 25 that we document --
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
173 1 MR. STATTEL: So, it would be it becomes 2 some NEI --
3 MS. GOLUB: Yes, that's right.
4 MR. STATTEL: That the licensee would take 5 up and --
6 MS. GOLUB: It would be an example. This 7 wouldn't be an exact template, but it would be an 8 example that the staff has provided input on.
9 And we'll have to kind of take this back 10 ourselves and contemplate this a little bit, about if 11 we try to build out one of these sections more with 12 more information, but I want to be careful. Because 13 if we include this in an NEI document, licensees are 14 going to create something similar. And so, we want to 15 make sure that the target that we're providing is a 16 target that makes some sense.
17 MR. STATTEL: If we don't put it right, 18 it's just going to be copied and pasted into 19 somebody's amendment, and that's not going to cut it.
20 So, what I would suggest is something like this. I 21 don't like the references to ISG or BTP-14. But what 22 I would suggest is, you know, I started an activity of 23 kind of boiling down the BTP-14 guidance. I'll 24 complete that, and I can provide you some words. And 25 maybe we can put them in brackets or something.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
174 1 Essentially, something like this is what we would 2 expect, and to tell the licensee: don't copy this, 3 but write what fits for you -- or the vendor, you 4 know.
5 MS. GOLUB: Your input would be welcomed.
6 MR. STATTEL: Do you see what I mean?
7 MS. GOLUB: Yes.
8 MR. STATTEL: So, I'll take that task on.
9 MS. GOLUB: And it doesn't have to be for 10 all of it. I mean, you gave us a couple of examples.
11 I think that would be helpful.
12 MR. STATTEL: It's not that much really.
13 MR. HERB: When you say that, on our side 14 we think you're asking, it almost feels like you're 15 asking us to replicate your platform acceptance if it 16 has already been done. And then, part of that may be 17 a software application manual and it plans it. So, 18 when we purchase that, we come with an assurance that 19 that's probably a very rigorous development process 20 that they've already done, and we're not going to 21 accept it at blind faith, but I don't know that we 22 have to go to the degree that you already have done, 23 to say that's an acceptable process, as per the LAR.
24 MR. STATTEL: Well, I'm really not 25 offering that.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
175 1 MR. HERB: Are you asking us to do that?
2 MR. STATTEL: No. I'm really not talking 3 about platform.
4 MR. HERB: Okay. Well, I'm talking about 5 the platform and the application manual.
6 MR. ODESS-GILLETT: I think he meant, when 7 he said "platform," I think it's like the software 8 program manual.
9 MR. HERB: The software program manual.
10 MR. STATTEL: Well, right, and that's 11 true. There are -- what? -- three software program 12 manuals approved.
13 MR. HERB: Right, but if we had one that 14 wasn't an approved one, do we have to go back and 15 reconfirm that --
16 MR. STATTEL: It's hard. It's going to be 17 hard. If you have an approved software program 18 manual --
19 MR. BURZYNSKI: No, but you just need to 20 address if you've implemented it.
21 MR. ODESS-GILLETT: What Rich is saying is 22 that, okay, just list a few activities as it's kind of 23 described in BTP-714. How would you do that to the 24 SPM, for instance?
25 MR. STATTEL: So, I refer to these as the NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
176 1 Lozier words, because in the revision to BTP-14 in 2 2007, Paul wrote these words, right? He added those 3 into BTP-14. And I can recognize them. I key-in on 4 those.
5 MR. HERRELL: It's not hard -- it's harder 6 to find them and the other words that are there aren't 7 as useful.
8 MR. STATTEL: That's right. That's right.
9 But, no, going back to the SPM, I think a 10 lot of credit can be taken for an approved SPM.
11 There's really no need to re-review the plans unless 12 we know of significant changes, right? But the 13 implementation is going to be every time you have a --
14 MR. HERB: No, I understand that. So, 15 we're confirming that they're following they're 16 following the plans that were probably approved and 17 the processes. And we want to get deep into that 18 process through questioning, through interviews, 19 through inspection of our products when we get back.
20 And so, I understand. That's what I thought you were 21 asking, but when you started those other things you 22 were talking about, I was thinking, wait a second; are 23 we questioning the validity of the plan that was part 24 of the --
25 MS. GOLUB: Yes. No, I don't think that's NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
177 1 what Rich is saying.
2 MR. HERB: Are we going to look to see 3 what the criteria of that plan and compare it to the 4 original BTP-714 criteria?
5 MS. GOLUB: Just the implementation.
6 MR. HERB: Or just the implementation?
7 MS. GOLUB: Right.
8 MR. HERB: So, I didn't think it was just 9 the implementation.
10 MS. GOLUB: But I think Rich is looking 11 for the Vendor Oversight Plan to contain some criteria 12 for how you plan to verify that implementation, that 13 it's not just, you know, "I did it in accordance with 14 the plan." I mean, it's something with a few --
15 MR. STATTEL: I don't think we can write 16 this in a generic way --
17 MS. GOLUB: Right.
18 MR. STATTEL: -- that it can just be cut 19 and pasted into --
20 MS. GOLUB: Right. You cannot, yes. You 21 can provide a couple of examples, but, yes, it's got 22 to be specific.
23 MR. STATTEL: So, I think we're going to 24 have to just bracket it and just something like this.
25 Otherwise, we're just going to be transferring BTP-14 NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
178 1 into here.
2 MS. GOLUB: Right. That's why I was 3 thinking, you know, if we could get an actual example 4 of something that would be a criteria, we could pick 5 it from one of the vendors. I don't know.
6 MR. ODESS-GILLETT: Or we could just --
7 you know, we've gotten feedback, and Mark has already 8 highlighted this for us, those sections of BTP-714.
9 We could come up with those little pieces of criteria 10 for each one of these.
11 MR. STATTEL: Well, I've already started 12 down that path of boiling it down. So, why don't we 13 finish that activity?
14 MS. GOLUB: Yes, if you could give us 15 that, that would be great. And then, we could kind of 16 bounce it off of what we have. But Mark actually went 17 through and did the exercise of highlighting certain 18 sections. You actually boiled it down a little bit.
19 But what Mark had done --
20 MR. HERB: But we actually got into the 21 plan rather than the actual summary of a plan.
22 MS. GOLUB: But Mark went through and kind 23 of did something similar.
24 MR. STATTEL: And I took some liberties.
25 Even in these examples, I did change some words NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
179 1 around. So, it's not a quote.
2 MS. GOLUB: Right, right. That's, I 3 think, the difference. But we did kind of start down 4 that path, just to see if in general it made some 5 sense.
6 MR. HERB: It's good feedback.
7 MS. GOLUB: Uh-hum, good feedback, really 8 good feedback.
9 MR. DARBALI: So, I need some 10 clarification. Earlier, before the break, right 11 before the break, we were talking about the licensee 12 as the term vendor oversight for the process. All 13 right, but the technical -- you have your design 14 acceptance of documents --
15 MR. HERB: That's called owner acceptance.
16 MR. DARBALI: Right.
17 MR. HERB: Yes.
18 MR. DARBALI: So, we agree they're all 19 license activities of vendor activities -- licensee 20 oversight activities of vendor development activities.
21 MS. GOLUB: Right.
22 MR. DARBALI: But what I want to be clear 23 is that the Vendor Oversight Plan and the summary is 24 going to include both.
25 MR. HERB: Okay. I understand.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
180 1 MS. GOLUB: Right. I think we got that 2 message, yes.
3 MR. DARBALI: Okay.
4 MR. HERB: I think we got that, yes.
5 MR. DARBALI: Okay.
6 MR. BURZYNSKI: And some discussion in the 7 vendor oversight template that married your part 3 8 expectations with what we do in our standard design 9 process, that that would illustrate what we're doing 10 and how we're doing it.
11 MR. DARBALI: Right. Because when we're 12 talking about inspection and vendor oversight, there 13 was a disconnect. "Oh, we don't call that vendor 14 oversight."
15 MS. GOLUB: Yes.
16 MR. DARBALI: And regardless of what you 17 call it, we're looking at the vendor oversight.
18 MS. GOLUB: That's right. That's exactly 19 right, yes. No, good feedback. That's good feedback.
20 MR. HERB: And we're saying that's part of 21 owner acceptance reviews and that would be here. And 22 when you all came to inspect, you would know where to 23 go to get it.
24 MR. DARBALI: Right.
25 MR. HERB: Exactly.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
181 1 MR. FANELLI: They would be careful. This 2 would become less -- put in there and accepted, right?
3 MR. STATTEL: Well, in the actual plan, 4 yes.
5 MR. ODESS-GILLETT: Yes, the summary of 6 the plan.
7 MR. FANELLI: So, be very careful of the 8 words.
9 MR. STATTEL: And you'll be using that.
10 (Laughter.)
11 MS. GOLUB: It carries a little more 12 weight then, doesn't it?
13 But your point is well taken. It's almost 14 like vendor oversight, the capital "VOP" versus like 15 the activity of vendor oversight.
16 Are you okay with that, Steve?
17 MR. DRAGOVICH: Yes, I think it's a good 18 plan.
19 MS. GOLUB: Putting a couple of 20 paragraphs?
21 MR. STATTEL: So, if it's okay with you, 22 I think we have a Word version of this. So, I'll go 23 ahead and edit right into it?
24 MS. GOLUB: That would be great.
25 MR. WATERS: We've got to be careful when NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
182 1 we share information.
2 MR. STATTEL: Okay.
3 MR. WATERS: I want to --
4 MR. STATTEL: Either way.
5 MR. DARBALI: We can try to have another 6 meeting or it could be a call, but I would suggest 7 after next year, after the holidays.
8 MS. GOLUB: Yes, this activity, the 9 inspection process, this whole meeting has been super-10 helpful, a lot of clarity, and I think we feel a 11 little bit better about the direction this whole thing 12 is going.
13 MR. DARBALI: Okay.
14 MS. GOLUB: But it's a really important 15 activity for the industry.
16 MR. WATERS: Let me clarify. I have no 17 problem with sharing his insights on this, but I don't 18 want it coming across as we are developing a plan for 19 you. The plan should be -- just to share this is all.
20 MS. GOLUB: Yes, right, and we would take 21 it in that context. But the point I'm trying to make 22 is that we're happy to support another meeting, you 23 know, another phone call, whatever it takes to make 24 sure that we're in a good place.
25 We just don't want the first licensee, who NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
183 1 happens to be sitting here, to have to wade through 2 these waters on their own.
3 MR. SCHRAYE: That won't hold up the 4 issuance of an ISG, will it?
5 MR. DARBALI: No. No, no, this is 6 separate.
7 MR. ODESS-GILLETT: This is the industry 8 guidance --
9 MS. GOLUB: Yes.
10 MR. ODESS-GILLETT: -- that follows the 11 ISG.
12 MS. GOLUB: That's right. Well said.
13 MR. DARBALI: So, unless there's any more 14 questions on the topics today, could you expand a 15 little bit on that industry guidance document? When 16 do you expect it to be complete?
17 MS. GOLUB: Yes. So, right now, we're 18 targeting end of first quarter 2019.
19 MR. DARBALI: Okay.
20 MS. GOLUB: So, it's not too far off.
21 MR. DARBALI: Okay.
22 MS. GOLUB: I'm writing it and, then, Dave 23 Houghton is writing it. Dave went through some 24 health-related issues.
25 MR. DARBALI: Right.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
184 1 MS. GOLUB: And so, he's doing well. When 2 he comes back -- you know, that's why we're kind of 3 shifting when we're writing it a little bit. But 4 we're sort of targeting that timeframe.
5 MR. DARBALI: Okay. And will you be 6 working with the new, the first use of ISG-06 licensee 7 on this?
8 MS. GOLUB: Yes.
9 MR. HERB: To the extent we can. To the 10 extent we can.
11 MR. DARBALI: Okay. Because it would be 12 helpful --
13 MS. GOLUB: Yes.
14 MR. DARBALI: -- both the learning 15 experience, but also help improvement --
16 MS. GOLUB: I agree. I agree.
17 MR. DARBALI: -- and for them to use the 18 document.
19 MS. GOLUB: Yes, agreed. Absolutely.
20 MR. DARBALI: Okay. And are there any 21 plans for the staff to get to see that?
22 MS. GOLUB: Yes. So, we've talked about 23 that before. I mean, the industry would welcome your 24 thoughts, your feedback. Obviously, we're not giving 25 it to you for review per se.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
185 1 MR. DARBALI: Right, right.
2 MS. GOLUB: But, yes, we would 3 absolutely -- because, again, this is going to be 4 what's guiding licensees as they go down this path.
5 And it's an expensive path. It's an important path, 6 a path to better safety, you know, plant longevity, 7 asset management, all of that good stuff. So, we 8 would welcome input, so that the guidance is really 9 good guidance; it's guidance that makes sense.
10 And hopefully, it will be a living 11 document. So, as the first licensee, the second 12 licensee goes through, we can amend that guidance with 13 those lessons learned.
14 MR. DARBALI: Well, the next topic on the 15 agenda is the closing comments and any follow-up 16 actions. So, before the comments, we have an action 17 to take a look at that ISG-06 language that says use 18 NQA-1 as a reference. Like we said, the ISG is 19 currently with OGC. So, we will internally see what 20 language we can agree with that cannot be 21 misunderstood, and see if we can incorporate that into 22 the ISG.
23 MR. ODESS-GILLETT: Or the alternative is 24 to document the interpretation in meetings --
25 MR. DARBALI: Right, right.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
186 1 MS. GOLUB: We prefer the former, though.
2 MR. ODESS-GILLETT: Yes, we prefer the 3 former, but if that's --
4 MS. GOLUB: Exactly. If we don't have to --
5 MR. DARBALI: Right.
6 MS. GOLUB: So, everybody can see it.
7 MR. DARBALI: Mike, I'm going to put you 8 on the spot. Do you think we can take five minutes 9 here to try to agree on the language or?
10 MR. WATERS: Sure.
11 MR. ODESS-GILLETT: Well, actually, it was 12 either you or Mike who suggested changing "vendor" to 13 "licensee".
14 MR. DARBALI: Well, yes, I did write that 15 down. I think it was Rich or Mike that said that.
16 MR. ODESS-GILLETT: Maybe we could put it 17 up and take a look.
18 MR. DARBALI: So, I'll read it.
19 MS. GOLUB: Yes, the wording is different 20 in the actual document.
21 MR. DARBALI: It is, yes. I mean, you can 22 type it.
23 MR. STATTEL: This isn't a prerequisite to 24 using the alternate review process?
25 MR. DARBALI: It's repeated in the NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
187 1 prerequisite, yes. Yes, it's also on the page before 2 it.
3 MR. STATTEL: Okay.
4 MR. DARBALI: Joe, can we type? Can we 5 change? Can we go out and edit the slide?
6 MR. GALLETTI: Hey, Warren, could you 7 repeat? I mean, I'm trying to get my hands around why 8 originally the industry proposed having any reference 9 to NQA-115, Subpart 2.7.
10 MR. ODESS-GILLETT: Gosh, the team, can 11 you help me remember?
12 MS. GOLUB: Yes. At the time, we were 13 thinking about -- and I don't actually have it here.
14 We wanted a common framework. You know, we wanted 15 sort of a common base that we were both working from.
16 So, we weren't just kind of committing to something 17 arbitrary, but we were committing to, the licensees 18 were committing to examine their vendor to a certain 19 process, to a certain level, right.
20 So, the idea was to have a common 21 framework, and that's why we ended up settling on this 22 Section 2.7 of this particular version. And we kind 23 of went back and forth on that, too. Some people 24 said, "Oh, we should change it to Reg Guide 1128."
25 Right?
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
188 1 MR. HERB: Right. Yes, and we thought 2 about just taking the year off of it completely, but, 3 then, they said --
4 MS. GOLUB: But I think the idea was a 5 common framework, you know.
6 MR. ODESS-GILLETT: Yes, as you had 7 mentioned, licensees all have different levels of --
8 MR. GALLETTI: Right, but, I mean, if you 9 were to remove this completely from this document, the 10 way this would, in practicality, get implemented at 11 each vendor consistently was to have some sort of 12 understanding that in the PO that you're sending them 13 you would specify this, whatever you want it to be.
14 But that's the formal mechanism to get that into the 15 requirements, if you will.
16 The problem, when we looked at this 17 originally, and it said something to the effect that 18 you'll ensure that it's consistent with this, it 19 implied that, hey, you, Licensee, you're an N45.2 20 plant. Well, you're not committed to this. So, all 21 of a sudden, now you're committing to something that 22 we don't want you to necessarily do, right?
23 MS. GOLUB: Yes, your point is well taken.
24 And we went kind of round and round on our side with 25 those words because, I mean, some of the folks sitting NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
189 1 at the table have N45.2 plants. And so, that 2 "consistent with" was chosen carefully not to be 3 "comply with" or --
4 MR. BURZYNSKI: Yes, we thought it was not 5 saying "conforms with".
6 MS. GOLUB: Right.
7 MR. FANELLI: So, you're just saying you 8 really want some software-specific quality assurance?
9 MS. GOLUB: Yes, that's exactly right.
10 MR. FANELLI: So, why don't you say that 11 instead of --
12 MS. GOLUB: Well, because we weren't sure 13 how that would be interpreted. So, having like kind 14 of a common framework we thought would make it easier 15 for licensees to understand what they were doing and 16 vendors to understand what that framework was, and NRC 17 to understand what was going on in licensee/vendor 18 space. So, right, wrong, or indifferent, that was the 19 idea.
20 MR. GALLETTI: But the purpose of doing 21 that was it was giving you, the licensee, who has 22 responsibility for oversight of the vendor, right --
23 basically, you're putting them on record as saying, 24 hey, Vendor, I'm going to come out and I'm going to 25 evaluate your program, and I'm going to use this as NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
190 1 guidance. Right? Is that sort of what you're saying?
2 MR. HERB: Yes.
3 MR. GALLETTI: Okay.
4 MR. HERB: Yes. It says the N45 plants 5 shouldn't have any --
6 MR. HERRELL: The reason we chose the 2015 7 version was (a) it was the most kind and (b) it was 8 the one place where we actually managed to put most of 9 the software requirements in one section.
10 MR. STATTEL: Right, and we endorsed it, 11 which is even better.
12 MS. GOLUB: Yes. Yes, that's right.
13 MR. ODESS-GILLETT: Right. And we wanted 14 something endorsed, but we didn't want to like say 15 IEEE 730 because that's not an endorsable standard.
16 MR. DRAGOVICH: I agree, though, couldn't 17 it be handled as part of the procurement specs? For 18 example, we send out a spec and we say you have to 19 comply with 2015, and they come back and say, well, we 20 have 2012. So, they write a conform spec. We agree 21 with that, and then, that's what we end up using.
22 Would that solve that problem, if we rely on that, or 23 would we have to?
24 MR. HERRELL: We really want to establish 25 a base --
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
191 1 MS. GOLUB: Right.
2 MR. DRAGOVICH: And why can't we use the 3 procurement spec to do that?
4 MR. HERRELL: You start with this, and 5 they come back and give you an alternative. You 6 accept the alternative.
7 MR. VAUGHN: What about a vendor uses --
8 you heard that referenced -- the appropriate software 9 standards and list as many examples as you want? I'm 10 not saying you should do that, but you can list more 11 than this one. Isn't that what you want, some sort of 12 software quality assurance standard and you want some 13 base level? Say that, and then, if you have a 14 different one -- and I don't even know what it was --
15 do that as an example. If this is the best example, 16 have this one first.
17 MS. GOLUB: Yes, so this was the best 18 example right now.
19 MR. VAUGHN: But, then, you make it -- I 20 mean, when the lawyers look at this, they're going to 21 think that it has to be this, I think. Don't make it 22 like that. Make it appropriate software quality 23 assurance standard. That's an "e.g.". "Such as" 24 this; "such as" this. Yes, "such as," even this.
25 MS. GOLUB: You're right. So, we still NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
192 1 get the function of having some kind of a base level, 2 but it softens the language to make -- yes, so that 3 somebody reading this five years from now doesn't 4 inadvertently --
5 MR. STATTEL: I would make the example a 6 separate sentence, right? So, in the middle bullet 7 you would say, "The vendor uses" an appropriate 8 reference QA program, whatever, something like that.
9 And then, in the final bullet you would say, "As an 10 example," NQA-1 could be the reference or something 11 like that. That way, you don't even put it in the 12 same sentence. There's no chance anyone is going to 13 interpret that as being the required criteria.
14 MR. HERRELL: Rich, as a vendor, I am 15 required to use the most current version of NQA-a, 16 unless I've got an SPM that's based on an earlier 17 version.
18 MR. GALLETTI: Well, you're required to 19 use whatever the licensee puts in their PO, right?
20 MR. HERRELL: It requires my program to be 21 based on the most current guidance.
22 MR. FANELLI: As a general rule.
23 MR. HERRELL: As a general rule.
24 MR. STATTEL: Well, that first statement 25 you made is not true for all vendors. I can guarantee NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
193 1 that.
2 MR. GALLETTI: Yes, I mean, we've got 3 vendors that are approved in 1994.
4 MR. STATTEL: Or foreign vendors that 5 don't even --
6 MR. GALLETTI: But, I mean, you may want 7 to put something in here, an appropriate QA software 8 program. You may want to consider something that's 9 been endorsed by the NRC, and that covers several 10 versions of NQA-1.
11 MS. GOLUB: I mean, I like this language.
12 What do you guys think? Dave? Mark? Frank? I want 13 to make sure I follow the -- yes, Frank, I know you've 14 had some energy about this and had some good comments 15 on this in the past.
16 MR. GALLETTI: What Rich said, I mean, end 17 the sentence there, and then, have another bullet "for 18 example," like another bullet. "For example," NRC-19 approved versions of NQA-1. And you can put in 20 parentheses "(2008, 2009, 2014-15 addenda)," et 21 cetera.
22 MR. ODESS-GILLETT: We didn't really like 23 "NRC-approved" earlier in the document.
24 MR. DARBALI: NRC-endorsed, right?
25 MS. GOLUB: I think even if we keep the NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
194 1 year on there in the subpart, like that Subpart 2, 2 just because --
3 MR. GALLETTI: Well, that is the subpart.
4 That's the one that's the current --
5 MS. GOLUB: No, but other versions have it 6 spread out in a different place. Right? So, that's 7 the reason that we picked 2015, was because it was 8 condensed.
9 MR. GALLETTI: I think we know we have 10 2.7, and '14-15 addenda are 2.7.
11 MR. HERB: '98 is like 2.7-plus.
12 MR. GALLETTI: Yes, but I don't think we 13 endorsed --
14 MS. GOLUB: I think if we have one 15 example, one example is sufficient.
16 MR. HERB: So, all the ones you've 17 endorsed, it's Subpart 2.7?
18 MR. GALLETTI: Well, at least for 2008 --
19 MS. GOLUB: For example, NRC endorsed the 20 2015 version of blah, blah, blah. I think that should 21 be sufficient.
22 MR. GALLETTI: We endorsed '94 and, then, 23 there was a long hiatus because the industry had done 24 a bunch of things to it that we didn't approve of.
25 Well, you basically have 2008-2009 addenda, and you NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
195 1 have 2014 to --
2 MS. GOLUB: Yes, I think that makes sense 3 in there. So, it's an example. We only need one 4 example.
5 MR. GALLETTI: It is an example.
6 MS. GOLUB: Right, yes. And it kind of 7 captures that NRC endorsed --
8 MR. HERB: So, you don't need the 2015?
9 MS. GOLUB: No, I think we do.
10 MS. ZHANG: I think, for example, the NRC 11 endorsed 2015, I think it's good.
12 MS. GOLUB: I do, too, yes.
13 MR. DRAGOVICH: So, how about if you 14 delete "can also be used to" and making sure plural --
15 because you have "used" twice in that first sentence.
16 Is it saying the same thing?
17 MR. DARBALI: Well, what we don't want to 18 say is that the VOP ensures something.
19 MS. GOLUB: Yes, that's right. That's 20 right.
21 MR. DARBALI: Yes, we're not striving for 22 perfect.
23 (Laughter.)
24 So, it seems that everybody is in 25 agreement with this?
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
196 1 MS. GOLUB: Yes.
2 MR. DARBALI: I mean, we're not going to 3 reconcur, but we need to see if that language changes 4 the concurrence. Fortunately, the folks that are 5 interested in that part are here. So, we'll take that 6 as the base and we'll do that after --
7 MS. GOLUB: We appreciate that. Thank 8 you. Thank you for your consideration.
9 MR. VAUGHN: If OGC has a comment on this, 10 it will be easy. If they don't, then you have to say, 11 well -- and they may or may not want to --
12 MS. GOLUB: But this was the intent.
13 MR. DARBALI: Yes.
14 MS. GOLUB: You know, this was the intent.
15 It may not have been worded quite as well, but it was 16 the intent.
17 MR. DARBALI: Okay.
18 MR. WATERS: Well, at a minimum, in the 19 meeting summary you can say we discussed this and --
20 MR. GALLETTI: You just say it was a 21 clarification.
22 MS. GOLUB: That's right, it is a 23 clarification. It's not changing the intent.
24 MR. DARBALI: All right.
25 MS. GOLUB: Thank you for doing that.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
197 1 MR. DARBALI: So, I guess we were doing 2 the follow-up actions. So, this is some of our 3 follow-up actions.
4 I believe, Rich, you took the action to 5 look at those specific activities and update the 6 Vendor Oversight Plan?
7 MR. STATTEL: Yes, yes.
8 MR. DARBALI: And I guess I had written 9 industry was to consider how to add technical aspects 10 into the VOP.
11 MS. GOLUB: Yes, that's right. That's 12 right.
13 MR. DARBALI: And again, the ISG is almost 14 done. We're not going to be doing tracking of action 15 items.
16 MS. GOLUB: That's right.
17 And then, maybe, Samir, we can kind of 18 contemplate if we want a phone call or something to 19 follow up on this.
20 MR. DARBALI: Right, right. So, follow-up 21 just to schedule follow-up on it.
22 MS. GOLUB: Right.
23 MR. DARBALI: Okay. And with that, I 24 think the other item was just to do closing arguments.
25 So, I'll turn it to Mike.
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
198 1 (Laughter.)
2 Well, Eric's not here.
3 MR. WATERS: No, I have nothing to say.
4 I think, as you know, it was a really beneficial 5 meeting, and I think we learned a lot from both sides 6 and come to a clearer understanding.
7 My real question -- and I don't have an 8 answer -- is a little clarification, and we'll look at 9 inspection programmatic issues to make sure. The ISG 10 is going to be issued in December. That's our plan.
11 What's next? I don't want to leave on this. I've 12 heard rumor of a licensee coming in next year, which 13 may push a test of this. But the question to think 14 about is, what do we do next? What is our next? We 15 don't need to interact anymore on this or not? So, 16 something to think about.
17 MS. GOLUB: Yes, let's think about that.
18 And, Mike, I know you've asked this several times, and 19 we kind of keep dodging it a little bit, about what 20 the schedule is. But I know licensees aren't 21 providing you a schedule, but I think I can speak to 22 just, on Tuesday-Wednesday of this week, we had a big 23 meeting at EPRI, a lot of licensees present. There's 24 a huge amount of interest in moving forward with these 25 modernization plans. And at this point, it's just NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
199 1 figuring out the "how" and the "when". But it's not 2 a possibility. I mean, it sounds --
3 MR. HERB: No, these are big plans that 4 take a lot of planning. We've changed the way the 5 industry, I think, is viewing this process, largely 6 through this effort. And so, these things take a 7 while to spin up. They're large capital projects.
8 Capital projects and budgets are out 5 to 10 years.
9 And so, to do these projects, you have to move other 10 capital projects out of the way. So, that takes time 11 and effort. And a large part of that assurance that 12 we have a good process goes a long way to --
13 MS. GOLUB: Yes.
14 MR. WATERS: And I hear you, and NRC's 15 business is to tell you when it's in the application 16 and not be critical --
17 MS. GOLUB: No, but --
18 MR. HERB: We want it as soon as we can.
19 MS. GOLUB: That's right. That's right, 20 but this effort is going a huge way toward providing 21 licensees that assurance, though.
22 MR. HERB: I would like to submit more, if 23 I had a magic ball to be able to go back five years 24 and be prepared.
25 (Laughter.)
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433
200 1 But I don't.
2 MR. WATERS: We have inspection folks 3 here. Do you have any comments to make to his remarks 4 or any further observations?
5 MR. GALLETTI: I thought it was a very 6 fruitful meeting. I appreciate the candor and their 7 discussion.
8 MS. GOLUB: Yes, we thank you very much 9 for your time. We appreciate all of you guys coming 10 in and participating.
11 And, Samir, thank you. I know this was a 12 lot to put together. We really, really appreciate all 13 of your time on this. It was really helpful for us.
14 So, thank you.
15 MR. GOLLA: Okay. Are there are any 16 comments from members of the public on the phone?
17 (No response.)
18 All right. Hearing none, anyone else on 19 the phone, if you would like to make a comment?
20 (No response.)
21 Okay. Thanks for a very good meeting.
22 MS. GOLUB: Thank you very much.
23 MR. GOLLA: We really enjoyed it.
24 (Whereupon, at 4:38 p.m., the meeting was 25 adjourned.)
NEAL R. GROSS COURT REPORTERS AND TRANSCRIBERS 1323 RHODE ISLAND AVE., N.W.
(202) 234-4433 WASHINGTON, D.C. 20005-3701 (202) 234-4433