IR 05000382/2021014
| ML22067A159 | |
| Person / Time | |
|---|---|
| Site: | Waterford  |
| Issue date: | 03/09/2022 |
| From: | Nick Taylor NRC/RGN-IV/DORS/EB2 |
| To: | Ferrick J Entergy Operations |
| References | |
| IR 2021014 | |
| Download: ML22067A159 (13) | |
Text
March 9, 2022
SUBJECT:
WATERFORD STEAM ELECTRIC STATION, UNIT 3 - DIGITAL INSTRUMENTATION AND CONTROL MODIFICATION INSPECTION REPORT 05000382/2021014
Dear Mr. Ferrick:
On January 27, 2022, the U.S. Nuclear Regulatory Commission (NRC) completed an inspection at Waterford Steam Electric Station, Unit 3 and discussed the results of this inspection with Matt Lewis, General Manager - Plant Operations and other members of your staff. The results of this inspection are documented in the enclosed report.
No findings or violations of more than minor significance were identified during this inspection.
This letter, its enclosure, and your response (if any) will be made available for public inspection and copying at http://www.nrc.gov/reading-rm/adams.html and at the NRC Public Document Room in accordance with Title 10 of the Code of Federal Regulations 2.390, Public Inspections, Exemptions, Requests for Withholding.
Sincerely, Nicholas H. Taylor, Chief Engineering Branch 2 Division of Operating Reactor Safety Docket No. 05000382 License No. NPF-38 Enclosure:
Inspection Report 05000382/2021014 Signed by Taylor, Nicholas on 03/09/22
ML22067A159 X SUNSI Review By: NHT ADAMS:
X Yes No Sensitive X Non-Sensitive Non-Publicly Available X Publicly Available Keyword NRC-002 OFFICE DORS/EB2/RI NRR/DEX/ELTB NSIR/DPC/CSB NRR/DRO/IQVB NAME SMakor SDarbali KLawson-Jenkins DZhang SIGNATURE
/RA/
/RA/
/RA/
/RA/
DATE 3/8/2022 3/8/2022 3/8/2022 3/8/2022 OFFICE DORS/RPBD/C DORS/EB2/C NAME JDixon NTaylor SIGNATURE
/RA/
/RA/
DATE 3/8/2022 3/9/2022
Enclosure U.S. NUCLEAR REGULATORY COMMISSION
Inspection Report
Docket Number:
05000382
License Number:
Report Number:
Enterprise Identifier:
I-2021-014-0002
Licensee:
Entergy Operations, Inc.
Facility:
Waterford Steam Electric Station, Unit 3
Location:
Killona, LA
Inspection Dates:
January 3, 2022 to January 27, 2022
Inspectors:
S. Makor, Reactor Inspector, Team Lead
S. Darbali, Electronics Engineer
K. Lawson-Jenkins, IT Specialist (Cyber)
D. Zhang, Sr Reactor Ops Engineer
Approved By:
Nicholas H. Taylor, Chief
Engineering Branch 2
Division of Operating Reactor Safety
SUMMARY
The U.S. Nuclear Regulatory Commission (NRC) continued monitoring the licensees performance by conducting a Digital Instrumentation and Control Modification Inspection at Waterford Steam Electric Station, Unit 3, in accordance with the Reactor Oversight Process.
The Reactor Oversight Process is the NRCs program for overseeing the safe operation of commercial nuclear power reactors. Refer to https://www.nrc.gov/reactors/operating/oversight.html for more information.
List of Findings and Violations
No findings or violations of more than minor significance were identified.
Additional Tracking Items
None.
INSPECTION SCOPES
Inspections were conducted using the appropriate portions of the inspection procedures (IPs) in effect at the beginning of the inspection unless otherwise noted. Currently approved IPs with their attached revision histories are located on the public website at http://www.nrc.gov/reading-rm/doc-collections/insp-manual/inspection-procedure/index.html. Samples were declared complete when the IP requirements most appropriate to the inspection activity were met consistent with Inspection Manual Chapter (IMC) 2515, Light-Water Reactor Inspection Program - Operations Phase. The inspectors reviewed selected procedures and records, observed activities, and interviewed personnel to assess licensee performance and compliance with Commission rules and regulations, license conditions, site procedures, and standards.
OTHER ACTIVITIES
- TEMPORARY INSTRUCTIONS, INFREQUENT AND ABNORMAL
52003 - Digital Instrumentation and Control Modification Inspection Digital Instrumentation and Control Modification Inspection
(1)
Digital Instrumentation and Control (I&C) Modification Inspection (1 Sample)
The NRC issued Amendment No. 260 to Renewed Facility Operating License No. NPF 38 for the Waterford Steam Electric Station, Unit 3. This amendment revises various technical specifications in order for the licensee to implement a planned modification that will replace the existing core protection calculator (CPC)system and the control element assembly calculator (CEAC) system with a Westinghouse Electric Company (WEC) Core Protection Calculator System (CPCS).
The inspectors performed this inspection to evaluate the licensees performance of Site Acceptance Testing (SAT) activities through record review and direct observation. The inspectors focused on the licensees conduct and performance of the SAT and verified open Vendor Oversight Plan (VOP) inspection activities from the Factory Acceptance Testing (FAT). The team evaluated as available whether the licensee developed, implemented, tested, installed, operated, and maintained the design according to the license amendment (LA), safety evaluation (SE), and in accordance with the manufacturers recommendations, and license commitments, including the commitments provided in the licensees cyber security plan (CSP).
The inspectors confirmed that the licensees upgrade to the system conformed to the plant-specific licensing basis and satisfies applicable guidance for this phase of the modification given that the site is actively modifying the CPCS system.
This confirmatory inspection focused on the SAT which is normally performed by rerunning selected portions of the FAT and verification of the licensees performance of VOP activities through direct observation of the licensees VOP performance at the vendor facility. The inspectors reviewed supporting documents including implementing procedures, work orders, inspection reports, engineering evaluations, and condition reports; and conducted interviews with licensee staff.
INSPECTION RESULTS
Observation: Site Acceptance Testing (SAT)
52003 Site Acceptance Testing (SAT)
1. The inspectors conducted the SAT Phase of the digital instrumentation and modification as
described below: 1.1 Licensee's Vendor Oversight Plan (ref. 2.01)
a. Inspection Scope
The inspectors verified that the licensees audit of vendor activities to ensure establishment and implementation of each Quality Assurance (QA) program attribute were performed in accordance with the licensees NRC-approved QA program and the VOP. The inspectors reviewed the results of oversight activities Entergy performed on WECs implementation phase development activities and design outputs for the CPCS. The inspectors confirmed that Entergy conducted the audit activities identified in the Waterford 3 (WF3) CPCS VOP Summary, Revision 2, for the implementation phase development activities and design outputs with respect to the CPCS configuration release record documented prior to the FAT.
The inspectors discussed with the licensee the receipt inspection process and reviewed the licensees material receipt procedure (EN-MP-120, Material Receipt, Revision 20). The licensee explained that the receipt inspection process for the CPCS replacement starts when the equipment is received from WEC and is finalized upon completion of the SAT. The licensee explained that all parts and components were moved from the shipping area to a cyber-secure testing facility to maintain proper cyber control of the equipment, and that components were individually checked by technicians to ensure all components had arrived.
The inspectors reviewed WEC document WNA-CC-04265-CWTR3, Certificate of Conformance, Revision 0, and the WEC purchase order deviations for a change to the cable dimensions and for other changes to the purchase order language that do not apply to the CPCS replacement. The inspectors also reviewed WEC document CCF#:01 to WNA-CC-04265-CWTR3, Contingent Certificate of Conformance Control, dated September 22, 2021, which details that the CPCS equipment is shipped with the software version used for FAT. The inspectors reviewed the Entergy letters accepting the purchase order deviations and the software release letter.
The inspectors reviewed a working copy of OSD&D-0040223, Assembly, CPC Four Channel Assembly, EA Channel 1 of 4 of the Four-Channel Assembly, dated January 22, 2022, which captures several non-conformances identified by the licensee during the receipt inspection process. The inspectors discussed the non-conformances with the licensee and requested further details. The inspectors will review the resolution of these non-conformances and the receipt inspection report during the site installation inspection.
This portion of the inspection partially completed section 2.01 of Inspection Procedure 52003, Digital Instrumentation and Control Modification Inspection, dated July 1, 2021.
b. Observations and Findings
The inspectors reviewed the capturing and tracking of commitments made in the VOP and were unable to confirm that the licensee verified that the Technical Manual satisfies the requirements for the Software Operations Plan per the Common Q Software Program Manual and that the requirements adopted from 00000-ICE-30158, Revision 14 without modification have been satisfied in the as-built WF3 CPCS.
The inspectors will complete the review of this item during the installation of the CPCS modification during Spring 2022, Refueling Outage 24.
1.2 Design and Documentation Verification (ref. 2.02)
a. Inspection Scope
The inspectors verified that the licensee had documented the performance of vendor oversight in accordance with the VOP and that surveillance, abnormal operating, emergency operating, and annunciator response procedures had been reviewed by the licensee.
The inspectors reviewed the security test and evaluation plan required by NEI 08-09 Appendix E.11.5, Developer Security Testing. The licensee provided WEC documents Software Development Plan for the Core Protection Calculator System Upgrade Project, Revision 3 and Core Protection Calculator System Independent Verification and Validation Summary Report, Revision 0. Upon review of the documents, the inspectors determined that while the Independent Verification and Validation Summary addressed testing for primarily safety requirements and not specifically security requirements, the documentation provided for the CPCS testing was sufficient for the cyber security plan requirements for NEI 08-09 Appendix E.11.5, Developer Security Testing and for NEI 08-09 Appendix E.11.3, Trustworthiness that requires that software developers employ software quality and validation methods to minimize flawed or malformed software.
This portion of the inspection partially completed section 2.02 of IP52003.
b. Observations and Findings
The inspectors were unable to verify that the site is meeting the requirement to perform impact analyses or assess changes by WEC such as Post-FAT design changes using EN-DC-115, Engineering Change Process, Revision 30.
The inspectors will complete the review of this item during the installation of the CPCS modification during Spring 2022, Refueling Outage 24.
1.3 Review of Testing, Operations, and Training (ref. 2.03)
a. Inspection Scope
The inspectors reviewed the as-built and as-delivered hardware and software configuration documentation and verified that the software and hardware configurations were aligned to the software configuration management release records and hardware/software bill of materials.
The inspectors also reviewed design changes to the WF3 CPCS that have been implemented post FAT and the documented rationale for the design changes. The inspectors held discussions with Entergy and WEC staff to verify the design changes were conducted in accordance with the design change process documented in the WF3 CPCS licensing documents. The inspectors also held discussions with Entergy staff to verify that Entergy will perform oversight activities for these design changes in accordance with the WF3 CPCS VOP.
The inspectors reviewed WNA-TR-05585-CWTR3, Waterford 3 Core Protection Calculator System Four-Channel Test Report, Revision 0, and WNA-TR-05584-CWTR3, Waterford 3 Core Protection Calculator System One-Channel Test Report, Revision 0 to verify that the system passed all the tests.
The inspectors then reviewed SAT procedures ECT-83843-03, Waterford 3 (WF3) Four-Channel Site Acceptance Test (SAT), Revision 1 and ECT-83843-09, Waterford 3 (WF3)
One-Channel Site Acceptance Test (SAT), Revision 0.
The inspectors compared the Four-Channel SAT procedure (ECT-83843-03) with the Four-Channel FAT report (WNA-TR-05585-CWTR3) and verified that - except for the power supply fan test - the same FAT tests are included in the SAT procedure. The inspectors also verified that the acceptance criteria for the Four-Channel SAT is the same as that of the Four-Channel FAT. The licensee explained that the power supply fan test was not performed during SAT given that it passed during the FAT and it is an intrusive test that requires disassembly of the equipment. The inspectors verified that the Four-Channel SAT procedure is sufficiently detailed to perform the tests for the proposed CPCS modification.
The inspectors also reviewed a redline version of the One-Channel SAT Procedure (ECT-83843-09, Revision 0) that was used to perform the test on the CPC channel A. The licensee explained that as a result of Revision 0 of this procedure, multiple test improvements and efficiencies were identified. The licensee further explained that ECT-83843-09, Revision 1, is currently under internal review and will be performed on an additional channel (CPC channel C or D) starting the week of January 24, 2022. The inspectors will review the completed ECT-83843-09, Revision1, as part of the site installation inspection.
The inspectors discussed with the licensee how the following regulatory commitment included in the license amendment request (LAR) was addressed: Entergy will evaluate Waterford CPCS Replacement Project Site Acceptance Test (SAT) and Installation Test Plans using the software process testing characteristics described in Branch Technical Position (BTP) 7-14 Section B.3.2.4. This is Plant-Specific Action Item #5 per WCAP 16096, Software Program Manual for Common QTM Systems.
The licensee explained that the BTP 7-14, Section B.3.2.4 testing characteristics were applied during the CPCS life cycle by WEC development and testing activities, and by Entergys oversight of WEC during the various life cycle phases, which helped in the development of the SAT plans (some of which are based of the FAT plans). The licensee also credited requirements traceability and regression analysis activities, which were not completed by the end of the SAT inspection. The regulatory commitment also applies to the site installation test plans, which were also not complete by the end of the SAT inspection.
Therefore, the inspectors will complete this inspection activity during the site installation inspection.
This portion of the inspection partially completed section 02.03(a) of IP52003.
The inspectors discussed with the licensee the plans to revise the operations and maintenance procedures impacted by the CPCS modification. For the operations procedures, the licensee identified those procedures that are being revised. The inspectors reviewed the changes made to alarm response procedures OP-500-008, Annunciator Response Procedure Control Room Cabinet H, Revision 45 and OP-500-010, Annunciator Response Procedure Control Room Cabinet L, Revision 38. The licensee explained that OP-903-001, Technical Specification Surveillance Logs, will be updated to incorporate the CPCS self-diagnostic checks. The inspectors will review the procedure for self-diagnostics checks during the site installation inspection.
This portion of the inspection partially completed section 02.03(b) of IP52003.
The inspectors discussed with the licensee the plans to train operators, maintenance technicians, and system engineers. The licensee explained that training to engineering occurred throughout the design and development of the CPCS. This included an Advant Controller 160 class with engineering and maintenance, and a WEC-provided site hands-on training with the single channel to maintenance, engineering (Major Fleet Projects Engineering, System Engineering, Reactor Engineering), site operational information technology, operations, and training representatives. Additional hands-on training is to occur after completion of the SAT with similar attendees.
For licensed operators, the license explained that formal instructor-led classroom training will be provided starting in February of 2022 for the CPC/CEAC and technical specification changes. Hands-on training in the simulator (with rack-mounted CPC/CEACs) will cover the different tasks associated with CPCs/CEACs, while utilizing the draft procedures for the new equipment. Simulator training during the refueling outage will include Shutdown training (with rack-mounted CPC/CEACs) and Startup training (with simulator-installed CPC/CEACs).
The licensee explained that I&C maintenance technicians will take formal training in February of this year, including training provided by WEC at Waterford Unit 3. The lesson plan had not been finalized by the time the SAT inspection ended. The inspectors will review the training plan documents and training records for operators and maintenance technicians during the site installation inspection.
This portion of the inspection partially completed sections 02.03(c) and 02.03(g) of IP52003.
The inspectors witnessed selected portions of the Four-Channel SAT and verified that the SAT was performed using the licensees Engineering Change Test (ECT) Procedure (ECT-83843-03). The inspectors reviewed the results and verified that the test validated the CPCS response time assumptions and acceptance criteria in the system requirements specification, including the Tcold response time acceptance criteria.
The inspectors reviewed corrective action CR-WF3-2022-00291, Condition Report for ECT-83843-03 Step 7.7.2, dated January 18, 2022, regarding an anomaly for a plant computer data link test that was identified as having not passed the acceptance criteria.
The inspectors discussed this test anomaly with the licensee, who explained that after further evaluation the test is considered as having passed the acceptance criteria. The inspectors will review the closure of CR-WF3-2022-00291 during the site installation inspection. The licensee confirmed that no changes to the CPCS have been made as a result of the Four-Channel SAT. The inspectors will also review the results of the One-Channel Test as part of the site installation inspection.
This portion of the inspection partially completed section 02.03(d) of IP52003.
Section 3.4 of the SE, the Waterford Unit 3 CPCS Equipment Qualification Summary Report (WEC EQ-QR-400-CWTR3, Core Protection Calculator System Primary Digital Components Qualification Summary Report for Waterford Unit 3, Revision 0) analyzes the equipment qualification of the components that make up the replacement CPCS and concludes that the testing and results encompass Waterford Unit 3 site requirements for the CPCS.
The NRC staffs review of the LAR verified that plant environmental data for the locations in which the Common Q equipment is to be installed are enveloped by the environment established in the equipment qualification summary report. Therefore, this portion of the inspection is completed for section 2.03(f) of IP52003.
Section 3.3.2.3 of the SE, the setpoints for the replacement are not changing because of the CPCS replacement and the uncertainty terms have been adequately evaluated by the licensee. The SE states that processing uncertainties of the replacement CPCS would continue to be bounded (as was the case with the existing CPCS) by those used in the safety analysis. Therefore, this portion of the inspection is completed for section 2.03(g) of IP52003.
b. Observations and Findings
During the tour of the testing facility, the inspectors made several observations regarding NEI 08-09, Appendix E.11.2 for Supply Chain Protections focusing primarily on chain of custody. Specifically, it was not clear what procedures were in place to maintain the integrity of acquired potential Critical Digital Assets (CDAs) and what are the procedures for an established trusted distribution path used from receipt of the equipment for site acceptance testing through installation of the CDAs. The inspectors will ask to see documentation of these procedures during the installation of the CPCS modification during the Spring 2022, Refueling Outage 24.
1.4 Review of Plans for Maintenance and Repair (ref. 2.04)
a. Inspection Scope
The inspectors verified that plant procedures were being actively updated by reviewing the licensees schedule for procedure development, procedure updates/additions, and deletions.
The inspectors reviewed the licensees plans for maintenance and repair and verified that the procedures had been updated, and correctly reflect the new system attributes.
The licensee identified the I&C maintenance procedures that were updated and explained that the deleted procedures were the result of technical specification changes or because of equipment being eliminated. The inspectors reviewed the updated I&C Maintenance Procedure MI-003-125, Core Protection Calculator Calibration, Revision DH12.15.21a, and verified that it is sufficiently detailed and clear to allow the I&C maintenance personnel to perform the CPCS calibration.
The inspectors evaluated the licensees review of the CPCS modification for impact on the maintenance rule functions. A qualified system engineer for the licensee reviewed the engineering change for maintenance rule impact and documentation of maintenance rule review and approval was provided in the Configuration Management Interface Standard Design Process (CMISDP) list.
The procedural and programmatic reviews were documented in CMISDP, which is the Entergy system used to document engineering changes per the Industry Standard Design.
The inspectors reviewed the consolidated equipment list from the Engineering Change 83843 document that included each CPCS associated component, safety classification, seismic classification, and CDA status. The list also contained manufacturer and model numbers for input into the licensees electronic database.
This portion of the inspection completed Section 2.04 of IP52003 for the SAT Phase.
b. Observations and Findings
No findings were identified.
1.5 Identification and Resolution of Problems
a. Inspection Scope
The inspectors reviewed the licensees corrective action program and verified that they were identifying issues related to the SAT at an appropriate threshold. The inspectors also reviewed the licensees condition reports related to the FAT oversight activities and verified that the licensee documented conditions adverse to quality identified during the FAT and implemented appropriate corrective actions to resolve these conditions.
This portion of the inspection completed Section 2.01(e) of IP52003 for the SAT phase.
b. Observations and Findings
No findings were identified.
EXIT MEETINGS AND DEBRIEFS
The inspectors verified no proprietary information was retained or documented in this report.
On January 27, 2022, the inspectors presented the Digital Instrumentation and Control Modification Inspection results to Matt Lewis, General Manager of Plant Operations, and other members of the licensee staff.
DOCUMENTS REVIEWED
Inspection
Procedure
Type
Designation
Description or Title
Revision or
Date
Corrective Action
Documents
Resulting from
Inspection
CR-WF3-2022-
291
Condition Report for ECT-83843-03 Step 7.7.2
01/18/2022
Cyber Security Scheduled Activities
11/03/2021
CCF#:01 to WNA-
CC-04265-
CWTR3
Contingent Certificate of Conformance Control Form
ENT-WF3-CPC-
394
WNA-MWO-00180-01, Rev. A Deviation Notice for Cable
Dimension
10/12/2021
ENT-WF3-CPC-
396
WEC Contingency Release for issuing the software per
purchase order 10575450
10/18/2021
ENT-WF3-CPC-
401
WNA-MWO-19-00180-02, Revision A, Deviation Notice for
PO 10587546 Changes
10/21/2021
OSD&D-0040223
Assembly, CPC Four-Channel Assembly, EA Channel 1 of 4
of the Four-Channel Assembly
01/22/2022
WNA-CC-04265-
CWTR3
Certificate of Conformance
WNA-MWO-19-
00180-01
Deviation Notice for Cable Dimension
WNA-MWO-19-
00180-02
Deviation Notice for PO 10587546 Changes
WNA-PD-00594-
CWTR3
Software Development Plan for the Core Protection
Calculator System Upgrade Project
WNA-TR-05584-
CWTR3
Waterford 3 Core Protection Calculator System One-Channel
Test Report
WNA-TR-05585-
CWTR3
Waterford 3 Core Protection Calculator System Four-
Channel Test Report
Miscellaneous
WNA-VR-00581-
CWTR3
Core Protection Calculator System Independent Verification
and Validation Summary Report
2003
Inspection
Procedure
Type
Designation
Description or Title
Revision or
Date
ECT-83843-03
Waterford 3 (WF3) Four-Channel Site Acceptance Test
(SAT)
ECT-83843-09
Waterford 3 (WF3) One-Channel Site Acceptance Test (SAT) 0
Engineering Change Process
Post Modification Testing and Special Instructions
Control of Hot Work and Ignition Sources
Control of Combustibles
EN-IT-03
Nuclear Cyber Security Program
Software Quality Assurance Program
Corrective Action Program
Critical Procurements
Material Receipt
MI-003-125
Core Protection Calculator Calibration
01/05/2022
OP-500-008
Annunciator Response Procedure Control Room Cabinet H
Procedures
OP-500-010
Annunciator Response Procedure Control Room Cabinet L
38