IR 05000335/2023010: Difference between revisions
StriderTol (talk | contribs) (StriderTol Bot change) |
StriderTol (talk | contribs) (StriderTol Bot change) |
||
Line 1: | Line 1: | ||
{{Adams | {{Adams | ||
| number = | | number = ML24025A058 | ||
| issue date = | | issue date = 01/26/2024 | ||
| title = | | title = NRC Quadrennial Focused Engineering Inspection (FEI) Commercial Grade Dedication Report 05000335/2023010 and 05000389/2023010 | ||
| author name = | | author name = Baptist J | ||
| author affiliation = NRC/RGN-II/DRS | | author affiliation = NRC/RGN-II/DRS/EB1 | ||
| addressee name = Coffey | | addressee name = Coffey B | ||
| addressee affiliation = Florida Power & Light Co | | addressee affiliation = Florida Power & Light Co | ||
| docket = 05000335, 05000389 | | docket = 05000335, 05000389 | ||
| license number = | | license number = NPF-067, NPF-016 | ||
| contact person = | | contact person = | ||
| document report number = IR 2023010 | | document report number = IR 2023010 | ||
| document type = Letter | | document type = Letter, Inspection Report | ||
| page count = | | page count = 12 | ||
}} | }} | ||
Line 18: | Line 18: | ||
=Text= | =Text= | ||
{{#Wiki_filter: | {{#Wiki_filter:==SUBJECT:== | ||
ST. LUCIE PLANT UNITS 1 AND 2 - NRC QUADRENNIAL FOCUSED ENGINEERING INSPECTION (FEI) COMMERCIAL GRADE DEDICATION REPORT 05000335/2023010 AND 05000389/2023010 | |||
==Dear Bob Coffey:== | |||
On December 15, 2023, the U.S. Nuclear Regulatory Commission (NRC) completed an inspection at St. Lucie Plant Units 1 and 2 and discussed the results of this inspection with Mr. | |||
Robert Craven and other members of your staff. The results of this inspection are documented in the enclosed report. | |||
Two findings of very low safety significance (Green) are documented in this report. Two of these findings involved violations of NRC requirements. We are treating these violations as non-cited violations (NCVs) consistent with Section 2.3.2 of the Enforcement Policy. | |||
If you contest the violations or the significance or severity of the violations documented in this inspection report, you should provide a response within 30 days of the date of this inspection report, with the basis for your denial, to the U.S. Nuclear Regulatory Commission, ATTN: | |||
Document Control Desk, Washington, DC 20555-0001; with copies to the Regional Administrator, Region II; the Director, Office of Enforcement; and the NRC Resident Inspector at St. Lucie Plant Units 1 and 2. | |||
If you disagree with a cross-cutting aspect assignment in this report, you should provide a response within 30 days of the date of this inspection report, with the basis for your disagreement, to the U.S. Nuclear Regulatory Commission, ATTN: Document Control Desk, Washington, DC 20555-0001; with copies to the Regional Administrator, Region II; and the NRC Resident Inspector at St. Lucie Plant Units 1 and 2. | |||
January 26, 2024 This letter, its enclosure, and your response (if any) will be made available for public inspection and copying at http://www.nrc.gov/reading-rm/adams.html and at the NRC Public Document Room in accordance with Title 10 of the Code of Federal Regulations 2.390, Public Inspections, Exemptions, Requests for Withholding. | |||
Sincerely, James B. Baptist, Chief Engineering Branch 1 Division of Reactor Safety Docket Nos. 05000335 and 05000389 License Nos. DPR-67 and NPF-16 | |||
===Enclosure:=== | |||
As stated | |||
==Inspection Report== | |||
Docket Numbers: | |||
05000335 and 05000389 License Numbers: | |||
DPR-67 and NPF-16 Report Numbers: | |||
05000335/2023010 and 05000389/2023010 Enterprise Identifier: | |||
I-2023-010-0052 Licensee: | |||
Florida Power and Light Company Facility: | |||
St. Lucie Plant Units 1 and 2 Location: | |||
Jensen Beach, FL. | |||
Inspection Dates: | |||
November 27, 2023 to December 15, 2023 Inspectors: | |||
T. Fanelli, Senior Reactor Inspector J. Lizardi-Barreto, Reactor Inspector T. Su, Reactor Inspector Approved By: | |||
James B. Baptist, Chief Engineering Branch 1 Division of Reactor Safety | |||
=SUMMARY= | |||
The U.S. Nuclear Regulatory Commission (NRC) continued monitoring the licensees performance by conducting a NRC inspection at St. Lucie Plant Units 1 and 2, in accordance with the Reactor Oversight Process. The Reactor Oversight Process is the NRCs program for overseeing the safe operation of commercial nuclear power reactors. Refer to https://www.nrc.gov/reactors/operating/oversight.html for more information. | |||
===List of Findings and Violations=== | |||
Failure to Justify the Quality and Reliability of Agastat Relays Cornerstone Significance Cross-Cutting Aspect Report Section Mitigating Systems Green NCV 05000335,05000389/2023010-01 Open/Closed | |||
[H.3] - Change Management 71111.21N. | |||
The NRC identified a Green finding and associated NCV of TS 6.8.1.a, "Procedures," and Regulatory Guide (RG) 1.33, Quality Assurance Program Requirements (Operation), dated November 1978, for failure to implement an effective maintenance strategy to ensure the continued quality and reliability of energized Agastat E7000 relays for an extended period was consistent with the manufactures ratings. | |||
Failure to Address Critical Characteristics for Design Cornerstone Significance Cross-Cutting Aspect Report Section Mitigating Systems Green NCV 05000335,05000389/2023010-02 Open/Closed | |||
[H.3] - Change Management 71111.21N. | |||
The NRC identified a Green finding and associated NCV of Title 10 CFR Part 50, Appendix B, | |||
Criterion III, Design Control, for the failure to ensure that reversed engineered electrical components met the critical characteristics for design and thus the plants design requirements. | |||
===Additional Tracking Items=== | |||
None. | |||
=INSPECTION SCOPES= | |||
Inspections were conducted using the appropriate portions of the inspection procedures (IPs) in effect at the beginning of the inspection unless otherwise noted. Currently approved IPs with their attached revision histories are located on the public website at http://www.nrc.gov/reading-rm/doc-collections/insp-manual/inspection-procedure/index.html. Samples were declared complete when the IP requirements most appropriate to the inspection activity were met consistent with Inspection Manual Chapter (IMC) 2515, Light-Water Reactor Inspection Program - Operations Phase. The inspectors reviewed selected procedures and records, observed activities, and interviewed personnel to assess licensee performance and compliance with Commission rules and regulations, license conditions, site procedures, and standards. | |||
==REACTOR SAFETY== | |||
===71111.21N.03 - Commercial Grade Dedication Commercial Grade Dedication (section 03.01)=== | |||
{{IP sample|IP=IP 71111.21|count=11}} | |||
The inspectors reviewed the implementation of the licensees processes for dedicating commercial-grade items/services as basic components and/or for procuring safety-related structures, systems, and components for the following samples: | |||
(1) | |||
* Commercial Dedication Package (CGD) CID 321605-2 for Swagelok 1/2 inch Ball Valve (Replacement for safety-related valve tag number SH15440) | |||
(2) | |||
* Commercial Dedication Package CID 366270-1 for Mobil SHC-824OIL Hydraulic, 1-GALLON (i.e. hydraulic fluid consumable for safety related-functions) | |||
(3) | |||
* Commercial Dedication Package CID 366498-2 Regal Oil (i.e. oil consumable for safety relate-functions) | |||
(4) | |||
* Commercial Dedication Packages CID 366598-2 for Grout, and CID # | |||
301609-1 for steel reinforcing bars (i.e. for reinforced concrete repairs on the Ultimate Heat Sink Barrier Wall) | |||
(5) | |||
* Commercial Dedication Package CID 366598-1 for Grout, and CID 381606-2 for steel reinforcing bars (i.e. for reinforced concrete repairs on Unit 2 ICW Pump Pedestal) | |||
(6) | |||
* Commercial Dedication Package CID 1720223 Module: RPS, Aux Trip Unit (7) | |||
* Commercial Dedication Packages CID 197197 Board, Tri Bistable Trip PCA (8) | |||
* Commercial Dedication Packages CID 309 923 Board, Control "B" (9) | |||
* Commercial Dedication Packages CID 408443 Board, Printed Circuit: | |||
Synchronizing, Static Inverter (10) | |||
* Commercial Dedication Package CID 84949-2 Communication Isolation (11) | |||
* Commercial Dedication Package CID 345075-1 Agastat Relay, Timing, 110-120 VAC, 5060 HZ, 20 to 200 Second | |||
==INSPECTION RESULTS== | |||
Failure to Justify the Quality and Reliability of Agastat Relays Cornerstone Significance Cross-Cutting Aspect Report Section Mitigating Systems Green NCV 05000335,05000389/2023010-01 Open/Closed | |||
[H.3] - Change Management 71111.21N.0 The NRC identified a Green finding and associated NCV of TS 6.8.1.a, "Procedures," and Regulatory Guide (RG) 1.33, Quality Assurance Program Requirements (Operation), dated November 1978, for failure to implement an effective maintenance strategy to ensure the continued quality and reliability of energized Agastat E7000 relays for an extended period was consistent with the manufactures ratings. | |||
The | =====Description:===== | ||
The team reviewed the sites licensing basis for Class 1E components, which specified that their quality and reliability shall be assured (IEEE 279-1971, Criteria for Protection Systems for Nuclear Power Generating Stations Sections 4.3 and 4.4). The inspectors reviewed normally energized Agastat relays model E7022AET004. The manufacturer qualified the relay for 25000 cycles or 10 years from the date of manufacture. | |||
The licensee extended the shelf life to an unlimited life and the replacement life while in service to 15 years. The licensee used a UL thermal index value to justify this in JPN-PSL-89-1579, Related to Qualified Life of Agastat Relays at Nuclear Power Plants, dated 6/7/1989. Based on this analysis, the relays were determined to have an unlimited life while energized in service. However, operating experience has shown this to be incorrect with many failures in the industry. The site had recorded failures of these types of relays, within the last six years, in the corrective action program (e.g. AR 02391272 and AR02330068). The thermal index values do not account for the dynamic properties of the relay. | |||
In addition, the licensee justified the change with EPRI 3002000541, Relay Series-Specific Guidance: Generic Service Life Analyses (GSLA) and Preventive Maintenance (PM). | |||
However, Appendix A for Agastat 7000 relays states in part, for E7000 relays, all recommended preventive maintenance, inspections, and surveillances are performed as intended, and the applicable relays demonstrate acceptable performance during normal operation as well as during testing, and that the replacement interval needs to be established consistent with the manufacturers ratings. The inspectors determined that the licensees preventive maintenance, inspections, and surveillances were not performed as intended and replacement interval needs were not established consistent with the manufacturers ratings. | |||
Records indicated that the Unit 1 relays were last replaced in 11/9/1991; approximately 32 years ago. The inspectors determined that the relays were not replaced within the interval required by the preventative maintenance program and was inconsistent with the manufacturers ratings. | |||
Corrective Actions: The licensee entered the issue into the corrective action program to determine a resolution. | |||
Corrective Action References: AR 02474574 | |||
=====Performance Assessment:===== | |||
Performance Deficiency: The licensees failure to implement an effective maintenance strategy, to ensure the continued quality and reliability of energized Agastat E7000 relays for an extended period consistent with the manufactures ratings, was a performance deficiency. | |||
Screening: The inspectors determined the performance deficiency was more than minor because it was associated with the Design Control attribute of the Mitigating Systems cornerstone and adversely affected the cornerstone objective to ensure the availability, reliability, and capability of systems that respond to initiating events to prevent undesirable consequences. Specifically, the failure to ensure the quality and reliability resulted in a condition where the team had reasonable doubt that the Agastat relays could continue to perform their safety functions. | |||
Significance: The inspectors assessed the significance of the finding using IMC 0609 Appendix A, The Significance Determination Process (SDP) for Findings At-Power. Exhibit 2 | |||
- Mitigating Systems Screening Questions, Item A. The finding was a deficiency affecting the design or qualification of a mitigating SSC, and it maintained its operability or PRA functionality - Screen to Green. | |||
Cross-Cutting Aspect: H.3 - Change Management: Leaders did not use a systematic process for evaluating and implementing a change to relay shelf and in service life for Agastat relays that have a specified lifetime so that nuclear safety remains the overriding priority. | |||
=====Enforcement:===== | |||
Violation: TS 6.8.1.a requires in part, that written procedures shall be established, implemented, and maintained covering the applicable procedures recommended in RG 1.33, Appendix A, February 1978. Section 9.b, requires in part, that preventive maintenance schedules should be developed to specify inspection of replacement parts that have a specified lifetime. The vendor specified lifetime is 25,000 operations or 10 years from the date of manufacture, whichever occurs first, which was exceeded for some relays. | |||
Contrary to the above, since 11/9/1991, the licensee failed to develop and implement a preventive maintenance schedule, or an effective maintenance strategy to ensure the quality and reliability of Agastat control relays in the safety related applications. | |||
Enforcement Action: This violation is being treated as a non-cited violation, consistent with Section 2.3.2 of the Enforcement Policy. | |||
Failure to Address Critical Characteristics for Design Cornerstone Significance Cross-Cutting Aspect Report Section Mitigating Systems Green NCV 05000335,05000389/2023010-02 Open/Closed | |||
[H.7] - | |||
Documentation 71111.21N.0 The NRC identified a Green finding and associated NCV of Title 10 CFR Part 50, Appendix B, Criterion III, Design Control, for the failure to ensure that reversed engineered electrical components met the critical characteristics for design and thus the plants design requirements. | |||
=====Description:===== | |||
The inspectors reviewed the reverse engineering program and sampled six electronic board assemblies used in safety related analog and digital systems. The licensees used the equivalency evaluation process to address the rebuild and manufacturing of the new boards. The inspectors reviewed the Quality Assurance Topical Report (QATR) and identified that Electric Power Research Institute (EPRI) 3002002982, Plant Engineering: Guideline for the Acceptance of Commercial-Grade Items in Nuclear Safety-Related Applications, was a regulatory commitment. Based on the review, deficiencies were identified in the program. | |||
The inspectors sampled a small number of critical characteristics of design to determine how the reverse engineered components were verified and validated (temperature and humidity). | |||
The licensee did not verify or validate these two characteristics even though many board components were not identical to the originals. The licensee determined that the equivalency evaluation process negated the requirement to verify the formal performance specifications for the components being reverse engineered. Thus, the licensee could not demonstrate the development of a technical evaluation and acceptance plan or a failure modes and effects analyses (FMEAs) to address the critical characteristics for design (e.g. systems and environments, temperature, interfaces, humidity etc.). | |||
The inspectors determined that EPRI 3002002982, Section 6.1.3 Critical Characteristics for Design and Critical Characteristics for Acceptance, does specify that equivalency evaluations require critical characteristics for design. Critical characteristics for design ensure that the reverse engineered boards will meet the plants design requirements. For example, the characteristics address the items interaction with other parts, components, systems, and structures; the items interchangeability; and the items safety function, and dependability. | |||
These characteristics are essential to the quality and reliability of these reverse engineered products. | |||
Corrective Actions: The licensee entered the issue into the corrective action program. | |||
Corrective Action References: AR 02474576 | |||
=====Performance Assessment:===== | |||
Performance Deficiency: The failure to ensure that reversed engineered electrical components met the critical characteristics for design and thus the plants design requirements was a performance deficiency. | |||
Screening: The inspectors determined the performance deficiency was more than minor because it was associated with the Design Control attribute of the Mitigating Systems cornerstone and adversely affected the cornerstone objective to ensure the availability, reliability, and capability of systems that respond to initiating events to prevent undesirable consequences. Specifically, the failure to ensure the critical characteristics for design increased the likelihood of embedded latent flaws, which is averse to the availability, reliability, and capability of the components developed by the program and created a reasonable doubt about their operability. | |||
Significance: The inspectors assessed the significance of the finding using IMC 0609 Appendix A, The Significance Determination Process (SDP) for Findings At-Power. Exhibit 2 | |||
- Mitigating Systems Screening Questions, Item A. The finding was a deficiency affecting the design or qualification of a mitigating SSC, and it maintain its operability or PRA functionality - Screen to Green. | |||
Cross-Cutting Aspect: H.7 Documentation: The organization did not create and maintain complete, accurate and up-to-date reverse engineering procedures and technical evaluation instructions for equivalency evaluations. (WP.3). | |||
=====Enforcement:===== | |||
Violation: 10 CFR 50, Appendix B, Criterion III, Design Control, states in part, that measures shall be established to assure that applicable regulatory requirements and the design basis, as defined in § 50.2 and as specified in the license application, for those structures, systems, and components to which this appendix applies are correctly translated into specifications, drawings, procedures, and instructions. | |||
Contrary to the above, since 2017 the licensee failed to ensure that applicable design basis for safety related electronic circuit boards were correctly translated into specifications, drawings, procedures, and instructions for reverse engineering activities. | |||
Enforcement Action: This violation is being treated as a non-cited violation, consistent with Section 2.3.2 of the Enforcement Policy. | |||
==EXIT MEETINGS AND DEBRIEFS== | |||
The inspectors verified no proprietary information was retained or documented in this report. | |||
* On December 15, 2023, the inspectors presented the NRC inspection results to Mr. | |||
Robert Craven and other members of the licensee staff. | |||
* On December 14, 2023, the inspectors presented the Inspection Debrief inspection results to Tim Falkiewicz St. Lucie Plant, N Regulatory Affairs Manager and other members of the licensee staff. | |||
=DOCUMENTS REVIEWED= | |||
Inspection | |||
Procedure | |||
Type | |||
Designation | |||
Description or Title | |||
Revision or | |||
Date | |||
AR 02471898 | |||
EXPIRED GREASE FOUND IN CABINET DURING NRC | |||
WALKDOWN | |||
11/8/2023 | |||
AR 02473098 | |||
PROCESS IMPROVEMENTS IN DOCUMENTING | |||
DEDICATION TESTING | |||
11/27/2023 | |||
AR 02473617 | |||
23 NRC CGD INSPECTION - ISC TS ASE - TECH | |||
EVAL - 84949-2 | |||
2/4/2023 | |||
AR 02474574 | |||
23 CGD INSP - GREEN NCV - QUAL AND REL OF | |||
AGASTAT RELAYS | |||
2/14/2023 | |||
AR 02474576 | |||
23 CGD INSP - GREEN NCV - RE DESIGN CRITICAL | |||
CHARAC. | |||
2/14/2023 | |||
AR 02474577 | |||
23 CGD INSP - MINOR VIOLATION - DOC ERROR IN | |||
RE BOARD | |||
2/14/2023 | |||
Corrective Action | |||
Documents | |||
Resulting from | |||
Inspection | |||
AR 02474578 | |||
23 CGD INSP - MINOR VIOLATION - APPEND B | |||
MAT'L STORAGE | |||
2/14/2023 | |||
Drawings | |||
2998-A-451-1000 | |||
Equipment Qualification Documentation Package | |||
rev. 15 | |||
00427168 | |||
Procurement Engineering Evaluation Record Agastat | |||
04/11/2011 | |||
2357321 | |||
Technical Evaluation Document CID 366598-1 for Grout | |||
9/28/2016 | |||
2379992 | |||
Technical Evaluation Document CID 366270-1 for Mobil | |||
SHC-824 Hydraulic Fluid/Oil | |||
7/13/2020 | |||
273211981 | |||
Technical Evaluation Document CID 366498-2 Regal Oil | |||
6/11/2022 | |||
CPED-RE- | |||
20223 | |||
Technical Evaluation Document CID 1720223-2 Auxiliary | |||
Trip Unit | |||
10/18/21 | |||
CPED-RE- | |||
197197-2022 | |||
Technical Evaluation Document CID 197197 Tri-Bistable | |||
Trip PCB | |||
11/02/22 | |||
CPED-RE- | |||
309923-2017 | |||
Technical Evaluation Document CID 309923 C&D Control | |||
B PCB | |||
2/19/17 | |||
CPED-RE- | |||
347646-2019 | |||
Technical Evaluation Document CID 347646 Synchnizing | |||
Board | |||
2/11/2019 | |||
PEER 00465158 | |||
Technical Evaluation Document CID 301609-1 for | |||
Concrete Steel Reinforcing Bars | |||
07/17/2015 | |||
71111.21N.03 | |||
Engineering | |||
Evaluations | |||
PEER 00477369 | |||
Technical Evaluation Document CID 381606-2 for | |||
Concrete Steel Reinforcing Bars | |||
08/10/2017 | |||
Inspection | |||
Procedure | |||
Type | |||
Designation | |||
Description or Title | |||
Revision or | |||
Date | |||
PEER 00488497 | |||
Technical Evaluation Document CID CID 366598-2 for | |||
Grout | |||
10/14/2020 | |||
SS-63TF8 | |||
Technical Evaluation Document CID 321605-2 Swagelok | |||
1/2 inch Ball Valve | |||
10/01/2020 | |||
Technical | |||
Evaluation | |||
CPED-RE- | |||
84949-2022 | |||
Communication Isolation Device | |||
rev 4 | |||
}} | }} |
Latest revision as of 20:41, 24 November 2024
ML24025A058 | |
Person / Time | |
---|---|
Site: | Saint Lucie |
Issue date: | 01/26/2024 |
From: | James Baptist NRC/RGN-II/DRS/EB1 |
To: | Coffey B Florida Power & Light Co |
References | |
IR 2023010 | |
Download: ML24025A058 (12) | |
Text
SUBJECT:
ST. LUCIE PLANT UNITS 1 AND 2 - NRC QUADRENNIAL FOCUSED ENGINEERING INSPECTION (FEI) COMMERCIAL GRADE DEDICATION REPORT 05000335/2023010 AND 05000389/2023010
Dear Bob Coffey:
On December 15, 2023, the U.S. Nuclear Regulatory Commission (NRC) completed an inspection at St. Lucie Plant Units 1 and 2 and discussed the results of this inspection with Mr.
Robert Craven and other members of your staff. The results of this inspection are documented in the enclosed report.
Two findings of very low safety significance (Green) are documented in this report. Two of these findings involved violations of NRC requirements. We are treating these violations as non-cited violations (NCVs) consistent with Section 2.3.2 of the Enforcement Policy.
If you contest the violations or the significance or severity of the violations documented in this inspection report, you should provide a response within 30 days of the date of this inspection report, with the basis for your denial, to the U.S. Nuclear Regulatory Commission, ATTN:
Document Control Desk, Washington, DC 20555-0001; with copies to the Regional Administrator, Region II; the Director, Office of Enforcement; and the NRC Resident Inspector at St. Lucie Plant Units 1 and 2.
If you disagree with a cross-cutting aspect assignment in this report, you should provide a response within 30 days of the date of this inspection report, with the basis for your disagreement, to the U.S. Nuclear Regulatory Commission, ATTN: Document Control Desk, Washington, DC 20555-0001; with copies to the Regional Administrator, Region II; and the NRC Resident Inspector at St. Lucie Plant Units 1 and 2.
January 26, 2024 This letter, its enclosure, and your response (if any) will be made available for public inspection and copying at http://www.nrc.gov/reading-rm/adams.html and at the NRC Public Document Room in accordance with Title 10 of the Code of Federal Regulations 2.390, Public Inspections, Exemptions, Requests for Withholding.
Sincerely, James B. Baptist, Chief Engineering Branch 1 Division of Reactor Safety Docket Nos. 05000335 and 05000389 License Nos. DPR-67 and NPF-16
Enclosure:
As stated
Inspection Report
Docket Numbers:
05000335 and 05000389 License Numbers:
DPR-67 and NPF-16 Report Numbers:
05000335/2023010 and 05000389/2023010 Enterprise Identifier:
I-2023-010-0052 Licensee:
Florida Power and Light Company Facility:
St. Lucie Plant Units 1 and 2 Location:
Jensen Beach, FL.
Inspection Dates:
November 27, 2023 to December 15, 2023 Inspectors:
T. Fanelli, Senior Reactor Inspector J. Lizardi-Barreto, Reactor Inspector T. Su, Reactor Inspector Approved By:
James B. Baptist, Chief Engineering Branch 1 Division of Reactor Safety
SUMMARY
The U.S. Nuclear Regulatory Commission (NRC) continued monitoring the licensees performance by conducting a NRC inspection at St. Lucie Plant Units 1 and 2, in accordance with the Reactor Oversight Process. The Reactor Oversight Process is the NRCs program for overseeing the safe operation of commercial nuclear power reactors. Refer to https://www.nrc.gov/reactors/operating/oversight.html for more information.
List of Findings and Violations
Failure to Justify the Quality and Reliability of Agastat Relays Cornerstone Significance Cross-Cutting Aspect Report Section Mitigating Systems Green NCV 05000335,05000389/2023010-01 Open/Closed
[H.3] - Change Management 71111.21N.
The NRC identified a Green finding and associated NCV of TS 6.8.1.a, "Procedures," and Regulatory Guide (RG) 1.33, Quality Assurance Program Requirements (Operation), dated November 1978, for failure to implement an effective maintenance strategy to ensure the continued quality and reliability of energized Agastat E7000 relays for an extended period was consistent with the manufactures ratings.
Failure to Address Critical Characteristics for Design Cornerstone Significance Cross-Cutting Aspect Report Section Mitigating Systems Green NCV 05000335,05000389/2023010-02 Open/Closed
[H.3] - Change Management 71111.21N.
The NRC identified a Green finding and associated NCV of Title 10 CFR Part 50, Appendix B,
Criterion III, Design Control, for the failure to ensure that reversed engineered electrical components met the critical characteristics for design and thus the plants design requirements.
Additional Tracking Items
None.
INSPECTION SCOPES
Inspections were conducted using the appropriate portions of the inspection procedures (IPs) in effect at the beginning of the inspection unless otherwise noted. Currently approved IPs with their attached revision histories are located on the public website at http://www.nrc.gov/reading-rm/doc-collections/insp-manual/inspection-procedure/index.html. Samples were declared complete when the IP requirements most appropriate to the inspection activity were met consistent with Inspection Manual Chapter (IMC) 2515, Light-Water Reactor Inspection Program - Operations Phase. The inspectors reviewed selected procedures and records, observed activities, and interviewed personnel to assess licensee performance and compliance with Commission rules and regulations, license conditions, site procedures, and standards.
REACTOR SAFETY
71111.21N.03 - Commercial Grade Dedication Commercial Grade Dedication (section 03.01)
The inspectors reviewed the implementation of the licensees processes for dedicating commercial-grade items/services as basic components and/or for procuring safety-related structures, systems, and components for the following samples:
(1)
- Commercial Dedication Package (CGD) CID 321605-2 for Swagelok 1/2 inch Ball Valve (Replacement for safety-related valve tag number SH15440)
(2)
- Commercial Dedication Package CID 366270-1 for Mobil SHC-824OIL Hydraulic, 1-GALLON (i.e. hydraulic fluid consumable for safety related-functions)
(3)
- Commercial Dedication Package CID 366498-2 Regal Oil (i.e. oil consumable for safety relate-functions)
(4)
- Commercial Dedication Packages CID 366598-2 for Grout, and CID #
301609-1 for steel reinforcing bars (i.e. for reinforced concrete repairs on the Ultimate Heat Sink Barrier Wall)
(5)
- Commercial Dedication Package CID 366598-1 for Grout, and CID 381606-2 for steel reinforcing bars (i.e. for reinforced concrete repairs on Unit 2 ICW Pump Pedestal)
(6)
- Commercial Dedication Package CID 1720223 Module: RPS, Aux Trip Unit (7)
- Commercial Dedication Packages CID 197197 Board, Tri Bistable Trip PCA (8)
- Commercial Dedication Packages CID 309 923 Board, Control "B" (9)
- Commercial Dedication Packages CID 408443 Board, Printed Circuit:
Synchronizing, Static Inverter (10)
- Commercial Dedication Package CID 84949-2 Communication Isolation (11)
- Commercial Dedication Package CID 345075-1 Agastat Relay, Timing, 110-120 VAC, 5060 HZ, 20 to 200 Second
INSPECTION RESULTS
Failure to Justify the Quality and Reliability of Agastat Relays Cornerstone Significance Cross-Cutting Aspect Report Section Mitigating Systems Green NCV 05000335,05000389/2023010-01 Open/Closed
[H.3] - Change Management 71111.21N.0 The NRC identified a Green finding and associated NCV of TS 6.8.1.a, "Procedures," and Regulatory Guide (RG) 1.33, Quality Assurance Program Requirements (Operation), dated November 1978, for failure to implement an effective maintenance strategy to ensure the continued quality and reliability of energized Agastat E7000 relays for an extended period was consistent with the manufactures ratings.
Description:
The team reviewed the sites licensing basis for Class 1E components, which specified that their quality and reliability shall be assured (IEEE 279-1971, Criteria for Protection Systems for Nuclear Power Generating Stations Sections 4.3 and 4.4). The inspectors reviewed normally energized Agastat relays model E7022AET004. The manufacturer qualified the relay for 25000 cycles or 10 years from the date of manufacture.
The licensee extended the shelf life to an unlimited life and the replacement life while in service to 15 years. The licensee used a UL thermal index value to justify this in JPN-PSL-89-1579, Related to Qualified Life of Agastat Relays at Nuclear Power Plants, dated 6/7/1989. Based on this analysis, the relays were determined to have an unlimited life while energized in service. However, operating experience has shown this to be incorrect with many failures in the industry. The site had recorded failures of these types of relays, within the last six years, in the corrective action program (e.g. AR 02391272 and AR02330068). The thermal index values do not account for the dynamic properties of the relay.
In addition, the licensee justified the change with EPRI 3002000541, Relay Series-Specific Guidance: Generic Service Life Analyses (GSLA) and Preventive Maintenance (PM).
However, Appendix A for Agastat 7000 relays states in part, for E7000 relays, all recommended preventive maintenance, inspections, and surveillances are performed as intended, and the applicable relays demonstrate acceptable performance during normal operation as well as during testing, and that the replacement interval needs to be established consistent with the manufacturers ratings. The inspectors determined that the licensees preventive maintenance, inspections, and surveillances were not performed as intended and replacement interval needs were not established consistent with the manufacturers ratings.
Records indicated that the Unit 1 relays were last replaced in 11/9/1991; approximately 32 years ago. The inspectors determined that the relays were not replaced within the interval required by the preventative maintenance program and was inconsistent with the manufacturers ratings.
Corrective Actions: The licensee entered the issue into the corrective action program to determine a resolution.
Corrective Action References: AR 02474574
Performance Assessment:
Performance Deficiency: The licensees failure to implement an effective maintenance strategy, to ensure the continued quality and reliability of energized Agastat E7000 relays for an extended period consistent with the manufactures ratings, was a performance deficiency.
Screening: The inspectors determined the performance deficiency was more than minor because it was associated with the Design Control attribute of the Mitigating Systems cornerstone and adversely affected the cornerstone objective to ensure the availability, reliability, and capability of systems that respond to initiating events to prevent undesirable consequences. Specifically, the failure to ensure the quality and reliability resulted in a condition where the team had reasonable doubt that the Agastat relays could continue to perform their safety functions.
Significance: The inspectors assessed the significance of the finding using IMC 0609 Appendix A, The Significance Determination Process (SDP) for Findings At-Power. Exhibit 2
- Mitigating Systems Screening Questions, Item A. The finding was a deficiency affecting the design or qualification of a mitigating SSC, and it maintained its operability or PRA functionality - Screen to Green.
Cross-Cutting Aspect: H.3 - Change Management: Leaders did not use a systematic process for evaluating and implementing a change to relay shelf and in service life for Agastat relays that have a specified lifetime so that nuclear safety remains the overriding priority.
Enforcement:
Violation: TS 6.8.1.a requires in part, that written procedures shall be established, implemented, and maintained covering the applicable procedures recommended in RG 1.33, Appendix A, February 1978. Section 9.b, requires in part, that preventive maintenance schedules should be developed to specify inspection of replacement parts that have a specified lifetime. The vendor specified lifetime is 25,000 operations or 10 years from the date of manufacture, whichever occurs first, which was exceeded for some relays.
Contrary to the above, since 11/9/1991, the licensee failed to develop and implement a preventive maintenance schedule, or an effective maintenance strategy to ensure the quality and reliability of Agastat control relays in the safety related applications.
Enforcement Action: This violation is being treated as a non-cited violation, consistent with Section 2.3.2 of the Enforcement Policy.
Failure to Address Critical Characteristics for Design Cornerstone Significance Cross-Cutting Aspect Report Section Mitigating Systems Green NCV 05000335,05000389/2023010-02 Open/Closed
[H.7] -
Documentation 71111.21N.0 The NRC identified a Green finding and associated NCV of Title 10 CFR Part 50, Appendix B, Criterion III, Design Control, for the failure to ensure that reversed engineered electrical components met the critical characteristics for design and thus the plants design requirements.
Description:
The inspectors reviewed the reverse engineering program and sampled six electronic board assemblies used in safety related analog and digital systems. The licensees used the equivalency evaluation process to address the rebuild and manufacturing of the new boards. The inspectors reviewed the Quality Assurance Topical Report (QATR) and identified that Electric Power Research Institute (EPRI) 3002002982, Plant Engineering: Guideline for the Acceptance of Commercial-Grade Items in Nuclear Safety-Related Applications, was a regulatory commitment. Based on the review, deficiencies were identified in the program.
The inspectors sampled a small number of critical characteristics of design to determine how the reverse engineered components were verified and validated (temperature and humidity).
The licensee did not verify or validate these two characteristics even though many board components were not identical to the originals. The licensee determined that the equivalency evaluation process negated the requirement to verify the formal performance specifications for the components being reverse engineered. Thus, the licensee could not demonstrate the development of a technical evaluation and acceptance plan or a failure modes and effects analyses (FMEAs) to address the critical characteristics for design (e.g. systems and environments, temperature, interfaces, humidity etc.).
The inspectors determined that EPRI 3002002982, Section 6.1.3 Critical Characteristics for Design and Critical Characteristics for Acceptance, does specify that equivalency evaluations require critical characteristics for design. Critical characteristics for design ensure that the reverse engineered boards will meet the plants design requirements. For example, the characteristics address the items interaction with other parts, components, systems, and structures; the items interchangeability; and the items safety function, and dependability.
These characteristics are essential to the quality and reliability of these reverse engineered products.
Corrective Actions: The licensee entered the issue into the corrective action program.
Corrective Action References: AR 02474576
Performance Assessment:
Performance Deficiency: The failure to ensure that reversed engineered electrical components met the critical characteristics for design and thus the plants design requirements was a performance deficiency.
Screening: The inspectors determined the performance deficiency was more than minor because it was associated with the Design Control attribute of the Mitigating Systems cornerstone and adversely affected the cornerstone objective to ensure the availability, reliability, and capability of systems that respond to initiating events to prevent undesirable consequences. Specifically, the failure to ensure the critical characteristics for design increased the likelihood of embedded latent flaws, which is averse to the availability, reliability, and capability of the components developed by the program and created a reasonable doubt about their operability.
Significance: The inspectors assessed the significance of the finding using IMC 0609 Appendix A, The Significance Determination Process (SDP) for Findings At-Power. Exhibit 2
- Mitigating Systems Screening Questions, Item A. The finding was a deficiency affecting the design or qualification of a mitigating SSC, and it maintain its operability or PRA functionality - Screen to Green.
Cross-Cutting Aspect: H.7 Documentation: The organization did not create and maintain complete, accurate and up-to-date reverse engineering procedures and technical evaluation instructions for equivalency evaluations. (WP.3).
Enforcement:
Violation: 10 CFR 50, Appendix B, Criterion III, Design Control, states in part, that measures shall be established to assure that applicable regulatory requirements and the design basis, as defined in § 50.2 and as specified in the license application, for those structures, systems, and components to which this appendix applies are correctly translated into specifications, drawings, procedures, and instructions.
Contrary to the above, since 2017 the licensee failed to ensure that applicable design basis for safety related electronic circuit boards were correctly translated into specifications, drawings, procedures, and instructions for reverse engineering activities.
Enforcement Action: This violation is being treated as a non-cited violation, consistent with Section 2.3.2 of the Enforcement Policy.
EXIT MEETINGS AND DEBRIEFS
The inspectors verified no proprietary information was retained or documented in this report.
- On December 15, 2023, the inspectors presented the NRC inspection results to Mr.
Robert Craven and other members of the licensee staff.
- On December 14, 2023, the inspectors presented the Inspection Debrief inspection results to Tim Falkiewicz St. Lucie Plant, N Regulatory Affairs Manager and other members of the licensee staff.
DOCUMENTS REVIEWED
Inspection
Procedure
Type
Designation
Description or Title
Revision or
Date
EXPIRED GREASE FOUND IN CABINET DURING NRC
WALKDOWN
11/8/2023
PROCESS IMPROVEMENTS IN DOCUMENTING
DEDICATION TESTING
11/27/2023
23 NRC CGD INSPECTION - ISC TS ASE - TECH
EVAL - 84949-2
2/4/2023
23 CGD INSP - GREEN NCV - QUAL AND REL OF
AGASTAT RELAYS
2/14/2023
23 CGD INSP - GREEN NCV - RE DESIGN CRITICAL
CHARAC.
2/14/2023
23 CGD INSP - MINOR VIOLATION - DOC ERROR IN
RE BOARD
2/14/2023
Corrective Action
Documents
Resulting from
Inspection
23 CGD INSP - MINOR VIOLATION - APPEND B
MAT'L STORAGE
2/14/2023
Drawings
2998-A-451-1000
Equipment Qualification Documentation Package
rev. 15
00427168
Procurement Engineering Evaluation Record Agastat
04/11/2011
2357321
Technical Evaluation Document CID 366598-1 for Grout
9/28/2016
2379992
Technical Evaluation Document CID 366270-1 for Mobil
SHC-824 Hydraulic Fluid/Oil
7/13/2020
273211981
Technical Evaluation Document CID 366498-2 Regal Oil
6/11/2022
CPED-RE-
20223
Technical Evaluation Document CID 1720223-2 Auxiliary
Trip Unit
10/18/21
CPED-RE-
197197-2022
Technical Evaluation Document CID 197197 Tri-Bistable
Trip PCB
11/02/22
CPED-RE-
309923-2017
Technical Evaluation Document CID 309923 C&D Control
B PCB
2/19/17
CPED-RE-
347646-2019
Technical Evaluation Document CID 347646 Synchnizing
Board
2/11/2019
PEER 00465158
Technical Evaluation Document CID 301609-1 for
Concrete Steel Reinforcing Bars
07/17/2015
71111.21N.03
Engineering
Evaluations
PEER 00477369
Technical Evaluation Document CID 381606-2 for
Concrete Steel Reinforcing Bars
08/10/2017
Inspection
Procedure
Type
Designation
Description or Title
Revision or
Date
PEER 00488497
Technical Evaluation Document CID CID 366598-2 for
10/14/2020
SS-63TF8
Technical Evaluation Document CID 321605-2 Swagelok
1/2 inch Ball Valve
10/01/2020
Technical
Evaluation
CPED-RE-
84949-2022
Communication Isolation Device
rev 4