IR 05000400/2013009

IR 05000400-13-009, 04/01/2013 07/15/2013, Shearon Harris Nuclear Power Plant, Unit 1, Evaluations of Changes, Tests, and Experiments and Permanent Plant Modifications Baseline Follow-up

ML13224A290
Person / Time
Site:	Harris
Issue date:	08/12/2013
From:	Nease R L NRC/RGN-II/DRS/EB1
To:	Kapopoulos E J Carolina Power & Light Co
References
IR-13-009
Download: ML13224A290 (16)

Text

August 12, 2013

Mr. Ernest Kapopoulos, Jr. Vice President Shearon Harris Nuclear Power Plant Carolina Power and Light Company P.O. Box 165, Mail Code: Zone 1 New Hill, NC 27562-0165

SUBJECT: SHEARON HARRIS NUCLEAR POWER PLANT UNIT 1 - NRC EVALUATION OF CHANGES, TESTS, AND EXPERIMENTS AND PERMANENT PLANT MODIFICATIONS BASELINE INSPECTION FOLLOW-UP REPORT 05000400/2013009

Dear Mr. Kapopoulos:

On July 15, 2013, the U.S. Nuclear Regulatory Commission (NRC) completed an inspection at your Shearon Harris Nuclear Power Plant, Unit 1. The enclosed inspection report documents the inspection results which were discussed on July 15, 2013, with you and other members of your staff. The inspection examined activities conducted under your license as they relate to safety and compliance with the Commission's rules and regulations and with the conditions of your license. The inspectors reviewed selected procedures and records, observed activities, and interviewed personnel. One NRC-identified finding of very low safety significance (Green) was identified during this inspection. This finding was determined to involve a violation of NRC requirements. Additionally, the NRC has determined that a traditional enforcement Severity Level IV violation occurred with the associated finding. The NRC is treating this violation as a non-cited violation (NCV) consistent with Section 2.3.2 of the Enforcement Policy. If you contest the violation or significance of this NCV, you should provide a response within 30 days of the date of this inspection report, with the basis for your denial, to the Nuclear Regulatory Commission, ATTN: Document Control Desk, Washington DC 20555-0001; with copies to the Regional Administrator, Region II; the Director, Office of Enforcement, United States Nuclear Regulatory Commission, Washington, DC 20555-0001; and the NRC Resident Inspector at the Shearon Harris facility. If you disagree with a cross-cutting aspect assignment in this report, you should provide a response within 30 days of the date of this inspection report, with the basis for your disagreement, to the Regional Administrator, Region II; and the NRC Resident Inspector at the Shearon Harris facility.

E. Kapopoulos, Jr. 2 In accordance with 10 CFR 2.390 of the NRC's "Rules of Practice," a copy of this letter, its enclosure, and your response (if any) will be available electronically for public inspection in the NRC Public Document Room or from the Publicly Available Records (PARS) component of NRC's Agencywide Document Access and Management System (ADAMS). ADAMS is accessible from the NRC Web site at http://www.nrc.gov/reading-rm/adams.html (the Public Electronic Reading Room).

Sincerely,RA Rebecca Nease, Chief Engineering Branch 1 Division of Reactor Safety Docket No.: 50-400 License No.: NPF-63

Enclosure:

Inspection Report 05000400/2013009 Supplementary Information cc: (See page 3)

SUMMARY

IR 05000400/2013009; 04/01/2013 - 07/15/2013; Shearon Harris Nuclear Power Plant, Unit 1; Evaluations of Changes, Tests, and Experiments and Permanent Plant Modifications Baseline Follow-up. Two Nuclear Regulatory Commission (NRC) inspectors from Region II conducted the inspection. One Severity Level (SL) IV non-cited violation (NCV) with an associated finding was identified. The significance of inspection findings is indicated by their color (i.e., greater than Green, or Green, White, Yellow, Red) and determined using Inspector Manual Chapter (IMC) 0609, "Significance Determination Process (SDP)," dated 06/02/11. All violations of NRC requirements are dispositioned in accordance with the NRC's Enforcement Policy dated 1/28/13. The NRC's program for overseeing the safe operation of commercial nuclear power reactors is described in NUREG-1649, "Reactor Oversight Process," (ROP) Revision 4, dated December 2006.

Cornerstone: Mitigating Systems

SL IV: The inspectors identified a SL IV Green NCV of 10 CFR 50.59, "Changes, Tests, and Experiments," for the licensee's failure to obtain a license amendment before implementing a change that created the possibility of a malfunction of a system, structure, or component important to safety with a different result than previously evaluated. The licensee did not follow guidance in Nuclear Energy Institute document NEI 01-01, "Guidelines on Licensing Digital Upgrades," Rev. 1, (referenced in licensee Procedure EGR-NGGC-0157, "Engineering of Plant Digital Systems and Components," Rev. 7), which resulted in the licensee implementing a change that created the possibility of common cause software malfunctions of the reactor protection system and engineered safety features actuation systems not previously evaluated in the Updated Final Safety Analysis Report. This failure to follow NEI guidance when implementing a change was a performance deficiency. The licensee entered this issue into their corrective action program, performed an evaluation that provided a reasonable expectation of operability, and initiated development of a license amendment request.

The performance deficiency was determined to be more than minor because it was associated with the design control attribute of the Mitigating Systems cornerstone and adversely affected the cornerstone objective of ensuring the availability, reliability, and capability of systems that respond to initiating events to prevent undesirable consequences (i.e., core damage). Additionally, in accordance with the guidance in the NRC Enforcement Manual, the 10 CFR 50.59 violation was more than minor because there was reasonable likelihood that the change would require NRC approval prior to implementation. The inspectors evaluated the significance of the finding using IMC 0609, "The Significance Determination Process," and determined the finding was of very low safety significance (Green). In accordance with the Enforcement Policy, the violation of 10 CFR 50.59 was determined to be a SL IV violation because it resulted in a condition evaluated as having very low safety significance (i.e., Green) by the SDP. The finding had a cross-cutting aspect in the "Decision Making" component of the "Human Performance" area because the most significant causal factor of the performance deficiency was that the licensee failed to oversee the work activities of vendors such that nuclear safety was supported H.4(c). (Section 1R17)

B. Licensee-Identified Violations

None

REPORT DETAILS

REACTOR SAFETY

Cornerstones: Initiating Events, Mitigating Systems, and Barrier Integrity 1R17 Evaluations of Changes, Tests, and Experiments and Permanent Plant Modifications (Closed) Unresolved Item (URI)05000400/2013002-03, "Solid State Protection System Digital Modification." (ML13120A340)

a. Inspection Scope

During the 2013, baseline inspection performed in accordance with Inspection Procedure 71111.17, "Evaluations of Changes, Tests, and Experiments and Permanent Plant Modifications," the team identified a URI related to the licensee's implementation of a permanent plant change that replaced the solid state protection system (SSPS) control circuit boards with digital complex programmable logic device (CPLD)-based boards. As referenced in site procedures, the licensee reviewed the plant change in accordance with the guidance and process described in Nuclear Energy Institute (NEI) 96-07, "Guidelines for 10 CFR 50.59 Implementation," Rev. 1. The licensee determined the change could be implemented without performing a formal 10 CFR 50.59 evaluation to determine if a license amendment request (LAR) was required to be submitted to the Nuclear Regulatory Commission (NRC) prior to implementation. The licensee failed to recognize that the software used in the replacement boards had the potential to adversely affect the design functions of the SSPS; therefore, erroneously concluded that the change could be implemented without performing a formal 10 CFR 50.59 evaluation, and without obtaining a license amendment. Subsequent to the team's questioning, the licensee performed a 10 CFR 50.59 evaluation and concluded the change did not require a LAR prior to implementation. The inspectors reviewed the evaluation and could not verify the licensee's bases for concluding that the change did not meet the 10 CFR 50.59 (c)(2)(vi) criterion for requiring a license amendment. Specifically, the inspectors could not confirm the licensee's conclusion that they could eliminate consideration and effects of software-based common cause failures (CCF) by meeting the Standard Review Plan (SRP) criteria contained in Branch Technical Position (BTP) 7-19, "Guidance for Evaluation of Diversity and Defense-in-Depth in Digital Computer-Based I&C Systems," Rev. 6. This item was unresolved pending further inspection to determine if the licensee's performance constituted a violation of 10 CFR 50.59, "Evaluation of Changes, Tests, and Experiments." The team determined that additional information from the licensee and consultation with the Office of Nuclear Regulation (NRR) was warranted before reaching a final disposition of the URI.

On April 5, 2013, the NRC staff conducted a meeting with the licensee and vendor of the replacement boards (Westinghouse) to discuss the design, development, qualification, testing, and implementation of the SSPS circuit board replacements.

On April 16, 2013, the licensee provided additional information regarding the analyses and testing of the boards. The NRC staff conducted an in-office review of additional information provided by the licensee and vendor.

b. Findings

Introduction:

The inspectors identified a SL IV Green NCV of 10 CFR 50.59, "Changes, Tests, and Experiments," for the licensee's failure to obtain a license amendment before implementing a change that created the possibility of a malfunction of a system, structure, or component important to safety with a different result than previously evaluated. The licensee did not follow guidance in Nuclear Energy Institute document NEI 01-01, "Guidelines on Licensing Digital Upgrades," Rev. 1, (referenced in licensee Procedure EGR-NGGC-0157, "Engineering of Plant Digital Systems and Components," Rev. 7), which resulted in the licensee implementing a change that created the possibility of common cause software malfunctions of the reactor protection system (RPS) and engineered safety features actuation systems (ESFAS) not previously evaluated in the Updated Final Safety Analysis Report (UFSAR). The licensee's failure to follow NEI guidance when implementing this change was a performance deficiency.

Description:

The SSPS circuit boards provide the coincidence logic to produce trip signals for the RPS and actuation signals for the ESFAS. Engineering Change 78484, "Replace SSPS boards with new Westinghouse design boards," Rev. 6, examined a digital modification to the existing SSPS circuit boards. Unlike the original circuit boards, which used fixed logic devices, the replacement boards were digital CPLD-based boards that required an application-specific software (data file) to configure the board's logic functions. These data files placed in the board's CPLD memory perform a specified design basis safety function in the SSPS. Because potential software related failures represent a new failure mode, and could occur on each of the redundant SSPS safety trains, there is a potential increase in the likelihood of software common cause failure (CCF) of the safety function performed by the CPLDs and ultimately, the SSPS. Licensee procedure EGR-NGGC-0157, "Engineering of Plant Digital Systems and Components," Rev. 7, described the licensee's process for complying with the requirements of 10 CFR 50.59 when implementing modifications of instrumentation and control systems employing digital equipment technology. The procedure referenced the use of guidelines contained in NEI 01-01, "Guideline on Licensing Digital Upgrades," Rev. 1, to evaluate digital modifications against the 10 CFR 50.59 (c)(2)(i - viii) criteria in order to determine if a LAR was required to be submitted to the NRC prior to implementation. Section 4.4.6, "Does the activity create a possibility for a malfunction of an SSC important to safety with a different result?" of NEI 01-01, provided guidance on evaluating digital modifications against criterion (c)(2)(vi) of 10 CFR 50.59 with respect to software CCFs. This section stated that engineering evaluations of the quality and design processes should determine if there is reasonable assurance that the likelihood of failures due to software (including software CCF), are sufficiently low and whether or not they should be considered further in the 10 CFR 50.59 evaluation process. These evaluations are described further in Sections 5.1, "Failure Analysis," and 5.3, "Assessing Digital System Dependability," of NEI 01-01. Section 5.1 provides guidance to analyze potential failures and consequences of the digital equipment and associated software to determine if they represent an acceptable risk level. Section 5.3 provides guidance to evaluate the dependability of the digital equipment and its associated software. A highly dependable digital device that is developed (including its software) in accordance with a defined life-cycle process and complies with applicable industry standards and regulatory guidance discussed in Section 5.3.3, "Digital System Quality," of NEI 01-01, should provide reasonable assurance of quality and low likelihood of failures. In addition to the evaluations of the quality and design processes, Section 3.2.2, "Software Common Cause Failures," of NEI 01-01 states, in part, that additional measures are appropriate for systems that are highly safety significant (e.g., the RPS and ESFAS) to achieve an acceptable level of risk. For digital modifications to such systems, defense-in-depth and diversity (D3) in the overall plant design are analyzed (in accordance with Section 5.2, "Defense-in-Depth and Diversity Analysis," of NEI 01-01) in order to assure that where there are vulnerabilities to software CCF, the plant has adequate capability to cope with vulnerabilities to software CCF. The inspectors reviewed the licensee's 10 CFR 50.59 evaluation, in action request (AR) 588797, design documentation, and additional information provided by Westinghouse (the CPLD boards' vendor) and identified that the licensee failed to recognize the CPLD boards used software to control their safety functions and the human system interface (HSI) used by operations and maintenance. As a result, the licensee did not perform the engineering evaluations and analyses (described in Sections 5.1 and 5.3 of NEI 01-01) to evaluate the digital device quality and design processes. In addition, the licensee did not perform the D3 analysis (described in Section 5.2 of NEI 01-01) to demonstrate that D3 in the overall plant design was adequate to cope with the possibility of software CCFs. Specifically, the inspectors identified that the failure modes and effects analysis performed by Westinghouse did not analyze potential software failures. Additionally, the development of the CPLD boards was outsourced to commercial vendors who used commercial software design practices and tools to design and program the CPLD boards which did not meet the quality identified in Section 5.3.3, "Digital System Quality," of NEI 01-01. The inspectors also identified that the new software-based HSI for the CPLD boards resulted in an additional burden to control room operators because it resulted in changes to indicators in the control room. Specifically, a warning in the Westinghouse vendor manuals advised of a new possible software failure mode for the HSI when maintenance personnel interfaced with the communication port on the safeguards driver CPLD board. The inspectors could not find any evidence that the licensee had performed an evaluation of this warning. The licensee's evaluation of criterion (c)(2)(vi) of 10 CFR 50.59 used guidance contained in NUREG-0800, "Standard Review Plan for the Review of Safety Analysis Reports for Nuclear Power Plants: Light Water Reactor Edition," to evaluate software CCF for the CPLD boards. Specifically, the licensee concluded that the 'Testability' criteria in Section 1.9, "Design Attributes to Eliminate Consideration of CCF," of BTP 7-19, "Guidance for Evaluation of Diversity and Defense-in-Depth in Digital Computer-Based I&C Systems," Rev. 6, could be used to eliminate consideration of software CCF because of the hardware functional testing performed by Westinghouse. Following consultation with NRR, the inspectors determined that the criteria in the BTP was intended to provide guidance to NRC staff in performing reviews of operating license applications (including LARs) and not as criteria to implement digital modifications under the 10 CFR 50.59 process without prior NRC review and approval. As a result, the inspectors determined that the lack of engineering evaluations of the quality and design processes did not provide reasonable assurance that the replacement CPLD boards did not create the possibility of a software CCF of the SSPS, which was a malfunction not previously evaluated in the UFSAR. Additionally, in failing to perform a D3 analysis the licensee did not demonstrate the capability to mitigate the effects of a software CCF, as specified by NEI 01-01, for highly safety significant systems.

The licensee entered this issue into their corrective action program as AR 617061 and initiated development of a LAR. In addition, the licensee performed an operability evaluation. Based on the functional testing performed by the vendor and satisfactory surveillance testing, the licensee determined the SSPS was operable. This determination, along with the boards' operating experience, provided a reasonable expectation that the system was operable.

Analysis:

The licensee's failure to follow the guidance in NEI 01-01 (referenced in licensee Procedure EGR-NGGC-0157), which resulted in the licensee implementing a change that created the possibility of common cause software malfunctions of RPS and ESFAS not previously evaluated in the UFSAR was a performance deficiency. The performance deficiency was determined to be more than minor because it was associated with the design control attribute of the Mitigating Systems cornerstone and adversely affected the cornerstone objective of ensuring the availability, reliability, and capability of systems that respond to initiating events to prevent undesirable consequences (i.e., core damage). Specifically, implementation of the new design CPLD boards affected the objective of ensuring the availability, reliability, and capability of the SSPS because the CPLD boards created the possibility of common cause software failures that were outside the current licensing bases of the SSPS. Additionally, in accordance with the guidance in the NRC Enforcement Manual, the 10 CFR 50.59 violation was more than minor because there was reasonable likelihood that the change would require NRC review and approval prior to implementation. The finding was screened using the traditional enforcement process because violations of 10 CFR 50.59 are considered to be violations that potentially impede or impact the regulatory process. Although this traditional enforcement violation is associated with a finding that can be evaluated and communicated with a Significance Determination Process (SDP) color reflective of the safety impact of the deficient licensee performance, the SDP does not specifically consider the regulatory process impact. Thus, although related to a common regulatory concern, it is necessary to address the traditional violation and finding using different processes to correctly reflect both the regulatory importance of the violation and the safety significance of the associated finding.

The inspectors used Inspection Manual Chapter (IMC) 0609, "Significance Determination Process," dated 6/2/11, to determine the safety significance of the finding.

Using IMC 0609, Attachment 4, "Initial Characterization of Findings," dated 6/19/12, Table 2, the inspectors determined that the finding affected the Mitigating Systems cornerstone. The inspectors then evaluated the finding using IMC 0609, Appendix A, "The Significance Determination Process for Findings At-Power," dated 6/19/12, Exhibit 2, for the Mitigating Systems Cornerstone. The inspectors determined the finding was of very low safety significance (Green) because the deficiency affected the design of the SSPS and was confirmed not to result in loss of operability of the system. In accordance with the NRC Enforcement Policy, Section 6.0, "Violation Examples," dated 1/28/13, a traditional enforcement violation of 10 CFR 50.59 that results in conditions evaluated as having very low safety significance (i.e., Green) by the SDP is considered a SL IV violation (Section 6.1.d). The finding had a cross-cutting aspect in the "Decision Making" component of the "Human Performance" area because the most significant causal factor of the performance deficiency was that the licensee failed to oversee the work activities of vendors such that nuclear safety was supported H.4(c).

Enforcement:

Title 10 of the Code of Federal Regulations, Part 50.59(c)(2) states, in part, that the licensee shall obtain a license amendment prior to implementing a proposed change, if the change would create a possibility of a malfunction of an SSC important to safety with a different result than any previously evaluated in the UFSAR. Contrary to this, the licensee failed to obtain a license amendment prior to implementing a change that created a possibility of a malfunction of the SSPS with a different result than previously evaluated in the UFSAR. Specifically, since the spring of 2012 (when the CPLD boards were installed), the licensee implemented a change to the SSPS circuit boards which created a possibility of common cause software malfunctions of the RPS and ESFAS not previously evaluated in the UFSAR. After the team identified this issue, the licensee performed an operability evaluation and determined the SSPS was operable. Additionally, at the time of the inspection, the licensee had initiated development of a LAR. This violation is being treated as an NCV, consistent with Section 2.3.2 of the Enforcement Policy. The violation was entered into the licensee's corrective action program as AR 617061. (NCV 05000400/2013009, Failure to Submit a License Amendment Request for a Digital Modification to the Solid State Protection System)

4OA6 Management Meetings

.1 Exit Meeting Summary

On July 15, 2013, the team presented the inspection results to Mr. Ernest Kapopoulos, Jr., Site Vice President, and other members of the licensee's staff. The team verified that no proprietary information was retained by the inspectors or documented in this report.

SUPPLEMENTARY INFORMATION

KEY POINTS OF CONTACT

Licensee personnel

D. Corlett, Supervisor, Licensing/Regulatory Programs

J. Caves, Site Licensing

NRC personnel

J. Thorp, Chief, Instrumentation & Controls (I&C) Branch, Division of Engineering, NRR

N. Carte, Senior Electronics Engineer, I&C Branch, Division of Engineering, NRR

S. Arndt, Senior Technical Advisor for Digital I&C, Division of Engineering, NRR

J. Austin, Shearon Harris Senior Resident Inspector

P. Lessard, Shearon Harris Resident Inspector

LIST OF ITEMS OPENED, CLOSED, AND DISCUSSED

Opened and Closed

05000400/2013009-01 NCV Failure to Submit a License Amendment Request for a Digital Modification to the Solid State Protection System (Section 1R17)

Closed

05000400/FIN-2013002-03 URI Solid State Protection System Digital Modification (Section 1R17)

Section 1R17: Evaluations of Changes, Tests, and Experiments and Permanent Plant Modifications

Engineering Change

EC 78484, Digital Modification to SSPS Control Boards, Rev. 6

Basis Documents Technical Specifications, Current Updated Final Safety Analysis Report, Current

Condition Reports

Reviewed

AR 588797

Other Documents

Branch Technical Position 7-19 (NUREG-0800), Guidance for Evaluation of Diversity and Defense-in-Depth in Digital Computer-Based Instrumentation and Control Systems, Rev.6

MDES-EDS-A-418A Eng. Data Sheet Universal Logic Board Configuration Settings

MDES-EDS-A-511A Eng. Data Sheet Safeguards Driver Boards Configuration Settings

MDES-EDS-A-515A Eng. Data Sheet Under voltage Output Board Configuration Settings Nuclear Energy Institute,

NEI 01-01, Guideline on Licensing Digital Upgrade - EPRI

TR-102348, Rev.1

Nuclear Energy Institute,

NEI 96-07, Guidelines for 10

CFR 50.59 Implementation, Rev.1

WCAP-16769-P, WEC SSPS Universal Logic Board Replacement Summary Rpt, Rev. 2

WCAP-16770-P, WEC SSPS Safeguards Driver Board Replacement Summary Rpt, Rev. 0

WCAP-16771-P, WEC SSPS Under voltage Driver Board Replacement Summary Rpt, Rev. 1

WNA-TR-02644-SCP, SSPS New Design Circuit Boards Final Logic Test Rpt, Rev. 0 Z05R0 Questions to Westinghouse (EC 70350) Z20R5 Westinghouse Email on Frozen MCB (EC 70350)

Westinghouse Electric Co. letter to John Caves, Duke Energy - Reg. Affairs, March 7, 2013 Westinghouse Electric Co. letter to John Caves, Duke Energy - Reg. Affairs, April 16, 2013

Action Requests

Written as a Result of the Inspection AR 617061