Text

Distributed Control System (DCS) Data Storm Test Results
	ML15282A537
Person / Time
Site:	Watts Bar
Issue date:	10/09/2015
From:	James Shea; Tennessee Valley Authority
To:	; Document Control Desk, Office of Nuclear Reactor Regulation
Shared Package
ML15282A536	List: CNL-15-193, Distributed Control System (DCS) Data Storm Test Results;
References
	CNL-15-193
	Download: ML15282A537 (12)
	v • d • e

Security-Related Information Withhold Under 10 CFR 2.390.

This letter is decontrolled when separated from Attachment 4 of the Enclosure Tennessee Valley Authority, 1101 Market Street, Chattanooga, Tennessee 37402 CNL-15-193 October 9, 2015 10 CFR 50.4 ATTN: Document Control Desk U.S. Nuclear Regulatory Commission Washington, D.C. 20555-001 Watts Bar Nuclear Plant, Unit 2 Construction Permit No. CPPR-92 NRC Docket No. 50-391

Subject:

WATTS BAR NUCLEAR PLANT (WBN) UNIT 2 - DISTRIBUTED CONTROL SYSTEM (DCS) DATA STORM TEST RESULTS

References:

1. Letter from TVA to NRC, CNL-15-130, Watts Bar Nuclear Plant (WBN)

Unit 2 - Distributed Control System (DCS) Data Storm Test Plan, dated July 10, 2015 (ML15196A515)

2. NUREG-0847, Supplement No. 23, Safety Evaluation Report Related to the Operation of Watts Bar Nuclear Plant, Unit 2, dated July 2011 (ML11206A499)

3. Electronic Mail from NRC to TVA, Clarification Call: Data Storm Testing for Open Item 83, dated September 15, 2015 On July 10, 2015, Tennessee Valley Authority (TVA) submitted the Distributed Control System (DCS) Data Storm Test Plan to the Nuclear Regulatory Commission (NRC) and committed to provide the results of the Data Storm Test after completion of the test (Reference 1). The completion of the Data Storm Test is the subject of Supplemental Safety Evaluation Report (SSER) 23, Appendix HH, Open Item No. 83 (Reference 2).

On September 17, 2015, a teleconference was held between the NRC and TVA to discuss eleven questions provided by the NRC in Reference 3. TVA committed to provide a response to these questions with the Data Storm Test results.

Security-Related Information Withhold Under 10 CFR 2.390.

This letter is decontrolled when separated from Attachment 4 of the Enclosure U.S. Nuclear Regulatory Commission CNL-15-193 Page 2 October 9, 2015 The purpose of this letter is to provide the results of the DCS Data Storm Test to the NRC.

The Data Storm Test was completed on October 2, 2015. The Data Storm Test was completed with four test deficiencies. All test deficiencies have been addressed and all acceptance criteria were met. The enclosure to this letter provides the DCS Data Storm Test Summary Report. Attachment 3 of the enclosure provides TVAs response to the eleven questions requested in Reference 3.

The material in Attachment 4 of the enclosure contains security-related information identified by the designation Security-Related Information - Withhold Under 10 CFR 2.390. TVA requests this information be withheld from public disclosure in accordance with the provisions of 10 CFR 2.390(d).

There are no new regulatory commitments made in this letter. Should you have questions regarding this submittal, please contact Gordon Arent at (423) 365-2004.

I declare under penalty of perjury that the foregoing is true and correct. Executed on the 9th day of October 2015.

Respectfully, J. W. Shea Vice President, Nuclear Licensing

Enclosure:

Distributed Control System (DCS) Data Storm Test Summary Report cc (Enclosure):

U.S. Nuclear Regulatory Commission, Region II NRC Project Manager - Watts Bar Nuclear Plant, Unit 2 NRC Senior Resident Inspector - Watts Bar Nuclear Plant, Unit 2 J. W. Shea Digitally signed by J. W. Shea DN: cn=J. W. Shea, o=Tennessee Valley Authority, ou=Nuclear Licensing, email=jwshea@tva.gov, c=US Date: 2015.10.09 15:02:14 -04'00'

Security-Related Information Withhold Under 10 CFR 2.390.

This letter is decontrolled when separated from Attachment 4 of the Enclosure ENCLOSURE WATTS BAR NUCLEAR PLANT, UNIT 2 DISTRIBUTED CONTROL SYSTEM (DCS) DATA STORM TEST

SUMMARY

REPORT

Security-Related Information Withhold Under 10 CFR 2.390.

This letter is decontrolled when separated from Attachment 4 of the Enclosure Page E-1 ENCLOSURE DISTRIBUTED CONTROL SYSTEM (DCS) DATA STORM TEST

SUMMARY

REPORT Test Summary:

The DCS Data Storm Test was performed under Nuclear Construction Project Instruction (PI)

NC-PI-98-03, Revision 0. Testing started on September 29, 2015 and was completed on October 2, 2015. The Data Storm Test was completed with four test deficiencies. All test deficiencies have been addressed and all acceptance criteria were met.

The data storm was performed on all DCS network switches, the two root switches in the Auxiliary Instrument Room, and the two edge switches in the Auxiliary Control Room. The data storm consisted of one broadcast storm and one multicast storm injected at each switch with all broadcast and multicast suppression disabled, and one broadcast and multicast storm injected at each switch with broadcast and multicast suppression enabled. The data storms were injected into a spare port in each switch in order to limit the amount of switch configurations required (see Attachment 1 for Network Layout).

During the data storm test, each Control Process (CP) was monitored by embedded indicating lights to determine CP status, and verify CP functionality (inputs through processor and through outputs) by using the hard-wired hand-auto stations for W201CP through W205CP, and W207CP through W215CP (see Attachment 2 for Hand-Auto Station with CP). The W206CP does not have a hard-wired hand-auto station. Functionality for W206CP was monitored by injecting a known waveform from the DCS into a spare analog input, passing the signal through the processor to a spare analog output, and monitoring the resultant waveform with a data recorder (Astromed) to verify functionality. In addition to the above, the following items were monitored during the Data Storm performance: DCS workstation status, DCS connection to Integrated Computer System (ICS), and the DCS Trouble Alarm.

The following responses were expected and observed during the DCS data storm with broadcast and multicast suppression disabled for each network switch.

Broadcast Storms:

x The connection between DCS and ICS did not pass points.

x DCS workstations had no network communications.

x Received DCS trouble alarm (82-F).

x Controllers tested continued to function.

x W206CP wave continued functioning normal.

x No change in Reactor Trip Status Lights.

x No CP pair had both processors fail.

Security-Related Information Withhold Under 10 CFR 2.390.

This letter is decontrolled when separated from Attachment 4 of the Enclosure Page E-2 Multicast Storms:

x The connection between DCS and ICS did not pass points.

x DCS workstations had no network communications.

x Did not receive DCS trouble alarm (82-F).

x Controllers tested continued to function.

x Some CPs went single, but no CP pair had both processors fail.

x No change in Reactor Trip Status Lights.

x W206CP output waveform had intermittent flat line indications (output frozen) of approximately 800 msec duration. These indications were likely due to "hot remarries" of the CP pair caused by the data storm. On one occasion, W206CP failed single after several "hot remarries." This is expected based on the Foxboro fault detection for the CPs.

The following responses were expected and observed during the DCS data storm with broadcast and multicast suppression enabled for each network switch.

Broadcast Storms:

x The connection between DCS and ICS did pass points.

x DCS workstations had network communications.

x Did not receive DCS trouble alarm (82-F).

x Controllers tested continued to function.

x W206CP waveform continued functioning normally.

x No change in Reactor Trip Status Lights.

x No CPs had both processors fail and No CPs went single.

Multicast Storms:

x The connection between DCS and ICS did pass points.

x DCS workstations had network communications.

x Received DCS trouble alarm (82-F).

x Controllers tested continued to function.

x W206CP waveform continued functioning normally.

x No change in Reactor Trip Status Lights.

x No CPs had both processors fail. Note: During multicast storm on W2SW04, W214CP and W215CP (Auxiliary Control Room) went single. No other CP pairs went single.

Security-Related Information Withhold Under 10 CFR 2.390.

This letter is decontrolled when separated from Attachment 4 of the Enclosure Page E-3 Test Deficiencies:

The following Test Deficiency Notices (TDN) were recorded during the performance of the Data Storm Test in NC-PI-98-03 Revision 0. All test deficiencies have been addressed in accordance with Startup Manual Procedure, SMP-14.0, Test Deficiency Notices, and all acceptance criteria were met.

TDN 15-1763 Description of Deficiency: During performance of NC-PI-98-03 Section 4.1 [1] while reviewing drawing 2-45W2635-131-1 Rev 2, found patch panel cable on W2SW01 Port 51 & W2SW02 Port 51 rolled. Also found Cable ID on W2SW01, Port 51 incorrect on Drawing 2-45W2635-131-1 Rev 1. The cable ID should be R184C16 not R184C18 to match drawing 2-45W2635-131A-1 Rev 2.

Resolution: Correct patch panel cable by rolling connections at Port 51 on W2SW01 and W2SW02. Drawing Deviation 15-0313 corrects cable ID number on 2-45W2635-131-1.

TDN 15-1855 Description of Deficiency: During performance of NC-PI-98-03, patch panel cables from W2SW01 & W2SW02 to the patch panels were found not labeled and some fiber optic patch cables were bent in the patch panel.

Resolution: Label patch panel cables as necessary and correct bent fiber optic patch cables in Work Order (WO) 116065019 (Switch Installation WO).

TDN 15-1790 Description of Deficiency: During performance of NC-PI-98-03, 2-PC-46-20 output meter would slightly stutter during ramp increase and decrease. This was seen during all sections of test performance and after the test was completed.

Resolution: The 2-PC-46-20 output meter would slightly stutter during ramp increase and decrease during data storm test performance and when the data storm test was not in progress. The 2-PC-46-20 was still able to increase to 100% and decrease to 0% in approximately 5 seconds. This is due to W205CP being a heavily loaded processor and occurs irrespective of the data storm. WO 117236141 was generated to review W205CP processing.

TDN 15-1791 Description of Deficiency: During performance of NC-PI-98-03 Section 6.2.8 (only),

received DCS Critical Loop to Manual Alarm 81-F. The 2-LIC-2-9 (HW Level Makeup Controller) went from Auto to Manual during test performance.

Resolution: The controller swapped from Auto to Manual as designed. The alarm was received due to transmitter lineup issues and plant conditions. The 2-LIC-2-9 was returned to Auto following data storm completion in section 6.2.8. The DCS Critical Loop to Manual Alarm was not received in any section prior to 6.2.8 or any section after. The WO 117236166 was generated to review processing of inputs and outputs of 2-LIC-2-9.

Security-Related Information Withhold Under 10 CFR 2.390.

This letter is decontrolled when separated from Attachment 4 of the Enclosure Page E-4 Conclusions During the DCS Data Storm Test, DCS controllers continued to function. There were no CP pairs where both processors failed during any section of data storm testing. Reactor Trip Status Lights did not change during any section, and thus the data storm had no impact on the Reactor Protection System. All the required acceptance criteria were met as delineated below:

o A data storm on the DCS network has no effect on the Reactor Protection System per clause 6.3 of IEEE Standard 603-1991.

o A data storm on the Auxiliary Control Room Network does not cause a functional failure of any Auxiliary Instrument Room CP pair due to the simulated data storm.

Loss of CP redundancy does not constitute a functional failure.

o A data storm on the DCS network does not cause a functional failure of any CP pair due to the simulated data storm. Loss of CP redundancy does not constitute a functional failure.

See Attachment 3 for the response to NRC questions regarding the Data Storm Test. A copy of the performed PI is provided in Attachment 4.

Security-Related Information Withhold Under 10 CFR 2.390.

This letter is decontrolled when separated from Attachment 4 of the Enclosure Page EA1-1

Security-Related Information Withhold Under 10 CFR 2.390.

This letter is decontrolled when separated from Attachment 4 of the Enclosure Page EA2-1 WBN Unit 2 DCS Data Storm Hand-Auto Station with CP (2-NC-PI-98-03 Rev 0)

System Hand Station Description CP Location 3

2-FIC-3-35 SG 1 MFW REG VLV W201CP MCR 3

2-FIC-3-48 SG 2 MFW REG VLV W202CP MCR 3

2-FIC-3-90 SG 3 MFW REG VLV W203CP MCR 3

2-LIC-3-103A SG 4 MFW BYP REG W204CP MCR 46 2-PC-46-20 MFPT Master SPD W205CP MCR 1

2-PIC-1-6A SG 1 PORV PCV-1-5 W207CP MCR 1

2-PIC-1-13A SG 2 PORV PCV-1-12 W208CP MCR 1

2-PIC-1-24A SG 3 PORV PCV-1-23 W209CP MCR 1

2-PIC-1-31A SG 4 PORV PCV-1-30 W210CP MCR 1

2-PIC-1-33 STM Dump Press Control W211CP MCR 62 2-FC-62-142 PW to Blender FCV-62-143 Control W212CP MCR 62 2-HIC-62-78A Letdown HX Outlet Temp TCV-70-192 Control W213CP MCR 1

2-PIC-1-6C SG 1 PORV PCV-1-5 W214CP ACR 1

2-PIC-1-13C SG 2 PORV PCV-1-12 W215CP ACR

Security-Related Information Withhold Under 10 CFR 2.390.

This letter is decontrolled when separated from Attachment 4 of the Enclosure Page EA3-1 WBN Unit 2 DCS Data Storm Response to NRC Questions (2-NC-PI-98-03 Rev 0)

NRC Questions a) Verify that an approved test procedure was used. If changes to the approved procedure were identified during testing, verify that these changes were reviewed and approved.

Response: NC-PI-98-03 Revison 0 was used as approved. No procedure changes were required.

b) Verify that TVA used a controlled document accurately defining the network layout architecture.

Response: Controlled drawings 2-45W2635-131-1 Rev 2, 2-45W2635-131-2 Rev 1, and 2-45W2635-115 Rev 3 were used.

c) TVA performed the Broadcast and Multicast tests at each of the locations identified in the test plan.

Response: See Attachment 1 for Network Layout and the discussion in the enclosure.

d) Document how live process data was simulated for verification of test acceptance criteria for response time evaluation.

Response: Two methods were used.

1. Hand-Auto Stations for W201CP to W205CP and W207CP to W215CP output demand signal were raised and lowered full scale.

2. W206CP was monitored by injecting a known waveform from the DCS into a spare analog input, passing the signal through the processor to a spare analog output, and monitoring the resultant waveform with a data recorder.

e) Verify that the test documentation recorded specifically where each packet generator was connected for each portion of the test. (Provide common controller pair function name so the staff understands the controller function involved for each portion of the test.)

Response: See Attachment 1 for Network Layout and Attachment 2 for Hand-Auto Station with CP

Security-Related Information Withhold Under 10 CFR 2.390.

This letter is decontrolled when separated from Attachment 4 of the Enclosure Page EA3-2 f) Verify that the test documentation recorded specifically where each packet sniffer was connected for each portion of the test. (Provide common controller pair function name so the staff understands the controller function involved for each portion of the test.)

Response: See Attachment 1 for Network Layout and Attachment 2 for Hand-Auto Station with CP for system connections and disconnections. Connections and disconnections are documented in Section 6 of NC-PI-98-03.

g) Verify that the test documentation identified the specific test result achieved, and how that result compared against the test acceptance criteria for that portion of the test.

Response: See Test Summary Report. Sections 6.2 and 6.3 of NC-PI-98-03 document the specific test result achieved against the test acceptance criteria.

h) Verify that the test documentation recorded accurately and with appropriate level of detail any degradation in performance of the control function while the broadcast or multicast tests were being conducted. (i.e., including response time to simulated input signals before and during the application of the data storm test)

Response: See Test Summary Report. Section 6.2 and 6.3 of NC-PI-98-03 document any degradation in performance of the control function while the broadcast or multicast tests were being conducted.

i)

Verify that for each portion of the test, the data storm speed of broadcast or multicast was higher than the setting employed on each protective speed limiter switch.

Response: For data storms in sections 6.2.3 to 6.2.10, multicast and broadcast switch limits were disabled using Appendix B in the test instruction. For sections 6.3.2 to 6.3.9, switch limits were enabled using Appendix B in the test instruction.

j)

Verify whether any processor pair failures resulted. If so, identify whether the analog signals for that processor pair were appropriately sent to the control room hand/auto stations and that the simulated control function was still being maintained.

Response: No CP pairs failed during the test. During some testing, some CP pairs did go single.

k) Verify that anomalies observed were carefully and thoroughly documented, and then followed up with appropriate engineering evaluation.

Response: The Test Deficiency Notice (TDN) Process was used in accordance with Startup Manual Procedure, SMP-14.0, Test Deficiency Notices. See Test Deficiencies section of Test Summary Report.

Security-Related Information Withhold Under 10 CFR 2.390.

This letter is decontrolled when separated from Attachment 4 of the Enclosure Page EA3-3 l)

Verify that any unresolved anomalies were documented and entered into the Corrective Action Program for the plant.