ML083030312

Submittal of Supporting Documentation for September 16, 2008 Regulatory Conference, Cover Letter Through Root Cause Investigation Report Content and Format, Page 20 of 73

ML083030312
Person / Time
Site:	Byron
Issue date:	09/08/2008
From:	Grundmann W Exelon Generation Co, Exelon Nuclear
To:	Document Control Desk, Office of Nuclear Reactor Regulation
Shared Package
ML083030310	List: BYRON 2008-0096, Submittal of Supporting Documentation for September 16, 2008 Regulatory Conference, Root Cause Investigation Report Content and Format, Page 45 of 73 Through End ML083030312 ML083030316
References
BYRON 2008-0096, IR-08-003
Download: ML083030312 (39)

Text

Nuclear September 8,2008 LTR: BY RON 2008-0096 File: 1.10.01 01 U. S. Nuclear Regulatory Commission ATTN: Document Control Desk Washington, DC 20555-0001 Byron Station, Unit 2 Facility Operating License No. NPF-66 NRC Docket No. STN 50-455

Subject:

Submittal of Supporting Documentation for September 16, 2008 Regulatory Conference

Reference:

Letter from Cynthia D. Pederson (NRC) to C.G. Pardee (Exelon Generation Company, LLC), "Byron Station, Units 1 and 2 NRC Integrated Inspection Report 0500045412008-003 0500045512008-003 Preliminary White Finding," dated August 14, 2008 In the referenced letter, the Nuclear Regulatory Commission (NRC) issued an inspection report with respect to the April 5, 2008 event involving the Byron Station Unit 2 inadvertent entry into an elevated risk condition. The emergent status of two Unit 1 Essential Service Water (SX) valves (i.e., 1SX033 and 1 SX034) impacted the Unit 2 risk condition. This report concluded that the failure to perform an updated risk evaluation for Unit 2 based on existing plant conditions was an apparent violation of 10 CFR 50.65(a)(4). Using the Significance Determination Process, the NRC has preliminarily determined that this finding for Unit 2 is to be of low to moderate safety significance (White). However, a risk evaluation completed by Byron Station Engineering staff concluded the finding to be of low safety significance (Green).

Consequently, a Regulatory Conference has been scheduled for September 16, 2008, to present to the NRC our perspective on the facts and assumptions used by the MRC to arrive at their preliminary risk conclusion. The NRC encouraged, in the referenced letter, that suppoding documentation for our presentation be submitted at least one week prior to the conference. This letter provides the MRC with the requested supporting documentation.

September 8, 2008 U.S. Nuclear Regulatory Commission Page 2 of 2 provides the risk evaluation entitled, "Byron Significance Determination Process (SDP) Evaluation of Failure to Conduct a Risk Evaluation Prior to Disabling 1 SX033 and 1 SX034 Remote Isolation Capability." This evaluation concludes the core damage frequency associated with configuration of concern to be well below the NRC established threshold for Green significance. Attachment 2 provides the root cause evaluation for this event entitled, "Unplanned Unit Two On-line Risk (OLR) Orange Condition Root Cause lnvestigation Report."

This letter contains no regulatory commitments. If you have any questions concerning this letter, please contact Mr. William Grundmann at (815) 406-2800.

William Grundmann Regulatory Assurance Manager Byron Station :

Byron Significance Determination Process (SDP) Evaluation of Failure to Conduct a Risk Evaluation Prior to Disabling 1 SX033 and 1 SX034 Remote Isolation Capability, June 2008 :

Unplanned Unit Two On-line Risk (OLR) Orange Condition Root Cause lnvestigation Report cc:

Regional Administrator, USNRC, Region Ill NRC Resident Inspector, Clinton Power Station A. M. Stone, USNRC Region Ill Byron Significance Determination Process (SDP) Evaluation of Failure to Conduct a Risk Evaluation Prior to Disabling 1 SX033 and 1 SX034 Remote Isolation Capability, June 2008

Byron PRA APPLICATION NOTEBOOK BB PRA-017.91 B Revision 0 Byron SDP Evaluation of Failure to Conduct a Risk Evaluation Prior to Disabling 1 SX033 and 1 SX034 Remote Isolation Capability June 2008

PURPOSE This evaluation examines the risk significance associated with the failure to conduct a risk evaluation in accordance with the requirements of the Maintenance Rule section a(4) before removing power to 1 SX033 and 1 SX034 valves as part of maintenance activities during the refueling outage at Byron Unit

1. Byron Unit 2 was operating at power during the outage at Unit 1.

BACKGROUND On April 6, 2008, Byron staff members were making preparations to replace the 1 SX034 valve with a newer and improved valve. In the process of isolating SX flow to the valve, the 1 SX033 valve was closed using its motor operator.

Although closed, the 1 SX033 valve did not provide a completely water tight seal sufficient to allow removal and replacement of the 1 SX034 valve without water leakage into the work area. For large (36") butterfly valves of this type, small leakage is not an unusual occurrence.

A decision was made to use the manual operator for 1 SX033 to tighten the seal between the butterfly and the seat by manually closing the valve operator further than the motor operator could. While manually tightening the closure of the valve, operators notice a "pop" noise followed by a decrease in torque needed turn the valve operator. At that point, the effort to replace 1 SX034 was stopped and troubleshooting to determine the status of the 1 SX033 operator was begun.

As part of the troubleshooting efforts, both the 1 SX033 and 1 SX034 valves were fully opened using their motor operators. Power was then removed from both valves as part of the effort to investigate the condition of the 1 SX033 operator.

The open position is the normal operating position of both the 1 SX033 and 1 SX034 valves. This alignment cross ties the SX pump supply to the A and B headers within Unit 1. This allows one SX pump to provide flow for both trains during normal operations.

However, operators failed to recognize that removal of power for these valves disabled the remote isolation capability normally relied on in the event of an auxiliary building flooding condition (IR # 759945 [8]). Abnormal Operating Procedure OBOA PRI-8 [I] directs operators to isolate the trains as part of steps to determine which train contains the leak so that it can be effectively isolated to limit the impact of flooding in the auxiliary building. Auxiliary Building flooding could impact both the unit in outage as well as the unit at power because of the nature of sharing inherent in the design of the SX system and the layout of the Auxiliary Building. Therefore, a risk evaluation for the condition should have been conducted for both the unit in outage and the unit at power

BB PRA-017.91 B Page 3 METHOD and ACCEPTANCE CRITERIA An evaluation of the condition for both units should have been made prior to entering a condition where the both the 1 SX033 and 1 SX034 valves could not be isolated remotely from the control room in accordance with OBOA PRI-8. NRC Inspection Manual Chapter 0609 Appendix K [Z] is the guiding document for conducting the significance determination process (SDP) for "findings related to licensee assessment and management of risk associated with performing maintenance activities under all plant operating or shutdown conditions".

In the event that no assessment of risk was performed prior to maintenance activities, App. K provides a flow chart to assess the impact of the failure to assess the risk implications of the maintenance. Portions of that flow chart applicable to this occurrence are reproduced below, App. K indicates that for cases where no risk assessment was performed, the risk deficit is defined as follows:

"If the licensee did not perform a risk assessment at all, the actual risk increase (ICDPactual ) is the product of the incremental CDF and the annualized fraction of the duration of the configuration [i.e., ICDPactual = ICDFactual x (duration in hours) t (8760 hours0.101 days <br />2.433 hours <br />0.0145 weeks <br />0.00333 months <br /> per reactor year)], where ICDFactual = CDFactual - CDFzero-maintenance The risk deficit, ICDPD, is equal to ICDP when the licensee's performance deficiency involves not conducting a risk assessment."

App K identifies that the number of RMAs (risk management activities) is also a factor in determining the SDP "color". The flowchart shows how the combination of the risk deficit and the number of RMAs taken affects the final SDP evaluation.

Byron does not use a quantitative shutdown risk model for assessing and managing risk for outage situations, but does use a risk model for assessing and managing risk for operations at power. App K notes that qualitative assessments are done for the former in a Note in prior to Section 4.0 which invokes the flowchart noted above.

Page 4 Flowchart 1 Assessment of Risk Deficit tMck (Step 4.2) r.,

,,H~

/

i \\.

,' - -- -- - -.,~

'\\

,,,"is Risk ~ e f l f ~

+

,' Is Rb* fiefkit \\..,

i > 1 E-6 (EDPOj w -

No-:

h

~inli~;+~es-<,'

c 5 E-6 [ICM"D) w >-%

'-q 1 E.7 $LERPDJ;,-.,."'

'-. $,5 E.7 (iLERP0)Y

-,/

1'.

I Yes I

Ya8 4

,~

Yes x

/'

\\,

3efW;'"

Is Risk I

~aii-l' 2 1 E 4 [K

<.,- 3a m a e RM&...

.-4.,,,."

[ten?

,',l w 2 RMAS '\\..-

I J"

Y e "r"

.t

,I

,,-'.IS Rhk Deficit".,.

< 5 E i (iiDPD) cs, -

NI

.",s E.6 (ILERPq)J,

.< 5 E-5 flLEr7PD.u.. /'

. L..,

,,+'

BB PRA-017.91 B Page 5 To evaluate the implications of having the 1 SX033 and 1 SX034 valves' power removed there are tvvo analyses that need to be performed. The first one involves the impact on the unit in outage and the second for the unit at power.

T&RM ER-AA-600-1041 [6] provides guidance for performing SDP analyses.

T&RM ER-AA-600-1012 [3] provides the guidance for documenting this evaluation.

LERF values are more than an order of magnitude lower than CDF values for Byron. The specific failure modes of the 1 SX033 and 1 SX034 valves have no impact on the LERF except through their impact on CDF. Therefore, calculations based on CDF are the more limiting cases.

ANALYSIS INPUTS and RESULTS For unit 1 which was in a refueling outage at the time that the valves had power removed, a qualitative assessment is required. During this period, the reactor head was removed and was flooded up so that fuel could be moved back into the reactor. The spent fuel pool was at normal levels. Decay heat was being removed via the component cooling heat exchangers to the essential service water system.

According to the plant status reports prepared for each shift turnover during the outage in accordance with Attachment 1 to OU-BY-104 Revision 10 [7], there was in excess of 16 hours1.851852e-4 days <br />0.00444 hours <br />2.645503e-5 weeks <br />6.088e-6 months <br /> to core boiling in the event of a loss of cooling and over 24 hours2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br /> to core damage. Even if one were to assume a total loss of essential service water due to an auxiliary building flooding event with failure of both 1 SX033 and 1 SX034 to close, the amount of water required to keep the core and spent fuel pool covered and cooled is minimal. Fire protection water, an alternate cooling water source, alone would be more than adequate for those purposes. In addition recent changes made in response to the NRC security orders under section B.5.b would also be available if needed. These are not included in this evaluation due to the sensitive nature of information related to those orders.

Given an SX leak in the Auxiliary Building at a rate of 7.6E-04 per year based on the flooding analysis notebook (BE3 PRA-012 Rev. 4 [4j) and a period of interest of 42.3 hours3.472222e-5 days <br />8.333333e-4 hours <br />4.960317e-6 weeks <br />1.1415e-6 months <br />, the maximum frequency of loss of SX would be 3.7E-06 per year.

Given the fact that the times to boil and uncovery were very long, and that fire protection water was available (along with other sources identified as part of the recent security inspections associated with section B.5.b of the NRG orders following the 911 1 events), it is qualitatively presumed that the probability of core damage frequency deficit for the outage unit would be significantly below the 1 E-6 value noted in the App. K flowchart. Based on these qualitative insights, the SDP assessment for the outage unit would be Green.

BB PRA-017.91B Page 6 For the unit operating at power, there are two scenarios where the loss of integrity of SX piping in the Auxiliary Building could affect risk. These are risks associated with leaks (flow rate ~ 2 0 0 0 gpm) and with ruptures (flow rates >2000 gpm. In order to prevent core damage, the flooding analysis presumes that loss of the SX pumps and inability to maintain charging pump flow will lead to core damage. This is due to the potential reactor coolant pump (RCP) seal LOCA that could occur if charging (RCP seal injection) and component cooling water (CCW barrier cooling) were both lost. Flooding induced failures of the SX pumps could lead to the loss of the RCP thermal barrier cooling capability via loss of CCW cooling while inundation of the charging pumps would lead to failure of RCP seal injection. Loss of SX would also prevent operation of RCS injection systems so that a RCP Seal LOCA would eventually lead to core damage due to lack of injection.

The Flooding Analysis [4] indicates that the frequency of SX pipe ruptures (flow rates ~2000 gpm) is 9.6E-06 per year. For the 42.3 hour3.472222e-5 days <br />8.333333e-4 hours <br />4.960317e-6 weeks <br />1.1415e-6 months <br /> duration, this is equivalent to a frequency of 4.6E-08 per year. Assuming that neither 1 SX033 nor 1 SX034 could be closed to isolate a rupture in accordance with OBOA PRI-8 in time to prevent loss of SX and loss of charging, the frequency would still indicate a Green condition per Appendix K of IMC 0609.

The frequency of leaks between 100 gprn and 2000 gprn is 7.6E-04 in the Flooding Analysis [4]. For the 42.3 hour3.472222e-5 days <br />8.333333e-4 hours <br />4.960317e-6 weeks <br />1.1415e-6 months <br /> duration when neither the 1SX033 nor the 1 SX034 valves were capable of closure in accordance with OBOA PRI-8, this equates to a frequency of 3.7E-06. However, even for the maximum flow rate among leaks (2000 gpm) it would take 10.8 hours9.259259e-5 days <br />0.00222 hours <br />1.322751e-5 weeks <br />3.044e-6 months <br /> to reach the point (1.29 million gallons per the Flooding Analysis) where the charging pumps would be inundated (215 hours0.00249 days <br />0.0597 hours <br />3.554894e-4 weeks <br />8.18075e-5 months <br /> at the minimum leakage rate). BOP SX-22 151 provides the procedure for isolating SX leaks at specific locations in the Auxiliary Building.

The probability of failure to isolate a leak is low for several reasons:

1. The time to isolate is between 10.8 hours9.259259e-5 days <br />0.00222 hours <br />1.322751e-5 weeks <br />3.044e-6 months <br /> for a 2000 gprn leak and 21 5 hours5.787037e-5 days <br />0.00139 hours <br />8.267196e-6 weeks <br />1.9025e-6 months <br /> for a 100 gprn leak.

2. A procedure exists to specify valves to isolate any particular piping segment in the Auxiliary Building. In addition, Operators are trained in use of P&IDs for troubleshooting problems with systems that are not functioning as would be expected in procedures such as OBOA PRI-8.

3. Complete isolation of the affected segment per BOP SX-22 is not needed in order to stop the flooding. The impact of failure of closing 1SX033 and 1 SX034 is that the A and B supply for the unit 1 trains are cross tied.

Other means exist to isolate the supply side of either the A or B train without having to perform the complete isolation of a leaking pipe segment. isolation of five valves (1 SX004, 1 SXO1 GA(B), 1 SX013A(B),

1 SX2103A ( I SX173), and 1 SX052AfB))in the major supply headers

66 PRA-017.91 B Page 7 downstream of 1 SX012A(B) would accomplish the same function as closing 1 SX033 or 1 SX034 for train isolation purposes.

4. With unit 1 in a refueling outage, there were more people on-site and in the Auxiliary Building than normally would be the case. Therefore, detection of the leak location and availability of staff to perform isolation steps would be enhanced.

5. Additional staff through manning of the Outage Control Center (and TSClEOF if needed) would be available in plenty of time to diagnose and effect isolation.

Using the SPAR-H methodology, the value for probability of failure to isolate would be about 6.OE-02 (Attachment 2). When combined with the frequency of the condition (leaks between 100 and 2000 gpm with failure of 1 SX033 and 1 SX034 for 42.3 hours3.472222e-5 days <br />8.333333e-4 hours <br />4.960317e-6 weeks <br />1.1415e-6 months <br />) the result is about 2.2E-07. When combined with the rupture failure to isolate probability of 4.6E-08, the total for leaks and ruptures that could not be isolated would be 2.7E-07 which is well below the 1.OE-06 value in Appendix K.

Using the nominal HRA methodology for Byron and Braidwood which involves the cause based decision tree (CBDT) method combined with the Accident Sequence Evaluation Program (ASEP) time response curves for cognitive error and the Technique for Human Error Prediction (THERP) method for execution errors, the probability of failure to isolate for leaks would be about 2.3E-02 (Attachment 1). This would further reduce the frequency of failure to isolate for leaks but would have no impact on failure to isolate for ruptures. This results in the actual incremental CDF of 8.4E-08 from the leaks. Thus, the combined value for leaks and ruptures would be about 1.3E-07 (4.66-08+8.4E-08) which is well below the 1.OE-06 threshold in Appendix K.

SUMMARY

Operators failed to perform a risk evaluation prior to engaging in maintenance activities that rendered portions of the SX system incapable of being used for isolating potential leaks or ruptures, In this condition, neither the 1 SX033 nor 1 SX034 valves could be operated remotely for purposes of train isolation in accordance with OBOA PRf-8. The plant was in the condition where both valves were open and incapable of remote operation for 42.3 hours3.472222e-5 days <br />8.333333e-4 hours <br />4.960317e-6 weeks <br />1.1415e-6 months <br />. Assuming that SX piping ruptures (>2000 gpm) could not be isolated by other means and assuming that operators could potentially isolate leaks (bemeen 100 gpm and 2000 gpm) before core damage would be assured, the actual core damage frequency associated with this configuration is about 1.3E-07 which is well below the 1.OE-06 threshold of Appendix 1( to IMC 0609 Figure 1 for evaluating the risk

BB PRA-017.918 Page 8 significance of such events. Therefore, the risk significance of this condition should be assessed as Green.

REFERENCES

1. Byron Abnormal Operating Procedure OBOA PRI-8, Revision 0.

2. USNRC Inspection Manual Chapter 0609 Appendix K.

3. T&RM ER-AA-600-1012, Rev. 7, Risk Management Documentation.

4. BB PRA-012 Rev, 4, Internal Flooding Analysis Notebook. March 2008.

5. Byron Operating Procedure BOP SX-22, Revision 1.

6. T&RM ER-AA-600-1041, Rev. 6, Risk Metrics - SDP & Event Analysis.

7. T&RM OU-BY-104, Rev. 10, Shutdown Safety Management Program Byron/Braidwood Annex.

AQachment 1 :

HEP for Isolation Failure for Leaks When 1 SX033 and 1 SX034 Fail Page 9 I Description I

HEP OSX-SX22fSO-HVOA I Operators Fail to Isolate Leak per SX-22 when 1SX033 and 1SX034 fail to close 1 2.33E.02 Boundarv Conditions Following a leak (100-2000 gpm) in the SX system in the Auxiliary Building durlng one unit in outage, failure of SX033 or SX034 to close prevents train Isolation due to cross tie on the supply side to both SX trains. OBOA PRI-8 Step 8 calls for identification of which traln the leak Is coming from and isolation of that train by closing the SX033 and SX034 valves to separate A and B trafn supplies. BOP SX-22 provides specific valve lists to isolate particular break locations. In this case, only the supply side for the affected traln needs to be isolated because the train isolation of the discharge path via lf2SX011 is presumed successful. There are five valves in each train downstream of the SX033 and SX034 valves whlch can accompllsh this function. They are SX004 and SXOl6(AIB) which are MOVs operable from the control room and SXOl3(AIB), SX052(AIB), and SX2103A(SX173) which are manual valves operable locally. More than 10 hours1.157407e-4 days <br />0.00278 hours <br />1.653439e-5 weeks <br />3.805e-6 months <br /> are avallabla to take this action before charging pumps are inundated.

I I

Evaluation of pc Causal Factor

-p-a-2 4-b-8

-P-C-l

-p-d-4 s-e-8 g

f 3

OBOA PRI-8 Step la.

Determine source and elect wrong control (Table 20-12-1) X 2 HEP-Nom 1.00E-04 1.00E-08 5.00E-04 5.00E-02 1.30E-02 3.00E-02 Compensating Factors / Comment Recovery factor; 'Extra crew' Recovery factor: 'Self review' and 'STA review' Recovery factor: 'STA review' Recovery factor: 'Extra crew' and 'STA review' Recovery factor: "elf review', 'Extra crew', and STA review' Recovery factor: 'Self Review', 'Extra crew' and 'STA review' Non-Recovery 5.00E-01 5.00E-02 1.00E-01 5.00E-02 HEP-Ffnal 5.00E-05.

5.00E-10 5.00E-05 2.50E-03 2.50E-02 j 3.25E-04 2.50E-02

/

7.50E-04

BB PRA-017.91 B Page 10 CBDT Justifications PcA I Data Not Available. Branch #2 (variable name is '-p-a-2")

is chosen because the procedures for isolating Auxiliary Building Flooding per BOA PRI-8 and BOP SX-22 are relatively new and high due to the fact that one unit is in a refueling outage. Branch #8 is selected since Auxiliary Building flooding is alarmed via sump PCB limited Gaining opportunities have been undertaken, -

Data Available But Not Attended To. The workload is assumed assessments. The Byron and Braidwood control room layouts have been subjected to formal human factors review & validation. The plant policy is to emphasize 3-way communications; this is stressed PcC level alarms.

Communications issues. Branch #I is a default for all HEP 1

I chosen because of the procedures are relatively new and training on 1 PcD in all training.

Available Information Misleading & Misinterpreted. Branch #4 is selected (Multiple procedures, E-0 and OA PRI-8, and BOP SX-22).

With one exception, there is no requirement for using any place keeping aids. The exception is the use of Status Trees (ST).

Except for 'boxed' procedure steps (immediate, memorized steps),

and steps identified by 'diamond symbol' (for continuous actions),

the procedure design does not include any feature that would prevent the operators from overlooking a procedure step - not "graphically distinct."

Misinterpretation of the Instruction. Branch #3 is selected because PcE determination of the actual break location is a function of the ability of the staff in the Auxiliary Building to locate the leak and communicate that to the control room. Also, training on the them is limited.

Skipping the Relevant Step in the Procedure. Branch #8 is with presence of logic statement(s) ih procedure. Branch #I0 is selected since the cited procedure steps include any written logic statements. Furthermore, it is assumed that the oDerators have PcG I received limited training on this action.

j relatively new OBOA PRI-8 and BOP SX-22 is limited.

Error in Interpreting the Decision Logic. This CBDT is concerned

BB PRA-017.91 B Page 11 Attachement 2: SPAR-H Estimate of HEP OSX-SX221SO-HVOA, Operators Fail to lsolate Leak per SX-22 when 1SX033 and 1SX034 fail to close Basic Event Summary 1:

, Method Plant: -

Byron Table 1 : OSX-SX221SO-HVOA

SUMMARY

Initiating Event:

SX flood leak (1 00-2000 gpm)

Cognitive Basic Event Context:

Following a leak (100-2000 gpm) in the SX system in the Auxiliary Building during one unit in outage, failure of 1SX033 and 1SX034 to close prevents train sola at ion due to cross tie on the supply side to both SX trains, OBOA PRI-8 Step 7 calls for identification of which train the leak is coming from and isolation of that train by closing the 1 SX033 or 1 SX034 valves to separate A and B train supplies. BOP SX-22 provides specific valve lists to isolate particular break locations, but no guidance related to addressing failures of particular isolation valves. However, the operators are trained to address SX leak isolation through review of the P&IDs to identify and secure isolation points. In this case, only the supply side for the affected train needs to be isolated because the train isolation of the discharge path via 1/2SX011 is presumed successful. There are five valves in each train downstream of the 1SX033 and 1 SX034 valves which can accomplish this function. They are SX004 and SXOlG(A/B) which are MOVs operable from the control room and SX013(A/B), SX052(A/B), and SX2103A (SX173) which are manual valves operable locally. More than 10 hours1.157407e-4 days <br />0.00278 hours <br />1.653439e-5 weeks <br />3.805e-6 months <br /> are available to take this action before charging pumps are inundated.

Execution 4.8e-02 1.2e-02 6.0e-02

66 PRA-017.91 B Timina:

Page 12 Timina Analvsis: Assuming isolation of a manual valve requires 20 minutes and that 1 minute is needed for a valve in the MCR, the 5 additional valve isolations that would be required to make up for 1 SX033 or 1 SX034 failure would contribute 62 minutes to the manipulation time if the valves were addressed sequentially, Most other isolation cases include only a few manual valve manipulations and 1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br /> would be a reasonable manipulation time for those valves. There are some cases, however, that require as many as 10 local valve closures. In these cases, 3.3 hours3.472222e-5 days <br />8.333333e-4 hours <br />4.960317e-6 weeks <br />1.1415e-6 months <br /> would be required to isolate them sequentially. For this case, minimal parallel work is assumed:

the 4.3 hours3.472222e-5 days <br />8.333333e-4 hours <br />4.960317e-6 weeks <br />1.1415e-6 months <br /> of ex MCR work is assumed to be split among two crews so that it could be completed in about 2.2 hours2.314815e-5 days <br />5.555556e-4 hours <br />3.306878e-6 weeks <br />7.61e-7 months <br />. The MCR isolations are considered to be completed in parallel with the ex-MCR work and no additional time is added to address those actions.

t=O The system window of 10 Hours is based on the time needed to reach critical flood volume 2 in the Flooding Analysis in BB PRA-012 Revision 3 for the worst leak rate (2000 gpm). For the smallest leak size (100 gpm) it would take about 215 hours0.00249 days <br />0.0597 hours <br />3.554894e-4 weeks <br />8.18075e-5 months <br /> to reach that level.

The delay time of 30 minutes, which is the length of time to the cue, is based on the time to reach sump alarm levels with minimum leak flow, but would more likely be much earlier due to visual identification from a crew member due to the fact that one unit was in a refueling outage and numerous staff were in the Auxiliary Building.

sw 10.00 Hours T

0.50 Hours TTJ2 0.00 Minutes T M 2.20 Hours Time available for recovery: 438.00 Minutes delay SPAR-H Available time (cognitive): 438.00 Minutes SPAR-H Available time (execution) ratio: 4.32 I

lrrevers~ble Cue Damages tate I

I Minimum level of dependence for recovery: ZD PART I, DIAGNOSIS

Page 13 Stress r Complexity r Procedures i Fitness for Duty I Work Processes k Low X

10 Nominal 1

High 0.5 lnsuff icient Information 1

Procedures OBOA PRI-8 and BOP SX-22 are relatively new and limited training has occurred due to their recent implekenfation. In addition, the operators would be required to use P&IDs and information from plant staff to identify the location of the leak and means of isolating the condition when lSXU33 and 123x034 failed to close. Therefore, a decrement to Low was chosen.

Not available Nominal 0.5 on their training to help the isolate the leak using alfernate valves.

No procedure can be written to address all failure cases and fhey are not expected to do so, but a degraded condition is used to account for the difficulties associated with dynamically identifying Unfit I P(failure) = 1.O

/

Page 14 Diaunosis HEP:

4.8e-02 [Adjustment applied: 1.OE-2

• 5.0e+00 1 (1.OE-2 * (5.0e+00 - 1) + I)]

PART ll. ACTION PSFs P

Available Time (recommended choice based on timing information in bold)

Extreme High Nominal Insufficient Information Complexity I

X Experienceflraining 5

2 1

1 Highly complex Moderately complex Nominal Insufficient Information 50 20 5

Procedures Nominal j X 11 X

3 1

0.5 1

Low Nominal High Insufficient Information Not available Incomplete Available, but poor Insufficient Information 5

2 1

1 X

1 1

Page 15 Action Probability:

1.2e-02 [Adjustment applied: 1.OE-3

• 1.2e+01 1 (1.OE-3 * (1.2e+OI - 1) + I)]

PART Ill. DEPENDENCY iddr nnai iou dbffersd

%4*,,,fi,,d CUPS 10 dddlilinat Mnderle sams Task Failure WITHOUT Formal Dependence:

6.0e-02 i:onolsite Complete high High no acdaiunal High Noderate Maderate

-ow L3CatlM i

re*

i I

c i j ~ e 1" 11718 Task Failure WITH Formal Deoendence:

6.0e-02 Tina tjd'tionsl Moderate

-0 addnwnal Mcderae different differed trlcdarate no addnional L a adddicnal L a

BB PRA-017.916 Page 16 RM DOCUMENTATION NO. 66 PRA-017.916 REV: 0 PAGE NO. 18 of I STATION: Byron UNIT(S) AFFECTED: UNITS I and 2 TITLE:

Byron SDP Evaluation of Failure to Conduct a Risk Evaluation Prior to Disabling 1SX033 and 1SX034 Remote Isolation Capability SUM MARY (Include URES incorporated):

This document evaluates the risk significance of operator failure to conduct a risk evaluation in accordance with Maintenance Rule section a(4) prior to beginning maintenance on the 1SX033 and 'lSX034 valves. Both valves were opened and power was removed by opening their supply breakers. This disabled the ability t o close the valves in response to SX floods in the Auxiliary Building.

Number of pages:

Total 46 pages, includin~ this page.

RM Document Level: Category 2, per ER-AA-600-1012.

[ 1 Review required after periodic Update

[ X 1 lnternaf RN mumenbtlon f ] External RN Documentatton Electronic Cafcu'lation Data Files: (Pmgmm Name, Versh, Fife Name extensfonfskefdateR\\ourImin)

Method of Review:

[ X ] Detailed [ ] Alternate f 1 Review of External Document This RM documentation supersedes:

in its entirety.

Prepared by:

-- Sfeven E. Mays -----

I 1 -- 6119108 Print Date Rev f ewe4 by:

f f

$11 9/08 Print Date Approved by:

Barry Sfoane I

I 6/19/08 Pfint Date Unplanned Unit Two On-line Risk (OLR) Orange Condition Root Cause Investigation Report

Root Cause Investigation Report Content and Format Unplanned Unit Two On-line Risk (OLR)

Orange Condition Root Cause lnvestigation Report 9/8/2008 Page 2 of 73

Root Cause Investigation Report Content and Format Table of Contents Executive Summary 4 Event Description 5

Analysis 8

Evaluation 10 Extent of Condition 11 Risk Assessment 12 Previous Events 14 Corrective Actions to Prevent Recurrence (CAPRs) 18 Corrective Actions 19 Effectiveness Reviews (EFRs) 22 Programmatic/Organizational Issues 22 Other Issues 25 Communications Plan 27 Root Cause Investigation Report Quality Checklist 28 - Root Cause Investigation Charter 30 - Event and Causal Factor Chart 31 - ~apRoot@

Trees 37 Attachment it - Risk Evaluation BB PRA-01'7.918 45 -- Cause and Effect Analysis -- Generic 6 1 - Barrier Analysis 63 Page 3 of 73

Root Cause lnvestigation Report Content and Format

Title:

Unplanned Unit 2 On-line Risk (OLR) Orange Condition Unit(s):

Unit 2 Event Date:

04/06/08 Event Time:

16:52 Action Tracking Item Number:

759945-04 Report Date:

05/02/08 Sponsoring Manager:

Bill Grundmann, Regulatory Assurance Manager Investigators:

1.l.

Name 1.2.

Department/Position Robert Lloyd Operations/Analyst*

Dave Coltman Work Management/Cycle Manager Joe Edom

• Root Cause Qualified Engineering ProgramslRisk Engineer EXECUTIVE

SUMMARY

Purpose of the lnvestigation The purpose of this investigation is to identify the causal factors and associated causes that led to an unplanned Unit Two OLR orange condition.

Scope of the Review The scope of the review includes:

Training and qualification associated with how well the Maintenance Rule (1 0CFR50.65 (a)(4) process is understood (i.e. as a Configuration Risk Management process) and of the level of understanding of risk and insights provided by the PRA, and review operator response actions to flooding, Technical review of risk evaluation sheets and other risk documents, Organizational and programmatic potential latent weaknesses associated with risk management, IRs related to operator response credited to mitigate flooding in the auxiliary building; 757507,757930, 759455, 759929, and 759930.

Root Cause Less than adequate enforcement of Operations OLR management roles, responsibilities, and expectations.

Corrective Actions to Prevent Recurrence (CAPRs)

Interim CAPR Develop and implement Byron specific revisions to LS-AA-120, Issue Identification and Screening Process, and WC-AA-101, On-line Work Control Process, to include expectations that contain clear guidance for operations shift management roles and responsibilities regarding risk 9/8/2008 Page 4 of 73

Root Cause Investigation Report Content and Format management.

CAPR Implement Byron specific revisions into fleet procedures LS-AA-120, Issue Identification and Screening Process, and WC-AA-101, On-line Work Control Process, to include expectations that contain clear guidance for operations shift management roles and responsibilities regarding risk management.

Extent of Condition Two other departments are involved in the risk management process; work management and engineering. Enforcement of process roles, responsibilities, and expectations may be transportable to other areas that operation's is responsible. An action has been created to identify other processes that operations personnel participate that may succumb to this type of cause.

Risk Assessment Unit one: Based on qualitative insights, the SDP assessment for the outage Unit would be Green.

Unit Two: The site took credit for local actions and used the auspices of Appendix K resulting in a SDP assessment of green, however, the NRC did not credit any local action and has assessed the SDP as white.

Reportability This event was not reportable.

Previous Events A search of the CAP database was performed using keywords "risk" and "online" in completed Exelon CAP investigations (Root Cause, Apparent Cause) from 01/01/2006 to present. This resulted in the identification of 13 investigations meeting these criteria. Of these 13 investigations one was relevant to this event and is documented in this report.

A search of the INPO plant events database using keywords "probabilistic risk assessmenf'and probabilistic safety assessment" was performed from 01/01/2006 to present. This resulted in the identification of 17 plant events that met these criteria. Of the 17 events, three were relevant to this event and are documented in this report.

3.

EVENT DESCRIPTION OtR Management Background Maintenance Rule 10CFR50.65 (a)(4) requires that before performing maintenance activities, the licensee shall assess and manage the increase in risk that may result from the proposed activity. Thus, conduct of risk evaluations are performed for all maintenance tasks (corrective maintenance, preventive maintenance, minor maintenance, suweillance tests and modifications) on all structure, system, or components (SSCs) within the scope of the Maintenance Rule (10CFR50.65 (a)(4)).

Risk management is the development, maintenance, and application of methods that provide risk insights to be used in the design, maintenance, and operation of Exelon nuclear power facilities. These methods may be either qualitative or quantitative. Overall risk consists of Wo elements; qualitative tools (plant transient assessment tree, and safety function assessment trees), and quantitative tools 9/8/2008 Page 5 of 73

Root Cause Investigation Report Content and Format probabilistic risk (core damage frequency and large early release frequency). These elements are further defined below.

Probabilistic risk analysis (PRA) is either the method that provides risk insights or the result of applying that method. (Also referred to as probabilistic risk assessment, probabilistic safety analysis, or probabilistic safety assessment. These terms are all equivalent). There are Two components associated with PRA; core damage frequency (CDF) and large early release frequency (LERF).

Plant Transient Assessment Tree (PTAT) represents the likelihood of a transient and the available equipment necessary to mitigate the effects of the transient.

Safety Function Assessment Tree (SFAT) represents the level of defense-in-depth of specific critical plant safety functions.

Configuration risk management criteria uses a color designation; green, yellow, orange, and red with green being low risk and red being high risk. Specific actions are required if a risk color of yellow, orange, or red is encountered. This event incurred an unplanned risk color change from green to orange.

Description of event A detailed time line is included in attachment 2, Event and Causal Factor Chart. Replacement of 1 SX034, 1 B Essential Service water pump discharge header crosstie motor operated isolation valve, was scheduled in B1 R15 as part of the SX system Butterfly Valve replacement project, under WO 8361 14. Two clearance orders (CO) 57893 and 57894 were implemented for isolation of the 1 SX034.

CO 57893 isolated the motor operator electrically for termination and determination and CO 57894 isolated the valve mechanically.

On 03/28/08, revision one of risk configuration sheet for Unit Two was implemented for the week of 03/31/08 that included the following relevant components and descriptions as components considered for overall risk:

1 SXOI PB - 1 SX034 replacement window 1 SX034 - valve replacement 1 SX005 and 1 SX033 - 1 SX034 valve replacement 2DSFS002 and 2DSFS003 removed - 1 SX034 replacement In addition to protecting opposite train equipment several other components were listed, if lost, would result in an orange or red risk condition while the 1 SX034 mechanical CO was placed. Moreover, if the 2SX01 PA or the 2SXO1 PB became unavailable while the same CO was placed or with both flood seal openings removed would result in OLR of orange or red.

in the process of isolating SX flow to the valve, the 1 SX033, 1 A Essential Service water pump discharge header crosstie motor operated isolation valve, was closed using its motor operator.

Although closed, the 1 SX033 valve did not provide a complete watertight seal sufficient to allow removal and replacement of the 1 SX034 valve without water leakage into the work area, For large (30 inch) bunerfly valves of this type, small leakage is not an unusual occurrence.

A decision was made to use the manual operator for 1 SX033 to tighten the seal between the bu"rtrf1y and the seat by manually closing the valve operator further than the motor operator could. While manually tightening the closure of the valve, operators heard an abnormal noise followed by a decrease in torque needed to turn the valve operator. At that point, the effort to replace 1 SX034 was stopped and troubleshooting to determine the status of the 1 SX033 operator was begun. Due to issues achieving isolation, 1 SX034 valve replacement was removed from B1 R15, On 04/01/08, revision two of the risk configuration sheet for Unit Two was implemented for the week of 9/8/2008 Page 6 of 73

Root Cause Investigation Report Content and Format 03/31/08 that included the following relevant components and descriptions:

0 1 SX033 unable to Open - emergent repairs 1 SX034 unable to Open - 1 SX033 repairs 0

-2DSFS002 and 2DSFS003 removed - 1 SX034 replacement In addition to protecting opposite train equipment two other components were listed, if lost, would result in an orange or red risk condition when the 1 SX033 or 1 SX034 were unavailable. Moreover, if the 2SX01 PA or the 2SXO1 PB became unavailable with both flood seal openings removed would result in OLR of orange or red. The ability of the 1 SX033 and 1 SX034 to close was not listed as a risk significant reason.

On 04102108, 1 SX034 and 1 SX033 were left in the closed position and electrically de-energized for emergent work. Mechanical isolation using 1 SX034 was added to clearance order (CO) 57893 to allow for investigation of 1 SX033 isolationlactuator issues.

On 04/04/08, the Unit one outage schedule contained a task to perform a CO manipulation to electrically open 1 SX033 and manually open 1 SX034 to support Diesel Generator testing. However, at this point both valves were in the closed position with power isolated from the operator (incapable of operating from the main control room [MCR]).

At 021 9 on 04105108, both the 1 SX033 and 1 SX034 valves were fully opened locally using their manual operators and equipment status tags (EST) were placed on respective main control board control switches. This configuration was instituted to support Unit One diesel generator (DG) testing. Power had been removed from both valves as part of the effort to troubleshoot the condition of the 1 SX033 operator. The open position is the normal operating position of both the 1 SX033 and 1 SX034 valves.

This alignment cross ties the SX pump supply to the A and B headers within Unit One. This allows one SX pump to provide flow for both trains during normal operations. To satisfy risk requirements associated with flooding, at least one of these valves need to be closed, capable of being closed from the MCR, or compensatory measures planned such as a specific prejob brief and an operator dedicated to close one of the two valves within 30 minutes. This configuration change was not evaluated for Unit Two OLR.

On 416108 Cycle Manager 1, while developing the risk evaluation for the week of 04107108, was informed from Cycle Manager 2 of the previous week, that the SX crosstie valves 1 SX033 and 1 SX034 work was carrying over to the week of 04/07/08. Cycle Manager 1 understood that these valves fed into the OLR profile for Unit Two. Cycle Manager 1 identified bounding cases for these valves. These cases included 1 SX033134 unable to open if closed and unable to close if open. These cases were discussed with the Site Risk Engineer who ran both bounding cases for the crosstie valves and determined that with both valves open and unable to be positioned closed from the MCR OLR profile would be orange. The Shift Manager was contacted to confirm the configuration of the valves and found that both 1 SX034 and 1 SX033 were electrically de-energized and in the open position thus unable to be positioned closed from the MCR without any compensating actions implemented.

At 1652 on 4106108, Byron Operations declared Unit Two OLR to be ORANGE due to inability to close 1 SX033 or 1 SX034 to prevent flooding. Both valves were OOSIDE-ENERGIZED OPEN. This was an unplanned risk condition, and a prompt investigation was initiated. At 1712 on 4/06/08, an operator was stationed locally at 1 SX033 and Unit Two OLR was declared to be GREEN. This equipment alignment resulted from efforts earlier in the Unit 1 outage to replace 1 SX034. Isolation of 1 SX034 was not obtained and replacement was aborted, and 1 SX033 MOV operator was determined to be defective, Later in the outage, 1A DG testing required the flowpath through 1 SX033 and 1 SX034 to be open, so both valves were taken open with power removed.

At no time during the event was Unit 1 Shutdown Risk compromised leading to a color change.

Page 7 of 73

Root Cause Investigation Report Content and Format

4.

ANALYSIS Event and Causal Factor Chart An events and causal factors chart (E&CF) is a graphically displayed flow chart of the entire event and is included as attachment two of this report. The heart of the E&CF chart is the sequence of events plotted on a time line. Beginning and ending points are selected to capture all essential information pertinent to the situation. Barrier analysis was used in conjunction with E&CF to identify four causal factors. These causal factors were then used for the various cause analysis techniques listed below.

Tap Roo Conditions and events, as listed on the E&CF, are analyzed to determine which are causal factors.

Each causal factor was analyzed by processing it completely through the Root Cause ~ree@.

Detailed results are listed in attachment three of this report. It was found that 38% of the causes were related to human engineering and 25% related to training. These categories were logically grouped to determine more specific causes. This analysis is documented in the next section of the report.

Cause and Effect Analysis Human engineering and training were aspects as suspected causes to this event were analyzed by cause and effect analysis. This analysis resulted in the identification of three causes:

1. Dual function components that if unavailable in conjunction with its redundant component, would result in an orange or red risk condition are not identified in rule-based guidance.

2. Licensed operator training learning objectives and lesson plan content does not address dual function high-risk components and their potential affect on OLR.

3. Cycle Managers and Outage Risk Managers do not receive training for risk assessment activities.

Generic Causes An aggregate review (extent of cause) of the causes identified using ~ a p R o o ~ @

and associated topical areas was performed to determine potential generic causes. This resulted in the identification of one area; Less than adequate understanding where knowledge based performance is required.

Cause and effect analysis was then used to arrive at a cause and is included in Attachment 5.

The results of this analysis have identified the following generic cause:

Ineffective risk management program administration oversight.

Barrier Analysis Barriers are devices employed to protect and enhance the safety and performance of the plant. They can be physical or administrative in form. Barriers are erected to ensure consistent and desired performance of the plant. A single barrier is rarely relied upon. Generally, barriers are diverse and numerous - a defense-in-depth concept.

A total of 33 barriers were evaluated using this methodology. A simplified analysis was performed to identify the applicable barriers to this event. This resulted in the identification and further analysis of ten barriers. The detailed results of this analysis are contained in Anachment 6, Barrier Analysis. The following causes have been identified through analysis:

1. No requirement exists to document the affect of the CO on OLR or SDR. Procedure guidance is generic in nature and does not discuss dual function risk components.

2. Inadequate task analysis resulting in incomplete training material content or lack of training for specific work groups.

9/8/2008 Page 8 of 73

Root Cause Investigation Report Content and Format

3. No formal mechanism in place to ensure that adequate transfer of knowledge related to OLR/ SDR when personnel changes are made.

4. OU-AP-104, Shutdown Safety Management Program ByronlBraidwood Annex, and OU-AA-101-1005, Exelon Nuclear Outage Scheduling are silent on opposite unit OLR considerations.

5. Less than adequate enforcement of Operations OLR management roles, responsibilities, and expectations.

6. Limited management oversight related to risk management process.

7. Unforeseen failure of isolation valve during manipulation forces work to be removed from the outage.

Page 9 of 73

Root Cause Investigation Report Content and Format

5.

EVALUATION Problem Statement Operators changed configuration of risk components without evaluating for risk significance.

Cause (describe the cause and identify whether it is a root cause or contributing cause)

Less than adequate enforcement of Operations OLR management roles, responsibilities, and expectations.

Management Systems SPAC not Used Enforcement Needs Improvement (Root Cause)

Less than adequate training material content, frequency, andlor lack of training for specific work groups have contributed to knowledge weaknesses with regard to risk management.

Training, Understanding Needs Improvement, learning objectives and lesson plan needs improvement (Contributing Cause)

Basis for Cause Determination Inadequate perception of roles, responsibilities, and expectations with regard to priority of OLR. Through interviews it was not clear who owns implementation of risk management at the site. Work management, contrary to how the procedure roles and responsibilities are presented, performs the majority of the risk analysis. The methodology is based on Operations owning this process since they are responsible for configuration of the plant components.

Upon review of learning objectives contained in materials for risk assessment in licensed operator continuing training, no content was found that addresses recognition of dual function high-risk components. Furthermore, training has been focused on the use of the risk management tool, Paragon. The same holds true for other support training material related to risk assessment for the licensed operator training programs. Additionally, training materials do not contain the basis for significant OLR components on why risk is affected.

Page t 0 of 73

Root Cause Investigation Report Content and Format Problem Statement Limited procedure content and usability.

1 SX034 Valve work removed from the outage.

Cause (describe the cause and identify whether it is a root cause or contributing cause)

No requirement exists to document the affect of the CO on OLR or SDR.

Work Management procedure guidance is generic in nature and does consider opposite risk in schedule development or planning and is generic in nature.

Procedures, Wrong, Situation Not Covered (Contributing Cause)

Unforeseen failure of isolation valve 1 SX033 during manipulation.

Equipment Difficulty (Contributing Cause)

Basis for Cause Determination Upon review of the COs associated with this event no documentation in the special instructions or other notes were found that informed the risk significance of the affected SX valves. However, there were annotations related to technical specifications and other applicable execution notes. OP-AA-109-101 was reviewed and found limit generic guidance related to risk. Risk is listed as an item on some of the preparationlapprover checklists but is generic in nature and often is associated with production or reactivity risk.

OU-AP-104, Shutdown Safety Management Program Byron/Braidwood Annex, and OU-AA-101-1005, Exelon Nuclear Outage Scheduling are silent on opposite unit OLR considerations.

Isolation of 1 SX034 was aborted by the failure of 1 SX033 to provide adequate isolation subsequently resulting in removal of work from the outage.

6.

EXTENT OF CONDITION enforcement of Operations OLR management roles, responsibilities, and expectations.

Less than adequate training material content, frequency, andlor lack of training for specific work groups have contributed to knowledge weaknesses with regard to risk Extent of Condition Review Extent of Condition Two other departments are involved in the risk management process; work management and engineering.

Actions created from this investigation will address these areas as well.

Extent of Cause Enforcement of process roles, responsibilities, and expectations is transportable to other areas that operation's is responsible. An action has been created to identify other processes that operations personnel participate that may succumb to this type of cause.

Extent of Condition From interviews with Operations and training personnel, it was determined that other complex tasks are included in licensed operator training such as problem identification, diagnosis, plant response evaluation, and operator actions to respond to plant equipment symptoms. These complex tasks are trained on the simulator and are based on infrequent performance and difficulty. Although risk assessment is a difficult task it is not trained on based on the frequency it is performed (daily in most cases). No other complex 9/8/2008 Page 11 of 73

Root Cause investigation Report Content and Format valve during manipulation forces work to be removed from the Cause being addressed management.

No requirement exists to document the affect of the CO on OLR or SDR Extent of Condition Review tasks were identified that are similar to risk assessment. No further review is necessary.

Work Management procedure guidance is generic in nature and does consider opposite unit risk in schedule development or planning and are generic in nature.

Extent of Cause This cause could apply to other major processes that operation's personnel are involved. An action has been created to identify other potential process knowledge weaknesses.

Extent of Condition The ability to Isolate components to perform maintenance has been challenged in the past especially in raw water systems. System improvement plans have been developed to help address these issues as the purpose of the 1 SX034 valve work until isolation problems occurred.

Extent of Cause Since the possibility of this failure to be recognized is not very likely before it was experienced improved contingency actions should be consider addressing major isolation valves failure to perform their function project plans/fragnets/schedules.

Extent of Condition There are other dual function components that affect OLR in a similar manner as 1 SX033 and 1 SX034.

Actions have been created to identify and include them in rule-based guidance. This condition also applies to Braidwood Units one and Two. As part of the OPEX actions associated with the root cause investigation process other sites will be informed of issues identified in this investigation. No further review is required.

Extent of Cause Other outage, work management, and operations procedures and TRMs were reviewed to determine if the causes identified related to procedure guidance was transportable in other areas. These causes could affect not online scheduling but outage scheduling. The actions implemented from this investigation will address this concern.

9/8/2008 Page 12 of 73

7.

RISK ASSESSMENT Plant-spcific risk consequence Basis for Determination

Root Cause Investigation Report Content and Format auspices of Appendix K resulting in a SDP assessment of green, Plant-specific risk consequence however, the NRC did not credit any local action and has assessed the SDP as white.

Basis for Determination with this configuration is about 1.3E-07 which is well below the 1.OE-06 threshold of Appendix K to lMC 0609 Figure 1 for evaluating the risk significance of such events. Therefore, the SDP assessment would be characterized as Green. However, the NRC did not provide credit for any local actions thus strictly adhering to appendix K and not conduct a phase three review. The NRC has rated this SDP as 'White".

Basis for the determinations are documented in attachment four, BB PRA-017.91 B.

Page 13 of 73

Root Cause Investigation Report Content and Format

10.

PREVIOUS EVENTS A search of the CAP database was performed using keywords "risk and "online" in completed Exelon CAP investigations (Root Cause, Apparent Cause) from 01/01/2006 to present. This resulted in the identification of 13 investigations meeting these criteria. of these'l3 investigations one was relevant to this event and is documented below.

A search of the INPO plant events database using keywords "probabilistic risk assessment" and probabilistic safety assessment" was performed from 01/01/2006 to present. This resulted in the identification of 17 plant events that met these criteria. Of the 17 events, three were relevant to this event and are documented below.

and PRA was incorrectly implement change.

It was found that responsibilities identified on "FIN Daily

. Perform training after completion were consistent between the performance of a FIN WO. The of the procedure reconciliation per various guidance documents cause was found to be that however, there is little controlled procedure. (SPAC confusing or evaluated in this investigation.

Moreover, training pertaining to this process is evaluated in this Page 14 of 73

Root Cause Investigation Report Content and Format Previous Events OE25667 -- Shutdown LC0 and Red PSA Risk Condition for Emergency Diesel Unavailability (Cooper Unit 1)

P SuNei'ianCe test procedure used

. Other procedures will be reviewed for troubleshooting contained required actions to declare DGI for similar issues and revised as inoperable following failure to necessary.

Event Description and Cause(s)

Both emergency diesel generators (DG) became simultaneously inoperable and unavailable, a condition that posed a RED risk in PSA. Causes included the meet acceptance criteria, when P

. The equipment performance issue other options were available.

with the float and solenoid valves Specifically, other system is being investigated separately.

Event Corrective Actions

1. Revised applicable EDG surveillance test procedures to direct splitting the day tank fuel oil supply subsystems when acceotance criteria are not met.

operating procedures allowed

. Revision to the risk evaluation splitting out the Two day tank fill process to provide better sub-systems via closing the cross-guidance for emergent issues.

connect valve and reconfiguring the fuel oil pump switch, thus preventing one DG fill evolution from potentially impacting the other DG.

Conclusion SX system operating procedure reviews were performed for guidance on the risk significance of 1 SX033 and 1 SX034; none was found. The actions implemented as a result of this event will be reviewed for use in this investigation.

Page 15 of 73

Root Cause Investigation Report Content and Format Previous Events without Required Risk (Three Mile Island Unit 1)

( Event Description and Cause(s)

Opportunities were missed to identify that work planned for a fire service pump did not include the required on-line risk assessment performed. Causes included the lack of knowledge that minor maintenance activities on any of the SSCs that fall within Maintenance Rule (a)(4) scope needs to be assessed with respect to managing on-line risk.

( Event Corrective Actions of this issue were addressed with members of the Operations Department.

. Knowledge that minor maintenance activities on any of the SSCs that fall within Maintenance Rule (a)(4) scope require assessment with respect to managing on-line risk was reinforced:

The site equivalent Maintenance Rule (a)(4) components list contained in the Risk Management Program exhibit was reviewed by all applicable personnel to impart the knowledge that minor maintenance on any of the SSCs, contained therein, needs to be assessed with respect to managing on-line risk.

The site equivalent Maintenance Rule (a)(4) components list contained in the Risk Management Program exhibit was reviewed by all licensed operators (SROs and ROs) and members of the Operations Department to impart the knowledge that minor maintenance on any of the SSCs, contained therein, needs to be assessed with respect to managing on-line risk.

1 Conclusion I

/ The actions implemented as a result of this event will be reviewed for use in this I investigation.

Page 16 of 73

Root Cause Investigation Report Content and Format 1 Previous Events I Event Description and Cause@) I Event Corrective Actions OE12240 -- Failure to Assess the Risk Associated With Testing Unit 1 HPCI While the 6 Emergency Diesel Generator Was Also Being Tested (Susquehanna 1)

While at I00 percent power, site personnel did not assess the risk of performing a Unit I high-pressure coolant injection (HPCI) system test concurrent with a test of the 'BE emergency diesel generator (EDG). The cause of this event was the use of an "Operations" schedule for PRA system related activities that is separate from the station schedule used to evaluate station risk.

There was no programmatic provision for the on-shift SROlSTA to perform a risk assessment. The controlling administrative procedure does not allow any other risk assessment besides what is performed using Sentinel and contacting the work week manager.

I Conclusion 1

J The action implemented as a result of this event will be A review of roles and responsibilities for operations supervision to reinforce the expectations and requirements of the maintenance rule risk assessment and management program, and communicating expectations for scheduling mismatches.

I reviewed for use in this investigation.

Conclusions The events above used primarily knowledge-based corrective actions to solve knowledge-based causes related to a complex system. Other corrective action methodologies including focused rule-based actions will be evaluated for use to address the causes identified in this investigation.

Page 17 of 73

Root Cause Investigation Report Content and Format Less than adequate enforcement of Operations OLR management roles, responsibilities, and expectations.

1. Interim CAPR to develop and implement Byron specific revisions to LS-AA-120, lssue ldentification and Screening Process, and WC-AA-101, On-line Work Control Process, to include expectations that contain clear guidance for operations shift management roles and responsibilities regarding risk management. These expectations shall include the following:

Specific guidance on the use of Paragon risk program and expectations on it's use, Specific guidance on review of risk summary sheet that the WWM developslmaintains clearly stating that if prior to any risk component status changes and if any risk significant component becomes unavailable to evaluate risk, Specific guidance related to communication of plant status changes for risk components to WWM.

Specific guidance related to the threshold of obtaining additional support,

a.

These changes shall be incorporated in LS-AA-120 by modifying section 4.4 Operations Shift Management Review, and WC-AA-101 by adding operations shift management roles and responsibilities to section 3 Responsibilities.

b.

Create TR, designate as a commitment, for licensed operator training (ILT and LORT) to train on these procedure changes.

Create and action to track completion of the TR.

1 I

Closure criteria: Procedures developed and implemented.

1

2. CAPR to implement Byron specific revisions into fleet procedures LS-AA-120, lssue Identification and Screening Process, and WC-AA-101, On-line Work Control Process, to include expectations that contain clear guidance for operations shift management roles and responsibilities regarding risk management. These expectations shall include the following:

A88 1 OOP A88 1 OOP Specific guidance on the use of Paragon risk program and expectations on it's use, 91812008 Page 18 of '73

Root Cause Investigation Report Content and Format Specific guidance on review of risk summary sheet that the WWM developslmaintains clearly stating that if prior to any risk component status changes and if any risk significant component becomes unavailable to evaluate risk, Specific guidance related to communication of plant status changes for risk components to WWM.

Specific guidance related to the threshold of obtaining additional support,

a.

These changes shall be incorporated in LS-AA-120 by modifying section 4.4 Operations Shift Management Review, and WC-AA-101 by adding operations shift management roles and responsibilities to section 3 Responsibilities.

Due Date Owner Root Cause Being Addressed Corrective Action to Prevent Recurrence (CAPR)

12.

CORRECTIVE ACTIONS Page 19 of 73 Less than adequate enforcement of Operations OLR management roles, responsibilities, and expectations.

Being Addressed Owner Corrective Action (CA) or Action Item (ACIT)

1. CA to perform and document one observation on a risk management activity per each shift manager and supervisor during 4th quarter 2008.

2. CA to perform and document one observation on a risk management activity per each SRO, Work Week Manager, and Cycle Manager during 1"' quarter 2009.

Due Date A881OOPSRM A88100PAIBICID

/E A881 OOPSRM A881 OOPA/B/C/D

/E 121'1 7/08 03/26/09

Root Cause Investigation Report Content and Format 1 Cause Being Addressed Corrective Action (CA) or Action Item (ACIT)

I Owner

/ Due Date I

material content, frequency, andlor determined through training system analysis related to risk lack of training for specific work management and flooding effects on risk.

groups have contributed to knowledge weaknesses with regard to risk management.

Unforeseen failure of isolation valve during manipulation forces work to be removed from the

outage, Sub assignments to CA:

a. ACIT to discuss this investigation to applicable CRCs (LORT/EO).

A8861 TROPP 12/01/08 in this assignment and take additional actions as necessary. /

I I

b. ACIT to consider a development and presentation of a case study on this event. Document results of the consideration

c. AClT to improve training content on risk management and the effects of flooding on OLR in equipment and licensed operator-training programs (LORTJILT, EOJEOI) based on OLR process procedure changes.

A8861TROPP 1 12/01/08 1

d. ACIT to develop training effectiveness evaluation worksheet (TEEW).

4. AClT to consider adding contingency actions if major isolation valves fail to perform their function in project plans/fragnets/schedules. Take additional actions based on the results of the review.

Page 20 of 73 L

No requirement exists to document the affect of the C 0 on OLR or SDR.

A881 OOP

5. ACIT to consider creating guidance that requires OLR considerations be referenced in applicable clearance orders. Take additional actions based on this review.

0911 6/08