ML111990346: Difference between revisions

From kanterella
Jump to navigation Jump to search
(Created page by program invented by StriderTol)
(StriderTol Bot change)
 
(One intermediate revision by the same user not shown)
Line 2: Line 2:
| number = ML111990346
| number = ML111990346
| issue date = 07/28/2011
| issue date = 07/28/2011
| title = 07/28/2011 Meeting Presentation, Prairie Island Nuclear Generating Plant - Failure to Maintain the Train a and B Direct Current Electrical Power Subsystems Operable
| title = Meeting Presentation, Prairie Island Nuclear Generating Plant - Failure to Maintain the Train a and B Direct Current Electrical Power Subsystems Operable
| author name =  
| author name =  
| author affiliation = Xcel Energy
| author affiliation = Xcel Energy
Line 18: Line 18:


=Text=
=Text=
{{#Wiki_filter:Prairie Island Nuclear Generating Plant NRC Region III Regulatory Conference Failure to Maintain the Train A and B Direct Current Electrical Power Subsystems Operable July 28, 2011
{{#Wiki_filter:NRC Region III Regulatory Conference Failure to Maintain the Train A and B Direct Current Electrical Power Subsystems Operable July 28, 2011 Prairie Island Nuclear Generating Plant Prairie Island Nuclear Generating Plant


Agenda Opening Remarks - Mark Schimmel Sequence of Events - Kevin Davison Causes and Corrective Actions - Kevin Davison Regulatory Significance - John Bickel Operator Actions - Darrell Lapcinski Conclusion - Mark Schimmel Closing Remarks - Dennis Koehl 2
2 Agenda
 Opening Remarks - Mark Schimmel
 Sequence of Events - Kevin Davison
 Causes and Corrective Actions - Kevin Davison
 Regulatory Significance - John Bickel
 Operator Actions - Darrell Lapcinski  
 Conclusion - Mark Schimmel
 Closing Remarks - Dennis Koehl


Opening Remarks Xcel Energy takes its obligation to protect the health and safety of the public very seriously Violation Failure to maintain the Train A and B direct current electrical power subsystems operable Agree with the performance deficiency and violation Disagree with the significance More than adequate time to diagnose charger lockup 3
3 Opening Remarks
 Xcel Energy takes its obligation to protect the health and safety of the public very seriously
 Violation
 Failure to maintain the Train A and B direct current electrical power subsystems operable
 Agree with the performance deficiency and violation
 Disagree with the significance  
 More than adequate time to diagnose charger lockup


Sequence of Events Kevin Davison - Plant Manager 4
4 Sequence of Events Kevin Davison - Plant Manager


Sequence of Events 1994 / 95           1997           1999              2005              Oct 2010      Feb 2012 Battery        Follow up           Test           12 Battery           TS 3.8.1.B     Unit 2 chargers        testing of      procedure        Charger mod          Entered for     battery installed          Unit 1        revised to        cancelled          D2 EDG;     chargers to battery      remove 12                            Exigent TS    be replaced chargers        Battery                               change Charger                                approved 1994                1997                2000            2003  2006 2009                2012 June 2010 Site staff 1996                                  2002                  enters 12 May 2011 12 Battery              1999          Improved                  Battery      Unit 1 Charger              Battery          Standard                Charger      battery lockup            charger mod      Tech Spec            testing issues chargers during ISI            requests      implemented              into CAP    replaced 5
5 Sequence of Events 1996 12 Battery Charger lockup during ISI 1994 / 95 Battery chargers installed 1994 1997 2000 2003 2006 2009 2012 1997 Follow up testing of Unit 1 battery chargers 1999 Test procedure revised to remove 12 Battery Charger 2005 12 Battery Charger mod cancelled Oct 2010 TS 3.8.1.B Entered for D2 EDG; Exigent TS change approved Feb 2012 Unit 2 battery chargers to be replaced 1999 Battery charger mod requests 2002 Improved Standard Tech Spec implemented June 2010 Site staff enters 12 Battery Charger testing issues into CAP May 2011 Unit 1 battery chargers replaced


Causes and Corrective Action A root cause analysis was performed to identify the direct cause and the organizational and programmatic breakdowns that led to failure to correct battery charger lockup.
6 Causes and Corrective Action
6
 A root cause analysis was performed to identify the direct cause and the organizational and programmatic breakdowns that led to failure to correct battery charger lockup.


Causes and Corrective Actions Root Cause - Direct The organization had developed a blind spot to this issue.
7 Causes and Corrective Actions
Over several years, multiple opportunities surfaced to identify and correct the condition 7
 Root Cause - Direct
 The organization had developed a blind spot to this issue.  
 Over several years, multiple opportunities surfaced to identify and correct the condition


Causes and Corrective Actions Root Cause - Programmatic RC-1: Equipment issues become long-standing when C severity level CAPs are allowed to be closed to a secondary work process deliverable, which may be canceled or extended beyond the original CAP due date.
8 Causes and Corrective Actions
Root Cause - Organizational RC-2: Behaviors necessary to drive long-standing equipment issues to timely resolution are neither modeled, nor required by Plant Health Committee (PHC).
 Root Cause - Programmatic
8
 RC-1: Equipment issues become long-standing when C severity level CAPs are allowed to be closed to a secondary work process deliverable, which may be canceled or extended beyond the original CAP due date.
 Root Cause - Organizational
 RC-2: Behaviors necessary to drive long-standing equipment issues to timely resolution are neither modeled, nor required by Plant Health Committee (PHC).


Causes and Corrective Actions Corrective Action - Equipment Implemented compensatory actions for all battery chargers Obtained an Exigent Technical Specification change Replaced and retested Unit 1 Battery Chargers (1R27)
9 Causes and Corrective Actions
Scheduled replacement of Unit 2 Battery Chargers (2R27)
 Corrective Action - Equipment  
Procurement of Spare and Unit 2 chargers in progress.
 Implemented compensatory actions for all battery chargers
9
 Obtained an Exigent Technical Specification change
 Replaced and retested Unit 1 Battery Chargers (1R27)
 Scheduled replacement of Unit 2 Battery Chargers (2R27)  
 Procurement of Spare and Unit 2 chargers in progress.


Causes and Corrective Actions Corrective Action - Process For C severity level CAPs, to ensure all open equipment issues remain captured in the corrective action process.
10 Causes and Corrective Actions
Establish a PHC Charter and changed behaviors that incorporates the knowledge gained from benchmarking industry leaders and make it available to System and Programs Engineering.
 Corrective Action - Process  
PHC needs to be defined as the champion of projects for presentation to PRG.
 For C severity level CAPs, to ensure all open equipment issues remain captured in the corrective action process.
10
 Establish a PHC Charter and changed behaviors that incorporates the knowledge gained from benchmarking industry leaders and make it available to System and Programs Engineering.
PHC needs to be defined as the champion of projects for presentation to PRG.  


Causes and Corrective Actions Corrective Action - Process Implement a periodic PHC roll-up to evaluate effectiveness of resolving equipment Legacy Issues Plan 11
11 Causes and Corrective Actions
 Corrective Action - Process
 Implement a periodic PHC roll-up to evaluate effectiveness of resolving equipment
 Legacy Issues Plan


Conclusion Station had multiple opportunities to identify Corrective actions have focused on:
12 Conclusion
Replacement of chargers Changing behaviors of PHC members Process changes to assure sustainability 12
 Station had multiple opportunities to identify
 Corrective actions have focused on:
 Replacement of chargers
 Changing behaviors of PHC members
 Process changes to assure sustainability


Regulatory Significance Scenarios & Risk Evaluation John Bickel - PRA Consultant Engineer Operator Actions Darrell Lapcinski - Shift Manager 13
13 Regulatory Significance Scenarios & Risk Evaluation John Bickel - PRA Consultant Engineer Operator Actions Darrell Lapcinski - Shift Manager


Regulatory Significance - Scenarios LOOP event with EDG D1 failed and 12 Battery Charger Lockup Scenario is most limiting time available prior to battery depletion Plant emergency lighting powered from the 12 battery Minimum design voltage on 12 battery will be reached in three hours Most risk significant sequence from the PINGP PRA evaluations Noted by the NRC to be significant risk contributor Failure of the EDG D1 would cause the operators to perform additional actions LOOP event with 11 and 12 Battery Charger Lockup EDG D1 successfully operates Scenario is not as limiting in time as scenario listed above Plant emergency lighting powered from the AC source (EDG D1 operates).
14 14 Regulatory Significance - Scenarios
 LOOP event with EDG D1 failed and 12 Battery Charger Lockup
 Scenario is most limiting time available prior to battery depletion

Plant emergency lighting powered from the 12 battery

Minimum design voltage on 12 battery will be reached in three hours
 Most risk significant sequence from the PINGP PRA evaluations  
 Noted by the NRC to be significant risk contributor
 Failure of the EDG D1 would cause the operators to perform additional actions
 LOOP event with 11 and 12 Battery Charger Lockup  
 EDG D1 successfully operates
 Scenario is not as limiting in time as scenario listed above

Plant emergency lighting powered from the AC source (EDG D1 operates).

Minimum design voltage on 11 battery reached in six and a half hours.
Minimum design voltage on 11 battery reached in six and a half hours.

Minimum design voltage on 12 battery reached in seven and a half hours.
Minimum design voltage on 12 battery reached in seven and a half hours.
14


Regulatory Significance - Scenarios LOOP, EDG D1 Failure & 12 Battery Charger Lockup T< 1 T=180 T=0        T<1                          T<60 Time for Operators to Diagnose, Cross-tie Buses & Reset Charger LOOP Occurs DG Start Signal                                                12 Battery Reaches EDG                                              Minimum Design loaded &                                                      Voltage Failed; Charger                      Timeline in minutes lockup 15
15 Regulatory Significance - Scenarios LOOP, EDG D1 Failure & 12 Battery Charger Lockup LOOP Occurs 12 Battery Reaches Minimum Design Voltage T=0 T< 1 DG Start Signal T<1 EDG loaded &
Failed; Charger lockup T=180 Timeline in minutes T<60 Time for Operators to Diagnose, Cross-tie Buses & Reset Charger


Regulatory Significance - Scenarios LOOP, 11 & 12 Battery Charger Lockup T< 1 T=390   T=450 T=0      T<1                        T<60 Time needed for Operators to Diagnose
16 Regulatory Significance - Scenarios LOOP, 11 & 12 Battery Charger Lockup LOOP Occurs 11 Battery Reaches Minimum Design Voltage T=0 T< 1 DG Start Signal T<1 EDG loaded; Charger lockup T=390 Timeline in minutes T<60 Time needed for Operators to Diagnose  
              & Reset Charger LOOP Occurs DG Start Signal 11 Battery 12 Battery EDG                                            Reaches    Reaches loaded;                                          Minimum   Minimum Charger                                            Design    Design lockup                    Timeline in minutes  Voltage   Voltage 16
& Reset Charger T=450 12 Battery Reaches Minimum Design Voltage  


Operator Actions - Immediate Immediate Actions Following LOOP Event (memorized actions) 1E-0 Reactor Trip or Safety Injection Step 1: Verify Reactor trip Step 2: Verify Turbine trip Step 3: Verify Both Safeguards Buses - Energized Lead Reactor Operator (RO) will recognize that EDG D1 has failed and Bus 15 is deenergized.
17 Operator Actions - Immediate
Response Not Obtained: IF one bus deenergized, THEN initiate action to restore power to deenergized safeguards bus per 1C20.5 AOP1, REENERGIZING 4.16KV BUS 15.
 Immediate Actions Following LOOP Event (memorized actions)
Step 4: Check if SI is Actuated Operators will transfer to 1ES-0.1, Reactor Trip Recovery No Safety injection 17
 1E-0 Reactor Trip or Safety Injection
 Step 1: Verify Reactor trip
 Step 2: Verify Turbine trip
 Step 3: Verify Both Safeguards Buses - Energized

Lead Reactor Operator (RO) will recognize that EDG D1 has failed and Bus 15 is deenergized.  

Response Not Obtained: IF one bus deenergized, THEN initiate action to restore power to deenergized safeguards bus per 1C20.5 AOP1, REENERGIZING 4.16KV BUS 15.  
 Step 4: Check if SI is Actuated
 Operators will transfer to 1ES-0.1, Reactor Trip Recovery
 No Safety injection


Operator Actions - Trip Recovery Transition to 1ES-0.1, Reactor Trip Recovery Control Room Supervisor (CRS) will restore normal alarm response protocol per FP-OP-COO-01 Conduct of Operations, Attachment 1 Alarm Response.
18 Operator Actions - Trip Recovery  
Crew Brief will be performed.
 Transition to 1ES-0.1, Reactor Trip Recovery
CRS will direct the ROs to perform a board walkdown and report any alarms that are unexpected for the LOOP/reactor trip.
 Control Room Supervisor (CRS) will restore normal alarm response protocol per FP-OP-COO-01 Conduct of Operations, Attachment 1 Alarm Response.
Operators are trained to look at electrical sources following LOOP event. Pattern of annuciators will also be evident.
 Crew Brief will be performed.
18
 CRS will direct the ROs to perform a board walkdown and report any alarms that are unexpected for the LOOP/reactor trip.  
 Operators are trained to look at electrical sources following LOOP event. Pattern of annuciators will also be evident.


Operator Actions - Emergency Plan Shift Manager Assumes Role of Emergency Director Within 15 minutes, Emergency Planning classification is required To determine escalation criteria, Shift Manager will evaluate alarms on electrical power panel Multiple people through multiple proceduralized paths give many opportunities to diagnose lockup of a battery charger.
19 Operator Actions - Emergency Plan  
19
 Shift Manager Assumes Role of Emergency Director  
 Within 15 minutes, Emergency Planning classification is required
 To determine escalation criteria, Shift Manager will evaluate alarms on electrical power panel
 Multiple people through multiple proceduralized paths give many opportunities to diagnose lockup of a battery charger.


Operator Actions - Control Room Unit 2           Unit 1 20
20 Operator Actions - Control Room Unit 2 Unit 1


Operator Actions - Control Room Control Room from the Unit 1 side. The electrical power annunciator 21 panel is shown between Unit 1 and Unit 2 Control Rooms where it can be easily seen and monitored by both units operators.
21 Operator Actions - Control Room Control Room from the Unit 1 side. The electrical power annunciator panel is shown between Unit 1 and Unit 2 Control Rooms where it can be easily seen and monitored by both units operators.  


Operator Actions After the plant is stabilized the operators will walk the panels.
22 Operator Actions
During a LOOP, the power panel will be a priority.
 After the plant is stabilized the operators will walk the panels.  
The difference in the pattern for Unit 1 and Unit 2 Annunciators will be quickly identified.
 During a LOOP, the power panel will be a priority.
Operators are trained to identify discrepancies to the norm.
 The difference in the pattern for Unit 1 and Unit 2  
22
 Annunciators will be quickly identified.  
 Operators are trained to identify discrepancies to the norm.



Unit 2 G-Panel Alarms Unit 1 G-Panel Alarms 23
23 Unit 2 G-Panel Alarms Unit 1 G-Panel Alarms


Operator Actions - Restore Bus 15 Per 1E-0, Step 3 RNO:
24 Operator Actions - Restore Bus 15
IF one bus deenergized,THEN initiate action to restore power to deenergized safeguards bus per 1C20.5 AOP1, REENERGIZING 4.16KV BUS 15.
 Per 1E-0, Step 3 RNO:  
Unit 1 SS will instruct Unit 1 Lead RO to enter 1C20.5 AOP1 per 1E-0.
 IF one bus deenergized,THEN initiate action to restore power to deenergized safeguards bus per 1C20.5 AOP1, REENERGIZING 4.16KV BUS 15.  
Dispatch outplant operator to locally check EDG D1.
 Unit 1 SS will instruct Unit 1 Lead RO to enter 1C20.5 AOP1 per 1E-0.  
Per 1C20.5 AOP1, it would be determined that no lockout exists on Bus 15 and is available for repowering Due to LOOP event and D1 EDG failure, Bus 15 will be reenergized via bus-tie breakers per 1C20.5 AOP4.
 Dispatch outplant operator to locally check EDG D1.
24
 Per 1C20.5 AOP1, it would be determined that no lockout exists on Bus 15 and is available for repowering
 Due to LOOP event and D1 EDG failure, Bus 15 will be reenergized via bus-tie breakers per 1C20.5 AOP4.


Operator Actions - Charger Restart Cues Available To Operator 12 DC System Trouble Alarm (47024-1105) 12 DC Panel Undervoltage Alarm (47024-1204)
25 Operator Actions - Charger Restart
Alarm Response Procedures (ARP) will instruct operator to review battery data indication on the Emergency Response Computer System (ERCS).
 Cues Available To Operator
25
 12 DC System Trouble Alarm (47024-1105)
 12 DC Panel Undervoltage Alarm (47024-1204)
 Alarm Response Procedures (ARP) will instruct operator to review battery data indication on the Emergency Response Computer System (ERCS).


Operator Actions - Charger Restart Operators are trained to use all available indication. The Emergency Response Computer System (ERCS) will be initiated to determine       26 status of the batteries. This screen is available on the power panel and would continue to have power during a LOOP.
26 Operator Actions - Charger Restart Operators are trained to use all available indication. The Emergency Response Computer System (ERCS) will be initiated to determine status of the batteries. This screen is available on the power panel and would continue to have power during a LOOP.  


27 28 29 30 31 Operator Actions - Charger Restart Procedure Instruction to Restart Battery Charger ARP will also refer operator to 1C20.9 AOP4: Failure of 12 Battery Charger.
27
1C20.9 AOP4 (Attachment B) - Rev 10, provides instruction to restart battery charger.

Outplant operator required.
Outplant Operators Available to Respond Two Turbine Building (TB) Operators per Unit (4 total).
Two of the operators (one per unit) perform immediate actions of 1ES-0.1, Attachment J.
Two TB Operators available to respond to EDG D1 failure and 12 Battery Charger.
In addition, although not credited in the PRA analysis, Auxiliary Building (AB) operator (2 total) can be available to respond to TB issues since this scenario would not require outplant actions in the AB.
32


Operator Actions - Charger Restart Complexity of Task and Time To Complete Action Simple action, open and close AC Input Breaker on front panel.
28
Both the control room and outplant action times were confirmed during two simulator case studies.

Approximate time to complete Bus 15 repower and restart of 12 battery charger actions based on two simulator case studies: less than 60 minutes 33


Operator Actions The only manual action required to physically reset the battery charger is to switch the AC Input breaker shown above on the left from the Off to the On position.                  34
29



Operator Actions - Conclusion T-0  LOOP w/ D1 Failure AOP to Restore 1E-0 Trip and SI  Bus 15              Bus 15 Restored 1ES-0.1 Trip Recovery 12 DC Trouble Alarm 12 DC Panel UV Alarm Operator Cues    Computer alarms / indications EAL Escalation Criteria ARP & AOP Failure of 12 Procedures      Battery Charger Simple Action Response        Directed by Procedure Battery Charger Reset T-60                                                          35
30



Risk Evaluation Key Risk Significance Question:
31 31

 
32 Operator Actions - Charger Restart
 Procedure Instruction to Restart Battery Charger
 ARP will also refer operator to 1C20.9 AOP4: Failure of 12 Battery Charger.
 1C20.9 AOP4 (Attachment B) - Rev 10, provides instruction to restart battery charger.
 Outplant operator required.
 Outplant Operators Available to Respond
 Two Turbine Building (TB) Operators per Unit (4 total).
 Two of the operators (one per unit) perform immediate actions of 1ES-0.1, Attachment J.
 Two TB Operators available to respond to EDG D1 failure and 12 Battery Charger.
 In addition, although not credited in the PRA analysis, Auxiliary Building (AB) operator (2 total) can be available to respond to TB issues since this scenario would not require outplant actions in the AB.
 
33 Operator Actions - Charger Restart
 Complexity of Task and Time To Complete Action
 Simple action, open and close AC Input Breaker on front panel.
 Both the control room and outplant action times were confirmed during two simulator case studies.
 Approximate time to complete Bus 15 repower and restart of 12 battery charger actions based on two simulator case studies: less than 60 minutes
 
34 Operator Actions The only manual action required to physically reset the battery charger is to switch the AC Input breaker shown above on the left from the Off to the On position.

 
35 Operator Actions - Conclusion Battery Charger Reset
    
1ES-0.1 Trip Recovery Operator Cues 12 DC Trouble Alarm 12 DC Panel UV Alarm Computer alarms / indications EAL Escalation Criteria Procedures ARP & AOP Failure of 12 Battery Charger
 
===Response===
Simple Action Directed by Procedure LOOP w/ D1 Failure 1E-0 Trip and SI AOP to Restore Bus 15 Bus 15 Restored T-0 T-60
 
36 36 Risk Evaluation
 Key Risk Significance Question:
Do operators have enough time to detect and reset tripped battery charger, before batteries become unavailable ?
Do operators have enough time to detect and reset tripped battery charger, before batteries become unavailable ?
Xcel HEP analysis: HEP=8.9E-4 (EPRI HRA Calc.)
 Xcel HEP analysis: HEP=8.9E-4 (EPRI HRA Calc.)
NRC analysis:     HEP=2.2E-2 (SPAR-H) 36
 NRC analysis: HEP=2.2E-2 (SPAR-H)


Risk Evaluation RASP Manual Vol.3 Section 2.3 requires:
37 37 Risk Evaluation
 RASP Manual Vol.3 Section 2.3 requires:
Technical reviews. Check whether a knowledgeable specialist reviewed the reasonableness and acceptability of the results of the thermal hydraulic, structural, or other supporting engineering bases that were used to justify the success criteria. (SC-B5)
Technical reviews. Check whether a knowledgeable specialist reviewed the reasonableness and acceptability of the results of the thermal hydraulic, structural, or other supporting engineering bases that were used to justify the success criteria. (SC-B5)
Fidelity to as-built design:
Fidelity to as-built design:
For CCF of Battery Chargers scenario with both diesels running (no emergency lighting loads on battery) - 7.5hr battery capacity For D1 diesel failure and 12 Charger trip - 3hr battery capacity 37
 For CCF of Battery Chargers scenario with both diesels running (no emergency lighting loads on battery) - 7.5hr 7.5hr battery capacity battery capacity
 For D1 diesel failure and 12 Charger trip - 3hr battery 3hr battery capacity capacity


Risk Evaluation RASP Manual Vol. 1, Section 6.3.1 requires considering following in the recovery analysis:
38 Risk Evaluation
Demonstration that the action is plausible and feasible for the scenarios to which recovery/repair action are applied (HLR-HR-H).
 RASP Manual Vol. 1, Section 6.3.1 requires considering following in the recovery analysis:
Availability of procedures, operator training, cues, and manpower (HR-H2).
 Demonstration that the action is plausible and feasible for the scenarios to which recovery/repair action are applied (HLR-HR-H).
Relevant scenario-specific performance shaping factors in the HRA (HR-H2 and HR-G3).
 Availability of procedures, operator training, cues, and manpower (HR-H2).
 Relevant scenario-specific performance shaping factors in the HRA (HR-H2 and HR-G3).
38
38


Risk Evaluation 39
39 39 Risk Evaluation


Risk Evaluation To obtain HEP=2.2E-2, NRC SPAR-H treatment omits consideration of available time With common cause charger failure and D1/D2 operable: 7.5 hrs available for recovery With D1 failing and lighting transferred to battery: 3 hrs available for recovery Per the SPAR-H model:
40 40 Risk Evaluation
1hr to detect/reset 3hr batteries (expansive time) 1.5hr to detect/reset 3hr batteries (expansive time) 1.5hr to detect reset 2.5hr batteries (extra time) 40
 To obtain HEP=2.2E-2, NRC SPAR-H treatment omits consideration of available time
 With common cause charger failure and D1/D2 operable: 7.5 hrs available for recovery
 With D1 failing and lighting transferred to battery: 3 hrs available for recovery
 Per the SPAR-H model:  
 1hr to detect/reset 3hr batteries (expansive time)
 1.5hr to detect/reset 3hr batteries (expansive time)
 1.5hr to detect reset 2.5hr batteries (extra time)


Risk Evaluation 41
41 41 Risk Evaluation


Risk Evaluation Peer Review of SPAR-H model Page I-17 of NUREG/CR-6883states:
42 42 Risk Evaluation
 Peer Review of SPAR-H model
 Page I-17 of NUREG/CR-6883states:
We do reduce the HEP for at-power situations where there is extra or expansive time available, beyond a certain time minimum corresponding to interpretation of the THERP diagnosis curves.
We do reduce the HEP for at-power situations where there is extra or expansive time available, beyond a certain time minimum corresponding to interpretation of the THERP diagnosis curves.
42


Risk Evaluation In conclusion, if NRC were to consider:
43 43 Risk Evaluation In conclusion, if NRC were to consider:
As-built, As-operated battery capacity Time available for resetting tripped charger for the dominant sequences of interest Per SPAR-H process there is Extra/Expansive time Risk significance is GREEN.
 As-built, As-operated battery capacity  
43
 Time available for resetting tripped charger for the dominant sequences of interest
 Per SPAR-H process there is Extra/Expansive time Risk significance is GREEN.


Conclusion Mark Schimmel - Site Vice President 44
44 44 Conclusion Mark Schimmel - Site Vice President


Closing Remarks Dennis Koehl - Chief Nuclear Officer 45
45 45 Closing Remarks Dennis Koehl - Chief Nuclear Officer


46}}
46}}

Latest revision as of 05:24, 13 January 2025

Meeting Presentation, Prairie Island Nuclear Generating Plant - Failure to Maintain the Train a and B Direct Current Electrical Power Subsystems Operable
ML111990346
Person / Time
Site: Prairie Island  Xcel Energy icon.png
Issue date: 07/28/2011
From:
Xcel Energy
To:
NRC/RGN-III
References
EA-11-110
Download: ML111990346 (46)


Text

NRC Region III Regulatory Conference Failure to Maintain the Train A and B Direct Current Electrical Power Subsystems Operable July 28, 2011 Prairie Island Nuclear Generating Plant Prairie Island Nuclear Generating Plant

2 Agenda

 Opening Remarks - Mark Schimmel

 Sequence of Events - Kevin Davison

 Causes and Corrective Actions - Kevin Davison

 Regulatory Significance - John Bickel

 Operator Actions - Darrell Lapcinski

 Conclusion - Mark Schimmel

 Closing Remarks - Dennis Koehl

3 Opening Remarks

 Xcel Energy takes its obligation to protect the health and safety of the public very seriously

 Violation

 Failure to maintain the Train A and B direct current electrical power subsystems operable

 Agree with the performance deficiency and violation

 Disagree with the significance

 More than adequate time to diagnose charger lockup

4 Sequence of Events Kevin Davison - Plant Manager

5 Sequence of Events 1996 12 Battery Charger lockup during ISI 1994 / 95 Battery chargers installed 1994 1997 2000 2003 2006 2009 2012 1997 Follow up testing of Unit 1 battery chargers 1999 Test procedure revised to remove 12 Battery Charger 2005 12 Battery Charger mod cancelled Oct 2010 TS 3.8.1.B Entered for D2 EDG; Exigent TS change approved Feb 2012 Unit 2 battery chargers to be replaced 1999 Battery charger mod requests 2002 Improved Standard Tech Spec implemented June 2010 Site staff enters 12 Battery Charger testing issues into CAP May 2011 Unit 1 battery chargers replaced

6 Causes and Corrective Action

 A root cause analysis was performed to identify the direct cause and the organizational and programmatic breakdowns that led to failure to correct battery charger lockup.

7 Causes and Corrective Actions

 Root Cause - Direct

 The organization had developed a blind spot to this issue.

 Over several years, multiple opportunities surfaced to identify and correct the condition

8 Causes and Corrective Actions

 Root Cause - Programmatic

 RC-1: Equipment issues become long-standing when C severity level CAPs are allowed to be closed to a secondary work process deliverable, which may be canceled or extended beyond the original CAP due date.

 Root Cause - Organizational

 RC-2: Behaviors necessary to drive long-standing equipment issues to timely resolution are neither modeled, nor required by Plant Health Committee (PHC).

9 Causes and Corrective Actions

 Corrective Action - Equipment

 Implemented compensatory actions for all battery chargers

 Obtained an Exigent Technical Specification change

 Replaced and retested Unit 1 Battery Chargers (1R27)

 Scheduled replacement of Unit 2 Battery Chargers (2R27)

 Procurement of Spare and Unit 2 chargers in progress.

10 Causes and Corrective Actions

 Corrective Action - Process

 For C severity level CAPs, to ensure all open equipment issues remain captured in the corrective action process.

 Establish a PHC Charter and changed behaviors that incorporates the knowledge gained from benchmarking industry leaders and make it available to System and Programs Engineering.

PHC needs to be defined as the champion of projects for presentation to PRG.

11 Causes and Corrective Actions

 Corrective Action - Process

 Implement a periodic PHC roll-up to evaluate effectiveness of resolving equipment

 Legacy Issues Plan

12 Conclusion

 Station had multiple opportunities to identify

 Corrective actions have focused on:

 Replacement of chargers

 Changing behaviors of PHC members

 Process changes to assure sustainability

13 Regulatory Significance Scenarios & Risk Evaluation John Bickel - PRA Consultant Engineer Operator Actions Darrell Lapcinski - Shift Manager

14 14 Regulatory Significance - Scenarios

 LOOP event with EDG D1 failed and 12 Battery Charger Lockup

 Scenario is most limiting time available prior to battery depletion



Plant emergency lighting powered from the 12 battery



Minimum design voltage on 12 battery will be reached in three hours

 Most risk significant sequence from the PINGP PRA evaluations

 Noted by the NRC to be significant risk contributor

 Failure of the EDG D1 would cause the operators to perform additional actions

 LOOP event with 11 and 12 Battery Charger Lockup

 EDG D1 successfully operates

 Scenario is not as limiting in time as scenario listed above



Plant emergency lighting powered from the AC source (EDG D1 operates).



Minimum design voltage on 11 battery reached in six and a half hours.



Minimum design voltage on 12 battery reached in seven and a half hours.

15 Regulatory Significance - Scenarios LOOP, EDG D1 Failure & 12 Battery Charger Lockup LOOP Occurs 12 Battery Reaches Minimum Design Voltage T=0 T< 1 DG Start Signal T<1 EDG loaded &

Failed; Charger lockup T=180 Timeline in minutes T<60 Time for Operators to Diagnose, Cross-tie Buses & Reset Charger

16 Regulatory Significance - Scenarios LOOP, 11 & 12 Battery Charger Lockup LOOP Occurs 11 Battery Reaches Minimum Design Voltage T=0 T< 1 DG Start Signal T<1 EDG loaded; Charger lockup T=390 Timeline in minutes T<60 Time needed for Operators to Diagnose

& Reset Charger T=450 12 Battery Reaches Minimum Design Voltage

17 Operator Actions - Immediate

 Immediate Actions Following LOOP Event (memorized actions)

 1E-0 Reactor Trip or Safety Injection

 Step 1: Verify Reactor trip

 Step 2: Verify Turbine trip

 Step 3: Verify Both Safeguards Buses - Energized



Lead Reactor Operator (RO) will recognize that EDG D1 has failed and Bus 15 is deenergized.



Response Not Obtained: IF one bus deenergized, THEN initiate action to restore power to deenergized safeguards bus per 1C20.5 AOP1, REENERGIZING 4.16KV BUS 15.

 Step 4: Check if SI is Actuated

 Operators will transfer to 1ES-0.1, Reactor Trip Recovery

 No Safety injection

18 Operator Actions - Trip Recovery

 Transition to 1ES-0.1, Reactor Trip Recovery

 Control Room Supervisor (CRS) will restore normal alarm response protocol per FP-OP-COO-01 Conduct of Operations, Attachment 1 Alarm Response.

 Crew Brief will be performed.

 CRS will direct the ROs to perform a board walkdown and report any alarms that are unexpected for the LOOP/reactor trip.

 Operators are trained to look at electrical sources following LOOP event. Pattern of annuciators will also be evident.

19 Operator Actions - Emergency Plan

 Shift Manager Assumes Role of Emergency Director

 Within 15 minutes, Emergency Planning classification is required

 To determine escalation criteria, Shift Manager will evaluate alarms on electrical power panel

 Multiple people through multiple proceduralized paths give many opportunities to diagnose lockup of a battery charger.

20 Operator Actions - Control Room Unit 2 Unit 1

21 Operator Actions - Control Room Control Room from the Unit 1 side. The electrical power annunciator panel is shown between Unit 1 and Unit 2 Control Rooms where it can be easily seen and monitored by both units operators.

22 Operator Actions

 After the plant is stabilized the operators will walk the panels.

 During a LOOP, the power panel will be a priority.

 The difference in the pattern for Unit 1 and Unit 2

 Annunciators will be quickly identified.

 Operators are trained to identify discrepancies to the norm.



23 Unit 2 G-Panel Alarms Unit 1 G-Panel Alarms

24 Operator Actions - Restore Bus 15

 Per 1E-0, Step 3 RNO:

 IF one bus deenergized,THEN initiate action to restore power to deenergized safeguards bus per 1C20.5 AOP1, REENERGIZING 4.16KV BUS 15.

 Unit 1 SS will instruct Unit 1 Lead RO to enter 1C20.5 AOP1 per 1E-0.

 Dispatch outplant operator to locally check EDG D1.

 Per 1C20.5 AOP1, it would be determined that no lockout exists on Bus 15 and is available for repowering

 Due to LOOP event and D1 EDG failure, Bus 15 will be reenergized via bus-tie breakers per 1C20.5 AOP4.

25 Operator Actions - Charger Restart

 Cues Available To Operator

 12 DC System Trouble Alarm (47024-1105)

 12 DC Panel Undervoltage Alarm (47024-1204)

 Alarm Response Procedures (ARP) will instruct operator to review battery data indication on the Emergency Response Computer System (ERCS).

26 Operator Actions - Charger Restart Operators are trained to use all available indication. The Emergency Response Computer System (ERCS) will be initiated to determine status of the batteries. This screen is available on the power panel and would continue to have power during a LOOP.

27



28



29



30



31 31



32 Operator Actions - Charger Restart

 Procedure Instruction to Restart Battery Charger

 ARP will also refer operator to 1C20.9 AOP4: Failure of 12 Battery Charger.

 1C20.9 AOP4 (Attachment B) - Rev 10, provides instruction to restart battery charger.

 Outplant operator required.

 Outplant Operators Available to Respond

 Two Turbine Building (TB) Operators per Unit (4 total).

 Two of the operators (one per unit) perform immediate actions of 1ES-0.1, Attachment J.

 Two TB Operators available to respond to EDG D1 failure and 12 Battery Charger.

 In addition, although not credited in the PRA analysis, Auxiliary Building (AB) operator (2 total) can be available to respond to TB issues since this scenario would not require outplant actions in the AB.

33 Operator Actions - Charger Restart

 Complexity of Task and Time To Complete Action

 Simple action, open and close AC Input Breaker on front panel.

 Both the control room and outplant action times were confirmed during two simulator case studies.

 Approximate time to complete Bus 15 repower and restart of 12 battery charger actions based on two simulator case studies: less than 60 minutes

34 Operator Actions The only manual action required to physically reset the battery charger is to switch the AC Input breaker shown above on the left from the Off to the On position.



35 Operator Actions - Conclusion Battery Charger Reset

    

1ES-0.1 Trip Recovery Operator Cues 12 DC Trouble Alarm 12 DC Panel UV Alarm Computer alarms / indications EAL Escalation Criteria Procedures ARP & AOP Failure of 12 Battery Charger

Response

Simple Action Directed by Procedure LOOP w/ D1 Failure 1E-0 Trip and SI AOP to Restore Bus 15 Bus 15 Restored T-0 T-60

36 36 Risk Evaluation

 Key Risk Significance Question:

Do operators have enough time to detect and reset tripped battery charger, before batteries become unavailable ?

 Xcel HEP analysis: HEP=8.9E-4 (EPRI HRA Calc.)

 NRC analysis: HEP=2.2E-2 (SPAR-H)

37 37 Risk Evaluation

 RASP Manual Vol.3 Section 2.3 requires:

Technical reviews. Check whether a knowledgeable specialist reviewed the reasonableness and acceptability of the results of the thermal hydraulic, structural, or other supporting engineering bases that were used to justify the success criteria. (SC-B5)

Fidelity to as-built design:

 For CCF of Battery Chargers scenario with both diesels running (no emergency lighting loads on battery) - 7.5hr 7.5hr battery capacity battery capacity

 For D1 diesel failure and 12 Charger trip - 3hr battery 3hr battery capacity capacity

38 Risk Evaluation

 RASP Manual Vol. 1, Section 6.3.1 requires considering following in the recovery analysis:

 Demonstration that the action is plausible and feasible for the scenarios to which recovery/repair action are applied (HLR-HR-H).

 Availability of procedures, operator training, cues, and manpower (HR-H2).

 Relevant scenario-specific performance shaping factors in the HRA (HR-H2 and HR-G3).

38

39 39 Risk Evaluation

40 40 Risk Evaluation

 To obtain HEP=2.2E-2, NRC SPAR-H treatment omits consideration of available time

 With common cause charger failure and D1/D2 operable: 7.5 hrs available for recovery

 With D1 failing and lighting transferred to battery: 3 hrs available for recovery

 Per the SPAR-H model:

 1hr to detect/reset 3hr batteries (expansive time)

 1.5hr to detect/reset 3hr batteries (expansive time)

 1.5hr to detect reset 2.5hr batteries (extra time)

41 41 Risk Evaluation

42 42 Risk Evaluation

 Peer Review of SPAR-H model

 Page I-17 of NUREG/CR-6883states:

We do reduce the HEP for at-power situations where there is extra or expansive time available, beyond a certain time minimum corresponding to interpretation of the THERP diagnosis curves.

43 43 Risk Evaluation In conclusion, if NRC were to consider:

 As-built, As-operated battery capacity

 Time available for resetting tripped charger for the dominant sequences of interest

 Per SPAR-H process there is Extra/Expansive time Risk significance is GREEN.

44 44 Conclusion Mark Schimmel - Site Vice President

45 45 Closing Remarks Dennis Koehl - Chief Nuclear Officer

46