ML061980109
| ML061980109 | |
| Person / Time | |
|---|---|
| Site: | Cooper  |
| Issue date: | 03/17/2004 |
| From: | Nebraska Public Power District (NPPD) |
| To: | Office of Nuclear Reactor Regulation |
| References | |
| 10295021, FOIA/PA-2006-0007, R4-5A31 SCR 2004-0077 | |
| Download: ML061980109 (25) | |
Text
(ý,-O Root Cause Investigation Service Water Gland Water Valve Mis-positioning Event SCR 2004-0077 DATE 3/17/04 VJ7
ý2, zr77 Notification(s) 10295021 Root Cause Team Responsible Manager Kevin Chambliss Root Cause Team Leader Dave Pease Root Cause Team Dan Snyder Travis Hoops Roy Radloff Kim Perry Report Type:
o bCRG Preliminary (3-day) Report Weekly CRG Update Draft 5CR Report Final 5CR Report
SCR 2004-0077 1 - PROBLEM STATEMENT..............................................................................................
3 2 - IMMEDIATE AND COMPENSATORY CORRECTIVE ACTIONS...................,.........
4 - EVENT
SUMMARY
................*....................................... 5 5 - EXTENT OF EFFECTS (SYMPTOMS, CONSEQUENCES)..........................
......... 5 6-1 NUCLEAR SAFETY SIGNIFICANCE.........................................................................
6 6-2 INDUSTRIAL SAFETY SIGNIFICANCE.....................
............ 6 6-3 BUSINESS SIGNIFICANCE..................
I 6
7--CAUSES.............................................................
8 - EXTENT OF CAUSES................................................................................................
8 ATTACHMENT A - ASSIGNED ACTIONS AND ENHANCEMENT.....
......,............ 9 EVENT DETAIL................................................................................
13 C-1iCAUSE CODES....,...,.......,... a...........
................... a...........................................
15 C-2 ROOT CAUSE DETERMINATION AND VALIDATION...
................................. 15 C-3 ROOT CAUSE ANALY1C'TOOL 'A"...........
15 C-4 ROOT CAUSE ANALYTIC TOOL "BN......................................
................. 15 C-s ROOT-CAUSE GRAPHICAL DISPLAY............................ o
............... 22 D-1 CNS (INTERNAL) OPERATING EXPERIENCE........... e............
................. 23 D-2 INDUSTRY (EXTERNAL) OPERATING EXPERIENCE.....................
.*moo.......... 23 D-3 EFFECTIVENESS OF CNSSELF-ASSESSMENT ACTIVITIES.......
.......... 23 E-1 DOCUMENTS REVIEWED........
............ I.........,...............................................
24 E-2 PERSONS CONTACTED...................
I
................... 25 3/17/2004 Page 2 of 25
. SCR 2004-0077 1 -.Problem Statement On 2-11-2004, a Station Operator was validating a valve line-up on the Service Water (SW) Gland Water System to assess the cause of low gland water supply pressure.
The Station Operator discovered SW-V-28 closed, SW-V-1 479 open, and SW-V-1 480 open which Was not the expected configuration. This resulted in SW Subsystem "B",
Diesel Generator #2 (DG #2), and the Control Room Emergency Filtration System (CREFS) being declared inoperable and entry Into a 7-day Shutdown LCO. Upon returning the valve line-up to normal the aforementioned equipment was declared operable and the LCO exited. Licensing determined that this was a reportable event because DG#2 and CREFS had been inoperable for greater than the allowed action time.
2 - Immediate and Compensatory Corrective Actions, Actions Completed Effectiveness System was restored to Its proper configuration.
The station was able to return the equipment to an operable status.
On 2/11/2004, an Event Review Board was held to This is an excellent means to gather evidence concerning the event, obtain personnel statements and evidence during the initial stage of an event discovery.
Createa yellow SW GLANDWATER CROSSTIED tags for Provides a caution to the the Service Water Pump Control Switches. Revised operating :crew at a location Procedure 2.2.71 to Include placement and removal of where they will be requlred the tags, and Verification Of valve lineups.
to read it.
Operations will use Interactive briefing techniques to Ensures that the person
,demonstrate understanding of the scope of.work, performing valve controlling documentation, and final configuration. This' manipulations understands requirement Is established through a night order.
the need to check cross tie line up.
Created requirement for* two separate SAP confirmation Establishes administrative entries -one for release of clearance order and one for checks that focus attention restoration of system. For safety-related systems, these on configuration control at confirmations also require independent Verification of the right time.
restoration. If there is no Work Order Operation, NOMS Narrative Logs will be used for these documentations.
3117/2004 Page 3 of 7-5 3/17/2004 Page 3 oV25
SCR 2004-0077 3-Scope Lines of Inquiry Pursued Why?
- 1. When and how did the Yes The LER depends on how long the valve mis-positioning valves were out of position. The occur?
reasons for their being out of position affect the preventive actions.
- 2. What factors contributed to Yes This will be necessary to help the error in restoirtion understand why the error occurred.
from maintenance?
- 3. Have similar mis-Yes To understand the effectiveness of positioning events occurred previous corrective actions and self-in the past at CNS?
assessments 'related to this event.
- 4. Have lessons learned by Yes This will allow CNS to benefit from others in the industry been corrective actions that have proven more effective at successful within the industry.
preventing mis-positioning events?
- 5. Did the error occur While Yes This Will aid In understanding the an operator was Inthe human factors associated with this knowledge-based realm?
event.
- 6. Were any latent Yes To identify preventive measures to organizational factors manage error-likely situations.
present thatcontributed to this event?
3117/2004 Page 4 of 25 3/17/2004 P8ge 4 of 25
SCR 2004-0077 4 - Event Summary On. February 11, 2004, operators discovered that the gland water for Service Water Pumps B & D was being supplied from the Subsystem A header. This condition had existed since the B Subsystem Zurn Strainer was returned to service 21.days earlier.
The Root Causes are process weaknesses In the Clearance Order program and an inadequate focus on the impact of emergent Work on Human Performance.
One Corrective Action will improve the clearance order process by providing clear instructions for the coordination of clearance order release sequences With applicable system operating procedures.
A second Corrective Action will provide an ongoing basisto improve the management-of competing priorities and 'emergent work. Observations during the periods of peak activity, particularly for:the Work Control Operators and Station Operators. These observations will also focus on the impact of emergent work on human performance.
For more details, see Attchment B.
5 - Extent of Effects (Symptoms, Consequences)
The failure to properly restore the gland water lineup affected the operability.of Service Water Subsystem B for approximately 21 days. The Required Action to
-restore the subsystem to an OPerable condition was therefore not completed within the seven days allowed by Technical Specifications. The Seivice Water Subsystem B also supports the operability of several safety systems, Including Diesel Generator 2.
and the Control Room Emergency Filtration System. Therefore the,consequences of
,this event are the unplanned and undetected In.perabillity of multiplesafety
,systems.
3/17/2004I Page 5 of 25 3/17/2004 Page 5 of 25
SCR 2004-0077 1.6 - Significance 6-1 Nuclear Safety Significance]
Risk Management evaluated the risk of this event, concluding that It was not risk significant.The change in the coredamageprobability was calculated as 3.85E-07, which is less than the threshold for classification as risk.significant. The risk assessment included assumptions that the'Service Water pumps can run 30 minutes with a loss of gland water flow, and that the average response time to low gland water flow Conditions was 20 minutes.
With the gland water for Service Water Subsystem B pumps supplied by the opposite subsystem, failure of both pumps in Subsystem A would, have resulted in a loss of gland water'fiOw to the Subsystem B pumps.This condition would have been alarmed in the Control Room, and the operating crew would have 30 minutes to respond before the abllity of the Subsystem B pumps to.complete their safety function would have been affected. The operator response time was conservatively validated by the Operations Department to be.20 minutes.
,6-2 Industrial Safety Significance There was no industrial safety significance to this event.
6-3 Business Significance The response to this event'includes the root cause investigation, risk assessment, and regulatory Interface. The direct costs of these efforts are estimated at $50,000.
The indirect costs of diverting management attention from prioritY Issues'are roughly equivalent to the loss of one management position for two months. Those efforts have been spent on preparing and administering interim actions, and oversight of the:root cause investigation, risk assessment and Licensee Event Report..
3/17/2004 Page 6 of.2S 3/17/2004 Page 6 of 25
SCR 2004-0077 7 - Causes Identifier Description Supporting Basis Root Cause I Administrative procedure for tagouts allowed the Attachment B use of practices that led to a loss of configuration Event Detail -
-control. The'tagout procedure allowed the use of Procedures.
notes instead of numbered steps to directactions.
The tagout procedure also does not require the coordination of the tagout release sequence with the sequence of steps In applicable system operating procedures.
Root Cause 2 The operators did not effectively balance their workload with personnel resource capabilities.
P The Work Control Operator did not focus adequately on the configuration control aspects for the release of the clearance order.
!Supervisory oversight of the clearance order release did not recognize that the Work Control Operator's preparation for the clearance order release das inadequate.
A When emergent conditions required the crew's attention, crew supervision did not use relative priorities to ensure that oversight and the Use of error prevention techniques were maintained.
.For root cause analysis details.ýsee Afttchment C.
3/17/2004 Page 7 of 25 3117/2004 Page 70(25
SCR 2004-0077 8 - Extent of Causes Other Human Performance Errors have related causal factors:
The Human Performance Station Clock is being reset based on management discretion due to the number of.recent events related to human performance. These
,events include: Reactor Recirculation Motor Generator Set Brush Wear Resulting in.a down-power transient (Although not specifically defined as a human error there are elements of this event that are performance related), the Battery Arc event that occurred on 2/26/2004 that had potential to seriously Injure personnel :and damage
'equipment Important to safety, the forklift operation that resulted in dropping a piece of eqUipmentbff of the forklift due to an Improperly balancedl load, and the group 6 and partial group,2 Isolation that resulted from an apparent, momentary loss
- of power (this event Is still in the Initial Investigation stages) 3/17/2004 Page 8 of 25 3/17/2004 Page 8 of 25
SCR 2004-0077 Attachment A - Assigned Actions and Enhancement Document No. SCR 2004- 0077 Page 1 of 4 Actions Assigned by: Kevin Chambliss Date: 3/17/2004 Corrective Action Description (for NAIlT assignment):
PMActType (Arr. 3 of COR Revise Procedure 0.9TAGOUT:
O.S.NA-r)
- 1) To eliminate the possibility for Improperly directing Schedule Type NON-OUTA the use of notes when numbered steps should be (ATr. 2 of uSed.
,O.S.NArn Priority
- 2) Require the tag release sequenceto be coordinated (O-NPG-4.12) with the applicable system operating procedure Overdue Date sequence.
4/19/2004 Firm Due Date?
'(YIN)
N Relationship to Causes:
Ucensing This action corrects Root Cause #1.
Concurrence?
(Y/N)
N Relationship to Extent of Condition:
Assigned Work None Center OPSMGR Expected Benefit and Suitable Effectiveness Measurement (Not for WO Related?
N NAlT assignment):
(Y/N)
The clearance order release sheet will provide step-by-step guidance to restore systems to the required configuration.
Work Center Acceptance: k-SAP CAP Order No:
Other Review:
3/17/2004 Page 9 of 25 3/17/2004 Page 9 or25
SCR 2004,0077 Document No.
SCR 2004- 0077 Page 2 of 4 Actions Assigned by: Kevin Chambliss Date: 3/17/2004 Corrective Action Description (for NAT[ assignment):
PMActType (ATr. 3 of PRE
- 1) Conduct weekly observations of Operating Crew o.5.NArT) activities with the following focus areas during peak Schedule Type activity hours:
NONOUTA
- a. Coordination of Station Operator and Work (Arr."2 of Control Operator Activities 0.5.NArTI'
- b. Coordination of workload and resource Priority availability (O-NPG-4.12) 3
- c. Revising crew priorities when emergent work Overdue Date 6/17/2004 that impacts.POD schedule arises
- 2) Document the results of the observations in the Operations Performance Assessment FiD a
(Y/N)
Relationship to Causes:
Licensing N
This action will Identify ways to manage competing Concurrence?
priorities and help us learn to improve managementof
'(YIN) emergent work. This addresses Root Cause #2.
Relationship to Extent of Condition:
Assigned Work This action will address other events where schedule Center OPSMGR pressure is a contributing cause.
Expected Benefit and Suitable Effectiveness Measurement (Not for WO Related?
N NArT assignment):
(YN)
This'is the beginning ofan ongoing effort to monitor performance. This action adds focus targeted on recent events.
Work Center Acceptance: '.
(iC-*,
Other Review:
3/17/2004 Page 10 of 25 3/17/2004 Page 10 of 25
SCR 2004-0077 Document No.
SCR 2004- 0077 Page 3 of 4 Actions Assigned by: Kevin Chambliss Date: 3/17/2004 Corrective Action Description (for NAIT assignment):
PMActType (Anr. 3 of COR Revise Procedure 0.40.1, 12 WEEK WORK CONTROL O.S.NAXT)
PROCESS to remove the'Operation's Manager's signature on the Schedule Impact Form and replace it with the Shift NONOUTA Manager's signature.
(A'T. 2 of O.S.NA1T)
Priority 3
(O-NPG-;4.12)
Overdue Date 6/17/2004 Firm Due Date?
N (Y/N)
Relationship to Causes:
Licensing N
This action is related to Root Cause #2 by providing the Concurrence?
Shift Manager approval authority for work scope additions.
(Y/N)
Relationship to Extent of Condition:
Assigned Work This action will address Other events where schedule Center WCMGR pressure Is a contributing cause.
Expected Benefit and Suitable Effectiveness Measurement (Not-for WO Related?
N NAIT assignment):
(Y/N)
This will give the Shift Manager the ability to adjust priorities for his crew.
Work Center Acceptance:
Other Review:
3/17/2004 Page 11 of 25 3/17/2004 Page ii of 25
SCR 2004-0077 Document No.
SCR 2004- 0077 Page 4 of 4 Actions Assigned by: Kevin Chambliss Date: 3/17/2004 Corrective Action Description (for NAIT assignment):
PMActType (ATT. 3 of Implement a Corrective Action Effectiveness Review,\\
O.5.NAI)
- X (CAER) Plan as follows: -.
Schedu.e Type
~
~
~~
ns:,.
Schedule TypeNoOT NONOUTA
- 1) SchedUle monthly follow-up meetings between (Arr. 2 of Root Cause Team Leader and Operations o.5.NA*)
Supervisor to discuss the results of the ongoing Priority 3
observations.
(0-NPG-4.12)
- 2) Review the Operations Departmental Human Overdue Date Performance Clock resets for configuration contr ol events.
Firm Due Date?
N (Y/N)
Relationship to Causes:
!Ucensing N
- fqe, Concurrence?
(YIN)
Relationship to Extent of Condition:
Assigned Work Center PIMGR Expected Benefit and Suitable Effectiveness Measurement (Not for WO Related?
N NAT assignment):
(Y/N)
Work Center Acceptance:
L Q0*.
Other Review:
3/17/2004 Page 12 of 25 3/17/2004 Page 12 of 25
SCR 2004-0077 Attachment B - Event Detail Event Detail On February 11, 2004, operators discovered that the gland water for Service Water Pumps B & D was being supplied from the Subsystem A header. This :condition had existed since'the B Subsystem Zurn Strainer was returned to service21 days earlier.
The clearance order used for that maintenance activity relied on a note referring the operator to use the system operating procedure instead of a specific step to, do the same. The station operator incorrectly interpreted the step to mean that when he was done with the clearance order, they system would be lined up correctly. When the clearance order had been hung on the previous shift, the crew crosstied the gland water supply to the Subsystem A header, using the procedure to direct the valve manipulations. Because the cearanceorder did not include these gland water Valves ini the release sequence and the reference to the procedure was misinterpreted, the station operator was unaware that the crosstie'lineup needed to be secured.
The day. shift crew that released the clearance order wasinvolved with a number of emergent activities, which affected the Shift Manager's oversight function and the Work Control Operator's focus 'One of those emergent activities was to support the repair of the screen wash discharge to the river, which required the crew to secure the screen wash system. The Work Control Operator had no experience in securing this system with the plant operating, and the clearance order for this activity was large. They did not know exactly when they.would be called on'to isolate the line, other than it would be near the end of the day and there would be a short window of opportunity for Projects to work before sunset. This job was a site priority and Involved a considerable level of industrial safety hazards due to working above water during cold weather.
The Work Control Operator'focised his attention on the screen wash activity, and as a result, he-was not as -prepared to coordinate the Zurn strainer clearance order as he should have been. In fact, that clearance order release was -not scheduled to be performed until the next shift, but the Shift Manager had made it a priority item to
ýrestore the strainer to'service as soon as maintenance was complete. As It turned out, the crew performed both activities in the last two hours of the day.
At the pre-job briefing for. releasing tags on the Zum strainer, the Work Control.
-Operator apparently did 'not Identify the giand water lineup as a critical step, and did not discuss the use of the system operating procedure. Neither the Shift Manager nor
'the Control Room Supevisor monitored the pre-job briefing. Both of these individuals were aware of the need to 'restore the gland water lineup, but missed the opportunity to discuss the critical 'nature of that lineup by not attending the pre-job briefing.
The Root Causes areprocess Weaknesses In theClearance Order. program and an inadequate focus on the impact of emergent Work on Human Performance.
One Corrective Action will imDrove the clearance order process by providinq clear 3/17/2004 Page 13 of 25 3/17/2004 Page 13 of 25
SCR 2004-0077 Event Detail Instructions for the coordination of clearance order release sequences with applicable system operating procedures.
A second Corrective Action will provide an ongoing basis to -improve the management of competing priorities and emergent work. Observations during the periods of peak activity, paiticularly for the Work Control Operators and Station Operators. These observations will also focus on the Impact of emergent work on human performance.
The third correctiveaction will change the approval authority for additions to the work schedule'from the Operations Manager to the Shift Manager. This will give that individual the information needed to adjust priorities for the crew when emergent work activities impact the potential for human performance errors.
3/17/2004 Page 14 of 25
SCR 2004-0077 Attachment C - Root Cause Analysis Detail C-1 Cause Codes M201 - Crew supervision did not monitor the pre-job brief.
PR203 -,Use of notes instead of numbered steps On the clearance order.
RM101 - Emergent work strainedavailable resources.
WP201:- System operating procedure was not used as required.
C-2 Root Cause Determination and Validation C-3, Root Cause Analytic Tool "A" After~gathering facts and conducting Interviews with the Involved individuals, the Root Cause Team used a Human Performance Evaluation Process Cause Tree from NUREG/CR-6751 to identify lines of inquiry.
The results are documented in Attachment C-5.
C-4 Root Cause Analytic Tool "B" The second analytic tool is a why staircase. Each potential causal factor was evaluated by asking'a series of "why" questions until the answer led to a condition that caused the event, and Is within the control of management. The answers to these questions are arranged in a cause'and effect format. Each cause is separated from'its effect by the phrase AS A RESULT. The effect then Is used as the cause for a second effect In the same manner, resulting In a chain of causes and effects. The causal factors were pi6ures, staffing, ýsupervlsion, communications, and coordination/control.These factors are discussed in separate sections In this Attachment. There is a brief explanation for why the causal factor was
-selected, the applicable standards for each function, :and the chain of cause and effects.
Procedures Procedures are Instructions for performing a task..The Instructions may be provided In formal written procedures or as hand-written Information included in a work package. Procedure-related errors are errors that occur because some charactedsUc of the procedure caused task performance to fail (NUREG/CR-6751, Human PerformanceEvaluation Process),
- 1) Use of notes Instead of numbered steos (Root Cause #1). The last step In the clearance order release Instructions contained a note as follows: "*Restore per 2.2.71."
Standard:
0 A note Is used for explanatory or descriptive information to aid the user in performing the Instructional step. Notes should not typically require action on the part of the user. Exceptions may be allowed for certain Items such as 3/17/2004 Page 15 of 25 3/17/2004 Page 15 of 25
ISCR 2004-0077 C-4 Root Cause Analytic Tool "B" contingency actions (0-PWG-01, Procedure Writer's Guide, Rev.5).
a) Procedure 0.9 provides guidance to annotate on release any procedures required to be performed in conjunction. AS A RESULT, the tagging operator Is allowed to use notes to coordinate procedures with dearance orders, Instead of numbered steps.
b) The'tagging operator is allowed to use notes to coordinate procedures with dearance orders, instead of numbered steps. AS ARESULT, The Clearance Order release Instructions relied on a note Instead of a numbered step to direct the use of Procedure 2.2.71 In returning the Zurn
!strainer to service.
c) The Clearance Order release Instructions relied on a note Instead of a numbered stepto direct the use of Procedure 2.2.71 in returning the Zurn strainer to service. AS A RESULT, the NLO assigned to release the tags did not refer to the procedure.
d) The NLO assigned to release the tags did not refer to Procedure 2.2.71. AS A RESULT, he was not aware of the steps to split the SW Pump gland water supplies.
e) The NLO assigned toyrelease tags was not aware of the steps to splitthe SW Pump gland water
- supplies. AS A RESULT, he did not perform the steps to restore the gland water lineup required for.operability
- 2) CO Release auencewasdfferent than orocedural steo seauence (Root Cause #1). Because it was associated with the last tag In the release sequence, even ýif the station operator had Interpreted the
,note correctly, the opportunity for error~was still there. The strainer bypass valve was already closed as directed by the clearance order release sheet. The steps leading to restoring gland water were conditional on the bypass valve being open.
Standard:
Tagging Operator shall initiate a Section Release as follows: Annotate on mrelease any procedures required to be performed in conjunction with release and any precautions required. Determine proper release sMuence (0.9, Tagout, Rev.
37).
ýa) Procedu're 0.9 reqUires anhotation of procedures to be performed In conjunction with dearance order release, but the co0rdination of sequences Is not required to be pre-planned. AS A RESULT, tagging operators are allowed to specify a sequence for releasing tags that is different than the cor.esponding system operating procedures.
b) The Tagging Operator specified a sequence for releasing tags that was different than the system operating procedure's sequence for placing the strainer in service. AS A RESULT, if the NLO had referred to Procedure 2.2.7i when picking up the last tag, he would have stopped after completing the step requiring him to check strainer differential pressure If the bypass valve was, closed.
c) If the NLO had referred to Procedure 2.2.71 when picking up the last tag, he would have stopped after completing the step requiring him to check strainer differential pressure if the bypass valve, was closed. AS A RESULT, he would not have performed subsequent steps to restore the gland water lineup required for operability.
- 3) Gland water valves were not designated for verification (no-taa). If the gland water valves had been designated for verification when the dearance order was prepared, the tagoUt system would have Induded steps to verify the valves were In the proper position.
Standard:
3117/2004 Page 16 of 25 3/17/2004 Page 16 of 25
. -SCR 2004-0077 C-4 Root Cause Analytic Tool "B" 0
Originator shall perform the following [to generate a tagout]. Determine which components within Tagout boundary will need to have their position verified when work Is complete. These components shall be designated as No-Tag (0.9; Tagout; Rev. 37).
a) Procedure 0.9 requires designating those valves within the tagout boundary to be verified when releasing tags, but does not require designating any valves outside the tagout boundary affecting operability. AS A RESULT, the tagging operator who prepared the clearance order did not designate the gland water valves for verification.
b) The tagging operator who prepared the dearance order did not designate the gland water valves for verification. AS A RESULT, the NOMS Clearance System Module did not Include the gland water valves on the clearance order release sheet.
c) The NOMS Clearance System Module did not include the GW valves on the dearance order release sheet. AS A RESULT, the NLO assigned to release the clearance order tags Was rot directed to check the position of the gland water valves.
Staffing
- 1) Peer check was not Used. If a peer checker had been present, he or she would have had an opportunity to notice that the clearance order release note was Intended to'direct them to refer to Procedure 2.2.71 for direction on how to start the strainer.
Standards:
0 Field Peer Checks should be used at every o0oortunitv (2.0.3 Conduct of Operations, Rev. 46).
Field Peer Checks should be used for first time performers and for tasks that have irrecoverable actions or affect main flow Wath Of ECCS Condensate, Reactor Feedwater, FP, or other sensitive equipment as determined by CRS (2.0.3 Conduct of Operations, Rev,. 46).
S RField Peer Checks can be waived as deemed reasonable and necessary for conditions Including: If, in the Station Operator's judgment, a peer check may cause unnecessary distraction to Peer Checking Operator, the peer check may be waived. Control Room Operator should be notified of waiving of peer check (2.0.3 Conduct of Operations, Rev. 46).
a) Procedure 2.0.3 allows valves that affect operability of Technical Specification systems to be operated without peer checks, if they are not in the main flow path of an ECCS. AS A RESULT,
-the operating crew did not designate a peer checker for the NLO, even though, he had never operated the gland water crosstie valves.
b) The operating crew did not designate a peer checker for the clearance order release. AS A RESULT, the opportunity for a second person in the field to prevent misinterpreting the CO release note was missed (contributing cause).
c) The opportunitytfor a second person In 'the field to prevent misinterpreting the clearance order release was missed. AS A RESULT, the station operator's error was not corrected at the time of commission.
- 2) High workload (Root Cause #2). The primary consequence of inadequate shif manning is task overload. Task overload exists when the number of tasks that must be accomplished In a given period of time exceeds the available personnel resources. One strategy for responding to task 3/17/2004 Page 17 of 25 3/17/2004 Page 17 of 25
SCR 2004-0077 C-4 Root Cause Analytic Tool "B"
- overload is to Increase the work pace. Increasing the work pace may lead to shortcuts or errors due to rushing (NUREG/CR-6751, Human Performance Evaluation Process).
Standards-0 The SM should ensure the on-shift staff Is adeauate to Derform the work scheduled for the shift and ensure sufficient manpower Is available for the activities planned for the next shift. The SM Is responsible to iecognize and prioritize all emergent Issues and contacting Senior Management and/or other plant disciplines, as necessary (0-HP-POLICY, Rev. 4).
The CRS should communicate to the WCO, WC SRO, STE, and Operations Shift the shift planned activities and their resoective grloritles (0-HP-POLICY, Rev.
4).
The WC-SRO shall ensure that the scheduled work activities can'be accomolished in a safe and efficient manner. When plant schedule conflicts occur or the POD activities cannot be accomplished as scheduled, the WC-SRO shall notify the SM and the Work Week Director (WWD)'so that appropriate contingencies can be Implemented as required (0-HP-POLICY, Rev. 4).
competing priritles exist When Individuals or Supervisors have more than one task to performsimultaneously.,Individual s given several tasks to complete without clearly defined Woritization have the tendency to work on multiple tasks to some dearee or to work on one task while tiInking abdutthe othersg This
,situation does not provide focus that each task deserves and leads to errors.
Manaaement must also understand the ability of the organization or department to resoond to changing priorities and the "cost" to the station when constant changing of prioritles Is allowed to occur (0-HP-POLICY, Rev. 4).
e
.Individuals are accountable fof counterina the effects of an error-likely situation by focusing on the task at hand (0-HP-POLICY, Rev. 4).
a) A number of high priority emergent activities dalmed the attention of the SM, CRS and WCO. AS A RESULT, key crewmembers were splitting their attention between multiple activities.
b) Key crewmembers were splitting theirattention between multiple activities. AS A RESULT, they had an Inadequate focus on system status control when restoring the Zum strainer to service.
c) The SM, CRS and WcO had an inadequate focus on system status control when restoring the Zurn strainer to service. AS A RESULT, crew supervision did not devote their attention to the pre-job briefing, and the WCO did not provide field supervision for'the release of the dearance order.
Supervision
- 1)
Competing Rriorities (Root Cause #2). The repairs to'the fish line were not included in the scheduling of maintenance activities for that week. The crew was required to tagout theScreen Wash System to support this work, but did not know exactly when that would happen."The activity to release the Zurn strainer clearance order was scheduled for the following shift, but the Shift Manager anticipated that the work would be completed ahead of schedule. To minimize the time that the Service Water subsystem was Inoperable, he wanted the crew to be ready to release the clearance order when maintenance was complete. Both of these activities deserved the undivided attention of the Work Control Operator. Securing the Screen Wash system Is not a routine task. Placing the Zurn strainer In 3i17/2004 Page 18 of 25 3/17/2004 Page 18 of 25
SCR 2004-0077 C-4 Root Cause Analytic Tool "B" service affects safety system operability. He was aware that the efforts to remove ice buildup around the fish line would provide a short window of opportunity to tagout the system before sunset.
Standards:
eComDeting riorities exist when an employee has more than one task to perform simultaneously. Individuals aiven 'several tasks to complete without clearly defined 2rloritization have the tendency to work on all the tasks to some dearee or to work on one task while thinking about the others: thus. not maintain focus on the task at hand. Eliminating this situation provides the focus that each task deserves and avoids the following TRAPS: time pressure, distractions/interruptions, multiple tasks, vague guidance, mental stress (0-HP-POLICY, Rev. 4).
a) The Work Control Operator (WCO) divided his attention between two significant tasks to be performed In the last two hours of the shift. AS A RESULT, he rfocused on the one with Industrial safety significance at the expense of preparing the release of tags on the Zum strainer.
b) The WCO focused on the Screen Wash idearance order at the expense of preparing the release of tags on the Zum strainer. AS A RESULT, he did not identify Procedure 2.2.71-as a critical step during the premJob briefing.
c) The Work Control Operator did not identify Procedure 2.2.71 as a criticaistep during the pre-job briefing for the clearance order release. AS A RESULT, the station operator was more likely to misinterpret the release note and not refer to the procedure.
d) The NLO assigned to release the tags did not refer to Procedure 2.2.714 AS A RESULT, he was not aware of the steps to split the SW Pump gland water supplies and did not perform them.
ý2).Supervisory oversight of Dre-job briefings (Root Cause #2). When Interviewed, the Shift Manager and Control Room Supenflsr recalled discussing the gland water line up early Inthe shift. Although they were aware of the abnormal lineup, neither one devoted their attention to the pre-job briefing, thus missing the opportunity to emphasize that Procedure 2.2.71 was a critical step for operability.
Standards:
r These "warning flags" represent conditions that may be precursors to station shortfalls and events. Operational Configuration Control: High standards and expectations are not set, are not reinforced, or are not commonly understood. Deviation from procedures is tolerated.
Multiple activities are conducted in the control room simultaneously, with potential breakdown of communications, status awareness, and oversight (INPO 01-002 Guidelines for the Conduct of Operations at Nuclear Power Stations).
Standards are developed to ensure desired job behaviors. Standards provide employees With dear expectations If communicated and enforced correctly. A standard such as place-keeping during procedure use Is developed to help an mindividual perform tasks correctly and successfully. But if the standard Is just established and not continually enforced and expected the practice will not always be performed. If not enforced, then chances foe errors Increase. Continuous and consistent enforcement of standards are a must.. 0-HP-POLICY, Rev. 4).
d) Crew supervision did not attend the pre-Job briefing for the Zum srainer clearance order release.
AS A RESULT, the failure to identify Procedure 2.2.71 as a critical step during the pre-job briefing was not corrected.
e) The Work Control Operator did not identify Procedure 2.2.71 as a critical step during the pre-job briefing for the clearance order release. AS A RESULT, the station operator was more likely to 3/17/20041 Page 19 of 25 3/17/2004 Page 19 of 25
SCR 2004-0077 C-4 Root Cause Analytic Tool "B" misinterpret the release note and not refer to the procedure.
f) The NLO assigned to release the tags did not refer to Procedure 2.2.71. AS A'RESULT, he was not aware of the steps to split the SW Pump'gland water supplies.
g) The NLO assigned to release tags was not aware of the steps to split the SW Pump gland water supplies. AS A RESULT, he did not perform the steps to restore the gland water lineup required for operability.
Communication
- 1) Verbal Communication. The Shift Manager knew that the gland water valve ineup had to be restored before declaring the subsystem operable. The station operator did not. Somewhere in between, the chain of communication was broken. Because the gland water valves were out of their normal position for three weeks before being detected, some of the operators were unable to recall the specifics of their communications. We were unable to determine if the standards for 3-part formal communication were followed. Based on interviews with the operators, the most likely cause was that
.the Work Control Operator did not pass the message along to the station operator.
Implementing the Shift Manager's requirement should have been directed through the clearance
ýorder release, and the causal factors for the failure to do so are explained In above section for Supervision: Competing priorities.
Coordination/Control
- 1) Emergent work Impact (Root Cause #2). The fish line repair was not a :scheduled activity. The operating crew did not know about this job when they prepared for their workweek. The Work Control Operator had not been Involved with it until that day. Additionally, the crew did not know when it would happn, other than sometime late in'the shift. The preventive maintenance task for cleaning the Zurn strainer was scheduled Well in advance. However, the clearance order release was started at 16:00 Instead of the scheduled time of 20:00. These two activities combined to create a peak demand on the operating crew resources that was not scheduled.
Standards:
- .Comoetino Drioritles exist when an employee has more than one task to perform simultaneously. Individuals aiven several tasks to complete without clearly defined orioritlzation have the tendency to Work on all the tasks to some dearee or to work
.on one task while thinking about the others: thus. not maintain focus on the task thn. Eliminating this situation provides the focus that each task deserves and
- avoids the following TRAPS: time pressure, distractions/interruptfons, mu!tiple tasks, vague guidance, mental stress (0-HP-POUCY, Rev. 4).
OWC-SRO confirms Operations can support the schedule as revised at the T-5 meeting. (0.40.1, Rev. 16).
4Shift Manager chairs the Daily Production Meeting and identifies issues that Impact the ability of Operations to support schedule Implementation. (0.40.1, Rev.
16).
- wc-SRo commits Operations to support the schedule that is approved (frozen) by the Work Week Director at the T-2 Meeting (0.40.1, Rev. 16).
a) The work order to repair the Screen Wash discharge line to the river was added to the work schedule by a Schedule Impact Form. AS A*RESULT, Operations departmerit committed 20 hours2.314815e-4 days <br />0.00556 hours <br />3.306878e-5 weeks <br />7.61e-6 months <br /> of support activity to Implement the required clearance orders,: without a complete 3/17/2004 Page 20 of 25 3/17/2004 Page 20 of 25
SCR 2004-0077 C-4 Root Cause Analytic Tool"B" understanding of the potential Impact on previously scheduled work.
b) Operations department committed 20 hours2.314815e-4 days <br />0.00556 hours <br />3.306878e-5 weeks <br />7.61e-6 months <br /> of support for emergent work without a complete understanding of the potential Impact on previously scheduled work. AS A RESULT, when previously scheduled work was completed earlier than anticipated, the Work Control Operator split his attention between two simultaneous tasks.
e) The Work Con'trol Operator split his attention between two simultaneous tasks. AS A RESULT, he focused on the job with industrial safety significance at the expense of:preparing to release tags on the Zurn strainer.
f) The WCO focused on the Screen Wash clearance order at the expense of preparing to release tags'on the Zurn strainer. AS A RESULT, he did not Identify Procedure 2.2.71 as a critical step during the pre-job briefing.
g) The Work Control Operator did not Identify Procedure 2.2.71 as a critical step during the pre-job briefing for the clearance order release. AS A RESULT, the station operator was more likely to misinterpret the release note and not refer to the procedure.
h) The NLO assIgned to release the.tags did not. refer to Procedure 2.2.71. AS A RESULT, he was not aware of the steps to split the SW Pump gland Water supplies.
The NLO assigned to release tags was not aware of the steps to split the SW Pump gland water supplies.
AS A RESULT, he did not perform the steps to restore the gland water lineup required for operability.
3/17/2004 Page 21 of 25 3/17/2004 Page 21 of 25
SCR 2004-0077 C-5 Root Cause Graphical Display.
Persnmel RrnSlnM Work Efvironment Human Performance Evaluation Process The shaded boxes are identified as causal factors and are further evaluated in a Why Staircase; 3/17/2004 Page 22 of 25
.SCR 2004-0077 Attachment D - Learning Organization Effectiveness D-,1 CNS (Internal) Operating Experience The root cause team reviewed self-assesSments and corrective actions related to the lossof configuration control. This review was conducted from the perspective of the causal factors and concluded that schedule pressure has been a contributing cause in many human performance events.
D-2 Industry (External) Operating Experience SOER 98-1; SAFETY SYSTEM STATUS CONTROL
.EAR ATL 94-014; CONTiAINMENTPRESSURE INSTRUMENT TAPS CAPPED DURING MAINTENANCE EAR PAR 97-011; PUMP DECLARED OPERABLE BUT STEAM INLET VALVELEFT CLOSED MER PAR 03-003; VALVE MIS-POSmTION EVENTS EVENT NUMBER: 298-'011102-1; BOTH REACTOR.BUILDING-TO-SUPPRESSION CHAMBER VACUUM RELIEF LINES MADE INOPERABLE D-3 Effectiveness of CNS Self-Assessment Activities After a reView of CNS Self Assessments it was determined that none of the assessments were applicable to this particularevent.
3/17/2004 Page 23 of 25 3/17/2004 Page 23 of 25
- SCR 2004-0077 Attachment E - Investigation Detail E-1 Documents Reviewed CNS Procedures ADMINISTRATIVE PROCEDURE 0-HP-POLICY; HUMAN PERFORMANCE POLICY ADMINISTRATIVE PROCEDURE O-PWG-01; PROCEDURE WRITER'S GUIDE.
ADMINISTRATIVE PROCEDURE 0.9; TAGOUT ADMINISTRATIVE PROCEDURE 0.31; EQUIPMENT STATUS CONTROL ADMINISTRATIVE PROCEDURE 0.40.1; 12-WEEK WORK CONTROL PROCESS CONDUCT:OF OPERATIONS PROCEDURE 2.0.3; CONDUCT OF OPERATIONS SYSTEM OPERATING PROCEDURE 2.2.71; SERVICE WATER SYSTEM OPERATIONS DESK GUIDE #3-NCS; NOMS CLEARANCE SYSTEM NCS OPERATIONS DESK GUIDE #3-NCCS; NOMS CONFIGURATION CONTROL SYSTEM NCCS INPO Documents GUIDELINE 01-002; GUIDELINES FOR THE CONDUCT OF OPERATIONS AT NucLEAR POWER STATIONS (OTHER DOCUMENT) EXCELLENCE IN HUMAN PERFORMANCE External Operating Experience SOER.98-1; SAFETY SYSTEM STATUS.CONTROL EAR ATL 94-014; CONTAINMENT PRESSURE INSTRUMENT TAPS CAPPED DURING MAINTENANCE EAR PAR'97-011; PUMP DECLARED OPERABLE BUT STEAM INLET VALVE LEFT CLOSED MER PAR 03-003; VALVE MIS-POSITION EVENTS EVENT NUMBER:298-011102-1; BOTH REACTOR BUILDING-TO-SUPPRESSION CHAMBER VACUUM RELIEF LINES MADE INOPERABLE 0E16753 -.HIGH PRESSURE CORE SPRAY PUMP BREAKER INADVERTENTLY RACKED OUT DURING PERFORMANCE OF ANOTHER TAGOUT
.OTHER Documents CLEARANCE ORDER SWB-1-4324147; SERVICE WATER ZURN STRAINER B PSA-ES062; PROBABILISTIC SAFETY ASSESSMENT: RISK SIGNIFICANCE OF SCR 2004-0077, SERVICE WATER GLAND WATER VALVE MIS-POSITIONING EVENT.
SNAPSHOT ASSESSMENT SS04030; EVENT REVIEW -SERVICE WATER VALVE LINEUP WORK ORDER 4324147; EXAMINE STRAINER OPERATIONS TURNOVER SHEETS 3/17/2004 Page 24 of 25 3/17/2004 Page 24 of 25
SCR 2004-0077 E-2 -Persons Contacted Brian Murphy Nate Beger Jamie Weiss Chris Carpenter Rod Penfleld Paul Tetrick Ken Nosbich Dave Montgomery Glen Seeman Mark Schaible Mike'Tackett 3/17/2004 Page 25 of 25 3117/2004 Page 25 of 25