ML051460494

Licensee Root Cause Report, CAP55527, Industrial Safety Issues and Poor Work Practices During Nozzle Dam Installation, with R. Alexander'S (Riii) Notes

ML051460494
Person / Time
Site:	Monticello, Palisades, Kewaunee, Point Beach, Prairie Island
Issue date:	04/09/2004
From:	Capristo A, Craft D, Denherder K, Hager J, Harden P, Peterson J, Schuelke D, Thomas Taylor, Zastrow K Consumers Power Co, Nuclear Management Co
To:	Office of Nuclear Reactor Regulation
References
CAP55527, FOIA/PA-2004-0282, FOIA/PA-2006-0113 RCE 253
Download: ML051460494 (42)

Text

'N .

I Cm lar -

Committed to Muclear Escelenco Point Beach Nuclear Plant IndustrialSafety Issues and Poor Work PracticesDuring Nozzle Dam Installation RCE 253 CAP55527 Event Date: 04/09104 Principle Investigators:

Team Leader Kristin Zastrow, Kewaunee Team Member John Peterson, Monticello Immediate Actions Paul Harden, Palisades Inmnediate Actions Kari DenHerder, Prairie Island Immediate Actions Joe Hager, Palisades Immediate Actions Don Schuelke, Prairie Island Immediate Actions Dan Craft, Hudson Immediate Actions Aldo Capristo, Hudson Immediate Actions Tom Taylor, Prairie Island Approvals:

Team Leader (Team RCE) or RCE evaluator Date Intormabon intis record was deleted inancocdar~e wWthfe Fresiom of Informabon Management Sponsor Date A-t Abl L-16

IndustrialSafety Issues & Poor Work PracticesDuring Nozzle Dam Installation- RCE2S3 Table of Contents I. Executive Summary ..................................... 3-4 II. Event Narrative ..................................... 5-7 III. Extent of Condition Assessment ..................................... 7-8 IV. Nuclear Safety Significance ................................ ...... 8 V. Reports to External Agencies & the NMC Sites ...................... 9 VI. Data Analysis.

Information & Fact Sources .9-10 Evaluation Methodology & Analysis Techniques .10-11 Data Analysis Summary .11-12 Failure Mode Summary .12-14 VII. Root Causes and Contributing Factors .15-17 ViI. Corrective Actions .17-21 IX. References .22-23 X. Attachments .24-42 2

Executive SummarV

Purpose:

This Root Cause Evaluation will determine the at-risk behaviors that did not meet Point Beach and Nuclear Management Company Expectations, and identify where personnel performed tasks outside of procedures and training to accomplish completion of work related to steam generator nozzle dam installation during the U1R28 Refueling Outage. This Evaluation will additionally identify the underlying causes of those behaviors and actions and recommend actions to correct those causes.

Event Synopsis:

On Friday, April 09, 2004, the Point Beach Nuclear Plant experienced four (4) separate incidents of personnel breathing air issues during steam generator nozzle dam installation evolutions. Two (2) incidents involved air line disconnections at the Snap-tite connection to the "bubble hood",

one (1) worker experienced low air pressure for unknown reasons, and one (1) air supply line was damaged during access to the manway. No personnel injuries occurred, however the incidents did lead to personnel contamination issues.

==

Conclusions:==

Through personnel interviews conducted by an immediate action team and the root cause evaluation team, various Task and Barrier Analyses, Event & Causal Factor Charting, Failure Analysis and Conclusions, Operating Experience review, Training records and Procedure reviews, facts detailed more than 20 inappropriate actions during Point Beach U1R28 Steam Generator Nozzle Dam Installation. Error-likely situations were not identified through adequate training, walkthroughs, and briefings, while procedures and proper communications were not used or did not provide the information for the team to be successful. In addition, Supervision throughout the event allowed development and approval of an inadequate work plan, and did not provide the necessary leadership to avoid tunnel vision and prevent incident.

Nuclear Safety Significance:

The nuclear safety significance of this event (provided by Plant Licensing) is minimal because Unit 1 was in the refueling shutdown condition (Mode 6) and the steam generators were open to atmosphere. However, the radiological significance of the event was elevated from an ALARA perspective due to three entries into Steam Generator B being required. Additionally, the industrial safety aspects of this event were more than minor due to the confined space of the task and the necessity to utilize a supplied air breathing system.

Root Cause & Significant Contributing Factors:

• Root Cause: Oversight by Supervisors/Managers during work planning development and task execution didn't assure compliance with procedures and processes, resulting in an inadequate work plan being developed and approved for use.

• Significant Contributing Factor #1: Work Order Processing per NP 10.2.4 and Outage Management Planning per NP 10.2.1 does not include logic ties (IF this, THEN that) to drive use of appropriate procedures during work plan development.

• Significant Contributing Factor #2: Program Engineering personnel and Radiation Protection personnel did not use and/or follow Work Order Processing, Risk Assessment, Briefing, or Radiation Protection procedures in preparing for and during execution of the steam generator nozzle dam project.

3

• Significant Contributing Factor #3: Training for steam generator nozzle dam installation was not adequate to identify the error-likely situations that existed upon the start of work.

• Significant Contributing Factor #4: Communications to the OCC of safety significant events was not delivered and Nuclear Oversight identification of an inadequate briefing was not delivered in a timely or effective manner.

• Significant Contributing Factor #5: Previous External Operating Experience on failures of quick-disconnect fittings was not adequately used to correct the similar failure mechanisms that existed on the equipment utilized at Point Beach.

Corrective Action Synopsis:

Interim: 1) Plant Stand-down or "Time Out" conducted on 04/09/04 (Excellence through Error Prevention). 2) Nozzle dam Lessons Learned meeting conducted the week of 04/26/04, prior to nozzle dam removal. 3) Following the incident, Radiation Protection brought in an independent team of NMC personnel to review procedures and processes for use of supplied air as breathing air and used their input to: update procedures, change out and replace nitrogen bottle back-up with certified Grade D breathing air, replace all bubble hoods, airlines and Snap-tite fittings to new CEJN type fittings. 4) Radiation Protection developed a Just in Time information sharing package which was used prior to restart of work with bubble hoods. 5) Mock-up training for nozzle dam removal included a review of procedure requirements, bubble hood issuance requirements, and manifold pressure requirements.

Corrective Actions to Restore (broke-fix): 1, 2, & 7) Perform a documented brief for Programs Engineering, Radiation Protection and Nuclear Oversight (separately) on the safety anomalies and poor work practices associated with this event - inappropriate actions taken on the part of their group, and their responsibility for assuring that their actions are corrected. 3)

Complete Lesson Plan HPC-04-LP203, Nozzle Dam Just-in-time Training, and implement Supervisory oversight corrective actions established by nozzle dam removal project plan. 4)

Develop Communications Protocol and include in work plan for nozzle dam removal. 5)

Prepare Expectations and Brief personnel on the adequate use and action to be taken for External Operating Experience that is assessed by Radiation Protection personnel. 6) Programs Engineering CRC perform a Task Analysis to determine personnel knowledge and training required to successfully lead and plan major projects.

Corrective Actions to Prevent Recurrence: 1) Develop a nozzle dam removal work plan in accordance with NP 10.2.1, Outage Management and NP 10.2.6, Work Order Processing, which also includes Supervisory independent approval. 2) Develop procedures for nozzle dam installation and removal that incorporates lessons learned from this event, supervisory and management requirements and stop work criteria, communications protocol, and external Operating Experience. 3) Develop, within NP 10.2.1, Outage Management, and NP 10.2.6, Work Order Processing, a process to determine when HIT teams or Project Managers should be assigned and include logic ties to drive use of appropriate procedures during work plan development.

Other Corrective Actions: 1) Create an "It Can Happen Here" article for distribution to plant personnel. 2) Develop Industry OE on the event 3) Perform INPO Nuclear Safety Culture Assessment to identify gaps and formulate corrective actions. 4) Develop and issue site-wide communication on the purpose of NP 1.1.7, Managing Work Activity Risk, and when to utilize it. 5) Work Week Coordinator review all existing High Risk work orders scheduled within the next five weeks for compliance with NP 1.1.7 and implement a review of High Risk work orders for compliance with NP 1.1.7 at the appropriate E- meeting. 6) OTH #6 - Take completed CRC Task Analysis results from CA #6 to the TOC to identify impacts on other work groups.

Effectiveness Reviews: 1-3) Complete separate Effectiveness Reviews for each CATPR.

4

Event Narrative On Friday, April 09, 2004, with the Point Beach Nuclear Plant Unit I in Refueling Outage U1R28, steam generator nozzle dam installation was to take place per Work Order 0400042, Safety Related Document # 83A7564 (PBNP Approval date of 03/07/04, however a work order plan per NP 10.2.4, Work Order Processing, and the associatedproceduresandfonns, wvas not completed).29 ,4249 On 04/08/04, this work was identified as a Critical Path/Near Critical Path Activity due to be completed within the next 24 hours2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br />, with indication on the Outage Status Report that the schedule was 9 hours1.041667e-4 days <br />0.0025 hours <br />1.488095e-5 weeks <br />3.4245e-6 months <br /> behind. 41 Work to install the nozzle dams in the generators, began in the early morning hours of the Friday (04/09) of Easter weekend. (This work commenced at 0417, with approximately eleven hours already into the 12-hourshift for the work group.)15 A briefing was held prior to the start of work and resulted in comments on the 04/08/04 Nights, Nuclear Oversight Rapid Trending Assessment Daily Report. This report documented that the "IPTE brief for nozzle dam installation was evaluated as less than adequate due to poor communications, failure to document brief on an approved form, and lack of interaction of personnel at the brief." 21 (An IPTE (Infrequently Performed Tests & Evolution) briefing was documented per a "Documentation of Information Sharing Worksheet" (QF-1060-02 Rev. I (FP-T-SAT-60) and was signed & dated by participantson 04/09/04, with Preparerand Approval signatures dated 04/10/04.29 Nuclear Oversight indicated that this was not the correctformfor the evolution.)

Upon start of work, Scientech personnel tasked with performance of the nozzle dam installation, dressed in the Radiation Protection area of the 8' elevation of Containment. Dress requirements per the approved Radiation Work Permit, required the use of "bubble hoods", (which utiTizes a constant air flow system from a service air manifold), where an airline would be connected to the suit or "bubble hood" for the necessary breathing air required by the worker. (Scientech personnel had expressed concerns with the clothing requirements (cloth hood requiredunder the plastic "bubble hood"), stay time requirements being too conservative, or not long enough to complete the work.27 Issuance of this "bubble hood" fell under HPIP 4.58, "Issuance of Respiratory Equipment" 3 , and Radiation Protection had a qualification task associated with it.35 18 However, the procedure and documentation of issuance was not performed as required.,

Personnel were dressed as necessary, (without use of guidanceprovided within HPIP4.58) with their hood air line connection connected to an air line regulated on the 8' elevation. (Scientech workers expressed concerns with low airpressure at the 8' level airlineconnection. Radiation Protection personnel discussed the low air pressure issue at the 8' level with the Scientech workers, and expressed the opinions that there would be more airflow at the platform level. 27 No formal investigation into the air pressure concerns was performed.). Workers then proceeded to the Steam Generator platforms where nozzle dam installation would commence. At the Steam Generator platforms, workers would disconnect their airline supplied by the 8' level regulator, and reconnect to an airline supplied by a regulator on the platform. First, workers would connect to a 100' hose at the bottom of the platform, climb up, then disconnect and allow the line to lower back to the bottom. After disconnection of the 100' airline, they (with the assistance of the RP Technician stationed on the platform) would then connect to a 50' airline that would be used for work while in the generator.

The first Scientech worker arrived at Ul Steam Generator B platform to perform cold leg nozzle dam installation. He expressed to the RP Technician that he had an air pressure concern, made a request to be wet down with a liquid previously staged on the platform (stagingof this liquid was not performed by the RP Technician assigned to the platfonn at the time, and he was not aware of its purpose orprevious delivery to the area. Who stadthe liquid, location of the liquid after

I the event, etc., could not be found.)2 6 , was wet down by the RP Technician as requested, and proceeded to enter the steam generator to install the nozzle dam. During the worker's stay time (1:48)'5, communications noted his dissatisfaction with the progress of his work (cursing, etc.),

along with a concern that he had low air pressure.' 2 The worker left the steam generator bowl based on his assessment of low air pressure, and was cut out of his bubble hood. This resulted in a personnel contamination event, and was documented per PBF-4039a. 3 0 (Communication between the RP Technician on the platform, as well as RP on the lower level, and individualsin the communications trailer were available throughout the workers stay time.12 ) When the worker exited the bowl, his bubble suit was stated to be inflated with minor fogging of the hoodl 2.26 27, while the worker stated that he was deflated and "sucking plastic"2 7 . The Snap-tite connection from the bubble hood air hose to the regulator air hose, was found to be connected, with no obvious failure identified. The RP Manager contacted the OCC (Outage Control Center) regarding the possible loss of breathing air (which was reported as a loss of breathing air)'6 .

This issue was logged by Operations approximately 16 minutes after the worker left the area.' 6 The RP Technician and the RP Supervisor discussed the air pressure issue and supervision approved an increase in the air manifold pressure to support the needs of the workers.15.26 (Whether known atLatime or not, the increase in airpressure violateP .5 Step 4.5.7 which states, "Adjust air supply pressure so that airflow is betveen 6 and Dcubic feet per "..1.

minute. Foran airline length of 50 feet, a pressurerange of2O to 28 psig correspondsto aflow 6(T rate range of 6 to 15 cfm. " As stated earlier, this procedure (Issuahce of RespiratorEquipment) was not performed when issuing the "bubble hoods" to workers, and therefore was violated priorto start of work on nozzle dam installation, and again upon increasingthe pressurebeyond procedural requirements. Again, no fonnal investigation was performed to confirm or dispute the concerns of low air pressure.) Upon worker leaving the area, a Personnel Contamination Event Report was filed due to the contamination received during cutout, however nasal smears were not performed as directed by PBF-4039a, and was not captured until a supervisory review one (1) day after the incident.30 (This was captured by the supervisor on CAP55565, and it was noted that this was not the only case where smears were not performed.)

Ten (10) minutes after the first Scientech worker lost air, the second Scientech jumper attempte to access the steam generator cold leg and failed to gain access upon two attempts. 2"'52 (Details: The worker had difficulty entering due to his size, and requested the RP Technician t wet him down. This request was acknowledged, and he was wet down, as well as provided wiJ assistance to access the manvay (RP Technician and other jumper that was stationed on t e platform.)r2 '22627 ) The jumper gained access on his third attempt, however, with a total logged jump time of 1:18 seconds, the jumper exited due to loss of air.' .26.27 (The worker details that :e remained in the bowl after he realized he had lost air. He believed he had 2 or 3 good breaths, and continuedworking.27 ). It was determined that the worker's quick disconnect fitting from the hose on his hood had become disconnected at the Snap-tite fitting, causing the loss of air. Like the first jumper, the worker was cut out of the bubble suit to allow breathing, which resulted in a personnel contamination event that was documented per PBF-4039a, again, however, nasal smears were not performed as directed, and it was not identified until the supervisory review one (1) day after the event. 30 Health Physics and an unnamed NMC Project Leader discussed allowance of the next jumper to complete the cold leg nozzle dam installation, and after air pressure was verified good and the airline connection challenged (five minutes from discussion to the time the worker entered cold leg'), the Scientech worker (with a total jump time of 1:14) completed the installation.

Ir

r1

¶ Two (2) other airline anomalies were unofficially communicated after installation of the nozzle dams was completed, though this was not documented in the Containment or Operations Logs.

One (1) additional air line disconnection occurred during nozzle dam installation on Steam Generator A, when a worker became disconnected and was quickly reconnected, while the other issue was described as an airline being damaged (due to the staging of equipment on the platfonn that caught the hose and cut it.) 1 8 9 x2,16 .17= 2 6 2,7 36 and then taped by Radiation Protection personnel to allow for continued work. Questions were then raised by site Nuclear Regulatory Commission staff, regarding the breathing air problems and a request for a formal investigation was made, leading to the following root cause evaluation.

Extent of Condition Assessment An Extent of Condition evaluation was conducted to determine how far the problem extended, what else the problem may affect and other programs, processes and equipment that may be vulnerable to the same condition. This was also performed to determine how wide spread the condition or its causes may be.

The condition assessment was limited to air line concerns and air line connection failures or concerns during the use of bubble hoods. The issuance of bubble hoods (according to HPIP 4.58), is for Radiological Use Only, and therefore only applies to tasks involving Radiation Protection's concerns for personnel radiological safety. Issuance of the bubble hoods and airline connection checks are part of Radiation Protection's Training Qualifications, and therefore is limited to Radiation Protection. Department Management has taken corrective actions to replace all bubble hoods, hoses and Snap-tite fittings to new CEJN type fittings. In addition, the Department created a Just in Time information sharing package to be used during a pre-job briefing for this task. This information included a review of procedure requirements, bubble hood issuance requirements, and manifold pressure requirements.

Similar equipment was also considered for this extent of condition. All supplied air for breathing is from the service air system. The Operations Department has an emergency breathing air'.

system for the control room, which is also used to fill SCBA bottles. This system has a compressor but also has X-connect valves with Service Air. No other systems were identified by the Radiation Protection Department.

System interrelations were considered for this assessment as well, and a possible relationship existed when a nitrogen bottle back-up to the air supply line was identified. This issue is being evaluated under CAP 55751, therefore no further action for this system interrelationship is required under this assessment.

This evaluation was a result of an Action Request that was screened as a Significance Level A issue, which is considered to be a Significant Condition Adverse to Quality (SCAQ). The cause of the event, and the organization effectiveness issues surrounding that, supports the initial assessment of a possible SCAQ. Due to the possibilities of an SCAQ in the organizational areas, Management has committed to performance of INPO Nuclear Safety Culture Assessment to bound the identified conditions. This action is being performed under CA57320.

Previous Operating Experience was identified throughout the INPO database5'1 , as well as external OE that was responded to internally by Point Beach staff' 3 relating to air hose disconnections of the type used at Point Beach, as well as problems associated with nozzle dam installation/removal. Further information on Operating Experience can be found under 7

Attachment D, OE Analysis. This OE was now utilized to effectively replace Point Beach bubble hoods, hoses and CEJN type fittings.

Extent of Cause For the extent of "Cause", any organization or process that performs high-risk activities where personnel safety awareness is required, may be affected and therefore must be addressed for this assessment. Plant Management and the Sponsor for this evaluation has committed to performing INPO Nuclear Safety Culture Assessment to determine the present condition for this extent of cause. This action is being performed under CA57320.

Previous Similar Events The TeamTrack database for the Corrective Action Program was queried and Plant Assessment personnel performed interviews of Radiation Protection personnel to determine previous similar events associated with this Root Cause Evaluation.

Statements gathered from individuals involved with the events, identified that air line disconnections may have occurred in a previous outage 27 , however, no formal documentation hrough the CAP process nor separate interviews found confirming information. No corrective actions were generated previously, therefore effectiveness of previous actions cannot be assessed.

Previous issues with HPIP 4.58 as well as requests for JflT topics and Task Analysis for specific Radiation Protection tasks related to this event had been captured in Curriculum Review Committee minutes and CAPs prior to this event 43'44. These identified the need for specific training and development of training for tasks associated with this event. However, the activities were post-poned or cancelled by Management sighting lack of support and time constraints. 43 ,4 The OE Analysis reveals that Radiation Protection personnel closed CAP actions without:

implementing changes to breathing air system connections/fittings, and were satisfied with then, ractices in place at the time of OE review.. Prior replacement of the Snap-tite fitting could have revented two of the failures asyociated with this event. This finding was addressed by the orrective Actions taken by Radiation Protection for replacement of the. Snap-tite onnections/fittings that were previously used for breathing air at Point Beach (Attachment 1). In

<$ addition, Corrective Action to Restore CA#5, will prepare/develop department expectations and perform a briefing for Radiation Protection personnel regarding adequate use of Operating Experience in the Radiation Protection department.

Nuclear Safety Significance The nuclear safety significance of this event (provided by Plant Licensing) is minimal because Unit I was in the refueling shutdown condition (Mode 6) and the steam generators were open to atmosphere. However, the radiological significance of the event was elevated from an ALARA perspective due to three entries into Steam Generator B being required. Additionally, the industrial safety aspects of this event were more than minor due to the confined space of the task and the necessity to utilize a supplied air breathing system.

8

i Reports to External Agencies Supplied by Plant Licensing: This event is not reportable to the NRC in accordance with 10 CFR 20, 10 CFR 50.72 or 10 CFR 50.73. The NRC has indicated that they have referred the circumstances associated with this event to OSHA, therefore there may be additional reporting that may be required in the future. INPO OE has been generated by Assessment personnel per site procedures and is required as documented in CAP 55527.

Data Analysis Information & Fact Sources Initial information was gathered by a "Rapid Response Team" that consisted of Point Beach, Hudson, Prairie Island, Palisades, and Monticello personnel. Interviews of personnel involved in the event included Radiation Protection personnel, Scientech personnel, and Program Engineering personnel. These interviews were summarized and captured for use in development of a timeline of events, and served as the initial information source by which further questions by the root cause evaluation team were developed."' 2' 7 .1924- 6 27 ,36 Questions were developed to gather information relevant to pre-staging, planning, training, concems/problems with the task, and execution. These interviews and questions detailed more than 20 inappropriate actions throughout planning and implementation of steam generator nozzle dam installation during the U1R28 Refueling Outage.

Data sources included (but were not limited to) Control Room and Containment Logs, Point Beach procedures that were relevant for work planning through to execution of work, Scientech procedures used for installation/removal of nozzle dams, Point Beach Job Files, Training Lesson' Plans and Qualification Matrixes, Corrective Action Process Action Requests and Corrective Action items. These data sources identified that personnel throughout planning and execution of the task, did not utilize the information available that would assist in successful task preparation and completion.

Similar equipment was gathered through the site Safety Department and Program Engineering personnel, who acquired bubble hoods and hoses that were the type used during the events, as well as video which allowed viewing of the type of activities that were taking place at the time of 4EX5 the event. Direct evidence - pictures of the air-line system, staged liquid used to wet-down workers, the equipment (bubble oods and hpsss), xvide f the actual event - could not be located and secured.

Previous External Operating Experience (Attachment lowing is a summary of the analysis performed by John Peterson, Monticello, for recent PBNP evaluations of external Operating Events (OE) regarding loss of supplied breathing air due to separation of supplied air line quick disconnect fittings. OE031454- 6/19/2003 -

Subject:

Worker lost air to his bubble-hood when one of the fittings unexpectedly became disconnected. The Point Beach evaluation noted that Point Beach uses quick disconnect fittings of a different manufacturer, and that the fittings are taped as further safe guard. Although the quick disconnect fittings made by the two different manufacturer's employed similar mechanisms and action for connecting/disconnecting the fittings, the evaluation was closed with no further action. Action should have been taken to challenge the connections used at Point Beach, to further justify closure without action.

OE048685- 8/25/2003 -

Subject:

Fittings became separated and an air line hose was accidentally cut by a co-worker. The Point Beach evaluation of this OE addressed the accidental 9

i cutting of the hose but did not address the separation of the quick disconnect fittings. The OE was closed by the activity performer with no further action other than stating that Point Beach RP technicians are trained to cut a person out of a bubble hood if they experience air loss.

Review and Approval by Supervision or Quality Check of this activity should have identified that all issues were not addressed in the Activity Completed section. OE010321- 10/10/2003 -

Subject:

Worker experienced a loss of air supply to an air hood because the quick disconnect fittings separated. The Point Beach evaluation of this OE "determined that the procedures and controls at Point Beach are adequate to minimize susceptibility to this event". A statement that "HPIP 4.51.3 have controls in place to address this issue" was used as justification to close the evaluation. Unfortunately, however, in this event, the controls that were in place were not used.

In summary, previous External Operating Experience identified a condition that existed at the site, however, no actions were taken to challenge site equipment and further safeguard against incidence. 1 Evaluation Methodology & Analysis Techniques An Event & Causal Factor Chart (E&CF). (Attachment A) was used to initially construct a timeline of events that occurred on April 9h, 2004, during steam generator nozzle dam installation. The timeline was used to determine actions that were considered inappropriate for the tasks being performed, which then required further analysis to determine a root and contributing causes. This timeline of events, or Event and Causal Factor Chart, identified more than 20 inappropriate actions during installation of steam generator nozzle dams. Those inappropriate actions were then used as problem statements and analyzed for cause using Tap Root methodology.

A Failure Analysis (Attachment B) was used to identify the underlying cause of the inappropriate actions that were identified on the E&CF Chart. Six common themes were identified for the inappropriate actions, with charting identifying the most common theme throughout the events.

Supervisory Oversight had the highest number of "inputs" aqd was identified as the cause of at least nine (9) of the inappropriate actions that occurred. Supervisions policies, administrative controls, and the use of corrective action were consistently not strict enbugh, not utilized, or not accountable for inappropriate actions that led to poor work practices and safety anomalies during a high-risk evolution. When asked "Why", management response was that they were focused on other elements such as the Radiological aspects of the job (Radiation Protection Management),

while Program Engineering Management stated that they did not recognize the task as being high-risk and therefore did not apply the necessary oversight. Considered to be Significant Contributors to the events was Work Direction and Procedures. Supporting those facts were:

Inadequate scheduling of training, poor worker selection, and inadequacies in pre-job briefings and walkthroughs. Also, procedures were not used, not followed or situations were not covered, and therefore significantly contributed to the unacceptable events that took place during planning and execution of nozzle dam installation.

A Barrier Analysis was performed (Attachment C) to determine what barriers were in place at

• the time of the event, and determine any failures that may have occurred in those established barriers. Six barriers were identified that, if utilized, would have prevented the inappropriate actions that took place during the steam generator nozzle dam installation. Barriers included:

Work Planning and Associated Outage Management Procedures, Training for nozzle dam installation/removal, Nuclear Oversight observations, Operating Experience, Radiation Protection Procedures, and Human Error Reduction Tools. Each failed barrier, independently, would have impacted performance, however, when all six in combination failed, it led to the 10

safety anomalies and inadequate work practices that were experienced during nozzle dam installation. The Analysis concludes that organizational effectiveness issues exist, as all barriers that were put in place for successful task performance, failed to be utilized to the extent necessary to prevent incident. This analysis then led to a Task Analysis of the process in place to plan this project, that should have included all barriers in it's performance.

The Task Analysis of the Outage Project Plan for this task (Attachment D) identified that a work plan (as directed to be developed by a planner) per NP 10.2.1, Outage Management and NP 10.2.6, Work Order Processing, was not appropriately developed and was subsequently accepted as adequate for performance. A Previous Action Request (CAP 31950) from a Benchmarking Trip in April of 2003, and its associated activities determined that Point Beach would need to develop a change management plan (CE11434) for ensuring personnel are prepared to properly install nozzle dams, however a project plan based on the needs of the contractor procedure was developed and approved instead (OTH 29264,29265). The Analysis further identified that the Outage Management document did not define a process for determining adequacy and acceptability of a work plan, methods or processes to accomplish the responsibilities detailed in I

the document, or references to perform procedures to assist in work plan development.

Task Analysis of HPIP 4.58 - Issuance of Respiratory Equipment (Attachment E) was also performed to identify the tasks associated with the use of bubble hoods, as was used during the events on April 9th, 2004, during nozzle dam installation. This Analysis identified that Radiation Protection personnel did not perform the steps outlined in the procedure, thereby bringing into question whether or not personnel were appropriated dressed and prepared for work in a bubble suit. This procedure directed verification of air line testing, setup, and on/off requirements (which would have confirmed or disputed worker concerns of "not enough air"), bubble hood and equipment condition, issuance and signature of the person dressed in the bubble suit (which would/could have confirmed or disputed workers acceptance of conditions), contamination cautions, taping of connections (which was not performed initially, or on the quick disconnect/reconnect issue), and air pressure requirements which were violated when pressure was increased beyond 28psig up to 60. This procedure is part of RP Training, and is considered a qualified task, however, the procedure was not used as required.

Data Analysis Summary In summary, data analyses performed for this root cause evaluation identified more than 20 inappropriate actions throughout the planning and execution of a high-risk outage task. Those inappropriate actions, once identified, then required a separate causal analysis. This causal analysis identified six common areas of concern. Those areas were Training, Work Direction, Supervisory Oversight, Procedures, Communications, and Human Engineering. Of those six areas of concern, the most inputs of the inappropriate actions directed cause to Supervision that ultimately approved an inadequate, narrowly focused work plan that wasn't appropriately overseen. Further process analysis identified weaknesses within work planning that contributed to the inadequately prepared outage work plan, however barriers in procedures, training, Nuclear Oversight, previous Operating Experience, and Human Performance tools were in place throughout planning and execution, but were not utilized.

The major contributors to the events were identified to be poor work direction and planning, and the non-use or not following of procedures for a high-risk evolution. Human engineering was a major contributor to the event, however, due to the nature of the evolution, much of the contributors must be considered in the planning, and cannot be changed (cramped and high 11

_ea radiation environment, etc.). Training and Communications contributed on two (2) occasions each, but were not considered major contributors to the event.

Data Analyses through Task Analysis of Radiation Protection and Work Planning and Outage Management procedures further identified that procedures were not followed or did not include checks and balances to promote success.

Failure Mode Identification The likelihood of Failure Modes occurs in a typical order. For the issues evaluated for this root cause, Organization & Management Failures occurred in two (2) Functional areas (F2 & F6) first, and were followed by two (2) Cultural Failures (C2 &C3). In addition, Human Performance Failures started in Attention (A9), followed by Judgment (J7), and then Knowledge Failures (K2 & K4). Supporting evidence is as follows:

F2 Organizational & Management I FUNCTIONAL - Inadequate Communication among Organizations Throughout the planning and execution of steam generator nozzle dam installation activities, there was a lack of defined interface requirements, expectations and responsibilities. Nuclear Oversight had identified weaknesses in briefings - neither they nor Radiation Protection followed up with concerns.21 In addition, upon identification of several breathing air issues, proper notification was not made, and notification that was made, was informal 1,1,236 Included in the improper notification was the fact that the project leader was not formally made aware of contamination events of the Scientech workers other than by an e-mail notification, and had previously not been.personally involved with the breathing air issues of those same workers. 3 0 12 IF6 I

Organizational & Management FUNCTIONAL - Inadequate Program Management

• Line Management was unfamiliar with the process that drove the requirements associated with breathing air pressures 2 6 , as well as the requirements associated with work planning, risk assessment, prejob briefings, and Radiation Protection procedures' 2'" 2 2629 30 36 40 . In addition, RP Outage JITT for air line testing, as well as Task Analysis for the air line system had been post-poned or cancelled citing time constraints, and therefore impacted successful performance of breathing air use in this event. 43 44 This ultimately led to an inadequate oversight of critical work processes to ensure they functioned smoothly and effectively. This results in program degradation over time or increased problems within those processes.

C2 Organizational & Management I CULTURAL -Inadequate Teamwork

• Interactions and information was informal and, at times, not tracked. Not all occurrences of breathing air problems were captured and identified in a formal manner, nor were activities stopped or communicated effectively to determine causes and preclude further incident. 9.15.16,2 In addition, interviews determined that the 12

-i separate groups involved in the project were each focused on their own specific aspects of nozzle dam installation/removal, without consideration of the impacts each one would have on the other.12' 27 C3 Organizational & Management CULTURAL - Inadequate Knowledge

) (

• -Line Management and personnel were unfamiliar with the process that drove th requirements associated with breathing air pressures 26 , as well as the requirement associated with work planning risk assessment, prejob briefings, and Radiatio Protection procedures' 23.9.21.269.23036.40 This lead to a work force that proceeded in Knowledge Based performance mode.

A9 Human Performance I I ATTENTIONAL- Time & Schedule Pressure I

• The nozzle dam installation evolution was identified as a Critical Path/Near Critica

. i Path Activity due to be completed that day, with indication on the Outage Status Report that the schedule was 9 hours1.041667e-4 days <br />0.0025 hours <br />1.488095e-5 weeks <br />3.4245e-6 months <br /> behind.4 ' Work on nozzle dam installation began in the early morning hours of the Friday (04/09) of Easter weekend, with work commencing at 0417, with approximately eleven hours already into the 12-hour shift for the work group.15 Workers had also expressed the fact that they held tickets for weekend travel home for the holiday, and they anticipated completion of the assigned task that shift.12.26 27 No actions were taken to heighten the level of awareness necessary for success, due to the time and schedule pressure that was evident, though not admitted as a contributor.

J7 Human Performance I I JUDGMENT- Shortcuts Taken

• 1) Task analysis identified that a work plan was not developed in accordance with NP 10.2.6, Work Order Processing, or NP 10.2.1, Outage Management. This detailed plan was necessary to promote teamwork among the various work groups involved with the task, identify the consequences associated with an IPTE task, and ultimately identify the critical actions necessary to successful implement the task. 2) HPIP 458, Issuance of Respiratory Equipment, would have identified breathing air pressure requirements, air line connection requirements, and verification of issues relevant to the problems that occurred during nozzle dam installation. 3 This procedure was not referred to or used in preparation and execution of the task, even though it is in a training lesson plan with a qualification associated with it.37 Line Management stated that there are many procedures and that they couldn't know all of them, and that they never read that one.26 3) RP Outage JfTI for air line testing, as well as Task Analysis for the air line system had been post-poned citing time constraints, and therefore impacted successful performance of breathing air use in this event.43 .44 13

I K2 Human Performance I KNOWLEDGE - Unfamiliar or Infrequent Task

(

• The nozzle dam installation evolution is identified as a Critical Path/Near Critical Path Activity that is only performed during Refueling activities. At the most, it gv would be performed every 18 months on one unit, or twice every 18 months for a two-unit facility such as Point Beach. It was successfully performed on the Unit 2

- steam generators in the Outage previously, but had not been performed by the site for years prior to that, making this an infrequently performed task. This work was not appropriately planned or prepared for as an unfamiliar or infrequent task, and awareness was therefore not elevated to a level necessary to avoid incident.

Il K4 lHuman Performance l l KNOWLEDGE -Tunnel Vision -

• Decisions were made without considering all the available information needed to adequately assess the situation. Radiation Protection was focused on Radiological impacts and did not consider the personal safety consequences associated with the task. Programs Engineering personnel were also focused on the task of installing the nozzle dams, and did not consider the personal safety consequences of the task.

There was a loss of the "big picture", which allowed individuals to make decisions without assessing the entire situation. 6 10 ,12.13.21.26,27,2936,37.40,43.44.47 14

Root Causes & Contributing Factors Oversight by Supervisors/Managers during work planning development and task execution didn't assure compliance with procedures and processes, resulting in an inadequate work plan being developed and approved for use.

Supporting details for Supervisory oversight issues: A Failure Analysis w<Attachment B) identified six (6) underlying causes of the inappropriate actions that were taken during the steam generator nozzle dam installation evolution. Supervisory Oversight had the highest number of "inputs" and was identified as the cause of at least nine (9) of the inappropriate actions that occurred. Adequate oversight by supervision could have identified an unacceptable work plan, a/ inadequate briefing, and additionally could have investigated identified low air pressure issues,

/ '5 prevented procedure violations or non-use, and could have prevented closure of inadequately 14 assessed OE CAP actions.

Supporting details for inadequate work plan: A work order to perform nozzle dam i n/removal was generated in December of 2003. This work order was written as a Priority 5, ype "Z' - Elective Maintenance activity, and categorized with a risk of "H - MU Multiple Risks". According to Planning personnel at Point Beach, this "H-MWJ"would have

• generated PBF-9812, Categorization and Mitigation of Risk (associated with NP 1.1.7, Managing Work Activity Risk). This form details that a risk category of High, as was the case or this work order, would have had to consider 12 different compensatory actions which ncluded, but were not limited to, a complete look-ahead plan, utilization of high risk pre-job riefing process, FLS and Manager attendance and conduct of pre-job briefing to ensure adequacy, critical step identification, etc. This.form, along with the associated PBF forms that would have been generated upon initiation, were not located and the evaluation did not identify that any mitigation or assessment of risk was performed. This detailed plan was necessary to promote teamwork among the various work groups involved with the task, identify the consequences associated with an IPTE task, and ultimately identify the critical actions necessary to successful implement the task.

Significant Contributing Factor #1: Work Order Processing per NP 10.2.4 and Outage Management Planning per NP 10.2. does not include logic ties (IF this, THEN that) to drive use of appropriate procedures during work plan development.

Supporting details: In accordance with Point Beach Nuclear Procedure NP 10.2.4, Work Order Processing and AM-3-15, Work Control Manual and Pont Beach forms associated pith work planning, a work plan by a planner is required for all Type "C" work orders (4.15.6). This would include an identification of the support requirements, activity risk, RWP, tools, testing, safety precautions, equipment ventilation, safety evaluation, and Operating Experience, to name a few.

This work order, however, was labeled a type '2" work order, and no project plan with inclusion or considerations of all elements was performed. This led to the evaluation questioning the process links associated with work order activity type in CHAMPS. This evaluation identified that the work order process does not identify procedures or references that need to be performed or referred to when planning a work package. Further, a Task Analysis (Attachment D) identified that the Outage Management document did not define a process for determining adequacy and acceptability of a work plan, methods or processes to accomplish the responsibilities detailed in the document, or references to perform procedures to assist in work plan development.

15

, Significant Contributing Factor #2: Program Engineering personnel and Radiation Protection personnel did not use and/or follow Work Planning, Risk Assessment, Briefing, or Radiation Protection procedures in preparing for and during execution of the steam generator nozzle dam project.

Supporting details: During documentation and procedure reviews for the evaluation, the work plan from inception to completion did not include nor reveal that a risk assessment per NP 1.1.7, and its associated PBF documents were utilized.21 2942 This Risk Assessment would have Le developed the heightened awareness that was necessary for this infrequently performed, high risk ,

task. In following the risk assessment, a briefing would have resulted in accordance with the /

assessment procedure, however, a briefing in accordance with the proper procedures was not' I

performed, and was deemed as inadequate by Nuclear Oversight during a Rapid Assessment. 21 -29 40 Radiation Protection procedures were not completed or were violated when!

bubble hoods were not issued in accordance with HPIP 4.58 and its associated PBF forms,'

regulator pressure was increased beyond procedural limits without a temporary procedure change performned, and nasal smears were not conducted as directed by the Personnel Contamination Event Report form PBF-4039a.2,15 26338 _~1J Significant Contributing Factor #3: Training for steam generator nozzle dam installation 5i was not adequate to identify the error-likely situations that existed upon the start of work.

Supporting details: Interviews and documentation reveals that training for steam generator a_

nozzle dam installation and removal was limited due to scheduling conflicts and monetary constraints. 27 Training limitations included the fact that Scientech jumpers practiced nozzle dam installation in street clothes. 27 This practice did not 1) address dress requirement issues that Scientech workers expressed once dressed for the evolution by Radiation Protection, 2) identify air pressure issues that were experienced by workers once full dress-out was complete, 3 identify the air line connection failure possibility that was experienced by two (2) workers on the platform, nor did it 4) identify the issues surrounding two individuals requiring to be "wet down" to assist them in gaining access to the steam generator bowl - one specifically who was too large to easily fit in the manway hole. 1,12,15,1626.27 Significant Contributing Factor #4: Communications to the OCC of safety significant' events was not delivered and Nuclear Oversight identification of an inadequate briefing was not delivered in a timely or effective manner.

Supporting details: Nuclear Oversight detailed in a Rapid Assessment Report, the fact that the IPTE briefing for nozzle dam installation was inadequate for various reasons. 21 Included, was the

-J fact that the briefing was not in accordance with procedural requirements. This information was not recorded in an Action Request, nor .was it brought to the attention of Management for action beyond the discussions that took place between the Assessor and the individual performing the brief. 2 1 Communications break-downs also existed between Radiation Protection, the nozzle dam project leaders, and the Outage Control Center, as evidenced by the fact that the OCC was only made aware of the first breathing air line issue, and that notification was logged after the second air line issue had already been experienced. No notification was made after first communication with the OCC.. 12 -.I. 16627 Additionally, project leaders attention was focused on the nozzle dam installation process itself, not the air line problems experienced by the workers, and therefore were not directly made aware of issues concerning the events. 12 27 16

Significant Contributing Factor #5: Previous External Operating Experience on failures of quick-disconnect fittings was not adequately used to correct the similar failure mechanisms that existed on the equipment utilized at Point Beach.

Supporting details: Several PBNP evaluations of external Operating Events (OE) were performed regarding loss of supplied breathing air due to separation of supplied air line quick disconnect fittings. OE031454- 6/19/2003 -

Subject:

Worker lost air to his bubble-hood when one of the fittings unexpectedly became disconnected. The Point Beach evaluation noted that Point Beach uses quick disconnect fittings of a different manufacturer, and that the fittings are taped as further safe guard. Although the quick disconnect fittings made by the two different manufacturer's employed similar mechanisms and action for connecting/disconnecting the fittings, the evaluation was closed with no further action. Action should have been taken to challenge the connections used at Point Beach, to further justify closure without action.

0E048685- 8/25/2003 -

Subject:

Fittings became separated and an air line hose was accidentally cut by a co-worker. The Point Beach evaluation of this OE addressed the accidental cutting of the hose but did not address the separation of the quick disconnect fittings. The OE was closed by the activity performer with no further action other than stating that Point Beach RP technicians are trained to cut a person out of a bubble hood if they experience air loss.

Review and Approval by Supervision or Quality Check of this activity should have identified that all issues were not addressed in the Activity Completed section. OE010321- 10/10/2003 -

Subject:

Worker experienced a loss of air supply to an air hood because the quick disconnect fittings separated. The Point Beach evaluation of this OE "determined that the procedures and controls at Point Beach are adequate to minimize susceptibility to this event". A statement that "HPIP 4.51.3 have controls in place to address this issue" was used as justification to close the evaluation. Unfortunately, however, in this event, the controls that were in place were not used.

din summary, previous External Operating Experience identified a condition that existed at the site, however, no actions were taken to challenge site equipment and further safeguard against

,incidence.

Corrective Actions Corrective Actions to Restore (broke - fix)

• CA #1 Conduct a documented briefing for Engineering on safety anomalies and poor work practices associated with this event - inappropriate actions taken on the part of their group, and their responsibility for assuring that their actions are corrected. (This will address the immediate needs associated with the Root Cause and Significant Contributing Factors 1-4, and Failure Mode F2)

(Responsible Group: Programs Engineering - Gary Sherwood, Priority 2, 60-day due date)

• CA #2 Conduct a documented Briefing for Radiation Protection on the safety anomalies and poor work practices associated with this event - inappropriate actions taken on the part of their group, and their responsibility for assuring that their actions are corrected. (This will address the immediate needs of the Root Cause, Significant Contributing Factors1-4, and FailureMode F2) 17

(Responsible Group: Radiation Protection - Stu Thomas, Priority 2, 60-day due date) o CA #3 Develop Lesson Plan HPC-04-LP203, Nozzle Dam Just-in-time Training, and implement Supervisory oversight corrective actions established by nozzle dam removal project plan (See Attachment I, second halo. (This will address Significant Contributing Factor #2, 3, & Failure Mode C2 as well as the immediate needs associated with the Root Cause and Failure Modes F6 and C3)

(COMPLETE - Approved 05/11/04)

• CA #4 Develop Communications Protocol and include in the work plan for nozzle dam removal. (This will address Signifcant Contributing Factor#4 and FailureMode F2)

(COMPLETE: Programs Engineering - 05/22/04)

• CA #5 Prepare Expectations and Brief personnel on the adequate use and action to be taken for External Operating Experience that is assessed by Radiation Protection personnel. (This will address Significant Contributing Factor#5)

(Responsible Group: Radiation'Protection - Stu Thomas, Priority 2, 60-day due date) o CA #6 Programs Engineering CRC perform a Task Analysis to determine personnel knowledge and training required to successfully lead and plan major projects. (This will address the long-term needs associated with the Root Cause and FailureModes C3 and J7)

(Responsible Group: Programs Engineering - Gary Sherwood, Priority 2, 60-day due date)

• CA #7 Conduct a documented briefing of Nuclear Oversight personnel on the safety anomalies and poor work practices associated with this event -

inappropriate actions taken on the part of their group (a pre-job briefing was conducted, and was determined by NOS to be inadequate, however, no other action other than a Rapid Assessment Report was documented - no CAP, etc.). (7This will address Significant Contributing Factor #5, and Failure Mode F2)

(Responsible Group: Nuclear Oversight - Mike Holzmann, Priority 2, 15-day due date) 18

Interim Corrective Actions (mitigation)

• CA #8 Plant Stand-down or 'Time Out" conducted on 04/09/04 (Excellence through Error Prevention). (This was to address the immediate site needs at the time of the event.)

(COMPLETE - Conducted 04/09/04)

• CA #9 Programs Engineering conduct Nozzle dam Lessons Learned meeting prior to nozzle dam removal. (This was to address the immediate Program Engineering Department needs prior to planning the nozzle dam removal evolution.)

(COMPLETE - Conducted week of 04/26/04 by Gary Sherwood) -

CA #10 Radiation Protection brought in an independent team of NMC personnel to review procedures and processes for use of supplied air as breathing air and used their input to: update procedures, change out and replace nitrogen bottle back-up with certified Grade D breathing air, replace all bubble hoods, airlines and Snap-tite fittings to new CEJN type fittings.

(This addressedSignificant ContributingFactor#5 and FailureMode F6.)

(COMPLETE - Conducted the month of April, 2004) e(j""

• CA #11 Radiation Protection conducted Just in Time information sharing before restart of the use of bubble hoods. (This partly addressed the Root Cause, Significant ContributingFactor#3, and FailureMode F6 and C3.)

(COMPLETE - Radiation Protection - 04/23/04)

• CA #12 - Mock-up training for nozzle dam removal included a review of procedure requirements, bubble hood issuance requirements, and manifold pressure requirements. (This partly addressed the Root Cause, Significant ContributingFactor#3, and FailureMode F6 and C3.)

(COMPLETE - During mock-up training sessions during 05/20-21/2004)

Corrective Actio ns to Prevent Recurrence (CATPRs)

CAI(PR #1 Develop a nozzle dam removal work plan in accordance with NP 0

10.2.1, Outage Management and NP 10.2.6, Work Order Processing, which also includes Supervisory independent approval. (This will address the 1*I" immediate needs associated with the Root Cause, and Failure Modes A9, J7, K2, and K4)

(COMPLETE: Programs Engineering - 05/18/04) 19

I;.

CATPR #2 Develop a procedure for nozzle dam installation and removal that incorporates lessons learned from this event, supervisory and management oversight requirements and stop work criteria, communications protocol, and external Operating Experience. (This will address the long-term needs associatedwith the Root Cause, Significant ContributingFactors#2, 4, 5, and FailureModes F6and K2.)

(Responsible Group - Programs Engineering - Gary Sherwood, Priority 2, Due date 11/15/04)

CATPR #3 Develop, within NP 10.2.1, Outage Management, and NP 10.2.6, Work Order Processing, a process to determine when HIT teams or Project Managers should be assigned, and include logic ties (IF this, THEN that) to drive use of appropriate procedures during work plan development. (This will address the long-tern needs of the Root Cause and Significant ContributingFactor#1)

(Responsible Group: Production Planning - Ron Davenport, Priority 2, 120-day due date)

Other Corrective Actions o OTH #1 Create an "It Can Happen Here" article for distribution to plant personnel. (This is an additionalactionforsite personnel infonnation.)

(Responsible Group: Assessment - Pat Russell, Priority 2, 90-day due date) o 0TH #2 Develop Industry OE on the event. (Required Action associated with an event of this type.)

(Responsible Group: Assessment - Pat Russell, Priority 2, 50-day due date (from the time of the event) 3 0TH #3 - CA057320 Perform INPO Nuclear Safety Culture Assessment to identify gaps and formulate corrective actions to improve PBNP performance.

(This will addressthe Extent of Condition issues identified in the evaluation.)

(Responsible Group: Assessment - Pat Russell, Priority 3, Due date

.09/01/04) i o OTH #4 - Develop and issue a site wide communication on the purpose of NP 1.1.7, Managing Work Activity Risk, and when to utilize it. (This will address the site's needs associatedwith performance of NP 1.1.7 during work planning.)

(Responsible Group - PPG - Ron Davenport, Priority 2, 7-day due date) o OTH #5 - Work Week Coordinator review all existing High Risk work orders scheduled within the next five weeks for compliance with NP 1.1.7 and implement a review of High Risk work orders for compliance with NP 1.1.7 at 20

the appropriate E- meeting. (This will address the short-term needs to ensure compliance with NP 1.1.7.)

(Responsible Group - PPG - Ron Davenport, Priority 2, 14-day due date) o OTH #6 - Take completed CRC Task Analysis results from CA #6 to the TOC to identify impacts on other work groups. (Th7is will address the long-term needs associatedwith personnelwork planning issues.)

(Responsible Group: Training - Chuck Sizemore, Priority 3, 120-day due date)

Effectiveness Reviews

• EFR #1 Perform an Effectiveness Review of CATPR #1 per Nuclear Management Company Root Cause Evaluation Guidelines immediately following U1R28 Refueling Outage.

(Responsible Group: Programs Engineering, Priority 3, Completion Due Date: 90-days)

• EFR #2 Perform an Effectiveness Review of CATPR #2 per Nuclear Management Company Root Cause Evaluation Guidelines.

(Responsible Group: Programs Engineering, Priority 3, Completion Due Date: 90-days after completion of CATPR #2)

• EFR #3 Perform an Effectiveness Review of CATPR #3 per Nuclear Management Company Root Cause Evaluation Guidelines.

(Responsible Group: Production Planning, Priority 3, Completion Due Date:

6-months after completion of CATPR #3) 21

References

1. Breathing Air Supply Issues (Kari DenHerder)

2. HPIP 4.51.3 - Airline Respiratory Equipment PBF4077d - Respiratory Protection Filter/Manifold Inspection & Maintenance Record

3. HPIP 4.58 - Issuance of Respiratory Equipment & PBF-2234 - Respirator Issue Record

4. Recommended Actions (Immediate Action Team Recommendations)

5. OE - Steam Gen Nozzle dam and cover installation & removal - INPO OE

6. Evaluation of Point Beach Nuclear Plant Safety Conscious Work Environment (Aldo Capristo)

7. IRMP 9391 - Connection of Unit/Nozzle Dam Control Console Remote Alarms to 1C20

8. PBNP Vent Path Timeline (from Vent Path RCE)

9. CAPs: 55527 - U1R28 Nozzle Dam Installation Supplied Breathing Air Problems 55565 - Facial Contamination Events without Nasal Swabs Being Taken 55587 - S/G Nozzle Dam Installation Dose Exceeded Estimate 55595 - Air supply to Bubble Hoods not within procedure limits 55645 - Bubble hoods not issued IAW HPIP 4.58

10. OE13365 - Two Separate Incidents where Loss of Breathing Air to Air-supplied Respirators (bubble hoods) occurred.

OE16239 - Separation of Air Line Coupling on Supplied Air Hood OE16908 - Separation of Air Line Coupling on Supplied Air Hood - related to OE16239 OE16368 - Airline Breathing Hood Fitting Disconnects Unexpectedly

11. Radiation Protection Outage Scope (U2R26) and Contractor Current Events Lesson Plans

12. Interviews (Aldo Capristo, Kristin Zastrow, Tom Klesper)

13. Point Beach OE Assessments of External OE CAP 46245 (1999)- OE10197 - Loss of Breathing Air During Steam Generator Nozzle Dam Removal OE10321 (2003) - Separation of Air Line Coupling on Supplied Air Hood OE31454 (2003) - Airline Breathing Hood Fitting Disconnects Unexpectedly OE48685 (2002) - Two Separate Incidents where Loss of Breathing Air to Air-supplied Respirators

14. Supplied Air Respirators - Technical Information from Nuclear Power Outfitters

15. Containment Log

16. Operations Log

17. Supporting Details (from Vent Path RCE)

18. Radiation Work Permit 04-141 - Nozzle Dam Install/Remove

19. E-mail correspondence (Aldo Capristo & Kristin Zastrow)

20. PBF-9157 - FME Material Control Log

21. Nuclear Oversight Rapid Trending Assessment Daily Report for 04/08/04 & Follow-up questions from Dennis Hettick to Brad Cole (Assessor)

22. Timetable from Vent Path RCE

23. NP 8.4.9 - Hose Control, Attachment A

-- 24. List of involved individuals (Tom Klesper, Aldo Capristo, Mark PeroutkaO)4

25. Site Communication effective 04/11 (Immediate actions)

26. Interviews (Dan Craft)

27. Interviews (Kari DenHerder)

28. PBF-4077d - Respiratory Protection Filter/Manifold Inspection & Maintenance Record PBF-4107 - Testing to Ensure Breathing Air Standards

29. PBF-4195a - Level 3 Pre-job ALARA review 22

s . s QF-1060-02 (FP-T-SAT-60) - Documentation of Information Sharing (This was considered the IPTE Brief)

30. PBF-4039a - Personnel Contamination Event Reports

31. PBF-8013, Test Gauge Calibration

32. 83A7564 - Steam Generator Nozzle Dam Installation and Removal, Test, Operation and Maintenance Manual - Scientech Procedure

33. Personal drawings of air supply hook-ups (Al Reiff & Pete VanLaarhoven)

34. NP 1.9.4 - Confined Spaces Procedure

35. Radiation Protection Qualification Matrix

36. Event "Apparent Cause Evaluation" performed by Immediate Action Team (Kari VanDenherder)

37. HPI-02-LP003, Respiratory Protection, Lesson Plan Requirements

38. PBF-4021, Radiological Surveys

39. Time Out, April 9,2004 -Point Beach Excellence Through Error Prevention

40. NP 1.1.7 - Managing work Activity Risk NP 1.2.5 - Special Test Procedures NP 1.2.6 - Infrequently Performed Tests or Evolutions (IPTEs)

NP 1.6.10 - Pre-and Post-Job Briefs PBF-4194a - Pre-Job Briefing Checklist PBF-9175 - Job Walkdown Checklist PBF-9175a - Job Walkdown Facilities Checklist PBF-9205 - High Risk Work Pre-Job Briefing Checklist PBF-9217& 9218 - Pre-Job Brief Checklist PBF-9811 - Look Ahead Process Planning Form

41. Outage Status Reports and U1R28 Today Articles

42. PBF-0039 - Confined Space Entry Permits

43. Internal Correspondence - Minutes for RP CRC Minutes

44. CAP 35015 - RP Outage JITT in jeopardy (too late to begin development of a lesson plan to support Airline training CAP 52203 - Implementation of Grade D aero test equipment inadequate OTH 11943 - Identify RPTs that maintain qualification of the Grade D Air Testing RFT 12018 - RP evaluate the post training feedback for topic selection OTH 12017 - RP Evaluate station improvement suggestions from 11/03

45. ANSI Z88.2-1992

46. HPIP 4.56 - Testing Supplied Air for Air-Line Respiratory Equipment

47. Job File 131 - Steam Generator Primary Manway Removal and Installation Job File 132 - Containment Setup for Steam Generator Work Job File 133 - Survey Schedule During Steam Generator Work Job File 134 - Requirements for Steam Generator Work Using Full Face Respirators Job File 135 - Requirements for S/G Work Using Supplied Air Respirator Hood

48. NP 10.2.1, Outage Management NP 10.2.2, Scheduling Planning & Implementing On-Line Work NP 10.2.4, Work Order Processing NP 10.3.6, Outage Safety Review and Safety Assessment AM 3-15, Work Control Manual

49. U1R28 Outage Risk Plan - PPG Outage Management

50. CAP 31950, Nozzle Dam Benchmarking Trip CE 11434 OTH 29264, 29265 23

Attachments A. Event & Limited Causal Factor Chart B. Failure Analysis & Conclusions C. Barrier Analysis D. Task Analysis of Planning Process for this Task E. Task Analysis of HPIP 4.58 - Respirator Issuance F. Operating Experience Analysis G. Quick Response Team "Apparent Cause Evaluation" H. Nozzle Dam Lessons Learned I. RP Immediate Actions & Actions to Address Supervisory Root Cause Issues 24

'I Attachment A - Event & Causail Factor Chart anner use0031204 tanrP11.

i otue 1320. nte0t07,4170 ceneh Breathing air hose tnadequate, SIG nozzle Nri orreotlopsi ddot and associated forms to Conducts training for a manifolds set dam IPTE brief brie t or repor to site seel l a( up on platforms (1) onducted. (8)

Odnot follow In adeuate Shdl IalIon b rieling ommunieatons, Old not follow conilicts (27) procedures J not timely J siaeurn precedues reea monetary lmitatiOnsg27 0428 -Ist lumperenters 'B' NoIvestigation of tack oa air by station personnel.

0417. SIG Jumpers arrive at Ut B- S/C to perform cold leg nozzle dam Installation. (1 5)

Jumper plastic suit wet-down per Sclentech request (26) 5IG cold leg to install nozzle dam. Jumpers size makes entry difficult.

Jumper requires assistance. (15) 0429-Jumper distressed, leaves bowtl air hood Is cut off. (IS /

0 LaCk of questing aeting a pastic suIt Preconceived mtndset attitude. notadrese I stationorcontractor byjumper regarding air proceures.

Uindset developed about Individual complaining about lack

Attachment A - Event & Cansal Factor Chart (continued) 1Spessure j to comodate jumper 0440.2nd jumper makes 2 Noinvestigatio h attempts to enter the SIG bowl jumpers are reponing Was not able to make entry towai pressr Into the SIG bowl. (I 5)

Proc tedmreulre Culn hihl not followed) c hood A Iontaminated R Lack of question ng causes facilat ) atttud contamInation. Airpressure contrary raiedM fo a procedure 0444-2nd lumpe r .ageI attempts and gains enrron Into SIG bowl. (15) 0445 -Jum per lost ai.on JuprcniusJumper alter realIzn air upply cuf. 0 (2 leaves bowl and bubble hood Is cut off.

umper bo contam nted u diconnect fitting aIlure (I ) (

vLc ofosaety Cuin highly oor ument contaminated Po qlmn hood causes facial design containtion Ovr onidnc Inadequate corrective action on previously ientIfie OE 26

U4*4.Htr ano NMm. Frofect Leader confer and agree to 0452.RPM repons to OCC allow 3rd jumper complete that I S/G jumper lost instatlation. Air supply 0455.3rd jumper enters S/G breathing ir. (OCC 1og) pressure Is verified good and leaves at 04:56 without 4 and air connection Is Incident. (15) chalenged. (15) oor project communication 0652-Commence removal of A S/G H/L 0537-Jumper Instalts A S/G Hi/L nozzle dam pleces.

nozzie dam. Jumper air hots fitting Installation of B SIG HtL

'connects and Is imrnmsdlal!st> nozzle dams continues.

\connected. (16 (16) commun att I

2neffective lokIng\ betwen fbidand mechanism (design) on quick disconnect correctve action on prevtously Identified 27

Industrial Safety Issues & Poor lWork Practices During Nozzle Dam Installation - RCE253 Attachment B - Failure Analysis & Conclusions Inappropriate Action *1 Inapproprlate Action *2 Inappropriate Action r3 Inapproptiate Acrbon 94 inappropriate Action 95 & II Inapproprlote Action 56 Inappropiatte Action ?7 Inaprprioea Action 5a 13 NWoaepercuh dw n do ca codsthg n e .adrssSo o le~

- e, 5.0 ,~~r \m..~ , , **

TRAINING: WORK DIRECTION: SUPERVISORY OVERStGHT: PROCEDURES:

OderolhndiogNeeds Prepaertlon AWorker StanderdaPottcyhAdmh NoUSedorFood COMMUNICATIONS HUMAN ENGINEERING:

tmprement 7 Selection Needs CSntrodsNeeds tAdnprNvemet NndWort NotstdAelFooloNo Environment ImpAvement toAreas: Are ropt vem Areas:

Areas: A 7 ArFISd Ar8as 2 Areas: Arias: Areas: No Used e6 f Ares Aras Cramped

• Practice Wakrugh Prejob Brie Needs

• Not Followed Not Tely *NotCommoicsted

• Radiation

• Testing *'SchduoliogTrainIng Improvement or* Equipment Guards and

• Wo~r Upselrm[* Accounabillty 'SituationNot Covered By Tools Need hmprovmentt

• Corrective Action_

-... Inepproprlale Action #AI IPrior to Eventq easoe rwab, Intocmdw workr dtloWd aware of I dagN "P ah o Jumper (__ Iw d _, Aesone

_. "0__ confin Mode* l )u d~w

_ouet W Ws A *r bn 2 .a n< a Aeo1 1 Insl net)ab, Achbous.. a 18 hon.. t -9 d Wee..-

- 00 - -Inaprorpdl.t Acto NA2 (Prior 50 Event) inappropriate Action 555II15 inapproprliate Action510 inapproprIate Action112 loapptoprlate Acbmo114 & 17 Inapproptlate Actio t nprpriate

-t Action 519 inapproprIateAction #20 28

Industrial Safery Issues & Poor WiYork Practices During Nozzle Dam Installation - RCE2s3 Attachment C - Barrier Analysis - Safety 1 - FAILED BARRIER Anomolles & Inadequate Work Practices During Rteir Assessment Procedure was not utitirted tsr this ptojec plan. as Management did not Steam Generator Nozzle Dam Installation consider this a high rik evolution.

I 2 - FAILED BARRIER Radleion Protection Human Performance tol *ar o o A lull mock-tp training session was not I

Previous 0E. relative to air proceduars were in piece and used to minimize the chance perlormed citing time and monetary Work Planning & Outage hme disconnections. and were pert Of qualftation tasks ot Incident during tasks, and for RP. These procedures constrain", and theretore did not Management Procedures & Similariies to the equlpment to try and put workers Inot a identify tW error likesy Situastions that used at Point Boach had beon Identified ir tOnenet-up and Rube-Based environment.

Associated PB i limits. issuantce o1 bubble occurred once work started.

previously evatuated by rather than a Knowtedo-1 Radiation Protection and hoods and dress Based environment Workers 3 - FAILED BARRIER tihoe issu were avaiabte requirementa. as woo Os had been awareof the Human Nuclear Oversiht observed the pno-bob briefing or review for this event. directione tot actions to be Performnce Toots thatcoutd that was Conducted. deemed it hiadequale tor taken upon peroonnet apply to work of this nature. various reasons, bat did not stop and address

. i TralnQ on No221e Dam cotrtamnation. the knadequsles, nor did theyraise the Issues IrtapproprltIAction AZ. above end beyond the rapid tsssosm ant report b 2 atinoclated With tili tnsre~aTton -I and the kIdhyidudt in charge of the meeting.

tafed bartir . Se Attachment B 4 - FAILED BARRIER Previous OE relative to SiWtine connection Issues was not part of the OE review for this Nuclear Ovebsbfht Observations event. A latent error existed when no formal actions were taken by Radiation L

Proleclion.and 0E Item s were cosed staling thai procedures and personnel wmre In place and trained to handle the Identified condotons.

OE on Air Line lnappropnate Action Discdo'hections 5 -FAILED BARRIER Radiation Protection personnel. trom management t worker. did not know the 0 procedunrl requirements tot air tine pressure.

Issuance reqeirementa for bubble hoods, or Re4lation Protection nsatl smear equirements upon taclat

-7 $dures contamination.

\ \naproriefe Action *7, 6 - FAILED BARRIER Human Pertormance tool%such as Slop When Unsure. Are You Ready Checklists. SloptThink. etc.

were not uslitzed to preclude Incident, nor wer they discussed with regard to personnel safety Issues These procedure wouid have Adequata training tor the lnappro1 opriaeAction Human PertbnTance Toolst haI Couid resuti from this task.

heightened the tvel 0o nozzle damens ltatationt Nuclear Overskiht provides Inpa.oi0,13. 15.20 Ouestioning Aqtude Swtesrias for consequt prnces romoval could have identified on-sit assessment end 1

astoclated with the la$ k. as the enor-tlikoy situstions that overnight opportunities which

,el as drive the protect 'letader eistled upon the start ot work, can Impact work performsnce.

Io pertorm the appropt at (ersprs C: size of worker end This group has the abitity (as procedures (examples: rtk need t be wet down to gain we Os any station personnei) _ Salety anaimois A inadequate work assessmnent, took ane ad access to the bowl. breathing to top wor ore bttthg practices process. high rlik brieIIng IUsas air binedlsconnections. based on their assessment Inappropriate Action *t4, S.

wtrakdowns. IPTE. el Ce.) at I 6. 11,12 29

Industrial Safety Issues & Poor Work Practices During Nozzle Dam Installation - RCE253 Attachment D - Task Analysis of Outage Project Plan for This Task Planning Process for this Project as Outlined in 10.2.1, NPIO2s.1utsgeManagement Outage Planning, Scheduling and Management Refueling Outaesobth PBNP 3.0 R.aponsibiwi,: 3.0- Responrishties: 1.2.5- Intil.2. 4.2.11IWCC Reisase of Outage J4 Outage Manager. 3 8.3 -Project Valdaon and Rfk a ely Wosk Packages . Work Group Responibilities Managers and HIT Teems: Successful All,t **m** An.aly Validatn Roeire and Approseal. a supervisor hplemenlalisn of

1) Determine need tor Prolct Vertical reiw of the by the Nuclear or lead person knowledgeable of Suaccesakl Outage Managers and HIT teams for planning and schedule. His-by.- Safety Analylss the rok shta revie outage work Steam Generator m ajor ouage projects or esecutton . h. tine Is to take place Group perNP packages Is accordance with NP Norrt Dam evolutlors. scordance will wtslwrgroups 10.36 10.2.4 . aeer approval lerwarded to InstanatlonfRoemoval

2) Ensur Independent sabyt reSponibhilhee fhhed In Afctsh t and C ... WCC

• they prepare tagouts. Ops Project reviews and risk assessment of Review and Approval by an SRO the outage schedule are for adequacy of te PMT. and Thihwas done . UIR28 Outage Risk performed and that safaty Issues detmlne t wort package may be are adequately renotred Plan . PPO Outgage Management.

however kI staled that *Indhivdual approved and pre-reteased to the aclItb,1ef wit be marked is the work work group for work preparations group bar Chad end detaNed kehdute documents to idenkriy when HP 10.2 D4iroetasrk Actrty Risk as wet

Conclusions:

himsa performance risk analysh as pack ge pbansn e. Is be performed This Analysis of the Planning of has shown a high level of potential bye Panner 4. td.thisphnnrib for error. (see NP 1.1.7) Reference reponskle for all a s ofth w.ork plan. the project, concludes that 1) A 49 This analyss was nol performed. homever. no work pln was developed per and therefore only nuclear safety risk NP 10.2.4 nor doslh Outage Procedure Work Plan per NP 10.2.4, Work j1 was.assessed under the risk psan. direct planning per th procedure prior to this

/ 42.11. a ,hen the r ie I relerenced. Order Processing, was not developed, and 2) that the Outage Management Planning per NP 10.2.1 does not Include Li methods and procedures for individuals to perform a Alternative Path Taken for Nozzle Dam Installation/Remi successful Outage plan, nor does it require checks and ties f(lIT team alety review Documented vork plan by the, project manager was not tndiedual work groups did ormat r, balances for assessing (for peroonnel establluhed to eccomphih the not Identily when human was not p acceptability of an Outage task were not reaponshiOl'ls detaaed In the performance risk analysis work pack shed by the affachments and a HIT would wcm real a high level of devetope plan.

agers defined m not dnsignated and poaentisl error . using NP disclo nro r t for those responsiblities a er required has no a HIT Team HNometods or ref ereoe to perlormanci ahsl(b procdret copihsc Risk 30

/

IndustrialSafety Issues & Poor Work PracticesDuringNozzle Dam Installation - RCE253 Attachment E - Task Analysis of TIPIP 4.58 - Respirator Issuance

/

Procedure Step Description Performed Responsible Person Comments\ >

4.5 Bubble Hood (Radiological Use Only) NA NA NA NOTE: Plant training records were found and detail Bubble hood users are NOT required to have a that the jumpers had fit test to wear the hood, however, they SHALL medical clearance, and have medical clearance and appropriate No one procedurally received the general RP training. YES identified training required.

NOTE: Radiation Protection personnel will perform the steps outlined below and provide assistance to This was not performed individuals using air supplied respiratory as required - eAP equipment as needed. NO RP Organization 55645 initiated 4.5.1 Verify that the air supply to be used with the bubble hood has been tested in accordance with, and met the requirements of HPIP 4.56, Is there a datasheet Testing Supplied Air for Air-line Respiratory associated with this Equipment. Unknown RP Personnel test??

4.5.2 Check the blue supply air hose, Bullard airline filter (if used), manifold and regulated air hoses for proper installation and ensure that all Chicago fittings are lock wired and pinned to prevent accidental separation of the connection.

Verify that the air supply pressure gauge is in current calibration and then turn on the air supply to the manifold (if off). Unknown RP Personnel NOTE: A shelf life of three (3) years is recommended as a safety precaution; however this is NOT mandatory if a visual inspection is made of all components and found to be satisfactory prior to putting the respirators in service. Unknown RP Personnel 4.5.3 Inspect all bubble hoods for material softness, clarity, cracks, and seam strength. Unknown RP Personnel 4.5.4 Place air distribution unit into the hood and snap retainers around the unit if required.

Inspect hood material integrity to ensure that the hood is in oeprating condition. Unknown RP Personnel 4.5.5 Log the issue of the bubble hood on PBF-4234 attached to the appropriate RWP by filling in the Date, writing "Bubblehood' in the Other RP Organization and This was not performed Equipment" column, and filling in the Issuer Respirator User as required - CAP and Wearer Name and Signature columns. NO Signature 55645 initiated CAUTION Take care NOT to coontaminate the supplied air hose quick disconnects. Unknown RP Personnel 4.5.6 Connect the air line fittings and establish air flow to the hood. Unknown RP Personnel NOTE: The following data is only applicable to the Nuclear Power Outfitters hood NIOSH certification. Use of other manufacturer's hood or other air line length requires approval of the Respiratory Protection Coordinator and strict compliance with the appropriate NIOSH hood This was applicable in certification. RP Personnel this case.

4.5.7 Adjust air supply pressure so that air flow is YES - RP Personnel This was set up prior to 31

IndustrialSafety Issues &Poor Work PracticesDuringNozzle Darn Installation- RCE253 Attachment E - Task Analysis of HPIP 4.58 - Respirator Issuance between 6 and 15 cubic feet per minute. For an HOWEVER, the start of this job air line length of 50 feet, a pressure range of 20 See comments however it was noted to 28 psig corresponds to a flow rate range of 6 that after the events, the to 15 cfm. configuration was not as it was left the night before the event, therefore voiding the verification done prior to the event.

4.5.8 Put on the outer protective clothing garment. Unknown RP Personnel 4.5.9 Place hood over the individual's head, setting the lower edge at the shoulders. Tuck the inner plastic cape inside of the outer protective clothing garment. Unknown RP Personnel 4.5.10 Smooth the outer cape against the outside of the upper outer garment. Unknown RP Personnel 4.5.11 Tape outer cape to outer protective clothing. Unknown RP Personnel 4.5.12 Tighten belt over outer cape and connect the air line to the belt. Unknown RP Personnel 4.5.13 Assure adequate freedom of movement by extending and bending the arms. Adjust the hood, if necessary. Run the air line from the belt around the shoulder to the hood. Secure the belt and air line with tape. Unknown RP Personnel 4.5.14 Verify that the pressure is within the required pressure range (20-28 psig), adjust if necessary.

In general, higher flow provides a higher protection factor. Unknown RP Personnel 4.5.15 Document air system PBNP equipment serial numbers and air supply pressure verification on Form PBF-4107a, and periodically check to Was this performed? If insure pressure remains constant. Unknown RP Personnel so, where's PBF-4 07a?

4.5.16 See NP 4.2.32 for standby rescue persons(s) requirements. (B-5) Unknown RP Personnel See NP 4.2.32.

Responsible Individuals: Specific Responsibilities within this procedure Radiation Protection Organization: Issurance of approved respiratory 3.3 protection to qualified individuals Respirator Users: Use equipment in accordance with this procedure and 3.4 training received.

32

IndustrialSafety Issues & Poor Work PracticesDuring Nozzle Damn Installation- RCE253 Attachment F - Operating Experience Analysis - John Peterson/Monticello Analysis of recent PBNP evaluations of external Operating Events (OE) regarding loss of supplied breathing air due to separation of supplied air line quick disconnect fittings.

OE031454- 6/19/2003 This OE involves an incident at the HOPE CREEK station where a worker lost air to his bubble-hood when one of the fittings unexpectedly became disconnected.

The Point Beach evaluation noted that Point Beach uses quick disconnect fittings of a different manufacturer, and that the fittings are taped as further safe guard. Although the quick disconnect fittings made by the two different manufacturer's employed similar mechanisms and action for connecting/disconnecting the fittings, the evaluation was closed with no further action.

OE048685- 8/25/2003 Two separate incidents at CATAWBA station. One incident where fittings became separated and one incident where the air line hose was accidentally cut by a co-worker. The Point Beach evaluation of this OE addressed the accidental cutting of the hose but did not address the separation of the quick disconnect fittings. The OE was closed with no further action because Point Beach RP technicians are trained to cut a person out of a bubble hood if they experience air loss.

OE010321- 10/10/2003 This OE involves an incident at St. Lucie Unit 2 where a worker experienced a loss of air supply to an air hood because the quick disconnect fittings separated.

The Point Beach evaluation of this OE "determined that the procedures and controls at Point Beach are adequate to minimize susceptibility to this event". A statement that "HPIP 4.51.3 have controls in place to address this issue" was used as justification to close the evaluation.

33

IndustrialSafety Issues & PoorWork PracticesDuringNozzle Danz Installation- RCE253 Attachment H - Nozzle Dam Lessons Learned - Gary Sherwood Event

Description:

During the U1R28 steam generator nozzle dam installation evolution (4/9/04), four (4) separate incidents involving personnel breathingair supply occurred. Of thosefour (4) events, one (1) occurrence was reported to the OCC. Thefollowing is a summary of those events:

1) Individual experiences low air pressure to bubble hood.

When authorized to install the cold leg nozzle dams, the RP Technician sprayed down the contractor who was to enter the B Steam Generator Cold Leg with 409 to lubricate him due to his size relative to the manway opening. While in the steam generator bowl, the worker reports low air pressure to bubble hood at numerous points during the nozzle dam installation. The individual exited the steam generator prior to completing nozzle dam installation due to the low air pressure. Upon exit, the RP Technician stationed on the B Steam Generator platform cut the worker out of the bubble hood. At this time, the condition of the worker upon bowl exit is unclear. RP personnel, a Scientech crewmember on the platform, and the contractor liaison interviewed have reported that the individual's bubble hood was inflated and no fogging was noted, indicating that the individual was still getting air. However, when interviewed, the individual in question reported that upon exit, the plastic suit and bubble hood were completely deflated and that he was "sucking plastic."

After being cut out of the bubble hood, the individual left the B Steam Generator platform and rested at the RP desk in containment. After a brief rest, the individual exited containment on his own power with an RP escort. Work on both steam generator platforms was stopped while RP's investigated the event and the OCC was notified of air supply issues. The airline in question was inspected and no problems (i.e. kinks, disconnected fitting, leaks, etc.) were noted.

In an effort to increase the air supply after more air was requested by the other two contract workers on the platform in bubble hoods, RP personnel increased the manifold air pressure to the. B Steam Generator platform workers from the as found setting of 25 psig. When interviewed, workers on the B

/, Steam Generator platform commented that the new setting "felt better," and that the RP Technician reported the manifold was now set-at approximately 64 psig.

The decision to increase the air pressure to workers on B Steam Generator platform was made by the RP

/77. Technician in consultation with his immediate supervisor per interviews of the two. When the workers had requested more air, the RP Technician asked his supervisor if he could increase air pressure to

\ which the supervisor responded to give them as much as they need. HPIP 4.51.3 and HPIP 4.58 limit manifold air pressure to a setting of 20-28 psig when using bubble hood respirators. In interviews, the RP supervisor indicated that he did not know the procedurally directed range at the time. The RP

'Technician stated that he was aware of the setting directed by the procedure, but felt that raising the pressure was acceptable because the workers requested more air and his supervisor authorized him to J J increase it.

2) Individual's air supply line disconnected.

As a result of the first event, described above, a second individual was tasked to finish B Steam Generator Cold Leg nozzle dam installation. This individual was physically larger than the first and had difficulty entering the manway even after being sprayed with 409. According to the individual in 34

IndustrialSafety Issues & Poor Work PracticesDuring Nozzle Dam Installation- RCE253 Attachment H - Nozzle Dam Lessons Learned - Gary Sherwood question and RP personnel stationed on B Steam Generator platform, it took a total of three attempts to successfully enter the steam generator through the manway.

On the first attempt, the individual got stuck at the shoulders. He then exited the manway and was sprayed with additional 409. On the second attempt, the worker got stuck at the waist. He then exited the manway and was again sprayed down with additional 409 to aid in entry. On the third attempt the individual was able to enter the steam generator bowl with some physical aid from personnel outside of the steam generator.

While inside B Steam Generator Cold Leg, the individual realized his air supply line had disconnected.

The individual continued the nozzle dam installation until he estimated he had "three to four good breaths" left. At this time, the individual exited the steam generator. The individual in question reported that his bubble hood was deflated, sucked against his face, and fogged on exit.

Upon exit, the RP Technician stationed on the B Steam Generator platform attempted to reconnect the air supply line. However, due to interference of tape on the fittings, the RP Technician was unable to restore air supply. The RP Technician then immediately cut the worker out of the bubble hood. After being cut out of the bubble hood, the individual left the B Steam Generator platform by his own power.

At this point, a third person was required to complete the B Steam Generator Cold Leg Nozzle Dam installation. The third worker did not experience any difficulties with air supply, and the installation/verification steps were completed.

3) Individual's air supply line disconnected upon attempted bowl entry.

While attempting to enter A Steam Generator manway, worker's airline disconnected. RP Technician stationed on the A Steam Generator platform reported that the fitting hit the manway on entry and disconnected. The worker, who was only halfway into the steam generator, immediately exited the manway and the RP Technician reconnected his air supply line.

At this point, the fittings on the individual's air supply line were taped and reconfigured. Fittings were taped in the horizontal direction on the jumper's back to prevent inadvertent disconnections caused by contact with the manway.

4) Individual's air supply line is damaged/cut.

While working on A Steam Generator platform, a strongback inadvertently cut an individual's air supply line. The strongback was in use to aid nozzle dam jumpers when entering the steam generator manways.

Individuals at the job site reported that the strongback rocked and pinched one worker's air supply line.

The air supply line was leaking air and was taped to control potential spread of contamination on the platform. The RP Technician asked the worker if the air pressure/supply was adequate and the individual replied that it was "okay" several times. Upon completion of his jump, the damaged hose was only used on the platform with an extra emergency hose on standby. As additional entries in the steam generator were completed, the hoses were switched between the worker entering the steam generator and those individuals remaining on the platform.

35

Industrial Safety Issues & PoorWork PracticesDuring Nozzle Damn Installation- RCE253 Attachment H - Nozzle Dam Lessons Learned - Gary Sherwood No personnel injuries occurred as a result of the above events, however the evolution resulted in multiple personnel contaminations. CAP 055527 has been initiated to document the events described above.

As a result of the events described above, plant management requested engineering personnel to complete walkdowns/inspection of the breathing air supply to nozzle dam installation personnel. Two non-conformances were found during the walkdown:

1. The "as found" manifold pressure on B Steam Generator platform was approximately 60 psig. CAP 055595 has been initiated to document this deviation.

2. Air supply to nozzle dam seals and eddy current equipment did not meet requirements of NP 8.4.9.

Procedure requires service air hoses to be yellow or a base color with yellow stripe. Original hoses were red, however yellow tape has been applied to the hose and verified by engineering. CAP 055560 documents this deviation.

Extent of Condition Assessment:

As a part of the initial investigation, a team of independent personnel was assembled. The team completed a review of station logs, work orders, plant procedures, action requests and conducted interviews with individuals involved in the evolution. Individuals interviewed included, but was not limited to, RP personnel, Scientech personnel, Engineering personnel, Safety personnel, and Point Beach station management.

In addition to the procedure violations described above, the team identified one other deviation. Bubble hood respirators used for the nozzle dam installation were not issued in accordance with HPIP 4.58. The procedure requires the issuance to be documented on PBF-4234 and this was not completed. CAP 055645 has been initiated to document this deviation.

Corrective Actions to Prevent Recurrence:

Prior to using bubble hood respirators at the station, a review of HPIP 4.51.3 and 4.58 should be completed.

The procedural limitations on air pressure (20-28 psig) does not give consideration to the number of individuals breathing off of one air station, the length of hose used, or the number of fittings on the hose(s). The bubble hood respirators are shipped with an information sheet containing guidance on required air pressures with consideration to all of these factors. HPIP 4.51.3 and 4.58 should be reviewed against the respirator information sheet and any changes necessary should be incorporated into plant procedures.

Prior to nozzle dam removal, station personnel should investigate new fittings or locking mechanisms for hoses used for breathing air supply. The hoses used during the nozzle dam installation did not have locking fittings.

The fittings could be easily disconnected when it they came in contact with a raised surface (i.e. manway lip).

Some type of locking mechanism or a complete change out of fittings should be incorporated into the nozzle dam removal evolution.

Management/Safety should complete a safety stand down or revision to confined space entry training to reiterate appropriate confined space entry practices. Workers on the B Steam Generator platform were lubricated to aid in entry to the steam generator. All station personnel need to be aware of acceptable confined space entry practices.

36

IndustrialSafety Issues & Poor Work PracticesDuringNozzle Dam Installation- RCE253 Attachment H - Nozzle Dam Lessons Learned - Gary Sherwood Management/Safety should complete a review of the circumstances surrounding this event, including the lubrication and physically pushing a large individual through the steam generator manways into a confined spaced, with regard to compliance with OSHA standards.

When planning/training for nozzle dam removal, consideration should be give to personnel physical size.

Individuals requiring the aid of physical force or lubrication to fit through the steam generator manway should not be allowed to enter the confined space. Oversight of mock-up training may help determine which individuals should not be allowed to enter the steam generator.

Additional management oversight of high-risk evolutions should be considered. When interviewed, personnel from each of the departments involved in the nozzle dam installation could not identify a single point of contact.

Roles and responsibilities of the individuals involved in the nozzle dam installation were unclear to a majority of the staff.

Apparent Cause Statement (Who, What, Why):

Through review of station logs, personnel interviews, work orders, plant procedures, and action requests, the following factors have been identified as an apparent cause:

1) Lack of Ownership

• It was unclear who owned the nozzle dam installation. When interviewing personnel from each of the departments involved, no single point of contact was identified.

• The station's contractor liaison responsible for nozzle dam installation was unaware of management expectations & responsibilities associated with nozzle dam installation.

• No management oversight of evolution was provided. There wasn't anyone stepping back to see the big picture.

2) Lack of Procedural Adherence

• Air supply regulator was adjusted outside the procedural requirements of HPIP 4.51.3 and HPIP 4.58. CAP 055595 was initiated to document this deviation.

• Air supply to nozzle dam seals and eddy current equipment does not meet requirements of NP 8.4.9.

Procedure requires service air hoses to be yellow or a base color with yellow stripe. Original hoses were red, however yellow tape has been applied to the hose and verified by engineering. CAP 055560 documents this deviation.

• During the steam generator nozzle dam installation bubble hoods used for the job were not issued in accordance with HPIP 4.58. The procedure requires the issuance to be documented on PBF-4234 and this was not completed. CAP 055645 has been initiated to document this deviation.

3) Lack of Questioning Attitude

• Confined Space Rescue Team (CSRT) members stated that they wanted to be at the job site. Due to dose concerns, the Radiation Protection department denied this request. CSRT did not question further.

• Worker had issues fitting through manway opening. Worker was wet down to aid in entry, and after three (3) attempts successfully entered the steam generator.

• Integrated training session was not completed. The station's contractor liaison stated that it was desired to complete integrated training, however did not actively advocate.

37

IndustrialSafety Issues & Poor Work PracticesDuring Nozzle Dawn Installation - RCE253 Attachment H - Nozzle Dam Lessons Learned - Gary Sherwood a Radiation Protection personnel did not question appropriateness of air supply regulator adjustments.

Initial personnel interviews indicated individuals were unaware of procedural requirements.

• The site's evaluation of previous Operating Experience bulletin on breathing air lines becoming disconnected was to tape the fittings, whereas many sites in the industry changed to a different style fitting.

4) Low Sensitivity for Raising Issues

• Untimely communications of anomalies. Four (4) separate incidents involving personnel breathing air occurred. One (1) issue reported to the OCC.

• The station's contractor liaison did not elevate events as the evolution progressed.

• NOS personnel assessed IPTE Briefing as "less than adequate." Feedback was not given to brief leader. Evaluation appeared in the 4/8/04 Nights, Nuclear Oversight Rapid Trending Assessment Daily Report, however issues were not raised at the time of initial assessment.

• CAP 055527 written by RP manager summarizing three (3) of the air supply issues. CAP was twice amended to include the fourth issue and provide clarification of original CAP.

38

IndustrialSafety Issues & Poor WMork PracticesDuring Nozzle Dam Installation- RCE253 Attachment H - Nozzle Dam Lessons Learned - Gary Sherwood Breathing Air System

• Procedures were not used to don bubble hoods and setup and adjust breathing air system. If procedures had been followed problems encountered could have been avoided.

• Breathing air system may not have been providing an adequate flow for jumpers.

• There were not checks on air system just prior to use. Procedure did not require it.

• Breathing air manifold pressure was adjusted higher than allowed by procedure without evaluating impact of adjustment

• The manufacturer of the air fittings had previously informed customers that the fittings could come apart.

• No clear basis why supplied air versus a respirator is needed Job preparations

• Mockup training was not done in bubble suits.

• High risk procedure, NP 1.1.7 was not used for this evolution o Not flagged as a high risk activity o Flagged as an IPTE o High risk pre-job readiness checklist and pre-job briefing checklist were not used o There was no stop work criteria developed

• There was no OSHA approved platform available for access to steam generators

• Human performance error prevention tools not identified for use prior to job Communications

• No communication protocol established with OCC

• OCC not kept informed of problems with air system

• Communications amongst team members was weak Management Oversight

• Roles and responsibilities of contractor liaison and RP supervision not clear

• Job not classified as high risk activity Supervisory Oversight

• Work was not stopped despite problems. No stop work criteria evident.

• Industrial safety was not a focus.

• Pre-job brief was characterized as weak by NOS

• Problems were not resolved and fixed appropriately.

o Fixes performed on air lines were not appropriate nor did they address problem o Workers chose to use manway cover as step vice using an appropriate OSHA approved step/platform.

• Team allowed to push on despite recurring problems with air system and access to steam generators o Perceived time pressure o Late on night shift on holiday weekend o Strong desire to finish job

• Jumpers not evaluated for fitness for job prior to job start 39

IndustrialSafety Issues & Poor Work PracticesDuring Nozzle Darn Installation- RCE2S3 Attachment H - Nozzle Dam Lessons Learned - Gary Sherwood o Some jumpers had difficulty accessing channel head o Some workers had breathing difficulty

• Human performance error tools not utilized effectively o Stop when unsure o Are you ready checklist o Effective Pre-job brief Worker Practices

• Worker continued working despite losing air supply RP Practices

• Nasal swabs were not done following facial contamination

• Personnel were contaminated when they were cut out of their bubble suits.

40

Industrial Safety Issues & Poor Work PracticesDuring Nozde Dam Installation- RCE253

. *Attachment I - Radiation Protection Immediate Actions & Actions to Address Supervisory Root Cause Issues Radiation Protection Immediate Corrective actions associated with the nozzle dam events:

1. Brought in an independent team of NMC personnel (Don Schuelke, Hudson RP and Joe Hager, Palisades) to review our procedures and processes for use of supplied air as breathing air.

2. Used their input to:

• Update applicable procedures (temp changes issued) (included manifold pressure versus hose section table)

• Recognize that it was unsat to have nitrogen backup bottles connected to the breathing air system via the nozzle dam control panels. We had the bottles changed out and replaced with certified Grade D Breathing air.

• Replaced all of our bubble hoods, airlines and fttings to new CEJN type fittings -- these fittings won't pop open, you have to push and pull them to open.

3. RP also developed a Just in Time information sharing package to be used in conjunction with the pre-job briefing. This included a review of our procedure requirements, bubble hood issuance requirement, and manifold pressure requirements.

These items were completed and we used bubble hoods very successfully on Friday, April 23, 2004 to perform hydrolazing decon of the reactor vessel underhead.

We are also working on getting a system set up to conduct flow testing through the breathing air hoses to confirm design air flow versus manifold pressure.

Actions to Address Supervisory Root Cause Issues:

The actions that have been taken to address supervisory oversight issues from the event are:

• RP Supervision participated in the planning for and development of the work plan for nozzle dam removal.

• Added a note in the work plan that the RP Supervisor at 8ft Control Point will direct performance of Steps 14 through 25. These are the steps that control placing the work crew on the S/G Manway platform and removing the nozzle dams. This ensures that all personnel involved with the job are cognizant of the chain of command while workers are in the S/G Channelheads. (Reference W.O.

0400042 and W.O. 0400043)

• RP Supervisors and RPTs are required to participate in mock-up training for nozzle dam removal prior to engaging in the actual job as supervisor or job coverage RPT on the platform or at the control point desk. The training department will provide a memo to the Program Engineering General Supervisor listing the RP and Scientech personnel who have successfully completed the training. Only the personnel designated on the memo will be allowed to perform the activities listed above.

Stop work criteria have been developed and will be communicated to the work crew during the pre-job briefing. Stop work criteria includes:

o Stop work dose rates specified on the Radiation Work Permit o Loss of breathing air o Inadequate breathing air o If a worker is not comfortable with breathing air o If communications are lost o If equipment problems are encountered such as a loss of lighting in the channelhead o If a worker exhibits heat stress symptoms or feels ill I o If shut down cooling is lost or Reactor Vessel Level changes unexpectedly, the control room will inform the Program Engineering General Supervisor who will stop the job. If j'ersonnel are in the bowls, they will exit the bowls immediately.

41

IndustrialSafety Issues & Poor Work Practices During Nozzle Dam Installation- RCE253 Attachment I - Radiation Protection Immediate Actions & Actions to Address Supervisory Root Cause Issues Communications criteria for nozzle dam removal have been set and will be communicated to the work crcw during the pre-job briefing. Communications between the workers and RP were discussed during the mock-up training.

The actions taken to monitor and maintain breathing air pressure to the workers are:

• Replaced the quick disconnect airline fittings to the respirator with CE.N double action fittings.

• Performed testing with the respirator manufacturer, NPO, to ensure that the MSA Manifold System, with the pressure regulator adjusted for pressure gauge readings as defined in NPO's instruction manual, will provide an air flow to each respirator user of between 6 and 15 scfm.

While workers are using breathing air during nozzle dam removal activities we will station a dedicated operator to monitor the pressure gauge reading on the MSA Manifold.

• Once the first respirator user is hooked up to breathing air and the pressure regulator has been adjusted for pressure gauge readings as defined in HPIP 4.58, Issuance of Respiratory Protection Equipment, the pressure regulator will not be adjusted for subsequent users. If pressure indicated on the MSA Manifold falls below the range specified by HPIP 4.58 the job will be stopped and workers will be cut out of the bubble hoods.

Please let me know if I can supply anymore information.

Brian J. Carberry GeneralSupervisor RadiologicalEngineering 42