IR 05000255/2002010

IR 05000255-02-010, on 11/04 - 22/2002, Nuclear Management Company, LLC, Palisades Nuclear Generating Station; Identification and Resolution of Problems. Non-Cited Violations Noted

ML023520392
Person / Time
Site:	Palisades
Issue date:	12/17/2002
From:	Anton Vegel NRC/RGN-III/DRP/RPB6
To:	Cooper D Nuclear Management Co
References
IR-02-010
Download: ML023520392 (28)

Text

ber 17, 2002

SUBJECT:

PALISADES NUCLEAR GENERATING PLANT NRC PROBLEM IDENTIFICATION AND RESOLUTION INSPECTION REPORT 50-255/02-10

Dear Mr. Cooper:

On November 22, 2002, the U. S. Nuclear Regulatory Commission (NRC) completed a team inspection at the Palisades Nuclear Generating Plant. The enclosed report documents the inspection findings which were discussed on November 25, 2002, with you and other members of your staff.

This inspection was an examination of activities conducted under your license as they relate to the identification and resolution of problems, compliance with the Commissions rules and regulations and with the conditions of your operating license. Within these areas, the inspection involved selected examination of procedures and representative records, observations of activities, and interviews with personnel.

On the basis of the sample selected for review, the team concluded that in general, problems were being properly identified, evaluated, and corrected. The team made several observations regarding the effectiveness of problem identification and resolution program implementation.

For example, the team identified several minor examples of implementation deficiencies such as some departments not routinely entering problems into the corrective action system, several examples of narrowly focused problem evaluations, and several examples of corrective actions not being effective in preventing problem recurrence. On the positive side, the team noted that corrective actions to alleviate a previously identified cross-cutting issue in human performance have, to date, been effective, and that Nuclear Oversight was effectively identifying performance issues. The team also noted that corrective action program improvements had only recently been implemented and therefore, the effectiveness of these initiatives could not be fully assessed. The team concluded that except for some isolated examples the corrective action program was effective in ensuring that conditions adverse to quality were being adequately addressed.

There was one Green finding identified during this inspection associated with the failure to follow the procedure for the control of scaffolding. The finding illustrated several corrective action problems involving the identification of the issue, extent of condition, and follow through of corrective actions. This finding was determined to be a violation of NRC requirements.

However, because of its very low safety significance and because it has been entered into your corrective action program, the NRC is treating this finding as a Non-Cited Violation, in accordance with Section VI.A.1 of the NRCs Enforcement Policy. If you deny this Non-Cited Violation, you should provide a response with the basis for your denial, within 30 days of the date of this inspection report, to the U. S. Nuclear Regulatory Commission, ATTN: Document Control Desk, Washington, DC 20555-0001, with copies to the Regional Administrator, Region III; the Director, Office of Enforcement, U. S. Nuclear Regulatory Commission, Washington, DC 20555-0001; and the NRC Resident Inspector at the Palisades Nuclear Generating Plant.

In accordance with 10 CFR 2.790 of the NRCs Rules of Practice, a copy of this letter and its enclosure will be available electronically for public inspection in the NRC Public Document Room or from the Publicly Available Records (PARS) component of NRCs document system (ADAMS). ADAMS is accessible from the NRC Web-site at http://www.nrc.gov/NRC/ADAMS/index.html (the Public Electronic Reading Room).

Sincerely,

/RA/

Anton Vegel, Chief Branch 6 Division of Reactor Projects Docket No. 50-255 License No. DPR-20

Enclosures:

Inspection Report No. 50-255/02-10

REGION III==

Docket No: 50-255 License No: DRP-20 Report No: 50-255/02-010(DRP)

Licensee: Nuclear Management Company, LLC Facility: Palisades Nuclear Generating Plant Location: 27780 Blue Star Memorial Highway Covert, MI 49043-9530 Dates: November 4 through November 22, 2002 Inspectors: L. Collins, Lead Inspector K. Coyne, Operations Engineer, NRR D. Schrum, Reactor Inspector Approved by: Anton Vegel, Chief Branch 6 Division of Reactor Projects 1 Enclosure

SUMMARY OF FINDINGS IR 05000255-02-010, on 11/04 - 11/22/2002, Nuclear Management Company, LLC, Palisades Nuclear Generating Station; identification and resolution of problems.

The inspection was conducted by two region-based inspectors and one headquarters-based inspector. This inspection identified one Green finding, which was a Non-Cited Violation. The significance of most findings is indicated by their color (Green, White, Yellow, Red) using Inspection Manual Chapter (IMC) 0609, Significance Determination Process (SDP). Findings for which the SDP does not apply may be Green or be assigned a severity level after NRC management review. The NRCs program for overseeing the safe operation of commercial nuclear power reactors is described in NUREG-1649, Reactor Oversight Process, Revision 3, dated July 2000.

Identification and Resolution of Problems In general, the plant identified issues and entered them into the corrective action process at an appropriate low-level, although some exceptions to this practice were identified. Nuclear Oversight assessment reports identified issues for the plant to resolve, including issues with corrective action follow through. The majority of issues reviewed were properly categorized and evaluated although some evaluations were narrowly focused, particularly for apparent cause evaluations and extent of condition reviews. Most corrective actions reviewed were appropriately implemented; however, some examples, including one inspection finding, were identified regarding corrective actions that were not fully implemented or fully effective in correcting the identified problem. Corrective action follow-through and effectiveness is one aspect of the corrective action process that could be strengthened to reduce repeat issues at the plant.

Cornerstone: Mitigating Systems Green. The inspectors identified a finding of very low safety significance that is being treated as a Non-Cited Violation of Technical Specification 5.4.1 Procedures. The licensee failed to adequately implement scaffold control requirements contained in procedure MSM-M-43, Scaffolding. Seismic scaffolding erected over Component Cooling Water (CCW) pump P-52A was anchored to a safety related pipe support for CCW pump P-52B without engineering evaluation and approval.

The finding was greater than minor because the finding would become a more significant concern if left uncorrected. The failure of scaffolding installed in the vicinity of safety-related equipment during a seismic event could result in damage to mitigating equipment. The finding was of very low safety significance because it did not result in the actual loss of the safety function of the train or system (4OA2.3).

2 Enclosure

Report Details 4. OTHER ACTIVITIES (OA)

4OA2 Problem Identification and Resolution

.1 Effectiveness of Problem Identification a. Inspection Scope The inspectors reviewed inspection reports issued over the last year, selected plant corrective action documents, Nuclear Oversight assessments, trend reports, operating experience and a sample of procedure change requests in order to determine if problems were being identified at the proper threshold and entered into the corrective action process. The inspectors also conducted a focused plant walkdown of the Component Cooling Water system to ensure that equipment problems were entered into the corrective action system. The documents listed in Attachment were used during the review.

b. Issues In general, the plant identified issues and entered them into the corrective action process at an appropriate low-level, although some exceptions to this practice were identified. Nuclear Oversight assessment reports identified issues for the plant to resolve, including issues with corrective action follow through. The trending program was inconsistently implemented with some departments effectively trending and others not performing trending at all. Details of these and other observations are described in the following sub-sections.

b.1 Identification Threshold The licensee had defined an adequate threshold for the identification of issues to be entered into the corrective action program in accordance with Palisades Nuclear Plant Administrative Procedure No. 3.03 Corrective Action Process. The current electronic database system was called TeamTrack and was implemented in August 2002. A corrective action document in Team Track was called an Action Request (AR) or CAP.

Prior to TeamTrack, corrective action documents were called condition reports or CPALs. The generation rate for CAPs was fairly high with almost 4000 CAPs generated to date in 2002. While the threshold appeared adequate and the generation rate was good, the inspectors found several examples of either the NRC identification of issues that were not entered into the corrective action system or specific plant programs or departments not effectively identifying issues within the corrective action system. While some of these issues represented conditions adverse to quality, none were considered to be significant conditions adverse to quality. The issues included the following:

C Deficiencies identified during fire drills were not always entered into the corrective action system 3 Enclosure

C Spent fuel cooling system pump 51A required frequent operations monitoring and oil addition for several months. A work request was initiated for maintenance but no CAP was written.

C The resident inspectors identified an inadequate operability evaluation and an inadequate procedure used during operator requalification training.

C The licensee identified that the CAP initiation threshold was too high for security department issues and emergency plan drill issues.

The license initiated CAPs on all of the issues listed above.

b.2 Procedure Change Requests The inspectors determined that the Procedure Change Request (PCR) program, which was used to track planned revisions to facility procedures, could allow conditions adverse to quality to bypass the corrective action program. Because the licensee maintained the PCR program independent of the corrective action program, procedure deficiencies could be entered into the PCR database without initiation of a corresponding CAP. The inspectors reviewed eleven PCR database entries not related to a condition report and identified one condition adverse to quality that was not identified within the corrective action program. Specifically, a drawing configuration control deficiency identified in PCR 21030 should have been documented in a condition report per the guidance contained in Administrative Procedure 3.03, Attachment 7, Palisades Corrective Action Process (CAP) Significance Guidelines. The licensee documented the failure to initiate a condition report associated with PCR 21030 in CAP032212. Although this issue was of minor significance, the inspectors concluded that the use of a PCR database separate from the corrective action program could allow conditions adverse to quality to be processed outside of the corrective action program.

b.3 Trending The licensee inconsistently implemented condition report trending requirements contained in plant procedures. Although Administrative Procedure 3.03, Section 6.13, Corrective Action Trending, required periodic condition report trending, the licensee failed to implement a station-wide condition report trending program. Specifically, the licensee was unable to demonstrate that the security, training, and engineering departments implemented these trending requirements. However, the inspectors noted that some individual departments, including the operations and maintenance departments, performed periodic reviews of condition reports to assess performance trends. The inspectors concluded that the failure to adequately implement procedural requirements for trending could result in the failure to identify and correct adverse performance trends. The licensee initiated CAP032007 to document this issue and stated that a station-wide trending program was being developed.

4 Enclosure

b.4 Operating Experience The inspectors reviewed a sampling of 13 industry operating experience (OE) items and concluded that the licensee adequately evaluated OE items and appropriately identified related plant issues. The inspectors noted some minor documentation weaknesses in operating experience evaluations and one minor inconsistency between an operating experience evaluation and plant practices associated with the use of high pressure plastic tubing during testing. The licensee initiated CAP032213 to further evaluate the latter issue.

b.5 Nuclear Oversight The inspectors reviewed the last four quarterly Nuclear Oversight (NOS) assessment reports and determined that the NOS staff, in general, effectively identified plant performance issues. In particular, the inspectors did not identify significant performance issues during the inspection that were not described in previous NOS assessment reports. However, the inspectors noted that the licensee did not consistently resolve NOS identified issues. For example, since the fourth quarter of 2001, quarterly oversight audits identified repetitive issues with corrective action adequacy and follow through that were not adequately resolved. Additionally, NOS staff identified issues related to control of contractor personnel and implementation of corrective actions by the maintenance department similar to repetitive issues identified by the inspectors.

The inspectors concluded that, although the NOS staff effectively identified performance issues, the licensee failed to consistently resolve NOS identified problems.

.2 Prioritization and Evaluation of Issues a. Inspection Scope The inspectors conducted an independent assessment of the prioritization and evaluation of a selected sample of corrective action program documents. The assessment included a review of the category assigned, operability and reportability determinations, extent of condition evaluations, cause investigations, and the appropriateness of the assigned corrective actions. The inspectors also attended several Condition Review Group (CRG) meetings during which CAPs are screened and assigned a significance level and Corrective Action Review Board (CARB) meetings which reviewed completed root cause evaluations. The documents listed in Attachment were used during the review.

b. Issues The majority of issues reviewed were properly categorized and evaluated. The team had several observations regarding narrow or limited evaluations, particularly for apparent cause evaluations and extent of condition reviews. Details of these and other observations are described in the following sub-sections.

5 Enclosure

b.1 Overview of Prioritization/Evaluation Process The corrective action process included a review of newly initiated CAPs by the Condition Review Group (CRG), which included senior plant management. The CRG would assign a significance level to each CAP, with A being a Significant Condition Adverse to Quality (SCAQ) requiring a root cause evaluation, B was a Condition Adverse to Quality (CAQ) requiring an apparent cause evaluation, and C was a CAQ requiring a condition evaluation to determine the proper corrective actions. A significance level D was also available for conditions that were not adverse to quality.

The backlog of open CAPs was at approximately 1770 at the time of the inspection.

This backlog included CAPs that required evaluation and CAPs for which the evaluation was complete but the corrective actions were not yet complete. This backlog did not meet the station goal for the backlog but appeared to be understood and was receiving appropriate management attention.

b.2 Evaluation Observations The inspectors observations regarding narrowly focused cause evaluations or extent of condition evaluations are described in the following paragraphs.

• The licensee did not promptly assess the condition of the CCW pump P-52B motor following a catastrophic failure of the CCW pump P-52C motor on January 2, 2002. In the CPAL020014 root cause evaluation for the P-52C motor failure, the licensee was unable to determine a specific cause, but postulated that an original winding defect or manufacturing issue contributed to the motor failure. The licensee noted that the pump P-52B motor was manufactured by the same vendor as the P-52C motor and both motors were supplied as original plant equipment. Although the licensee did not provide an adequate basis for concluding that the P-52B motor was not susceptible to a similar failure as P-52C motor, no follow-up actions for testing the condition of the of P-52B motor were identified in the CPAL020014 root cause evaluation.

On March 6, 2002, engineering personnel submitted work order 24210985 to perform motor testing on P-52B. However, the work order was not associated with a condition report or otherwise linked to the earlier failure of the P-52C motor within the corrective action program. Subsequently, on April 24, 2002, the licensee identified that performance of the P-52B motor testing had been inappropriately delayed and wrote CPAL0201619. Although CPAL0201619 noted that the cause of the P-52C motor failure could be a common mode failure mechanism for P-52B, engineering personnel failed to effectively communicate the importance of performing the P-52B motor testing to scheduling personnel when initiating work order 24210985. The P-52B motor testing was satisfactorily performed on July 30, 2002, approximately 7 months after the P-52C failure.

The inspectors concluded that the failure to link the P-52B motor testing to the previous failure of the CCW pump P-52C motor within the corrective action system contributed to the failure to promptly schedule and perform the P-52B testing.

6 Enclosure

• The apparent cause evaluation and corrective actions for CPAL0101551, associated with the use of an unapproved test procedure, were narrowly focused. On April 11, 2001, the inspectors identified that contractor personnel were performing testing on a CCW heat exchanger with a vendor supplied procedure that was not approved for use by the licensee. The apparent cause evaluation determined that the cause of this issue was that the CCW system engineer was not aware of administrative requirements for the review and approval of vendor procedures. The inspectors noted that the apparent cause evaluation was narrowly focused on the actions of the system engineer and failed to consider other process or procedural barriers that could have prevented use of an unapproved procedure in the plant. Additionally, the actions contained in CPAL0101551 did not specifically address the lack of knowledge by system engineering personnel concerning use of vendor procedures. The licensee stated that these issues would be corrected by stations service coordinator program, which is intended to provide additional oversight for vendor and contractor personnel. The licensee initiated corrective action CA017521 to evaluate the need for specific requirements in the contractor control program for the review of vendor work instructions.

C A National Fire Protection Association (NFPA) code compliance review was performed by an outside vendor as part of an extent of condition (EOC)

evaluation for a previous significant NRC finding. During a walkdown of the plant, the NRC identified an impaired sprinkler in the Electrical Equipment Room that was not identified and evaluated during this EOC. The licensee initiated CAP 031655 to document this in the corrective action program. Upon further review, the licensee identified several additional sprinkler issues. Except for this less than rigorous EOC, the licensee performed appropriate corrective actions for this NRC inspection finding.

.3 Effectiveness of Corrective Action a. Inspection Scope The inspectors reviewed corrective action documents and recent plant issues to determine if corrective actions were implemented in a timely, appropriate, and effective manner. The inspectors conducted a walkdown of the CCW system with the system engineer to assess the material condition of the system and verify that the licensee appropriately identified degraded conditions within the corrective action program.

Additionally, the inspectors evaluated the current status of corrective actions to improve previously identified substantive cross-cutting issues in the areas of corrective actions and human performance. The inspectors also reviewed the licensees corrective actions for eleven Non-Cited Violations (NCVs) documented by NRC inspections in the past year. The documents listed in Attachment were used during the review.

b. Issues One Green finding was identified involving the failure to adequately implement seismic scaffolding procedural requirements. This finding illustrated several corrective action 7 Enclosure

issues. Most notable is that the finding is a repetitive failure to properly implement scaffold control requirements and after appropriate corrective action was specified, the action was not completed. Other examples of corrective action effectiveness or follow-through observations were noted during this inspection and were exhibited in NRC findings in the past year related to repeat issues. Repetitive corrective action follow-through issues were also identified as a Nuclear Oversight finding in July 2002.

Licensee corrective actions in response to the Nuclear Oversight finding had been identified but had not yet been implemented. The current inspection finding and other observations are described in the following sections.

b.1 Repetitive Failure to Adequately Implement Seismic Scaffolding Control Requirements Introduction The inspectors identified one finding of very low safety significance (Green) associated with the failure to adequately implement scaffold control requirements contained in procedure MSM-M-43, Scaffolding. The finding was determined to be a violation of NRC requirements and was dispositioned as a Non-Cited Violation of Technical Specification 5.4.1, Procedures.

Description On November 5, 2002, the inspectors identified that seismic scaffolding erected over CCW pump P-52A was anchored to a safety related pipe support for CCW pump P-52B without engineering evaluation and approval. Step 5.5.1 of MSM-M-43 required that design engineering provide direction and approval for tie off of scaffolding to supports.

Although the MSM-M-43, Attachment 1, Scaffold Erection Control Checklist, for the scaffold installation was annotated that the scaffold was to be secured to plant equipment, design engineering approval was not obtained prior to scaffold construction.

Following identification of this issue, the licensee reconfigured the scaffolding to eliminate the tie off to the CCW pump P-52B suction piping support and initiated CAP 031961 to document this condition. The licensees immediate corrective actions included a temporary suspension of scaffolding activities, stand down and retraining of scaffold craft workers and supervisors, and extent of condition walk downs to identify other potentially deficient scaffold installations.

During extent of condition walkdowns on November 6, 2002, the licensee identified three additional scaffold installations that failed to comply with the requirements of MSM-M-43:

• Scaffold in 1-D switchgear room was not left in a seismically secure condition during an interruption in work activities (CAP 032013).

• Scaffold in electrical equipment room was not built to seismic requirements.

Specifically, a minimum separation of 1 inch between the scaffold and safety-related structures was not maintained and the scaffold was braced to equipment supports (CAP 032010).

• Scaffold in auxiliary feed water pump room was not built to seismic requirements in that a minimum separation of 1 inch between the scaffold and safety-related structures was not maintained (CAP 032012).

8 Enclosure

The inspectors assessed the effectiveness of the licensees immediate actions for these scaffolding control deficiencies during subsequent plant walkdowns. The inspectors identified two additional scaffold deficiencies that were not adequately addressed by the licensees immediate corrective actions:

• On November 8, the inspectors identified that scaffolding installed in the mezzanine adjacent to the 1-D switchgear bus area had not been adequately braced. The licensee previously identified deficiencies in this scaffold installation during initial extent of condition walkdowns on November 6 (CAP 032054).

• On November 18, the inspectors identified scaffold in contact with safety related control room ventilation piping. The licensee later determined that this scaffold deficiency had been identified during the extent of condition walkdowns on November 6, 2002, but no action had been taken to correct the identified deficiency (CAP 032153).

The inspectors concluded that these additional scaffold control issues indicated that the licensee failed to effectively implement the planned corrective actions for the initial scaffold deficiency identified on November 5.

The inspectors reviewed recent licensee scaffold control issues to determine if there had been prior opportunity to address scaffolding procedural adherence deficiencies. On April 19, 2001, the NRC issued NCV 50-255/01-06-02 for three examples of the licensees failure to satisfy seismic requirements specified in plant procedures.

Specifically, in February and March of 2001, the inspectors identified scaffolding and storage racks constructed near the auxiliary feedwater pump P-8B steam supply and the low pressure safety injection pump P-67A suction piping that did not meet procedural requirements. The licensee initiated condition reports C-PAL-01-00652 and C-PAL-01-00695 to document and evaluate the issues described in NCV 50-255/01-06-02. In the root cause evaluation for C-PAL0100695, the licensee determined that the site lacked a programmatic method to control scaffold design, erection, inspection, and approval. The licensee identified several corrective actions to prevent recurrence of these scaffold deficiencies, including revision to scaffold control procedures and additional training. On October 15, 2002, the licensee completed all C-PAL-01-00695 corrective actions to prevent recurrence, with the exception of an effectiveness review. Based on the identification of repetitive failures to adequately control seismic scaffolding between November 5 through November 18, 2002, the inspectors concluded that the corrective actions of CPAL0100695 were not effective in preventing recurrence of scaffolding control problems.

Analysis The inspectors concluded that the finding was greater than minor in accordance with IMC 0612, Power Reactor Inspection Reports, Appendix B, Issue Disposition Screening, because the finding would become a more significant concern if left uncorrected. The failure of scaffolding installed in the vicinity of safety-related equipment during a seismic event could result in damage to mitigating equipment.

Specific examples of inadequate scaffolding were identified in the vicinity of component 9 Enclosure

cooling water, auxiliary feedwater, and power system components. Therefore, continued inadequate control of seismic scaffold installation could affect the operability, availability, reliability, or function of mitigating systems during seismic events.

The inspectors evaluated the finding using Manual Chapter 0609, Significance Determination Process, Appendix A, Significance Determination of Reactor Inspection Findings for At-Power Situations, Phase 1 screening, and determined that the finding:

• was not a design or qualification deficiency;

• did not represent an actual loss of safety function of a system;

• did not represent an actual loss of a safety function of a single train for greater than Technical Specification outage time;

• did not represent an actual loss of a safety function of one or more Non-Technical Specification trains of equipment; and

• did not screen as potentially risk significant due to a seismic, flooding, or severe weather initiating event. Specifically, the finding does not involve the loss or degradation of equipment or function designed to mitigate a seismic initiating event or the total loss of any safety function Therefore, the finding was determined to be of very low safety significance (Green).

Enforcement Technical Specification 5.4.1, Procedures, requires, in part, that written procedures shall be established, implemented, and maintained covering the activities recommended in Regulatory Guide 1.33, Revision 2, February 1978. Regulatory Guide 1.33, Appendix A, Section 9.a, recommends that procedures should be written to cover maintenance that can affect the performance of safety-related equipment.

Procedure MSM-M-43, Scaffolding, Revision 6, was written to provide requirements for maintenance activities that can affect the performance of safety-related equipment, including scaffold erection, inspection, and tagging. Contrary to the above, licensee personnel failed to adequately implement the requirements of procedure MSM-M-43.

Specifically, on November 5, 2002, the inspectors identified scaffolding tied off to a suction piping support for CCW pump P-52B without prior engineering direction and approval, contrary to the requirements of step 5.5.1 of MSM-M-43. Additionally, between the period of November 6 and November 18, 2002, four additional examples of scaffolding installations that did not comply with the minimum separation requirements of step 5.4.3.a or the equipment tie off requirements of step 5.5.1 of MSM-M-43 were identified by the licensee and the inspectors. This violation is associated with an NRC identified finding that is characterized by the significance determination process as having very low risk significance (Green) and is being treated as a Non-Cited Violation of 10 CFR 50 Appendix B, Criterion V, consistent with Section VI.A.1 of the NRC Enforcement Policy (NCV 50-255/02-10-01). This finding is in the licensees corrective action program as CAP031961, CAP032010, CAP 032012, CAP032013, CAP032054, and CAP032153.

10 Enclosure

b.2 Observations on the Effectiveness of Corrective Actions The inspectors had several observations regarding corrective actions that were not fully implemented, not fully effective in correcting the identified issue, or were narrowly focused. These observations are described below.

• The inspectors identified that the corrective actions for a deficient maintenance procedure did not ensure that the procedure would not be used prior to issuance of a necessary procedure revision. Specifically, on July 31, 2002, during a rebuild of CCW pump P-52B, maintenance personnel noted that the pump rotating element had been installed backwards. The licensee performed an apparent cause evaluation for this condition and determined that the maintenance procedure instructions lacked adequate detail for the orientation of the pump rotating assembly. Consequently, an action was generated to revise the associated maintenance procedures. The inspector noted that the corrective action did not prevent further use of the deficient procedure until an appropriate revision could be made. The licensee initiated CAP032145 to address this issue.

• The corrective actions for CPAL0200101, associated with a maintenance error that resulted in a CCW leak of approximately 75 gpm, were inconsistently implemented. During a maintenance activity on January 7, 2002, for primary coolant pump P-50B, maintenance workers failed to tighten a flanged CCW system connection to a CCW lube oil cooler. When the cooler was later placed in service, the CCW system leakage from the flange resulted in an unexpected lowering of the CCW surge tank level. The licensee determined that the root cause of this event was the lack of formality and standards during the job turnover process and inaccurate place keeping in the maintenance work instructions. The corrective actions to prevent recurrence included implementation of a formal job turnover sheet for use by maintenance department supervisors and more formal maintenance procedure place keeping standards by craft workers. The inspectors assessed the licensees implementation of these actions while observing maintenance activities on Diesel Generator 1-1 on November 6, 2002. The inspectors determined that the actions from CPAL0200101 were inconsistently implemented. Specifically, the mechanical maintenance supervisor did not use the formal turnover sheet as required and a maintenance worker was not maintaining place keeping in the work procedure in accordance with station guidelines. The licensee initiated CPAL032001 and CPAL032027 to document these issues.

• The corrective actions for CPAL0200292 Main Feedwater Pump 1A Discharge Pressure Higher than Expected Following Main Feedwater Pump 1B Startup included a corrective action to prevent recurrence to conduct a training needs analysis, and if needed, conduct training on post-maintenance testing requirements. The action was closed as completed; however, only the needs analysis was done. The training had not yet been conducted although the event had occurred in January 2002. In fact, the training was not yet developed and the effectiveness review for this root cause and corrective actions had been extended to December 2003, almost 2 years after the event occurred.

11 Enclosure

• Some recently closed CAPs reviewed were closed although the identified problem was either not successfully corrected or the problem was accepted rather than fixed. Examples included an Emergency Diesel Generator exhaust temperature indicator which had been evaluated as a repetitive problem in CAP 030334. The CAP was closed to a work order which had been completed but was not effective in correcting the problem. A second example involved Primary Coolant Pump Motor Structural Web Vibration which was documented on CAP 030444 in July 2000. The corrective action was an engineering action request to evaluate performing a modification. The engineering action request was closed with no action, essentially accepting the condition, and the CAP closed.

• The licensee did not promptly enact monitoring actions to detect further CCW system degradation following the February 2002, failure of CCW pump P-52C due to foreign material ingestion. The licensee determined that the foreign material originated from a degraded rubber seating surface for CCW valve MV-CC923, the outlet valve from the spent fuel pool heat exchangers. The licensee did not begin hourly monitoring of CCW header pressure, to identify further pump degradation, until approximately 5 days after the failure of CCW pump P-52C. Additionally, the licensee did not measure CCW cooling flow to individual engineered safeguards pumps, in order to verify that foreign material had not blocked essential CCW cooling flow to the emergency cooling system pumps, until approximately 5 weeks after the P-52C failure. The licensee did not develop a formal CCW system monitoring plan until approximately six months after the failure of P-52C. The inspectors determined that these monitoring actions were particularly important due to the inability to promptly repair MV-CC923 because the degraded condition CCW system isolation valves prevented establishment of satisfactory isolation for the valve repair.

b.3 Cross-Cutting Issues The NRC identified a human performance cross-cutting issue in the area of engineering in November 2001 and in the area of maintenance in February 2002. Human performance as a substantive cross-cutting issue was also described in the NRCs annual assessment letter to the licensee in March 2002 and in the mid-cycle assessment letter in August 2002. A root cause analysis for the engineering human performance issue was completed in February 2002 and identified two root causes. The first root cause was that roles and responsibilities were not clearly defined, communicated and adhered to. The second root cause was that performance expectations were not clearly and effectively defined, communicated and upheld.

Corrective actions were specified to clearly define roles, responsibilities and expectations, to provide training to engineers, and to implement process changes to monitor the quality of engineering products. No separate evaluation was performed for the maintenance human performance finding. However, the licensee had a site-wide human performance improvement plan which required each department to have a specific plan. In addition to the site-wide and department-specific plans, the licensee had provided a series of training seminars to improve human performance. Based on 12 Enclosure

the lack of significant human performance issues identified in this and recent inspections, it appeared that the actions taken by the licensee had been effective in reducing human performance issues.

A cross-cutting finding in corrective action was identified by the NRC in February 2002 and was also discussed as a substantive cross-cutting issue in both the NRCs annual assessment letter and mid-cycle assessment letter. In response to the identified corrective action program deficiencies, the licensee revised the program procedure to improve the process. Additional improvement initiatives were tracked by the station Excellence Plan and focused on improving the quality of evaluations and strengthening CRG and CARB. Although many of these actions were taken since the cross-cutting finding was identified, corrective action effectiveness issues continue to occur. Several additional NRC findings related to inadequate corrective action involving repeat plant problems have been identified and documented in recent inspection reports (50-255/02-02, 50-255/02-07). Also, in July 2002, Nuclear Oversight identified a finding regarding inadequate corrective action follow through as a recurrent issue. This finding was entered into the corrective action program and a root cause evaluation was completed in October 2002. The corrective actions were not yet complete at the time of the inspection.

.4 Assessment of Safety-Conscious Work Environment a. Inspection Scope The inspectors conducted interviews with plant staff to assess whether there were impediments to the establishment of a safety conscious work environment. During these interviews, the inspectors used Appendix 1 to Inspection Procedure 71152, Suggested Questions for Use in Discussions with Licensee Individuals Concerning PI&R Issues, as a guide to gather information and develop insights. The inspectors also discussed the implementation of the Employee Concerns Program (ECP) with the plants ECP Coordinator.

b. Issues Plant staff interviewed did not express any concerns regarding the safety conscious work environment. The staff was aware of and generally familiar with the corrective action program and other plant processes including the Employee Concerns Program to raise issues.

4OA6 Management Meetings

.1 Exit Meeting Summary The inspectors presented the inspection results to M and other members of licensee management in an exit meeting on November 25, 2002. Licensee management acknowledged the findings presented and indicated that no proprietary information was provided to the inspectors.

13 Enclosure

PARTIAL LIST OF PERSONS CONTACTED Licensee L. Bogue Outage and Scheduling Manager Site Vice President B. Dotson Regulatory Analyst P. Harden Engineering Director N. Haskell Nuclear Oversight Manager G. Hettel Maintenance Manager L. Lahti Licensing Manager D.J. Malone Plant General Manager B. McKenzie Corrective Action Supervisor G. Packard Operations Manager P. Russell Performance Improvement Manager ITEMS OPENED, CLOSED, AND DISCUSSED Opened 50-255/02-10-01 NCV Green. Failure to adequately implement procedural requirements for the control of scaffolding in the vicinity of safety-related equipment, contrary to the requirements of TS 5.4.1, Procedures. Specifically, the licensee failed to obtain engineering direction and approval prior to anchoring scaffolding to a safety related pipe support for CCW pump P-52B.

Closed 50-255/02-10-01 NCV Green. Failure to adequately implement procedural requirements for the control of scaffolding in the vicinity of safety-related equipment, contrary to the requirements of TS 5.4.1, Procedures. Specifically, the licensee failed to obtain engineering direction and approval prior to anchoring scaffolding to a safety related pipe support for CCW pump P-52B.

1 Attachment

LIST OF ACRONYMS EAR Engineering Assistance Request EOC Extent of Condition NFPA National Fire Protection Association SQUG Seismic Qualification Utility Group USI Unresolved Safety Issue 2 Attachment

LIST OF DOCUMENTS REVIEWED The following is a list of licensee documents reviewed during the inspection, including documents prepared by others for the licensee. Inclusion of a document on this list does not imply that NRC inspectors reviewed the entire documents, but, rather that selected sections or portions of the documents were evaluated as part of the overall inspection effort. In addition, inclusion of a document on this list does not imply NRC acceptance of the document, unless specifically stated in the body of the inspection report.

4OA2 Identification and Resolution of Problems Plant Procedures EM-09-16 Engineering Manual Procedure, Heat Exchanger Revision 2 Condition Assessment Program EPS-M-14 Permanent Maintenance Procedure, Diesel Revision 3 Generator 1-1- Refueling frequency Maintenance EM-20 Performance Monitoring Program Revision 9 EM-25-01 Palisades Nuclear Plant Engineering Manual Revision 0, 1 Procedure MSM-M-43 Palisades Nuclear Plant Permanent Revision 6 Maintenance Procedure, Scaffolding PFM-E-1 Emergency Post-Fire Repair for Appendix R Revision 4 Equipment I-SC-88-022-1 Component Cooling Water Flow Balance for P- Revision 5 54A, P-54B and P-54C Procedure 1.09 Self-Assessment Program Revision 13 Procedure 5.09 Maintenance Cleanliness Standards Revision 7 Procedure 10.41 Administrative Procedure, Procedure Initiation Revision 34 and Revision Procedure 3.30 Palisades Nuclear Plant Administrative Revision 28, 29 Corrective Action Process Procedure 5.19 Palisades Nuclear Plant Administrative Revision 11 Procedure Post Maintenance Testing Procedure 5.30 Palisades Nuclear Plant Administrative Revision 1 Procedure, Rework Maintenance Procedure 3 Attachment

Procedure QO-20 Inservice Test Procedure - Low Pressure Safety Revision 12 Injection Pumps Procedure T-223 Component Cooling Water Flow Verification Revision 12 Miscellaneous Documents Scaffold Request Form #02-00316 November 4, 2002 Quality Program Description for nuclear Power Revision 21 Plants - Palisades Nuclear Power Plant Component Cooling Water Monitoring Plan for August 29, 2002 Debris Due to Potential Further Degradation of MV-CC923, Spent Fuel Cooling Heat exchanger Outlet Valve Equipment Reliability Watch List November 4, 2002 Shift Turnover Review Sheet Revision 1 Operations Department Focused Self February 2002 Assessment - Plant Status Control Mechanical Pump Seal Training - Training March 30, 2000 Needs Assessment Palisades Assessment Plan October 18, 2002 Palisades Off-Site Review Committee Meeting August 21, 2002 2002-02 Chairmans Report List of Mechanical Seal Problems; January 1997

- November 2002 List of Degraded and Nonconforming Conditions October 22, 2002 Underwriter Laboratories Inc. Memo; File October 16, 2002 NC4809, Project 02NK20855 SDR-99-1205; EA-CCW-87-01 Spent Fuel Pool November 15, 1999 Heat Load and Required Spent Fuel Pool Heat Exchanger Component Cooling Water Flow During a Design Basis Accident (10CFR50.59 Safety Review)

SDR-99-1506; Revise Final Safety Analysis November 12, 1999 Report Table 1-2 and Table 9-7 to Include the Volume of Empty Spent Fuel Pool Cavity and Volume Empty North Tilt Pit Cavity (10CFR50.59 Safety Review)

4 Attachment

EA-SC-94-090-02; Spent Fuel Pool Time to Boil Revision 2 Following a Loss of Cooling Palisades Nuclear Plant Root Cause Evaluation July 5, 2001 for CPAL-01-0251, CPAL-01-0252, and CPAL-01-0253 Condition Report Evaluation Maintenance Rule Refueling Periodic Assessment Observations List of Degraded Equipment Associated With November 5, 2002 Caution Tags TM-2000-022; Installation of Ultrasonic Flow June 20, 2000 Meter on Component Cooling Water to P-54A Containment Spray Pumps VTD-2241-0001; File Number M0008 0033; Revision 1 Graham Vacuum& Heat Transfer Installation, Operation, and Maintenance Instructions for Heat Exchangers; VTD-0271-0018; File Number M0008 0010; Revision B Ingersoll-Rand Co Instructions for Installation, Operation and Maintenance of Overhung Process Pumps Palisades National Fire Protection Association July 16, 2002 Code Compliance Review List of Changes and Response to Appendix A to August 24, 1996 Branch Technical Position APCSB 9.5-1 and Regulatory guide 1.78 and 1.101 Human Performance Improvement Plan Palisades Excellence Plan EAR-1997-0695 Evaluate CCW Differential Pressure Limits November 13, 1997 Across Component Cooling Water Heat Exchangers EAR-1998-0467 Approve te Tube Plugging Limit of the June 29, 1999 Component Cooling Water Heat Exchangers up to the Limit Specified in the Latest LOCA Analysis EAR-1999-0238 Identify and Evaluate the Use of an Improved September 23, 1999 Mechanical Seal for P-67A and P-67B EAR-1999-0337 Remove Check Valve Internals From CK-SW07, June 8, 2000 CK-SW08, and CK-SW409 5 Attachment

EAR-2000-0401 Containment Spray Pump (P-54A) Temporary June 20, 2000 Modification For Seal Cooling Flow Rate EAR-2000-0143 CCW Flow Balance Between Primary Coolant February 29, 2000 Pump Motor Bearing and Primary Coolant Pump Seals Nuclear Oversight Nuclear Oversight 1st Quarter 2002 Assessment Assessment Report for Palisades 2002-001-8 Nuclear Oversight Fourth Quarter 2001 Nuclear Oversight February 18, 2002 Assessment Assessment of the Palisades Plant 2001-004-8 Nuclear Oversight Nuclear Oversight 2nd Quarter 2002 Assessment Assessment Report for Palisades 2002-002-8 Nuclear Oversight Nuclear Oversight 3rd Quarter 2002 Assessment November 19, 2002 Assessment Report for Palisades 2002-003-8 Work Order Perform selected portions of refueling outage 24212951 frequency maintenance on diesel generator 1-1 Work Request Service Water Pump 288338 Work Request Perform characterization testing on P-52B motor, 292769 EMA-1208 and replace motor Condition Reports CA015154 Complete EAR 2000-0345 to Establish Alternate February 27, 2001 Safe Shut down Path for Safety Qualification Utility Group (SQUG)

CA016468 Explore the Various Options Available for May 2, 2000 Resolving the A-46 Outlier Issue Associated With the Seismic Adequacy of the Safety Injection and Refueling Water Tank CA017521 Evaluate the need to change the service November 20, 2002 coordinator responsibilities to include reviewing vendor work instructions or procedures for adequacy CAP000017 OE 11420 HPSI pump bearing experienced a September 25, 2002 leak of oil due to inability to drain from oil bubbler 6 Attachment

CAP029056 Maintenance Rule Refueling Periodic January 24, 2001 Assessment Observation - Adverse Trend in Resolution of Category (A)(1) Issues CAP029070 T-388 (CV-0824 D/P test) suspended due to April 19, 2001 difficulties with CV-0824 and CV-0847 CAP029142 Recommended Inspection/Repair of Traveling December 7, 2001 Screens in Summer 2001 Not Performed CAP029523 EDG 1-1 Shutdown during TS surveillance test July 6, 2002 MO-7A-1 due to leaking petcock on cylinder 7R CAP029678 Heat exchanger performance monitoring issue not documented by condition report CAP029749 Component Cooling Water System Maintenance July 24, 2002 Rule Category (a)(1) action plan CAP029842 CCW pump P-52B rotating element assembled July 31, 2002 incorrectly CAP029856 Concrete Cracking on E-9C Supports May 6, 1999 CAP029920 Component CV-5426 Valve Failed PMT June 10, 2002 CAP029941 Elevated Pump Seal Leakage on Main June 17, 2000 Feedwater Pump P-1A CAP029945 Removed Feedwater Pump P-1A From Service March 11, 2000 Due to Degraded Inboard Pump Seal CAP030041 NRC Residents Identified Human Performance February 8, 2002 as a Cross-Cutting Issue and Assigned a No Color Finding CAP030073 Labels Missing Off Various Auxiliary Building May 21, 2002 Components and Equipment CAP030334 Repeat Failure of 1-1Emergency Diesel June 8, 2002 Generator Cylinder 2R Exhaust Temperature Indicator CAP030377 Insulation on Condensate Tank Level Sensing December 12, 2001 Lines Not Installed in a Timely Fashion CAP030400 Inappropriate Result From the Analysis in March 29, 2002 Support of an Assigned Corrective Action CAP030456 Unsafe Access to Emergency Lighting Units June 23, 2000 7 Attachment

CAP030444 Resonance of the Vertical Structural Members July 17, 2000 that are Part of the PCP Motor Backstop Assembly Contributes to Oil Leaks CAP030770 The Limitations of a Fire Watch During Times July 17, 2000 When Actual Hot Work is Not Taking Place is Unclear CAP030790 Pump P-10B Disassembly following Mechanical May 5, 2001 Seal Failure - Inconsistencies Noted CAP030823 Spurious Control Room Alarm EK-0736, Boric May 5, 2000 Acid Critical Heat Trace System Trouble CAP030829 Drop in P-10A, Heater Drain Pump Amps With May 11, 2001 Flow Dropping to Zero CAP031405 Unexplained rise in containment gas monitor September 25, 2002 RIA-1817 counts CAP031655 National Fire Protection Association (NFPA) Fire October 14, 2002 Code Concern Regarding a Sprinkler Head in the Electrical Equipment Room 725 CAP031703 Service Water Pump P-7A Packing Leakage October 16, 2002 CAP031752 Service Water Pump P-7AShaft Appears Not to October 21, 2002 Have Been Changed to Stainless CAP031939 Scaffold erection begun without full authorization November 4, 2002 CAP031961 NRC identified that scaffold in component November 5, 2002 cooling water system room was improperly restrained CAP032007 NRC identified inconsistent application of November 7, 2002 procedural trending requirements CAP032010 Scaffold in electrical equipment room not built to November 7, 2002 seismic requirements CAP032012 Scaffold in auxiliary feed water pump room not November 7, 2002 built to seismic requirements CAP032013 Scaffold in 1-D switchgear room not left in a November 7, 2002 seismically secured condition CAP032032 NRC identified that characterization testing data November 7, 2002 on old P-52B was missing from archived work package CAP032053 NRC identified that ladders were improperly November 8, 2002 stored in the west safeguards room 8 Attachment

CAP032054 NRC identified that in progress scaffold erection November 8, 2002 in the 1-D switchgear room was not adequately braced CAP032145 NRC identified that deficient procedure was not November 18, 2002 segregated to prevent use until corrections can be made CAP032153 NRC identified that scaffold was in contact with November 18, 2002 safety-related piping for the control room emergency ventilation system CAP032179 Inadequate Evaluation of Heat Exchanger November 20, 2002 Condition CAP032158 NRC identified that the acceptance criteria for November 19, 2002 containment air cooler inspections was not revised to reflect increased inspection frequency CAP032179 Inadequate Evaluation of Heat Exchanger November 20, 2002 Condition CAP032186 Insight Regarding Floor Coating/Fire Hazard November 20, 2002 Analysis Generic Issue CAP032211 Failure to Perform an Adequate Apparent November 21, 2002 Cause Evaluation CAP032212 NRC identified that an action request was not November 21, 2002 initiated for a condition adverse to quality documented on Procedure Change Request 21030 CAP032213 NRC identified failure to adequately evaluate OE November 21, 2002 14230 - plastic tubing fails during check valve testing CIED0100227 SER 00-007 - BWR core power oscillations January 22, 2001 CIED0103143 SEN-222 - Emergency Diesel fuel oil storage October 1, 2001 tank water intrusion CIED0103288 SEN 223 - Debris in essential service water October 15, 2001 system results in low cooling flow to emergency diesel generators CIED0103639 SEN 224 - Recurring event, inadvertent reactor November 14, 2001 vessel inventory reduction during RHR cross tie line flushing CIED0104029 SEN 227 - Improper fuel reloading results in the December 18, 2001 incorrect locations for 113 fuel assemblies 9 Attachment

CIED0200225 SER 1-02: Intake structure blockage results in January 14, 2002 multi-unit transients and potential loss of heat sink CIED0200454 IN 02-05: foreign material in standby liquid February 1, 2002 control storage tanks CIED0201019 IN 02-10: non conservative water level setpoints April 13, 2002 on steam generators CPAL0000151 Diesel Generator Room Temperature Below January 17, 2000 System Operating Procedure Requirement CPAL0000320 Safety Injection and Refueling Water Tank February 1, 2000 Calculated Stresses Exceed Allowables Under Unresolved Safety Issue (USI) A-46 Criteria CPAL0100335 Some Maintenance Workers Unwilling to Report October 17, 2001 Injuries CPAL0100457 Less Than Required Component Cooling Water February 9, 2001 Cooling Flow For Containment Spray P-54B CPAL0100764 Performance of Containment Sump Check February 9, 2001 Valves During Post-Design Basis Accident Recirculation Mode May Not be Acceptable CPAL0101551 NRC identified that EPRI personnel performed April 11, 2001 single tube testing on CCW heat exchanger E-54A without an approved procedure CPAL0101826 Component Cooling Water Flow Rate to Spray May 5, 2001 Pump Seal Heat Exchanger Found Below Minimum Expected Value CPAL0102497 Cordless Drill for Backup Motor Operated July 24, 2001 Disconnect Operation Not Adapted to the Modification CPAL0102826 Weaknesses in Condition Report Evaluations August 29, 2001 and Corrective Actions CPAL0103100 Rotating Equipment Issues September 26, 2001 CPAL0103309 Potential Green Finding from Problem October 16, 2001 Identification and Resolution Inspection CPAL0103310 Potential Green Finding from Problem October 16, 2001 Identification and Resolution Inspection CPAL0103307 Potential Green Finding form Problem October 16, 2001 Identification and Resolution Inspection 10 Attachment

CPAL0103678 Lab Procedures Not Included in AP 10.41 November 19, 2001 Procedure Initiation and Revision as Requiring Periodic Reviews CPAL0103797 NRC Inspector Identified Potential Weaknesses November 30, 2001 in Engineering CPAL0103934 Insulation on Condensate Tank Level Sensing December 6, 2001 Lines Not Installed in a Timely Fashion CPAL0200101 Entered ONP-6.2, Loss of Component Cooling January 7, 2002 Water, while restoring P-50B lube oil coolers CPAL020014 Component cooling water P-52C breaker tripped January 2, 2002 open on time overcurrent CPAL0200292 Main Feedwater Pump 1A Discharge Pressure January 17, 2002 Higher Than Expected Following Main Feedwater Pump 1B Startup CPAL0200344 Equipment Control Processes/Practices January 23, 2002 Assessment Needed CPAL0200447 Final Results of Completing Surveillance February 1, 2002 Procedure FPSP-AE-4, Emergency Lighting Unit Battery Conductance and Discharge Test CPAL0200526 Component cooling water pump P-52C failed TS February 7, 2002 surveillance QO-15C CPAL0200580 LCO action time challenged due to February 12, 2002 implementation of emergent corrective maintenance CPAL0200586 Ineffective Corrective Actions for Action Follow- February 13, 2002 up Item (or 2-1) on Not Establishing and Reinforcing High Standards CPAL0200601 Fire Main Break at A Cooling Tower Results in February 14, 2002 all Fire Pumps Starting CPAL0200620 Effectiveness of Corrective Actions Generated February 15, 2002 from C/Rs Needs Improvement CPAL0200624 Incomplete Corrective Actions for P-10A/B March 6, 2002 Failure CPAL0200702 Attempts to disassemble MV-CC923 aborted; February 20, 2002 restoration identifies new condition 11 Attachment

CPAL0200756 Potential Containment Spray Pump Component February 21, 2002 Cooling Water Flow Rate Anomalies Recorded During 2001 Performance of Special Test T-223 CPAL0201025 Component Cooling Water Flow Rates to March 13, 2002 Containment Spray Pump P-54CBelow Expected Value CPAL0201099 Non-conservative Logarithmic Plotting Error in March 19, 2002 Excel Graphs in Three Engineering Analyses Which Effect Containment Response Calculations CPAL0201160 Diesel Generator Corridor Fire Door Frames are March 21, 2002 Not in Compliance With Our 1978 Safety Evaluation Report Licensing Basis CPAL 0201343 Untimely Implementation of Actions to Repair April 5, 2002 High Pressure Air Check Valve CPAL0201619 Difficulties obtaining extent-of-condition data on April 24, 2002 CCW pump P-52B CPAL0201838 Inspection of Spare Auxiliary Feed Water Motor May 8, 2002 Deficiencies Found CPAL0202351 Dispersed contamination in east engineering June 18, 2002 safeguards CPAL0202517 Plant Oversight of Non-Station Workers Lacks August 21, 2002 Rigor CPAL032007 NRC identified that expectation for use of November 7, 2002 Maintenance shift turnover sheet was not properly understood by department supervisors CPAL032027 NRC identified that Maintenance department November 7, 2002 procedural place keeping expectation were not met 12 Attachment