IR 05000361/2008012: Difference between revisions

From kanterella
Jump to navigation Jump to search
(Created page by program invented by StriderTol)
 
(Created page by program invented by StriderTol)
Line 154: Line 154:
During the inspection, the team noted that some corrective actions documented in action requests for equipment failures were inadequate to correct the stated deficiency. Upon further review, the licensee provided additional documentation to demonstrate that actual corrective actions taken were adequate. The team noted that while there was very poor documentation for corrective actions taken, the licensee was able to demonstrate that the actual actions taken were effective in addressing the equipment deficiencies appropriately. Additionally the team observed that while the licensee was aware of numerous failures to follow corrective action program procedures, the staff generally failed to self-identify ineffective or improper closeout of corrective actions and re-enter these issues into the corrective action program for further action.
During the inspection, the team noted that some corrective actions documented in action requests for equipment failures were inadequate to correct the stated deficiency. Upon further review, the licensee provided additional documentation to demonstrate that actual corrective actions taken were adequate. The team noted that while there was very poor documentation for corrective actions taken, the licensee was able to demonstrate that the actual actions taken were effective in addressing the equipment deficiencies appropriately. Additionally the team observed that while the licensee was aware of numerous failures to follow corrective action program procedures, the staff generally failed to self-identify ineffective or improper closeout of corrective actions and re-enter these issues into the corrective action program for further action.


3. Findings (a) Inadequate Maintenance Procedures Results in a Ground on the Safety-Related 125 VDC Distribution System
3. Findings
: (a) Inadequate Maintenance Procedures Results in a Ground on the Safety-Related 125 VDC Distribution System


=====Introduction.=====
=====Introduction.=====
Line 170: Line 171:
Technical Specification 5.5.1 requires that written procedures be established, implemented, and maintained covering the activities specified in Appendix A, "Typical Procedures for Pressurized Water Reactors and Boiling Water Reactors" of Regulatory Guide 1.33, "Quality Assurance Program Requirements (Operation)," dated February 1978. Regulatory Guide 1.33, Appendix A, Section 9.a, requires, in part, that procedures for performing maintenance be properly pre-planned and performed in accordance with documented instructions, appropriate to the circumstances. Contrary to the above, on January 24, 2008, the licensee failed to provide documented instructions to ensure that maintenance would not affect the performance of safety-related equipment. Specifically, Maintenance Order 07071321000A did not provide instructions to address the potential for maintenance practices to cause nicked wires during replacement of flooding switch covers, which resulted in an electrical ground on the safety-related Unit 2 electrical distribution Bus 2D2.
Technical Specification 5.5.1 requires that written procedures be established, implemented, and maintained covering the activities specified in Appendix A, "Typical Procedures for Pressurized Water Reactors and Boiling Water Reactors" of Regulatory Guide 1.33, "Quality Assurance Program Requirements (Operation)," dated February 1978. Regulatory Guide 1.33, Appendix A, Section 9.a, requires, in part, that procedures for performing maintenance be properly pre-planned and performed in accordance with documented instructions, appropriate to the circumstances. Contrary to the above, on January 24, 2008, the licensee failed to provide documented instructions to ensure that maintenance would not affect the performance of safety-related equipment. Specifically, Maintenance Order 07071321000A did not provide instructions to address the potential for maintenance practices to cause nicked wires during replacement of flooding switch covers, which resulted in an electrical ground on the safety-related Unit 2 electrical distribution Bus 2D2.


Because the finding is of very low safety significance and has been entered into the licensee's corrective action program as Nuclear Notification 200177574, this violation is being treated as noncited violation consistent with Section VI.A of the NRC Enforcement Policy: noncited Violation 05000361/2008012-01, "Inadequate Maintenance Procedures Results in Ground on the Safety-Related 125 VDC Distribution System." (b) Failure to Properly Implement Operability Determination Process
Because the finding is of very low safety significance and has been entered into the licensee's corrective action program as Nuclear Notification 200177574, this violation is being treated as noncited violation consistent with Section VI.A of the NRC Enforcement Policy: noncited Violation 05000361/2008012-01, "Inadequate Maintenance Procedures Results in Ground on the Safety-Related 125 VDC Distribution System."
: (b) Failure to Properly Implement Operability Determination Process


=====Introduction.=====
=====Introduction.=====
Line 189: Line 191:


=====Enforcement.=====
=====Enforcement.=====
Title 10 CFR Part 50, Appendix B, Criterion V, "Instructions, Procedures and Drawings," requires that activities affecting quality shall be prescribed by instructions, procedures, or drawings, and shall be accomplished in accordance with those instructions, procedures, and drawings. Procedure SO123-XV-52, "Functionality Assessments and Operability Determinations," Revision 8, requires, in part, the licensee to assess for operability any known degraded conditions of safety-related equipment needed to mitigate accidents. Contrary to the above, on August 9, 2007, operations personnel failed to adequately evaluate a degraded and nonconforming condition associated with the Emergency Diesel Generator 3G002. Specifically, operations personnel failed to consider all relevant information when determining whether the idle speed Relay 3L160-2-K52 affected the operability of the Emergency Diesel Generator 3G002. Because the finding is of very low safety significance and has been entered into the licensee's corrective action program as Nuclear Notification 200146292, this violation is being treated as noncited violation consistent with Section VI.A of the Enforcement Policy: noncited Violation 05000362/2008012-02, "Failure to Properly Implement Operability Determination Process." (c) Inadequate Auxiliary Feedwater Pump Room Heat Load
Title 10 CFR Part 50, Appendix B, Criterion V, "Instructions, Procedures and Drawings," requires that activities affecting quality shall be prescribed by instructions, procedures, or drawings, and shall be accomplished in accordance with those instructions, procedures, and drawings. Procedure SO123-XV-52, "Functionality Assessments and Operability Determinations," Revision 8, requires, in part, the licensee to assess for operability any known degraded conditions of safety-related equipment needed to mitigate accidents. Contrary to the above, on August 9, 2007, operations personnel failed to adequately evaluate a degraded and nonconforming condition associated with the Emergency Diesel Generator 3G002. Specifically, operations personnel failed to consider all relevant information when determining whether the idle speed Relay 3L160-2-K52 affected the operability of the Emergency Diesel Generator 3G002. Because the finding is of very low safety significance and has been entered into the licensee's corrective action program as Nuclear Notification 200146292, this violation is being treated as noncited violation consistent with Section VI.A of the Enforcement Policy: noncited Violation 05000362/2008012-02, "Failure to Properly Implement Operability Determination Process."
: (c) Inadequate Auxiliary Feedwater Pump Room Heat Load


=====Analysis.=====
=====Analysis.=====

Revision as of 07:43, 20 September 2018

IR 05000361-08-012 and 05000362-08-012; on 09/08/2008 - 10/03/2008; San Onofre Nuclear Generating Station: Biennial Baseline Inspection of the Identification and Resolution of Problems
ML083390399
Person / Time
Site: San Onofre  Southern California Edison icon.png
Issue date: 12/03/2008
From: Anton Vegel
NRC/RGN-IV/DRP
To: Ridenoure R T
Southern California Edison Co
Deborah Harrison
References
EA-07-232, FOIA/PA-2011-0221, FOIA/PA-2011-0157 IR-08-012
Download: ML083390399 (46)


Text

December 3, 2008

EA-07-232 Mr. Ross Senior Vice President and Chief Nuclear Officer Southern California Edison Company San Onofre Nuclear Generating Station P.O. Box 128 San Clemente, CA 92674-0128

SUBJECT: SAN ONOFRE, NUCLEAR GENERATING STATION UNITS 2 AND 3 - NRC PROBLEM IDENTIFICATION AND RESOLUTION INSPECTION REPORT 05000361/2008012 AND 05000362/2008012, AND CONFIRMATORY ORDER (EA-07-232) FOLLOW-UP INSPECTION

Dear Mr. Ridenoure:

On October 3, 2008, the U.S. Nuclear Regulatory Commission (NRC) completed a team inspection at your San Onofre Nuclear Generating Station. The enclosed inspection report documents the inspection findings which were discussed on October 3, 2008, with Mr. M. Short and other members of your staff during an exit meeting. A subsequent telephone exit was conducted on October 22, 2008, with Mr. D. Axline.

This inspection reviewed activities conducted under your license as they relate to the identification and resolution of problems, compliance with the Commission's rules and regulations and the conditions of your operating license. Within these areas, the inspection involved examination of selected procedures and representative records, observations of activities, and interviews with personnel.

The inspection team reviewed approximately 400 action requests and notifications, work orders, associated root and apparent cause evaluations, and other supporting documentation to assess problem identification and resolution activities. The team determined that once problems were identified, your staff usually entered the issues into the corrective action program, but your staff was inconsistent in ensuring that identified problems were thoroughly evaluated in a timely manner. The team identified several issues with the quality of cause evaluations and the completeness of corrective action documents. The team identified that operability assessments and reportability reviews were not being implemented consistent with procedural guidance and Southern California Edison Company - 2 -

many of these assessments did not demonstrate the appropriate level of technical rigor to support conclusions made for operability or reportability. Your ability to effectively evaluate problems has been and continues to be a concern to the NRC. This concern was documented in the past two NRC assessment letters, dated March 3 and September 2 of 2008. The NRC will continue to focus our inspections in this area until sustained improvement is demonstrated.

The team conducted interviews with 56 individuals, and conducted focus groups with approximately 50 others. On the basis of the interviews conducted during this inspection, observations of plant activities, and reviews of the corrective action and Nuclear Safety Concerns programs, the team determined that site personnel were willing to raise safety issues and document them in the corrective action program. The team observed that workers at the site felt free to report problems to their management, and were willing to use the Nuclear Safety Concerns program.

The inspection team also reviewed the corrective actions associated with the Confirmatory Order, dated January 11, 2008 (ADAMS Reference ML080110380). The team reviewed associated corrective actions and interviewed personnel to assess and evaluate the effectiveness of your performance improvement initiatives. The team determined that SONGS has demonstrated sufficient progress for closure of Items 1, 2d, 2e, 2f, 2i, 2j, 2k, and 3 of the Confirmatory Order. For Items 2a, 2b, 2c, 2g, 2h, and 2l, the team was unable to determine that your staff had taken sufficient actions; therefore, these items will remain open. Subsequently, your staff responded to the open items in a letter titled, "Response to Confirmatory Order EA 07-232 and Notice of Violation EA 07-141," dated November 14, 2008, to define and establish success criteria for closure of each of the Confirmatory Order Items. In this letter, Southern California Edison committed to submit a status report in January of 2009 discussing completion of each individual Confirmatory Order Item. Following receipt of the status letter, the NRC plans to perform additional inspections to evaluate your progress on the open items. Three findings were evaluated under the significance determination process as having very low safety significance (Green). These findings were determined to be violations of NRC requirements. However, because these violations were of very low safety significance and the issues were entered into your corrective action program, the NRC is treating these findings as noncited violations, consistent with Section VI.A.1 of the NRC's Enforcement Policy. The noncited violations are described in the subject inspection report. If you contest the violations or the significance of the violations, you should provide a response within 30 days of the date of this inspection report, with the basis for your denial, to the U. S. Nuclear Regulatory Commission, ATTN: Document Control Desk, Washington, D.C. 20555-0001, with copies to the Regional Administrator, U. S. Nuclear Regulatory Commission, Region IV, 612 East Lamar Blvd, Suite 400, Arlington, Texas, 76011; the Director, Office of Enforcement, U.S. Nuclear Regulatory Commission, Washington, D.C. 20555-0001; and the NRC resident inspector at the San Onofre Nuclear Generating Station facility.

Southern California Edison Company - 3 -

In accordance with 10 CFR 2.390 of the NRC's "Rules of Practice," a copy of this letter, its enclosure, and your response will be made available electronically for public inspection in the NRC Public Document Room or from the Publicly Available Records component of NRC's document system (ADAMS). ADAMS is accessible from the NRC Web site at http://www.nrc.gov/reading-rm/adams.html (the Public Electronic Reading Room).

Sincerely,/RA/ Anton Vegel, Deputy Director Division of Reactor Projects Docket: 50-361, 50-362 License: NPF-10, NPF-15

Enclosure:

NRC Inspection Report 05000361 and 05000362/2008012

w/Attachment:

1. Supplemental Information 2. Information Request cc w/enclosure: Chairman, Board of Supervisors County of San Diego 1600 Pacific Highway, Room 335 San Diego, CA 92101 Gary L. Nolff Assistant Director-Resources City of Riverside 3900 Main Street Riverside, CA 92522 Mark L. Parsons Deputy City Attorney City of Riverside 3900 Main Street Riverside, CA 92522 Dr. David Spath, Chief Division of Drinking Water and Environmental Management California Department of Health Services 850 Marina Parkway, Bldg P, 2nd Floor Richmond, CA 94804 Southern California Edison Company - 4 -

Michael J. DeMarco San Onofre Liaison San Diego Gas & Electric Company 8315 Century Park Ct. CP21G San Diego, CA 92123-1548 Director, Radiological Health Branch State Department of Health Services P.O. Box 997414 (MS 7610) Sacramento, CA 95899-7414 Mayor City of San Clemente 100 Avenida Presidio San Clemente, CA 92672 James D. Boyd, Commissioner California Energy Commission 1516 Ninth Street (MS 34)

Sacramento, CA 95814 Douglas K. Porter, Esq. Southern California Edison Company 2244 Walnut Grove Avenue Rosemead, CA 91770 Albert R. Hochevar Southern California Edison Company San Onofre Nuclear Generating Station P.O. Box 128 San Clemente, CA 92675 A. Edward Scherer Southern California Edison Company San Onofre Nuclear Generating Station P.O. Box 128 San Clemente, CA 92674-0128 Mr. Steve Hsu Department of Health Services Radiologic Health Branch MS 7610, P.O. Box 997414 Sacramento, CA 95899-7414

Southern California Edison Company - 5 -

Mr. James T. Reilly Southern California Edison Company San Onofre Nuclear Generating Station P.O. Box 128 San Clemente, CA 92674-0128 Chief, Radiological Emergency Preparedness Section National Preparedness Directorate Technological Hazards Division Department of Homeland Security 1111 Broadway, Suite 1200 Oakland, CA 94607-4052

Southern California Edison Company - 6 -

Electronic distribution by RIV: Regional Administrator (Elmo.Collins@nrc.gov ) DRP Director (Dwight.Chamberlain@nrc.gov) DRP Deputy Director (Anton.Vegel@nrc.gov ) DRS Director (Roy.Caniano@nrc.gov ) DRS Deputy Director (Troy.Pruett@nrc.gov) Senior Resident Inspector (Greg.Warnick@nrc.gov) Resident Inspector (John.Reynoso@nrc.gov ) Branch Chief, DRP/D (Michael.Hay@nrc.gov) Senior Project Engineer, DRP/D (Don.Allen@nrc.gov ) SO Site Secretary (Heather.Hutchinson@nrc.gov ) Public Affairs Officer (Victor.Dricks@nrc.gov ) Team Leader, DRP/TSS (Doug.Bollock@nrc.gov ) RITS Coordinator (Marisa.Herrera@nrc.gov ) DRS STA (Dale.Powers@nrc.gov) S. Williams, OEDO RIV Coordinator (Shawn.Williams@nrc.gov) ROPreports Senior Allegations Coordinator (Mark.Haire@nrc.gov)

SUNSI Review Completed: _RIT_ ADAMS: Yes No Initials: _RIT

_ Publicly Available Non-Publicly Available Sensitive Non-Sensitive

SRI/PBD RI/PBD RI/EB2 RI/EB2 HQ/OE SRI/PBD RI/PSB2 RITreadway JPReynoso EDUribe BKCorrell JXCai CCOsterholtz MTBaquera/RA/ /RA/ /RA/ /RA/ /RA/ /RA/ /RA/ 11/3/08 11/3/08 11/3/08 11/3/08 10/30/08 11/6/08 11/3/08 C/PSB2 D/DRS C/PBD DD/DRP GEWerner RJCaniano MCHay AXVegel /RA/ /RA/ /RA/ /RA

/ 12/2/08 12/2/08 12/2/08 12/3/08 OFFICIAL RECORD COPY T=Telephone E=E-mail F=Fax Enclosure U.S. NUCLEAR REGULATORY COMMISSION REGION IV Docket: 50-361, 50-362

License: NPF-10, NPF-15 Report: 05000361/2008012 and 05000362/2008012

Licensee: Southern California Edison, Co.

Facility: San Onofre Nuclear Generating Station, Units 2 and 3 Location: 5000 South Pacific Coast Highway San Clemente, California Dates: September 15, 2008 through October 3, 2008 Inspectors: R. Treadway, Senior Resident Inspector (Team Leader) J. Reynoso, Resident Inspector C. Osterholtz, Senior Resident Inspector J. Cai, Enforcement Specialist E. Uribe, Reactor Inspector B. Correll, Reactor Inspector M. Baquera, Reactor Inspector Approved By: Anton Vegel, Deputy Director Division of Reactor Projects Enclosure

SUMMARY OF FINDINGS

IR 05000361/2008012 and 05000362/2008012; 9/8/08 - 10/3/08; San Onofre Nuclear Generating Station: Biennial baseline inspection of the Identification and Resolution of Problems.

The report covered a 2-week period of inspection by two senior resident inspectors, a resident inspector, three region-based inspectors, and one enforcement specialist. Three Green findings of very low safety significance were identified during the inspection. The significance of most findings is indicated by their color (Green, White, Yellow, or Red) using Inspection Manual Chapter 0609, "Significance Determination Process." Findings for which the significance determination process does not apply may be Green or be assigned a severity level after NRC management review. The NRC's program for overseeing the safe operation of commercial nuclear power reactors is described in NUREG-1649, "Reactor Oversight Process," Revision 4, dated December 2006.

Identification and Resolution of Problems

The team selected and reviewed approximately 400 risk-informed action requests and notifications, work orders, associated root and apparent cause evaluations, and other supporting documentation to assess problem identification and resolution activities. The inspectors verified that the licensee had taken actions to address previous NRC findings. The team performed a five year review of the auxiliary feedwater system to determine whether problems were being effectively addressed and that the corrective action program was effective in identifying problems. As a result of these reviews, the team concluded that when site personnel identified problems, they entered them into the corrective action program at a low threshold; however, the team identified several issues with the quality of cause evaluations and overall documentation of corrective action documents. Corrective actions were generally implemented in a timely manner, although the team identified several corrective actions associated with conditions adverse to quality that were not completed in a timely manner. The team also identified that operability assessments and reportability reviews were not being implemented consistent with procedural guidance, and many of these assessments did not demonstrate the appropriate level of technical rigor to support conclusions made for operability or reportability.

The team determined that the licensee identified, reviewed, and applied industry operating experience relevant to the facility, and had entered applicable items into the corrective action program. The team noted that the licensee was evaluating industry operating experience when performing root cause and apparent cause evaluations. The team also noted that Quality Assurance audits and other self-assessment activities were generally effective.

The team reviewed the corrective action plans to address substantive cross-cutting issues in the areas of procedural adequacy and evaluations, and noted that both had recently been re-assessed and revised. The team concluded that the licensee made minimal progress in implementing corrective actions for these plans. The team further concluded that while the identified corrective actions should address the cross-cutting issues, the team could not assess and evaluate these corrective action plans because they were in the early stages of implementation.

Based on 56 interviews and six focus groups (consisting of approximately 50 people) conducted during this inspection, observations of plant activities, and reviews of the corrective action and nuclear safety concerns programs, the team determined that site personnel were willing to raise safety issues and document them in the corrective action program. The team observed that workers at the site felt free to report problems to their management, and were willing to use the Nuclear Safety Concerns program.

A. NRC-Identified and Self-Revealing Findings

Cornerstone: Mitigating Systems

Green.

A self-revealing noncited violation of Technical Specification 5.5.1, "Procedures," was identified for the failure of maintenance personnel to have adequate procedures. Specifically, on January 24, 2008, during maintenance on a flood detector switch, an inadequate procedure resulted in an inadvertent electrical ground on the safety-related Unit 2 electrical distribution Bus 2D2. This issue was entered into the licensee's corrective action program as Nuclear Notification 200177574. The licensee plans to revise the maintenance procedure and train maintenance personnel.

The performance deficiency associated with this finding involved the failure of maintenance personnel to ensure written guidance was provided in documented instructions to ensure nicked wires did not cause electrical grounds during maintenance activities. The finding was more than minor because it affected the procedure quality attribute of the mitigating systems cornerstone, and affected the cornerstone objective to ensure the reliability of systems that respond to initiating events to prevent undesirable consequences. Using the Manual Chapter 0609.04, "Phase 1 - Initial Screening and Characterization of Findings," the finding was determined to have a very low safety significance because it did not result in an actual loss of system safety function, did not result in a loss of a single train of safety equipment for greater than its technical specification allowed outage time, and did not screen as potentially risk significant due to seismic, flooding, or severe weather initiating events. The finding had a cross-cutting aspect in the area of human performance associated with work control because the work instruction was not planned appropriately to address the human-system interface and ensure grounds were not caused by maintenance activities H.3(a) (Section 4OA2.a.3(a)).

Green.

The team identified a noncited violation of 10 CFR Part 50, Appendix B, Criterion V, "Instructions, Procedures and Drawings," for the failure of operations personnel to follow procedures and adequately evaluate a degraded condition. Specifically, on August 5, 2007, operations personnel failed to properly evaluate a degraded relay that affected the operability of the Unit 3 Train A emergency diesel generator. This issue was entered into the licensee's corrective action program as Nuclear Notification 200146292. The licensee began performing a failure analysis on the relay and initiated an apparent cause evaluation for this relay failure. The performance deficiency associated with the finding was the failure of operations personnel to adequately implement the operability determination process. The finding was more than minor because it is associated with the equipment performance attribute of the mitigating systems cornerstone and affects the associated cornerstone objective to ensure the availability, reliability, and capability of systems that respond to initiating events to prevent undesirable consequences. Using the Manual Chapter 0609.04, "Phase 1 - Initial Screening and Characterization of Findings," the finding was determined to have a very low safety significance because it did not result in an actual loss of system safety function, did not result in a loss of a single train of safety equipment for greater than its technical specification allowed outage time, and did not screen as potentially risk significant due to seismic, flooding, or severe weather initiating events. The finding had a cross-cutting aspect in the area of human performance associated with decision-making because the licensee failed to use conservative assumptions for operability decision-making when evaluating a degraded and nonconforming condition H.1(b) (Section 4OA3.a.3(b)).

Green.

The team identified a noncited violation of 10 CFR Part 50, Appendix B, Criterion III, "Design Control," for the failure of engineering personnel to ensure the auxiliary feedwater pump room heat load calculation was adequate.

Specifically, since initial plant construction, engineering personnel failed to consider the impact to the auxiliary feedwater pump room's heat load design basis calculation for the most limiting scenario, in which all auxiliary feedwater pumps in the room have started and are running with only one emergency room cooler available. This issue was entered into the licensee's corrective action program as Nuclear Notification 200149442. The licensee plans to perform a revised auxiliary feedwater pump room heat load analysis.

The performance deficiency associated with this finding was the failure of engineering personnel to include the proper heat load scenarios and use of realistic assumptions for a design basis calculation. The finding was more than minor because it affected the design control attribute of the mitigating systems cornerstone and affected the objective to ensure the reliability and capability of systems that respond to initiating events to prevent undesirable consequences. Using the Manual Chapter 0609.04, "Phase 1 - Initial Screening and Characterization of Findings," the finding was determined to have a very low safety significance because it did not result in an actual loss of system safety function, did not result in a loss of a single train of safety equipment for greater than its technical specification allowed outage time, and did not screen as potentially risk significant due to seismic, flooding, or severe weather initiating events. This finding was evaluated as not having a cross-cutting aspect because the performance deficiency was not indicative of current performance (Section 4OA2.a.3(c)).

B. Licensee-Identified Violations

None

REPORT DETAILS

OTHER ACTIVITIES (OA)

4OA2 Problem Identification and Resolution

The team based the following conclusions, in part, on a review of issues that were identified in the assessment period, which ranged from September 22, 2006, (the last biennial problem identification and resolution inspection) to the end of the on-site portion of the inspection on October 3, 2008.

a. Assessment of Corrective Action Program Effectiveness 1. Inspection Scope The team reviewed approximately 400 action requests and notifications, including associated root cause, apparent cause, and direct cause evaluations, from approximately 40,000 that had been issued between September 2006 and October 2008 to determine if problems were being properly identified, characterized, and entered into the corrective action program for evaluation and resolution. The team reviewed a sample of system health reports, self-assessments, trending reports and metrics, and various other documents related to the corrective action program. The team evaluated the licensee's efforts in establishing the scope of problems by reviewing selected logs, work requests, self-assessments results, audits, system health reports, action plans, and results from surveillance tests and preventive maintenance tasks. The team reviewed work requests and attended the licensee's daily action request review committee and the management review committee meetings to assess the reporting threshold, prioritization efforts, and significance determination process, as well as observing the interfaces with the operability assessment and work control processes. The team's review included verifying the licensee considered the full extent of cause and extent of condition for problems, as well as how the licensee assessed generic implications and previous occurrences. The team assessed the timeliness and effectiveness of corrective actions, completed or planned, and looked for additional examples of similar problems. The team conducted interviews with plant personnel to identify other processes that may exist where problems may be identified and addressed outside the corrective action program.

The team considered risk insights from both the NRC's and San Onofre Nuclear Generating Station's (SONGS) risk assessments to focus the sample selection and plant tours on risk significant systems and components. The team selected the following risk significant systems: emergency diesel generators, auxiliary feedwater, 4160 and 480VAC electrical distribution, main steam, component cooling water, and salt water cooling systems. The samples reviewed by the team focused on, but were not limited to, these systems. The team also expanded their review to include five years of evaluations involving the auxiliary feedwater system to determine whether problems were being effectively addressed. The team conducted a walkdown of this system to assess whether problems were identified and entered into the work order process.

Additionally, the team reviewed the licensee's evaluations and corrective action plans associated with substantive cross-cutting issues in procedure adequacy and failures to thoroughly evaluate problems that were previously identified by the NRC. The team also reviewed the actions associated with the station's transition to a new corrective action program software tool: System's, Analysis and Programs (SAP). The team reviewed training plans and corrective action program metrics, conducted interviews, and reviewed the change management process to evaluate the overall effectiveness of this transition.

2. Assessments

Effectiveness of Problem Identification The team concluded that problems were generally identified and documented in accordance with the licensee's corrective action program guidance and NRC requirements. The team concluded the licensee was identifying problems at an appropriately low threshold. The licensee had written approximately 40,000 action requests and notifications during the two year period of review, which demonstrated that the licensee was identifying problems and entering them into the corrective action program.

Effectiveness of Prioritization and Evaluation of Issues The team reviewed action requests and notifications that involved operability reviews to assess the quality and timeliness of operability assessments. The team noted that the immediate and prompt operability assessments reviewed were completed in a timely manner; however, operability assessment documentation was limited, making it difficult for the team to evaluate the appropriateness of the operability assessments. Operability assessments were routinely performed without significant engineering input, generally involved little discussion about what function(s) were impacted and why performance of a component/system was sufficient to fulfill these functions as required by station procedures. The team concluded that the operability assessments which were based on engineering judgment were not labeled as such, and the procedural requirements for this type of result were not followed to confirm that the assumptions were correct. The team noted several of the immediate operability assessments did not include adequate justification and documented conclusions for prompt operability assessments in a separate field assignment, contrary to station procedures. Overall, the team determined that both immediate and prompt operability reviews did not include the appropriate technical rigor to support conclusions as required by station procedures. The team reviewed 56 of 246 operability assessments during the inspection period and identified 23 reviews that did not include an adequate engineering review or that were not completed in accordance with station procedures. Some examples included:

  • Operations personnel failed to evaluate the operability of the Emergency Diesel Generator 3A (noncited Violation 05000362/2008006-01) in accordance with station procedures for a degrading condition associated with unexplained load swings (Action Request 071201393).
  • Licensee personnel identified loose electrical wires for the Emergency Diesel Generator 2A emergency fan when the fan failed to start, but failed to assess the seismic effects appropriately during the operability review (Action Request 050601315).
  • Following a maintenance activity that identified a degraded relay, operations personnel failed to follow procedures and adequately evaluate the operability of the Emergency Diesel Generator 3A (Nuclear Notification 200146292).

Problems with the quality of operability assessments have been raised during prior NRC problem identification and resolution inspections (NRC report 05000361;362/2006012 and 05000361;362/2004009), and during recent special inspections conducted at SONGS (NRC report 05000361;362/2008006 and 05000361;362/2007013). The team reviewed the licensee's actions to improve in this area, and concluded that these actions have not been effective. While training was conducted, the team noted that the licensee had failed to recognize that procedures were not being followed for the operability assessments process, as described above.

The team reviewed the root cause evaluation, apparent cause evaluation, and direct cause evaluation procedures, including samples of these evaluations. The team noted that 31 of 73 cause evaluations reviewed were not completed within 60 days as required by procedures. The team noted that during the inspection period for 2006 and 2007, the root cause evaluation age was 52 days and the average apparent cause evaluation age was 36 days. The team observed during the 2008 period, the average root cause evaluation age was 30 days and the average apparent cause evaluation age was 25 days, indicating the trend is improving. The team also identified that 24 of 73 evaluations reviewed were not identifying specific, measurable, achievable, realistic and timely causes, which then subsequently resulted in developing inappropriate corrective actions. During the inspection, the team also identified several evaluations that did not identify the cause as required by station procedures. Some examples included:

  • Unit 3 main steam isolation valve root cause evaluation never identified a root cause for the valve's failure to close (Action Request 071000901).
  • The apparent cause evaluation for a failed Unit 2 control room controller did not correctly identify the cause and subsequently only identified corrective actions for the symptoms instead of the cause (Action Request 060200377).
  • The apparent cause evaluation for a failed Unit 3 atmospheric dump valve, identified during a surveillance test, was unsuccessful at identifying a cause because the valve was never quarantined as required by station procedures (Action Request 070900082).
  • The root cause evaluation for loose bolts associated with the 2A battery terminal never identified that the loose connection was caused by an inappropriate corrective action (Action Request 080400839).

The team determined that 5 of 30 root cause evaluations reviewed did not identify corrective actions to prevent recurrence as required by procedures. Two examples included:

  • Unit 3 main steam isolation valve root cause evaluation identified eleven corrective actions, but never specified which actions would prevent recurrence (Action Request 071000901).
  • The root cause evaluation for missing taper pins, affecting numerous component cooling water valves, identified that maintenance personnel failed to provide adequate oversight, but failed to identify a corrective action to prevent recurrence (Action Request 050200761).

Additionally, the team observed that the licensee did not track qualification records for cause evaluators, nor was there a training program established for cause evaluators.

Effectiveness of Corrective Actions Overall, the team concluded that the licensee developed appropriate corrective actions to address problems, although the team identified several corrective actions associated with conditions adverse to quality that were not completed in a timely manner. Some examples included:

  • The Unit 2 B emergency diesel generator air line support brackets (Action Request 040100080) were not corrected for greater than nine months.
  • Corrective actions for the broken studs for a Unit 2 auxiliary feedwater pump (Action Request 041100768) were not taken for greater than two years.
  • Corrective actions for the thermal overload failures (Action Request 070901311) were not taken for more than two years.
  • Corrective actions for degraded Unit 2 control room controllers (Action Request 060200377) were not taken for greater than 12 months.
  • Corrective actions to repair an erratic Unit 3 atmospheric dump valve (Action Request 070900082) took over six months to repair.
  • The installation of a new motor for the degraded circulating oil pump (Action Request 080300395) on the Unit 2 A emergency diesel generator was not complete for over six months.
  • The licensee identified that Unit 2 effluent releases were not measured in accordance with approved procedures. The procedure was not changed for over 12 months (Action Request 050200377).

During the inspection, the team noted that some corrective actions documented in action requests for equipment failures were inadequate to correct the stated deficiency. Upon further review, the licensee provided additional documentation to demonstrate that actual corrective actions taken were adequate. The team noted that while there was very poor documentation for corrective actions taken, the licensee was able to demonstrate that the actual actions taken were effective in addressing the equipment deficiencies appropriately. Additionally the team observed that while the licensee was aware of numerous failures to follow corrective action program procedures, the staff generally failed to self-identify ineffective or improper closeout of corrective actions and re-enter these issues into the corrective action program for further action.

3. Findings

(a) Inadequate Maintenance Procedures Results in a Ground on the Safety-Related 125 VDC Distribution System
Introduction.

A self-revealing Green noncited violation of Technical Specification 5.5.1, "Procedures," was identified for the failure of maintenance personnel to have adequate procedures. Specifically, on January 24, 2008, during maintenance on a flood detector switch, an inadequate procedure caused an inadvertent electrical ground on the safety-related Unit 2 electrical distribution Bus 2D2.

Description.

On January 24, 2008, during maintenance for a flood switch replacement, a 75 percent ground on the safety-related Unit 2 electrical distribution Bus 2D2 occurred when maintenance personnel replaced a flood switch cover. The team reviewed the flood switch replacement Maintenance Order 07071321000A and noted that a caution in the maintenance order stated, "In the past, personnel have received electrical shock from contact with the switch cover when leads were grounded internally. Low voltage gloves should be worn when removing and reinstalling switch cover." The team observed that this caution warns the technician to use electrical safety gloves to prevent electrical shock, but does not alert the technician to take measures to prevent nicked wires during switch replacement.

Upon their review of the maintenance order and the related work that caused the ground, the inspectors determined that the licensee had not provided sufficient instruction in the maintenance order document to ensure that grounds were not created during switch cover replacement. Additionally, the team noted that while this deficiency was entered into the licensee corrective action program, no corrective actions were identified for the inadequate maintenance procedure until the team discussed their observations with maintenance personnel. The licensee plans to revise the maintenance procedure and train maintenance personnel.

Analysis.

The performance deficiency associated with this finding was the failure of maintenance personnel to ensure written guidance was provided in documented instructions to ensure nicked wires (a known issue) did not cause electrical grounds during maintenance activities. The finding was greater than minor because it is associated with the mitigating systems cornerstone attribute of Procedure Quality and affects the cornerstone objective to ensure the reliability of systems that respond to initiating events to prevent undesirable consequences. Using the Manual Chapter 0609.04, "Phase I - Initial Screening and Characterization of Findings," the finding was determined to have very low safety significance (Green) because it did not represent a loss of system safety function, was not a design or qualification deficiency, did not involve an actual loss of safety function of a single train for greater than its technical specification allowed outage time, or screen as potentially risk-significant due to a seismic, flooding or severe weather initiating event. The finding had a cross-cutting aspect in the area of human performance associated with work control because the work instruction was not planned appropriately to address the human-system interface and ensure grounds were not caused by maintenance activities H.3(a).

Enforcement.

Technical Specification 5.5.1 requires that written procedures be established, implemented, and maintained covering the activities specified in Appendix A, "Typical Procedures for Pressurized Water Reactors and Boiling Water Reactors" of Regulatory Guide 1.33, "Quality Assurance Program Requirements (Operation)," dated February 1978. Regulatory Guide 1.33, Appendix A, Section 9.a, requires, in part, that procedures for performing maintenance be properly pre-planned and performed in accordance with documented instructions, appropriate to the circumstances. Contrary to the above, on January 24, 2008, the licensee failed to provide documented instructions to ensure that maintenance would not affect the performance of safety-related equipment. Specifically, Maintenance Order 07071321000A did not provide instructions to address the potential for maintenance practices to cause nicked wires during replacement of flooding switch covers, which resulted in an electrical ground on the safety-related Unit 2 electrical distribution Bus 2D2.

Because the finding is of very low safety significance and has been entered into the licensee's corrective action program as Nuclear Notification 200177574, this violation is being treated as noncited violation consistent with Section VI.A of the NRC Enforcement Policy: noncited Violation 05000361/2008012-01, "Inadequate Maintenance Procedures Results in Ground on the Safety-Related 125 VDC Distribution System."

(b) Failure to Properly Implement Operability Determination Process
Introduction.

The team identified a Green noncited violation of 10 CFR Part 50, Appendix B, Criterion V, "Instructions, Procedures and Drawings," for the failure of operations personnel to follow procedures and adequately evaluate a degraded condition. Specifically, on August 5, 2007, operations personnel failed to properly evaluate a degraded relay that affected the operability of the Unit 3 Train A emergency diesel generator.

Description.

On August 5, 2007, the Unit 3 Emergency Diesel Generator 3G002 was taken out of service for preventive maintenance. On August 9, 2007, the licensee performed preventive maintenance in the emergency diesel generator Cabinet 3L160. The maintenance activity instructs personnel to perform continuity checks for all associated contacts in the electrical cabinet to ensure they are in the correct position, and then perform relays checks to ensure the relays and associated contacts perform as expected when energized or de-energized. During performance of the maintenance activity, maintenance personnel reported (Action Request 070800466) that Relay 3L160-2-K52 (K52) failed because it was sluggish and would not rotate completely. Maintenance personnel replaced the failed relay and completed the maintenance activity with no other discrepancies noted. Operations personnel responded to the action request in an immediate operability determination and stated, in part, that the Cabinet 3L160 provides the control power to Emergency Diesel Generator 3G002 components and protective circuits and that the idle speed relay that failed gets overridden on a safety-injection actuation signal and does not have a true safety function. Based on this information, operations personnel determined that the Cabinet 3L160 was unaffected by the failed idle speed relay; the failure of the relay occurred during the maintenance activity; and the emergency diesel generator remained operable.

The team reviewed Action Request 070800466 and the associated immediate operability determination. The team reviewed Maintenance Order 07080493000 which the licensee used to perform relay checks on the Cabinet 3L160. After reviewing the maintenance order, the team observed in the notes section of the maintenance order that during functional testing, the relay failed to reset. The relay performed satisfactorily during the pick up function when energized; however, when de-energized, the relay would not return to its normal position. The team noted that if the relay would not return to its normal position, then associated contacts would not close as required. The team reviewed the electrical diagrams for the Cabinet 3L160 and associated contacts for Relay K52 and determined the relay could impact the safety function of the emergency diesel generator if associated contacts did not close once the relay was de-energized. The three contacts associated with the failed Relay K52 provide power to the field flashing, governor control, and excitation circuits. These circuits are relied upon during an emergency start of the emergency diesel generator and require these contacts to be closed for operation of the emergency diesel generator. Based on these observations, the team questioned the licensee on whether the failure of the relay truly had no impact on the safety function of the emergency diesel generator, and on September 29, 2008, the licensee generated Nuclear Notification 200146292 to track this concern.

Upon completion of their review, the team concluded that the relay does have an impact on the operability of the emergency diesel generator and this conclusion could have been determined by the licensee if the operations personnel would have performed a thorough operability assessment. The team determined that the assessment for operability of the emergency diesel generator did not meet the guidelines as delineated in Regulatory Issue Summary 2005-20, "Revision to guidance formerly contained in NRC Generic letter 91-18, "Information to Licensees Regarding two NRC Inspection Manual Sections on Resolution of degraded and nonconforming Conditions and on Operability,"" or follow the operability determination process in Procedure SO123-XV-52, "Functionality Assessments and Operability Determinations," Revision 8. As a result of the inspector's observations, a failure analysis and apparent cause evaluation is now being performed to evaluate the failure mechanism of the relay, and a Licensee Event Report is being drafted to document the inoperability of the emergency diesel generator.

Analysis.

The performance deficiency associated with the finding was the failure of operations personnel to adequately implement the operability determination process.

The finding was greater than minor because it was associated with the equipment performance attribute of the mitigating systems cornerstone and affects the associated cornerstone objective to ensure the availability, reliability, and capability of systems that respond to initiating events to prevent undesirable consequences. Using the Manual Chapter 0609, "Phase I - Initial Screening and Characterization of Findings

," the finding was determined to have a very low safety significance because it did not result in an actual loss of system safety function, did not result in a loss of a single train of safety equipment for greater than its technical specification allowed outage time, and did not screen as potentially risk significant due to seismic, flooding, or severe weather initiating events. This finding had a cross-cutting aspect in the area of human performance associated with decision-making because the licensee did not use conservative assumptions for operability decision-making when evaluating degraded and nonconforming conditions H.1(b).

Enforcement.

Title 10 CFR Part 50, Appendix B, Criterion V, "Instructions, Procedures and Drawings," requires that activities affecting quality shall be prescribed by instructions, procedures, or drawings, and shall be accomplished in accordance with those instructions, procedures, and drawings. Procedure SO123-XV-52, "Functionality Assessments and Operability Determinations," Revision 8, requires, in part, the licensee to assess for operability any known degraded conditions of safety-related equipment needed to mitigate accidents. Contrary to the above, on August 9, 2007, operations personnel failed to adequately evaluate a degraded and nonconforming condition associated with the Emergency Diesel Generator 3G002. Specifically, operations personnel failed to consider all relevant information when determining whether the idle speed Relay 3L160-2-K52 affected the operability of the Emergency Diesel Generator 3G002. Because the finding is of very low safety significance and has been entered into the licensee's corrective action program as Nuclear Notification 200146292, this violation is being treated as noncited violation consistent with Section VI.A of the Enforcement Policy: noncited Violation 05000362/2008012-02, "Failure to Properly Implement Operability Determination Process."

(c) Inadequate Auxiliary Feedwater Pump Room Heat Load
Analysis.

Introduction. The team identified a Green noncited violation of 10 CFR Part 50, Appendix B, Criterion III, "Design Control," for the failure of engineering personnel to ensure the auxiliary feedwater pump room heat load calculation was adequate. Specifically, since initial plant construction, engineering personnel failed to consider the impact to the auxiliary feedwater pump room's heat load design basis calculation for the most limiting scenario, in which all auxiliary feedwater pumps in the room have started and are running with only one emergency room cooler available.

Description.

During their review of corrective actions associated with an auxiliary feedwater pump room cooler taken out of service for corrective maintenance, the team observed that there was no action to take any of the three auxiliary feedwater pumps out of service as well. The auxiliary feedwater system consists of two motor driven pumps and one steam driven pump that are all located in the same room. There are two safety-related room coolers for each unit in these rooms that remove heat during certain event sequences to ensure the electrical motors and supporting electrical equipment will remain cool enough to perform their safety function. The team reviewed Calculation M-74-06, "Auxiliary Feedwater Pump Room Heat Load Calculation," and noted the calculation only required one room cooler to be operable to support cooling. The purpose of the calculation was to verify the actual heat loads for various plant operational modes are within capacity of the auxiliary feedwater room ventilation system with one cooler to ensure adequate cooling of the pump room equipment.

The team noted the design basis calculation for the auxiliary feedwater pump room assumed a heat load with only one or two auxiliary feedwater pumps running. The team questioned the adequacy of this calculation given the design of the auxiliary feedwater system is to have all three auxiliary feedwater pumps running during an emergency feedwater actuation signal or following a safety injection actuation signal. The team also noted that the calculation was inadequate in that the heat load contribution from the steam driven auxiliary feedwater pump and associated piping to the room was not realistic. The team concluded that since the auxiliary feedwater pump room heat load calculation did not consider all potential scenarios regarding exhaust fans and running equipment as well as using non-realistic heat loads for the turbine driven auxiliary feedwater pump, the results of the design calculation was determined to be non-conservative. The licensee initiated Nuclear Notification 200149442 to document the non-conservative conditions and to update the auxiliary feedwater heat load design basis analysis.

The licensee evaluated the effect of having three pumps running and only one exhaust fan and the non-conservative assumptions for the heat loads in the room. Engineering personnel performed a draft analysis and revised their calculation to consider these new heat load assumptions. During the initial revision to the calculation, the licensee determined that a significant decrease in margin would occur during an emergency feedwater actuation signal or safety injection actuation signal as a result of their non-conservative assumptions.

Subsequent analysis determined there were several other assumptions in the auxiliary feedwater pump room calculation that were also inadequate. The licensee took actual heat load measurements when the auxiliary feedwater pumps were running and were able to find some conservatism in several other assumptions that would allow for improving the margin needed in the calculation. Once the licensee completed their review of the analysis they concluded that there was no impact on operability of the auxiliary feedwater pumps. The team concurred with this determination. The licensee's corrective actions for this issue are to re-perform the auxiliary feedwater pump room heat load design analysis with only one room cooler including the specific items discussed above to gain additional margin and improve the auxiliary feedwater heat load calculation.

Analysis.

The performance deficiency associated with this finding was the failure of engineering personnel to include the proper heat load scenarios and use of realistic assumptions for a design basis calculation. The finding was more than minor because it affected the design control attribute of the mitigating systems cornerstone and affected the objective to ensure the reliability and capability of systems that respond to initiating events to prevent undesirable consequences.

Using the Manual Chapter 0609.04, "Phase 1 - Initial Screening and Characterization of Findings," the finding was determined to have a very low safety significance because it did not result in an actual loss of system safety function, did not result in a loss of a single train of safety equipment for greater than its technical specification allowed outage time, and did not screen as potentially risk significant due to seismic, flooding, or severe weather initiating events. This finding was evaluated as not having a cross-cutting aspect because the performance deficiency was not indicative of current performance. Enforcement

. Title 10 CFR Part 50, Appendix B, Criterion III, "Design Control," requires, in part, that design control measures shall provide for verifying or checking the adequacy of design, such as by the performance of design reviews, by the use of alternate or simplified calculation methods, or by the performance of a suitable testing program. Contrary to the above, the licensee failed to provide design control measures for verifying or checking the adequacy of design to evaluate the initial design assumptions assumed in the auxiliary feedwater pump room heat load design analysis. Specifically, Calculation M-74-06, "Auxiliary Feedwater Pump Room Heat Load Calculation," Revision 1, failed to consider the proper ventilation and pump scenarios and assumed non-realistic heat load assumptions which resulted in a non-conservative auxiliary feedwater pump room heat load design analysis. Because this violation was of very low safety significance and it was entered into the licensee's corrective action program in Nuclear Notification 200149442, this violation is being treated as a noncited violation consistent with Section VI.A.1 of the NRC Enforcement Policy: noncited Violation 05000361;05000362/2008012-03, "Inadequate Auxiliary Feedwater Pump Room Heat Load

Analysis.

"

b. Assessment of the Use of Operating Experience

1. Inspection Scope The team examined the licensee's program for reviewing industry operating experience, including reviewing the governing procedure and self assessments. A sample of operating experience notifications that had been issued during the assessment period were reviewed to assess whether the licensee had appropriately evaluated the notification for relevance to the facility. The team then examined whether the licensee had entered those items into their corrective action program and assigned actions to address the issues. The team reviewed a sample of root cause evaluations, action requests and notifications to verify if the licensee had appropriately included industry operating experience.

2. Assessment Overall, the team determined that the licensee was adequately evaluating industry operating experience for relevance to the facility, and had entered applicable items in the corrective action program in accordance with stations procedures. The team concluded that the licensee was evaluating industry operating experience when performing root cause and apparent cause evaluations. Both internal and external operating experience was being incorporated into lessons learned for training and pre-job briefs.

The team noted that root and apparent cause evaluations were required in order to evaluate whether internal or external operating experience was available associated with the event or failure being examined, and whether the evaluation and actions to address those items had been effective. Additionally, all root cause evaluations reviewed included an assessment as to whether the issue being evaluated had potential application to other similar component or plants. Several exceptions were noted where recent root cause evaluations identified relevant operating experience which had been ineffectively addressed.

3. Findings No findings of significance were identified.

c. Assessment of Self Assessments and Audits 1. Inspection Scope The team reviewed a sample of licensee self assessments and audits to assess whether the licensee was regularly identifying performance trends and effectively addressing them. The team reviewed audit reports to assess the effectiveness of assessments in specific areas. The specific self-assessment documents reviewed are listed in the Attachment.

. Assessment The team concluded that the licensee had an adequate self-assessment process. The team observed that SONGS management was involved in developing the topics and objectives of self assessments. The team observed that the assignment of the assessment team included members with the proper skills and experience to ensure an effective self assessment was conducted and the team members included individuals from outside organizations.

3. Findings

No findings of significance were identified.

d. Assessment of Safety Conscious Work Environment 1. Inspection Scope The inspection team conducted individual interviews with 56 individuals and six focus group discussions with approximately eight individuals in each group, for a total of approximately 100 individuals. The interviewees represented various functional organizations and ranged across contractor, staff, and supervisor levels. The team conducted these interviews to assess whether conditions existed that would challenge the establishment of a safety conscious work environment at SONGS. The team used the interviews to gather insights on communications regarding Confirmatory Order 07-232 and recent willful violations. In addition, the team reviewed the 2008 Nuclear Safety Culture Assessment and discussed the results and associated action plans with key owners. Finally, the team interviewed the manager responsible for the Nuclear Safety Concerns Program and reviewed a sample of the program files.

2. Assessment All of the individuals interviewed expressed a willingness to raise safety concerns and were able to provide multiple examples of avenues available, such as their supervisor, writing a notification, other supervisors/managers, the Nuclear Safety Concerns Program, and the NRC. Overall, the interviewees expressed positive experiences in raising concerns to their supervisors and through their chain of management. One of the focus groups expressed some concern that raising issues to supervisors of other departments was not as effective as going through their own supervision.

Across the board, all the interviewees provided negative feedback and shared concerns about their working knowledge of SAP which includes the site's new corrective action program system. The interviewees indicated that they either did not know how to write a notification or found the process to be very difficult. In particular, 38 of the 56 individual interviewees indicated they were uncomfortable using SAP. Regarding training on the new system, all the interviewees explained that they either did not receive any, or the training they received was of limited effectiveness. Some of the feedback regarding training was that it was not presented in a hands-on format and the trainers were not very knowledgeable. The interviewees provided examples of current workaround practices such as going directly to their supervisors or other individuals with safety issues instead of entering them into the system. There was general concern expressed by all the interviewees about not feeling comfortable using SAP for all the tasks needed for their specific job functions. The supervisors expressed the belief that with appropriate training, the system could be effectively used.

Regarding the nuclear safety concerns program, participants in four of the six groups did not have an opinion because they have not had any experience with the program or a need to use it. Two of the focus groups, one staff level and one contractor, expressed some concerns regarding the effectiveness of the program in resolving problems, believing it to be of limited effectiveness. However, they indicated they would use the nuclear safety concerns program if necessary.

Regarding effectiveness of problem resolution, interviewees in all the focus groups indicated that the difficulties of using SAP have started affecting their confidence to some degree. One of the individual interviewees expressed similar concerns. Regarding management response to issues raised, half of the focus groups, at the contractor and staff levels, explained that sometimes management does not have the proper understanding of problems due to limited presence in the field.

None of the interviewees expressed any concerns or awareness of retaliation for raising safety concerns. Two of the individual interviewees explained that due to perceptions around a recent management change, they had some concerns about potential negative reactions for raising safety issues in general. However, if the issues were related to nuclear safety, these individuals indicated that supervision or management would ensure they were appropriately addressed.

When asked about the nuclear safety culture assessment, only a few (less than 10%) of the focus group interviewees were able to recall receiving any information about the results. Of those who did recall, none were able to provide any specifics. The supervisors' group indicated awareness of receiving communications, but no one could provide any specifics. Of the individual interviews, only four were able to provide information on results of the survey, and these individuals were all in management positions. The licensee was continuing to develop and implement actions in response to the assessment results at the time of the inspection.

For communications on the Confirmatory Order, all the interviewees indicated they had received training and/or communications on the topic. All expressed that the message they received was if they committed deliberate misconduct, it would result in termination. None of the interviewees was aware of any details associated with the site's disciplinary process or factors that would be considered in determining disciplinary actions which may affect the outcome regarding any acts of deliberate misconduct.

Additionally, the inspection team made some general observations during the focus group discussions on the overall safety culture at SONGS. For instance, four of the focus groups at the staff and contractor levels mentioned recent schedule pressures, particularly related to the upcoming outage. Half of the focus groups, also at the staff and contractor levels, believed there should be increased supervisory presence in the field, and the supervisors' group expressed their desire to be engaged in the field for a larger portion of their time. Due to workload, these supervisors have not been able to spend as much time in the field as they prefer. One of the staff level groups brought up the safety department of a contractor organization being frequently visible in the field as a very positive practice.

3. Findings No findings of significance were identified.

e. Substantive Cross-Cutting Issues - Procedure Adequacy and Evaluation of Problems 1. Inspection Scope In the 2007 annual assessment for SONGS (NRC letter dated March 3, 2008), the NRC identified substantive cross-cutting issues associated with procedure adequacy and the licensee's failure to thoroughly evaluate problems such that resolutions address causes and extent of conditions. The licensee performed two separate root cause evaluations to determine what actions were necessary to resolve these substantive cross-cutting issues. The licensee then prepared improvement plans for each substantive cross-cutting issue and identified corrective actions needed to address and correct the each issue. During this inspection, the team reviewed the licensee's evaluations, actions, and plans to assess the progress in addressing these substantive cross-cutting issues.

2. Assessment The team reviewed the scope of information considered in the root cause evaluations, the details of the evaluations, and the planned corrective actions. These reviews included assessment of the scope and progress of the licensee's procedure improvement efforts and cause evaluation improvement efforts. During their review, the team determined that the analysis in the root cause evaluations did not adequately address appropriate causal evaluations to assist in identifying the reasons that the stations procedures or evaluations were inadequate. The team noted that the causal evaluations addressing the procedure adequacy issues were too broad and not focused on specific procedures and actions that are needed to resolve the cross-cutting issue and improve performance. The team also noted that the causal evaluations addressing the inadequate evaluations issue were not adequately identifying the causes for the stations' inadequate evaluations. The team noted that the licensee did not consider additional information, such as self assessments or corrective action program trends, to provide further insight on either of the substantive cross-cutting issues. The licensee agreed with the team's conclusions that the root cause evaluations did not include appropriate information and detail to identify the reasons for the insufficient progress in addressing the substantive cross-cutting issues. To address the results of the inadequate root cause evaluations and improvement plans, the licensee conducted subsequent root cause evaluations and common cause assessments to determine the underlying themes and causes for these issues. Additionally, the team noted that the licensee had recently developed revised improvement plans that include enhancements for procedure upgrades, the verification and validation processes, as well as actions to address human performance and change management methods. However, the team could not assess and evaluate the effectiveness of the corrective actions because the licensee was in the early stages of implementation of their improvement plans.

3. Findings No findings of significance were identified.

f. Annual Sample: Review of Operator Workaround 1. Inspection Scope The team conducted a cumulative review of operator workarounds for Units 2 and 3 and assessed the effectiveness of the operator workaround program to verify that the licensee was: 1) identifying operator workaround problems at an appropriate threshold; 2) and entering them into the corrective action program; and 3) identifying and implementing appropriate corrective actions. The review included walkdowns of the control room panels, interviews with licensed operators and reviews of the control room discrepancies list, the lit annunciators list, the operator burden list, and the operator workaround list.

2. Assessment The team identified one closed operator workaround that required operations personnel to periodically vent the component cooling water system for nitrogen gas because of a design deficiency. The team also noted that upon a loss of instrument air, operations personnel are required to manually isolate sections of the component cooling water piping and take actions to vent the piping. The team performed a walkdown of the component cooling water system and interviewed engineering personnel to understand why the operator workaround was closed.

During the inspection, the team questioned why the particular aforementioned operator actions were not considered an operator workaround. The licensee informed the team that this operator action was part of an approved procedure for loss of instrument air. The licensee also informed the team that according to the licensee operator workaround program Procedure, SO123-XX-6, Revision 5, it states, "if the compensatory actions have been proceduralized, but are not intended as part of the permanent plant design, then the issue should still be considered as an operator workaround." Contrary to this requirement, the operator action for isolating component cooling water loads upon a loss of instrument air was closed as an operator workaround on August 26, 2008. This issue was entered into the licensee's corrective action program as Nuclear Notification 200162648.

The team noted that the operations department developed the operator workaround list to help prioritize scheduling of maintenance and/or system upgrades. The operator workaround list allows station management to perform an aggregate review for the impact on the plant due to open operator workarounds. The team concluded that this finding was minor because the impact on the overall effectiveness of this metric was minimal.

3. Findings No findings of significance were identified.

4OA5 Other Activities

a. Confirmatory Order Follow-up for EA-07-232 [NRC INSPECTION REPORT 05000361/2007016; 5000362/2007016, AND OFFICE OF INVESTIGATIONS REPORT 4-2007-016]

1. Inspection Scope By letter dated January 11, 2008 (ADAMS Ref. ML080110380), the NRC issued a Confirmatory Order to the licensee as part of a settlement agreement through the NRC's alternative dispute resolution process. The settlement was in regards to the falsification, by a contract fire protection specialist, of firewatch certification sheets on numerous occasions from April 2001 to December 2006 at SONGS. The team utilized Inspection Procedures 92702 and 71152 to assess the licensee's completion of the items contained in the Confirmatory Order. The team reviewed information available about the licensee's progress on the Confirmatory Order items and interviewed site personnel regarding related training and communications received.

2. Assessment Based on the information provided by the licensee, the inspection team determined that the licensee has demonstrated sufficient progress for closure of Items 1, 2d, 2e, 2f, 2i, 2j, 2k, and 3 of the Confirmatory Order.

  • Item 1 of the Confirmatory Order states, "by January 31, 2008, SCE [Southern California Edison] will perform a common cause evaluation of known recent events, actually or potentially involving willful events to determine the root and contributing causes for the collective issues. This evaluation will include an analysis to determine if any deficiencies of safety culture components, as defined by NRC's Inspection Manual Chapter 0305, 'Operating Reactor Assessment Program,' were significant contributors. The results of this evaluation will be factored into the Corrective Action Program and addressed in other ongoing related efforts, as appropriate."

This common cause evaluation was completed on January 29, 2008. The analysis included a review of contribution of safety culture components, and the results were factored into the corrective action program and other related efforts (nuclear notification 071200432-1).

  • Item 2d of the Confirmatory Order states, "SCE will conduct a safety culture assessment by an independent outside organization by April 1, 2008, that will provide trends of key cultural performance indicators," and also states, "by June 30, 2008, the results of this assessment will be factored into the Corrective Action Program and addressed in other ongoing related efforts." The safety culture survey was completed in January 2008 and published in February 2008. The results of the survey identified corrective actions to address the concerns.
  • Item 2e of the Confirmatory Order states, "by April 1, 2008, SCE will further enhance the new employee orientation and the general employee training programs to better ensure that new and periodically retrained personnel clearly understand that deliberate acts of non-compliance with regulations or procedures will not be tolerated and could result in a significant disciplinary action up to and including termination." The new employee orientation and the general employee training modules were updated on March 4, 2008, to provide the specific information described above.
  • Item 2f of the Confirmatory Order states, "by September 30, 2008, SCE will develop and begin conducting training (for managers and supervisors) on techniques that can be used to monitor that workers are implementing procedures as instructed." Prior to September 30, 2008, SCE developed the training required for managers and supervisors and established plans to begin conducting this training for those enrolled.
  • Item 2i of the Confirmatory Order states, "SCE will take steps to develop and implement incentives for on-site service contractors to help SCE address the issues that have resulted in deliberate misconduct-related violations." SCE has established a contingency to ensure that new contractors accept these new guidelines, and old contracts will incorporate this guidance when they are renewed. As of July 25, 2008, sixty-seven contracts have been revised or renewed incorporating this new guidance.
  • Item 2j of the Confirmatory Order states, "by April 1, 2008, SCE will incorporate into the SONGS oversight surveillance program, periodic sampling of repetitive rounds and log-keeping activities to provide reasonable assurance that actions to deter and detect instances of deliberate non compliance are effective. This oversight will include sampling of both SCE and contractor activities."

Prior to April 1, 2008, SCE revised Procedure SO123-XV-18.15, "Surveillance Program," to incorporate this guidance which includes sampling of both SCE and contractor activities.

  • Item 2k of the Confirmatory Order states, "SCE will use multiple site-wide communication tools to emphasize to employees and contractors at SONGS the need to comply with job rules, regulations, and procedures and potential consequences when compliance does not occur." SCE has provided various site-wide communication tools to emphasize to employees and contractors at SONGS the need to comply with rules, regulations, and procedures and potential consequences when compliance does not occur including: all-hands meetings, department meetings, and blue book handouts that include specific guidance.
  • Item 3 of the Confirmatory Order states, "by 30 days after issuance of the order, SCE will provide to the NRC under separate letter its response to the three issues addressed by the NRC in its letter dated September 27, 2007, (EA 07-232) and for NRC letter dated November 7, 2007, (EA 07-141), the extent to which trainers may fail to follow the procedural requirements of Section 6.3.2 of SONGS Training Procedure SO123-XV-27." By letter, dated February 8, 2008, SCE provided a letter with the requested response (ADAMS Ref. ML080520266

).

Based on the information provided by the licensee, the inspection team was unable to determine that sufficient actions had been taken for closure of Items 2a, 2b, 2c, 2g, 2h, and 2l as described below.

  • Item 2a of the Confirmatory Order states, "SCE will develop a monitoring program to determine the effectiveness of the Corrective Action Plan developed pursuant to Item 2." SCE developed a monitoring plan that included: 1) enhanced surveillances, 2) emphasis of station requirements, 3) identifying potential willful violations, 4) trending of results, and 5) assessment of effectiveness of the corrective action plan. The team reviewed the monitoring plan and determined that while the monitoring plan identified the components needed to observe the effectiveness of the corrective action plan, the monitoring plan did not include specific measures that established success criteria such as site and/or departmental metrics or performance indicators. Because SCE failed to ensure the monitoring plan included success criteria, Item 2a will remain open.
  • Item 2b of the Confirmatory Order states, "by June 30, 2008, SCE will conduct multi-day interventions that reinforce fundamental company values. SONGS will ensure that this effort includes the elements of a strong nuclear safety culture to prevent deliberate violations. The intent of the interventions will be to focus leaders and managers on the importance of balancing accountability and encouraging workers to self-report errors and the importance of communicating this to their workers." Prior to June 30, 2008, SCE has conducted multi-day interventions; however, SCE failed to provide any means for the team to verify that the leaders and supervisors at SONGS attended these interventions. SCE had identified 458 employees that had attended the training, of which, 292 were identified as supervisors; however, because SCE failed to provide an adequate means for the team to understand who SCE had identified as leaders or managers and who attended the interventions, Item 2b will remain open.
  • Item 2c of the Confirmatory Order states, " SCE will expand the Corporate Ethics Program to encompass long-term (i.e., greater than 90 days) managers and supervisors of independent contractor workers at SONGS, who will be required to take the integrity training in 2008. SCE will conduct Corporate Ethics Training for SONGS managers and supervisors in 2008 and other SONGS employees in 2009." SCE expanded the Corporate Ethics Program to encompass long-term (i.e., greater than 90 days) managers and supervisors of independent contractor workers at SONGS and begun training for SONGS managers and supervisors.

However, because SCE is still training SONGS managers and supervisors and will not begin training for other SONGS employees until 2009, Item 2c will remain open.

  • Item 2g of the Confirmatory Order states, "by April 1, 2008, SCE will enhance it's existing disciplinary process to provide more detailed guidance in cases involving a deliberate misconduct-related violation. This process will communicate to the workforce specific escalating disciplinary actions that may be taken in response to initial and/or repeat deliberate misconduct by individual contributors and supervisors/managers. Communication of process enhancements will focus personnel on the importance of balancing accountability and encouraging workers to self-report errors and the importance of communicating this with their workers." Prior to March 30, 2008, SCE enhanced the procedures to include specific disciplinary guidance standards. Additionally, SCE communicated to the workforce during all hands meetings conducted between March 28 and March 30, 2008. The team reviewed what SCE communicated to the workforce and determined that SCE failed to ensure the message conveyed the importance of balancing accountability and encouraging workers to self-report errors, the importance of communicating this with their workers, and specific escalating disciplinary actions that may be taken in response to initial and/or repeat deliberate misconduct by individuals. Because SCE failed to ensure these items were properly communicated, Item 2g will remain open.
  • Item 2h of the Confirmatory Order states, "by April 1, 2008, SCE will revise the SONGS training lesson for on-the-job trainers and provide this training to all on-the-job trainers and trainees. The revised on-the-job training will reinforce the responsibilities of the trainer and the trainees. Emphasis will be placed on the expectations of a trainer while his/her trainee is performing work during and on-the-job session." Prior to March 30, 2008, SCE established a computer based training module for all on-the-job trainers and trainees. Additionally, SCE revised the on-the-job procedure to establish emphasis on the expectations of the on-the-job trainer while his/her trainee is performing work during the on-the-job training. However; SCE failed to provide an adequate means for the team to verify that all on-the-job trainers and trainees were provided the computer based training module, therefore, Item 2h will remain open.
  • Item 2l of the Confirmatory Order states, "Upon completion of the terms of the Confirmatory Order, SCE will provide the NRC with a letter discussing its basis for concluding the Order has been satisfied." Because all the items of the Confirmatory Order have not been completed, SCE has not provided the NRC with a letter; therefore, Item 2l will also remain open. At the end of the inspection period, the licensee continued to implement actions related to the open items of the Confirmatory Order. Additionally, the licensee submitted a response letter to the NRC titled, "Response to Confirmatory Order EA 07232," dated November 14, 2008, in which the licensee defined and established success criteria for closure of each of the Confirmatory Order Items. In this letter, the licensee commits to submittal of a status report in January of 2009 discussing completion of each individual Confirmatory Order item.

3. Findings The Confirmatory Order open items will be treated as an unresolved item pending licensee completion of the remaining actions and NRC inspection of the adequacy and effectiveness of those actions: URI 05000361;05000362/200812-04, "Open Confirmatory Order Items".

4OA6 Management Meetings

On October 3, 2008, a preliminary debrief was conducted. The results of the inspection were discussed with Mr. M. Short and other members of the staff. The licensee confirmed that no proprietary information was handled during this inspection.

On October 22, 2008, a telephonic exit was conducted with Mr. D. Axline to discuss the final results of the findings identified during the inspection.

s: 1. Supplemental Information 2. Information Request A1-1

SUPPLEMENTAL INFORMATION

KEY POINTS OF CONTACT

Licensee Personnel

J. Armas, Senior Nuclear Engineer
D. Axline, Nuclear Regulatory Affairs
T. Broughton, Project Manager
D. Breig, Manager, Engineering Standards and Excellence
D. Burda, Nuclear Engineer
C. Cates, Manager, Maintenance Project Management
S. Chun, Senior Nuclear Engineer
R. Corbett, Manager, Performance Improvement
M. DeMarco, SDG&E Representative
J. Fee, Manager, Site Emergency Preparedness Services
O. Flores, Manager, Chemistry
W. Frick, Manager, Nuclear Safety Concerns
R. George, Nuclear Oversight Consultant
M. Goettel, Director, Site Support Business Planning & Financial
M. J. Johnson, Manager, Environmental Protection
L. Kelly, Nuclear Regulatory Affairs
W. A. Lippett, Nuclear Engineer
C. McAndrews, Director, CPUC Regulatory Affairs
M. McBrearty, Technical Specialist, Nuclear Regulatory Affairs
C. Mell, Nuclear Engineer
R. Pate, Manager, Computer Engineering
M. Orewyler, Manager, SPM production
J. Osborne, Project Manager
N. Quigley, Manager, Mechanical/Nuclear Maintenance Engineering
J. Madigan, Manager, Health Physics
A. Shean, Manager, Nuclear Oversight
A. Scherer, Manager, Nuclear Regulatory Affairs
M. Short, Vice President, Engineering and Technical Services
W. Stevenson, Supervisor, Computer Engineering
C. S. Stinson, Manager, Nuclear Training
T. Vogt, Manager, Systems Engineering
D. Wilcockson, Manager, Operations and Engineering Training
C. Williams, Manager, Compliance
L. Wright, Manager, Regulatory Performance
T. Yackle, Manager, Operations

NRC Personnel

None.

A1-2

LIST OF ITEMS OPENED, CLOSED, AND DISCUSSED

Opened and Closed

05000361/2008012-01 NCV Inadequate Maintenance Procedures Results in Ground on the Safety-Related 125 VDC Distribution System (Section 4OA2.a.3(a))
05000362/2008012-02 NCV Failure to Properly Implement Operability Determination Process (Section 4OA2.a.3(b))
05000361;05000362/2008012-03 NCV Inadequate Auxiliary Feedwater Pump Room Heat Load Analysis (Section 4OA2.a.3(c))

Opened

05000361;05000362/2008012-04 URI Open Confirmatory Order Items (Section 40A5)

LIST OF DOCUMENTS REVIEWED

In addition to the documents called out in the inspection report, the following documents were selected and reviewed by the inspector accomplish the objectives and the scope of the inspection and to support any findings:

Section 4OA2: Identification and Resolution of Problems

Procedures

SO123-OR-1, Rev. 8, "Operational Experience Review Program" SO23-XV-40, Rev. 0, "Operating Experience Report: Sharing Industry Information" SO123-XIV-5.5, Rev. 2, "Operating Experience Report: Sharing Industry Information" SO123-XV-52, Rev. 8, "Functionality Assessments and Operability Determinations" SO123-XV-50, Rev. 8, "Corrective Action Process"
SO123-XV-50.39, Rev. 9, "Cause Evaluation Standards, Methods, and Instructions" SO123-XX-1 ISS2, Rev 21, "Notification Initiation and Processing" SO123-XII-17.1, Rev. 2, "Review and Transmittal of Quality Assurance Records" SO123-CA-1, Rev. 4, "Corrective Action Program" SO123-XV-3.3, Rev. 13, "NRC Reporting Requirements" SO123-O-A7, Rev. 7, "Reporting of Significant Events" SO23-XV-2, Rev.2, "Troubleshooting plant Equipment and Systems"
SO123-XV-50.2, Rev. 14, "Nuclear Safety Concerns Program" SO123-XV-51, Rev. 12, "Identifying and Assessing Impact to Site Programs and Procedures" SO23-3-3.23, Rev. 34, "Diesel Generator Monthly and Semi-Annual Testing" SO123-II-11.152, Rev. 9, "Circuit Device Tests and Overall Functional Test" SO123-II-11.177, Rev. 3, "Protective Relays and Devices Testing and Calibration"
SO123-XVI-24, Rev. 6, "Electrical Safe Work Practices"
Actions Requests and Notifications
060700747
070601194
070800454
080600438
021100392
030101970
060700753
050101702
060401258
060600393
060600500
060700841
060700765
060700895
070100264
070100989
070100379
070300161
070500468 070400732
070400632
070501196
070601180
070700181
070600873
070700909
070700319
070800291
070800466
070900082
070900496
070900925
071100632
071101401
071200614
080100919
080100637
080100226
080101185
080101507
080300395 080300550
080400246
080400170
080400125
080501424
080600212
040401344
040400696
060101868
200066209
050801414
060600500
071200614
021100392
030101970
050101702
060401258
060600393
060700841
080100919
080100637
070100264
070100379
070400732
070300161
060700895
070100989
080100226
070400632
070501196
070601180
080101185
070700181
070800291
070800454
070900925
071101401
080101507
070700909
070700319
070800466
070900082
070900496
080300550
071100632
080400246
080400170
080400125
080501424
080300395
080600212
020401425
020602171
020801305
030301589
030301792
030901303
040200173
040200918
040400153
040400184
040400696
040501589
040501657
040701564
040901424
041100768
041200179
050200357
050401486
050500105
050600290
050601315
050700472
050800946
051000116
051101380
060101764
060200231
060300848
060400064
060400097
060400414
060400870
060600089
060600644
060700765
060800415
060900154
060900158
061001129
061001131
061000838
070200597
070300300
070300508
070301266
070400447
070400518
070600840
070701029
070800038
070801399
071000309
071001426
080100376
080100428
080100581
080100919
080100984
080101033
080101035
080101116
080300336
080500430
080600214
080101431
070601250
070600873
070700291
061200640
060301125
060301125
071201393
070300161
071000309
070600872
070801151
070900160
070700291
070601194
070701029
061001379
070500468
060800603
070300161
070400873
070400998
061101250
061000859
060700753
060700765
060700747
060600500
060700471
060100172
060301594
070200583
070700545
061200747
070100370
061001435
060800249
060301191
070800454
Maintenance Orders
08052608
08030496
08030495
07021385
07041537
08030489
07010760
07070165
07080130
07101813
07061551
07060899
07110828
06120872
07051682
07111089
07061475
06100597
07040944
07070253
08011108
06091431
07061113
07010513
07080319
08040583
07080113
07120720
07121010
07080584
07030374
07071126
08011382
06091035
07110768
07040137
08022510
08032292
07061541
07081195
07071429
08011496
07011266
07090182
06100214
07081536
07071285
08020992
08050487
08010667
08052215
08040771
07011550
07100126
07010033
07040866
06091171
08050164
06121213
07120858
07111565
07050582
07080386
Operability Evaluations
040401344-08
040400696-09
050500999-46
050700064-08
051000204-21
060100738-01
060102099-02
060200377-05
060300822-08
060300881-05
060301103-04
060400474-31
060401140-04
060500306-02
060501384-01
060600089-21
060600109-01
060600367-01
060600644-02
060601064-01
060601355-05
060700696-02
060700747-01
060701285-02
060900059-02
060900154-03
060900158-01
060900493-01
060900500-03
060900840-01
060901383-01
061000135-03
061000181-02
061000414-01
061000859-25
061000859-18
061000865-03
061000963-03
061001154-04
061001379-49
061001379-01
061001431-03
061100073-01
061100129-01
061100268-02
061100289-01
061100411-04
061100533-05
061101004-01
061101006-02
061200549-02
061200897-04
070100063-04
070100110-01
070200058-01
070200602-01
070201148-03
070400933-06
070600482-02
070601385-04
070700381-01
071000125-04
071000391-10
071200139-01
080200842-02
080200264-01
080200949-01
080401404-06
080400273-13
080400954-01
080500430-01
080600001-01
080500128-03
Reportability Assignments
200059586
061200097
200081283
060301884
060401258
070600867
071100812
070801399
071000475
061001131
070101074
060901363
200065966
060600089
200062294
060801030
061200008
071000110
060801052
070100063
200065884
060600393
060900294
070100476
060600879
060600500
200060977
061101554
061000820
071000418
061200897
060700471
070800861
060800967
060900230
061101249
200077880
061001379
060900767
061000135
060801279
061100070
200079493
060700841
070700909
061100691
200069846
060600089
061100396
061101083
200079757
061000736
060900275
200114732
200116849
070100472
060600500
060901296
061000859
061100338
200118016
060700895
200083174
200096962
200088882
061000188
061100574
060901363
061100518
061001822
061001822
060900034
060901382
061000135
060900336
200101985
061000859
060900583
200116280
200116276
061000820
200105262
Root Cause Evaluations
RCE 040101536-06 (Crack identified on letdown line exiting of RHR)
RCE
050301091-06 (Calculation Clarification for TS required Offsite voltage)
RCE 070800313-01 (Multiple HP events adversely impacting SONGS)
RCE 060400888-36 (Overspeed trip circuit cause turbine trip)
RCE 060301594-31 (Leakage identified from PSI header to SIT)
RCE 070600074-21 (Fryquel oil leaks on hydraulic discharge header)
RCE 071201393-22 (EDG experienced MW spike while paralleled to grid during transient)
RCE 040400600-03 (Evaluate cause of Unit 2 Manual trip due to loss of both MFWP's)
RCE 070400783-13 (U3 Main XFMR Leak)
RCE 061200346-04 (Unexpected beam failure for material handling system monorail)
RCE 070500164-01 (Organizational Causes associated with recent EDG issues)
RCE 070600867-05 (Air Line Instrumentation Break)
A1-5RCE
070901311-03 (TOL spurious trips root cause evaluation)
RCE 040101536-06 (Crack identified on letdown line exiting of RHR) RCE
080074202-19 (Unit 2 stator water low during flow test cause Turbine/Reactor trip)
RCE 060301822-36 (MNSA leak, Boric acid residue on two of four pressurizer bolts)
RCE 071201393-22 (Evaluate timeliness of OD for EDG failure)
RCE 040101536-06 (RCS leakage increased due to crack in weld for letdown HX)
RCE 071101061-02 (Evaluate cause of security vehicle rollover)
RCE 071200432-01 (Evaluate cause of events potentially involving willfulness)
RCE 050200761-40 (Unit 2 train B CCW low flow due to valve failure)
RCE 050601315-05 (Loose wire caused EDG emergency cooling fan failure)
RCE 080400125-02 (Evaluate cause of Substantive Cross-cutting issue in PI&R)
RCE 070601237-01 (Operator license training program not providing adequate training)
RCE 080401137-03 (Ineffectiveness in CAP resolving conditions adverse to quality) RCE
070700319-01 (High consequence events at SONGS not being addressed Mgmt)
RCE 071000901-24 (MSIV failed to close when hydraulic dump solenoid valve closed)
RCE 070500164-01 (Organizational Causes associated with recent EDG issues)
RCE 070600867-05 (Air Line Instrumentation Break)
RCE 070901311-03 (TOL spurious trips root cause evaluation)
Apparent Cause Evaluations
ACE 021100392-05 (Emergency Chiller Trip on Low Refrigerant Temp)
ACE 040100080-03 (AC lube oil pump failure, not required for EDG Operability)
ACE 040800733-01 (Series of Non-1E class TOL Trips)
ACE 041100768-04 (Broken Stud on AFW Pump)
ACE 041200017-03 (Seat Ring Weld Crack)
ACE 050900772-32 (Offsite Siren; No audio)
ACE 060401658-04 (RCE low flow channel trip unexpectedly)
ACE 021100392-05 (Emergency Chiller Trip on Low Refrigerant Temp)
ACE 060500131-15 (SDC line vent crack in weld)
ACE 040100080-03 (AC lube oil pump failure, not required for EDG Operability)
ACE 040800733-01 (Series of Non-1E class TOL Trips)
ACE 041100768-04 (Broken Stud on AFW Pump)
ACE 060401471-04 (FWCS failure resulted in unexpected high level condition in both SG's)
ACE 051200354-02 (Fuel Handling Pump room A/C fans failed during test due to TOLs)
ACE 060100463-12 (Boric acid leak due to crack in weld on charging system)
ACE 060200113-02 (Containment Dome Air Circulating filter failure)
ACE 060200377-33 (Control room controllers failed due to dust in switches)
ACE 041200017-03 (Seat Ring Weld Crack)
ACE 060400236-02 (Unit 3 pressurizer heater failure)
ACE 060401260-12 (SBCS piping crack evaluation)
ACE 050900772-32 (Offsite Siren; No audio)
ACE 051200087-02 (CCW pump failed to rotate due to bearing problem)
ACE 051200288-08 (Unit 3 ADV broken support studs for silencer)
ACE 051001295-09 (Failure of two sirens during 2005 Annual test)
A1-6Direct Cause Evaluations
21200477-97
050100457-10
050100457-10
060400870-04
060400881-03
060500629-08
060900059-03
060900564-02
061001649-05
061101655-08
061101692-06
061200228-11
070100780-04
070100811-01
070300116-01
070300407-01
070401019-05
070500022-06
070500242-04
070500266-02
070500446-02
070600194-02
070600492-03
070900200-03
070900406-01
071001015-02
071001033-02
071200494-02
080101287-01
080200285-03
080301230-05
080301346-01
080400179-02
080400541-03
080400575-04
080400938-01
080401318-02
080501227-02
080501252-03
080501294-09
080600119-02
080600241-18
080600278-02
080600647-03 080600654=-2
Quality Assurance Audits
SCES-001-07, Operations Controls
SCES-005-08, Training
SCES-005-07, Special Processes ISI & IST
SCES-005-06, Maintenance Program
SCES-004-07, Environmental NPDES
SCES-003-08, Maintenance
SCES-003-07, Fire Protection
SCES-002-08, Access Authorization
SCES-002-07, Procurement & Material Control
SCES-001-08, Nuclear Regulations
SCES-010-07, Fitness for Duty
SCES-010-06, Emergency Preparedness
SCES-009-07, Chemistry Program
SCES-009-06, Fitness for Duty
SCES-008-07, Quality Assurance Program NIEP
SCES-008-07, NIEP Evaluation
SCES-008-06, Environmental ODCM
SCES-007-07, Design & Configuration Control
SCES-007-06, Modifications Program
SCES-006-07, Radiation Protection
SCES-015-07, Work Control
SCES-014-07, Equipment Reliability
SCES-013-07, Emergency Preparedness
SCES-012-07, Security Program
SCES-011-07, Technical Specifications
SCES-006-08, Fitness for Duty
SONGS Industry Operating Experience reviews
061200593 Loss of 400 kV Switchyard & 2 Electrical Trains
070401176 Unplanned BWR Rod Withdrawal
070500080 Resin Intrusion into Steam Generator
070600464 Degrading Plant Conditions
070700459 High Voltage Bushing Failure & Fault
070800887 Invalid SI with Failure to Reset
071001192 Earthquake at Kashewagaki
071200944 Inadequate Rigging Caused Injury
A1-7080200151 Digital Feed Control Failures
080301217 Underground Cable Ground Faults
080600342
Large Exposure to Fuel Handling Operator
060901026 Dual Unit Loss of Off-Site Power
061000564 Flow Accelerated Corrosion
061000648 Main Transformer Failure during Cleaning
061100022 Essential Service Water
061200596 Reactor Operation in Unanalyzed Range
070601114 Dual Unit Scram during Grid Disturbance
080300085 Loss of Grid / Subsequent Loss of 2 Electric Trains
200062673 BWR Fuel Channel Distortion
070800536 Reactivity Management
080101004 Intake Cooling Water Blockage
SONGS Submitted Operating Experience
OE23202
Unexpected Pressurizer Heaters Replacement
OE23227
Shutdown Cooling Leak from Vent Valve
OE23334 Three Negligent Weapons Discharges During Qualifications OE23341 Fire Sprinkler Activated During Battery Charger Load Testing OE23664 Premature Trip of Thermal Overload Device
OE23925 FME Plug Found Inside a Piping System During Refueling
OE24417 Contract Electrician Observed Within a Posted Radiography Boundary OE24576 Update to OE22929 - Siren Amplifier Miswiring OE24890 Anomalies Found in Software for Calculating Total Loop Uncertainty OE23202 Unexpected Pressurizer Heaters Replacement OE23227 Shutdown Cooling Leak from Vent Valve
OE24590 Update for OE22314 - Failure of Spiral Wound SIT Manway Gasket
OE24890 Anomalies Found in Software for Calculating Total Loop Uncertainty OE25206 Fisher Air Operated Valve Response to a Loss of Instrument Air OE25347 AFW DC Relay Corroded by Water Intrusion onto Zinc Plated Bolt OE25542 Unexpected Non-conservative Difference between TLD and PED Dose
OE25575 Contactor Failure during Battery Performance Test
OE25761 Update to OE25725 - Multiple MSIVs and FWIVs Disabled
OE26027 Misadjustment of Target Rock Solenoid Valves OE26158 Broken Valve Stem on AFW Steam Supply Check Valve OE26231 Sluggish Governor Control of a FWT Resulted from Misalignment
OE26240 AFW Turbine Valve Disabled by Stuck Open Auxiliary Contact OE26241 Diesel Generator Load Instability due to Speed Probe Connector Failure OE26251 Failure of EDG to Start Normally during Testing due to Fitting Leak OE26274 MFWP Trip due to Error in Selecting Direction for Turbine Oil Pressure OE26322 Update to OE25455 - Oil Leak from Switchyard Metering CT OE26342 Red Flakes Found on Fuel Assemblies and Core Internals OE26593 Analysis of Spent Fuel Pool Sediment
OE26645 Control Cable for Switchyard Breakers Left Disconnected after Modification OE26771 Loss of TDAFW Pump Speed Control due to Binding Sleeve Bushing
OE26821 Aging of Switching Power Supplies for MGPI Radiation Monitors OE27310 Injury from a Fall during Chemical Washdown Operation
A1-8Self Assessments and Independent Reviews Independent Review 2008 Nuclear Safety Concerns Independent Review 2006 Nuclear Safety Concerns 4Q07 Station Performance Report
4Q06 Station Performance Report 3Q07 Station Performance Report 2Q08 Nuclear Oversight Report
2Q07 Station Performance Report 1Q08 Nuclear Oversight Report
1Q07 Station Performance Report Self Assessment - 060301528-1 Technical Training
Self Assessment - 050801414-1.HU Procedure related events Self Assessment - 060900223-3.HP. PCEvents Self Assessment - 060900223-4.HPD Material Release Self Assessment - 060900223-5.HP Outage Dose Goals Self Assessment - 060900223-2.Control of Rad Material Self Assessment - 070301559-1.Nuclear Material Control Self Assessment - 070400748-1 SONGS ISI Self Assessment - 070100849-1 SONGS HU Self Assessment - 070100167-1 NRA Regulatory Commitment Program Self Assessment - 061001434-1.Unit 1 Safety Events Self Assessment - 061000083-13.ISI Program. Self Assessment - 060901317-1.Security Bi-annual DAR Self Assessment -
071100837 Work Control R2C15 Outage Assessment Self Assessment - 080100904-1 CPRS. Self Assessment - 071001454-2 EP.
Self Assessment - 070701409-1.HPD Safety Committee Self Assessment - 070601122-1.NTD Operations Training Programs Self Assessment - 080400042-2 Thermal Overload failures Self Assessment - 800126624-1 Loose Electrical Fasteners

Calculations

M-74-06, "AFW Pump Room Heat Load Calculation"

Other Documents

Guidelines on Component Evaluation and Classification to INPO AP-913

Section 4OA5: Confirmatory Order Review

Procedures

SO123-XV-27, "OJT Training Program," Revision 10, SO123-XII-18.15 , Rev. 9, "Surveillance Program" SO123-TN-1, Rev. 5, "Nuclear Organization Training"
A1-9Actions Requests and Notifications
071200432
200005201
200164513
200003538
200150236
080501344
080101442
200158877
Orders
800073513
800073533
800075497
800075498
800136474

Other Documents

Letter from J. T. Reilly to NRC dated 2/28/08
Letter from J. T. Reilly to NRC dated 2/8/08
SCE Briefing Paper - Confirmatory Order Corrective Action Plan
Synergy Executive Summary Report - SONGS 2008 Nuclear Safety Cultural Assessment and Appendices Analysis of the Results of the 2008 Nuclear Safety Culture Assessment in the Areas of the Nuclear Safety Concerns Program and the Safety Conscious Work Environment.
Email from Mike Short dated 8/8/08, description of training for Confirmatory Order item 2b Ethics Training Material - Building a Diverse and Ethical Culture, When to Question/When to Support, Ethics and Compliance Code
Email from J. Dale to L. Kelly re: MGTPLI/R Issue Date for WFV
Ethics training attendance report as of 9/19/08
MGTPLR training attainment list
MGTPLI training attainment list List of enrollees in training module PDWILV
C08RTT training attainment lists as of 3/31/08 and 9/7/08
Lists of OJT trainers and TPEs qualified as of 10/1/08
SONGS Disciplinary Guidance Standards
SCE Corporate Policy 302 - Employee Discipline List of onsite service vendors and status of new indemnity clause added Supply Chain Summary - (fraud prevention plan alternative)
2008 SONGS Human Performance Tools Handbook Memo for File dated 7/10/08, Disposition of 2008 Willful Violation Certification Exceptions Email from Bob Corbett dated 11/20/06
SCE Corporate Policy 301, Professional Conduct September 9-11, 2008 Stand Down Meeting materials, "At SONGS Nuclear Safety is Everyone's Responsibility" Training Modules
MGTPLR - Management Policies Retrain General Employee Training
MGTPLI - Management Policies Initial General Employee Training
PDWILV - Detecting and Preventing Willful Violations
C08RTT - Responsibilities of On-the-Job Training (OJT)/ Task Performance Evaluators (TPEs) Trainers and Trainees
Surveillances
SOS-001-08
7/29/08 Chemistry Division Unit 2 Cycle 15 Overtime
SOS-005-08
4/2/08 Fire Watch Surveillance for Monitoring of Willful Non-Compliances
SOS-006-08
4/2/08 Security Willful Non-compliance Surveillance
SOS-020-08
7/31/08 Operations Round Activities Monitored for Willful Non-compliance
SOS-026-08
9/8/08 HP Division Willful Non-compliance
SOS-024-08
9/11/08 Nuclear Coatings Application Documentation Review Confirmatory Order Communications at SONGS (all apply to item 2k)
November 2007

a. Manager and Supervisor Forums

b. Presentation to Nuclear Fuels Group

A1-11January 2008

c. 1/21 & 1/23 Manager and Supervisor Forums - Confirmatory Order

d. 1/28 & 1/29 All Hands Meetings - J. Reilly/R. Ridenoure/D. Rosenblum presentations

e. SONGS Human Performance Tools Handbook (blue book) distributed at All Hands

f. 1/14 Operational Alignment bulletin - mentions Confirmatory Order

g. 1/21 Operational Alignment bulletin - discusses Confirmatory Order February 2008

h. 2/11, 2/13, 2/14 & 2/15 Manager and Supervisor Forums - blue book & discipline matrix (2g)

i. 2/20 & 2/27 Engineering Training Sessions - Ed Scherer presentation

j. 2/28 E-mail from Ross Ridenoure - New Mandatory Training (2h)

March 2008

k. 3/3 Operational Alignment bulletin - Mandatory WBT training (2h)

l. 3/5 E-mail from Ross Ridenoure - Recap of NRC Annual Performance Review

m. 3/28 & 3/31 All Hands Meetings - Intolerance for deliberate noncompliance (2g)

April 2008

n. 4/8 & 4/10 Health Physics All Hands - Ed Scherer/Lloyd Wright presentations

o. 4/21 & 4/28 Operations retraining sessions - Ed Scherer/Lee Kelly presentations May 2008 p. 5/5, 5/12 & 5/19 Operations retraining sessions - Ed Scherer/Lee Kelly presentations

q. 5/16 & 5/19 All Leaders Meetings - Willful Violation Certification

r. 5/26 Operational Alignment bulletin - Willful Violation Certification

s. 5/27 Ross' Weekly Video Message - Willful Violation Certification

t. 5/27 Security Supervisors All Hands Meeting - Ed Scherer/Lee Kelly presentation

June 2008

u. 6/2 Operational Alignment bulletin - Willful Violations Certification

v. 6/9 & 6/12 All Leaders Meetings - Willful Violation Certification reminder, Nuclear Safety Culture survey

w. 6/16 Operational Alignment bulletin - status of Willful Violations Certification

x. 6/17 Ross' Weekly Video Message - must complete Willful Violations Certification

y. 6/19 Security Supervisors All Hands Meeting - Ed Scherer/Lee Kelly presentation

z. 6/24 Ross' Weekly Video Message - Willful Violations Certification update July 2008

aa. 7/7 All Leaders Meeting - Accountability and expectations for ethical behavior bb. 7/9 & 7/10 All Hands Meetings - Corporate ethics policy, nuclear safety concerns cc. 7/9 & 7/10 Security Supervisors All Hands Meeting - Ed Scherer/Lee Kelly presentation dd. FOCUS newsletter story - Nuclear Safety Culture survey results September 2008 ee. 9/29 All Leaders Meeting ff. Sept/Oct All Hands

Attachment 2 SONGS PI&R Inspection Document Request Information Request - June 2008 San Onofre Nuclear Generating Station (SONGS) PIR Inspection (IP 71152; Inspection Report 50-361 and 362/08-12)
The inspection will cover the period of September 22, 2006 to September 30, 2008.
All requested information should be limited to this period unless otherwise specified.
To the extent possible, please provide the information in electronic media.
The agency's document software is in Microsoft Office.
However, we can also accept Word Perfect suite files and Adobe Acrobat (.pdf) text files.
Please provide the following information electronically, if necessary this information can be sent to the following address by September 8, 2008.
USNRC Resident Inspector Office Palo Verde Nuclear Power Plant ATTN: Ryan Treadway
5801 South Wintersburg Road Tonopah,
AZ 85396
Note: On summary lists, please include a description of the problem, status, and initiating date.
I. Summary list of all condition reports related to significant conditions adverse to quality that were opened or closed during the period
II. Summary list of all condition reports related to conditions adverse to quality that were opened or closed during the period
III. Summary lists of all condition reports which were up-graded or down-graded during the period
IV. A list of all corrective action documents that subsume or "roll up" one or more smaller issues for the period
V. Summary lists of operator workarounds, engineering review requests and/or operability evaluations, temporary modifications, and control room and safety system deficiencies opened or closed during the period
VI. List of all root cause analyses completed during the period
VII. List of root cause analyses planned, but not complete at the end of the period
VIII. List of plant safety issues raised or addressed by the employee concerns program
IX. List of action items generated or addressed by the plant safety review committees during the period Attachment 2
X. All quality assurance audits and surveillances of corrective actions completed during the period
XI. All corrective action activity reports, functional area self-assessments, and non-NRC third party assessments completed during the period (do not include INPO assessments)
XII. Corrective action performance trending/tracking information generated during the period and broken down by functional organization
XIII. Governing procedures/policies/guidelines for:
A. Corrective action program/condition reports B. Apparent and root cause evaluation/determinations C. Employee concerns program
XIV. A listing of all external events evaluated for applicability at SONGS during the period
XV. Condition reports or other actions generated during the period for each of the items below:
A. Part 21 reports B. NRC Information Notices, Bulletins, and Generic Letters
C. LERs issued by SONGS D. Vendor Safety Information Letters or Equivalent
E. NCVs and Violations issued to SONGS
XVI. Security event logs and security incidents during the period
XVII. Radiation protection event logs during the period
XVIII. Condition reports generated as a result of emergency planning drills and tabletop exercises during the period
XIX. Current system health reports or similar information during the period
XX. Condition reports associated with maintenance preventable functional failures during the period
XXI. Condition reports associated with adverse trends in equipment, processes, procedures, or programs during the period
XXII. Corrective action effectiveness review reports generated during the period
XXIII. Corrective Action documents should include detailed documentation of the issue, resolution, corrective actions, and final disposition as applicable
Attachment 2 XXIV. List of emergency plan exercise and drill deficiencies during the period
XXV. Quality assurance audit reports during the period
XXVI. Copies of corrective action documents associated with the safety committee action items provided
XXVII. Employee Concern Program Files/ Reports
XXVIII.
List of Training deficiencies, requests for training improvements, and simulator deficiencies for the period Detailed evaluations of Vendor "Safety Information Letters" or Equivalent Additional requests to the licensee made during the inspection August 8, 2008 Request: -Plant tour with Operations during onsite visit

-Meeting to discuss Corrective Action Program and SAP with CAP manager or designee -Meeting to discuss actions for Order with manager or designee and August 18, 2008 Request: -Powerpoint overview of SONGS CAP

August 20, 2008 Request: -Summary list of RPT evaluations -List of all Maintenance Rule systems in A.1 status -Electronic copy of SONSG Technical Specifications

-Electronic copy of Design Basis Document.

August 27, 2008 Request: -Total numbers of employees at SONGS that are SCE employees -Schedule of CAP meetings including ARRC, MRC and CARB

-Safety Culture assessment performed for the NRC Confirmatory Order -List of all CAP systems used for tracking deficiencies -Schedule of interviews and focus groups during inspection September 9, 2008 Request:

-All SONGS CAP Procedures -Maintenance Orders for significant equipment deficiencies -Summary list of DARs -Maintenance orders involving any troubleshooting efforts