ML102110445
| ML102110445 | |
| Person / Time | |
|---|---|
| Site: | San Onofre  |
| Issue date: | 07/30/2010 |
| From: | Hay M Division of Reactor Safety IV |
| To: | Ridenoure R Southern California Edison Co |
| References | |
| EA-10-125 IR-10-006 | |
| Download: ML102110445 (73) | |
See also: IR 05000361/2010006
Text
UNITED STATES
NUCLEAR REGULATORY COMMISSION
REGION IV
612 EAST LAMAR BLVD, SUITE 400
ARLINGTON, TEXAS 76011-4125
July 30, 2010
Mr. Ross T. Ridenoure
Senior Vice President and
Chief Nuclear Officer
Southern California Edison Company
San Onofre Nuclear Generating Station
P.O. Box 128
San Clemente, CA 92674-0128
SUBJECT: SAN ONOFRE NUCLEAR GENERATING STATION - NRC PROBLEM
IDENTIFICATION AND RESOLUTION INSPECTION
REPORT 05000361/2010006; 05000362/2010006 AND NOTICE OF VIOLATION
Dear Mr. Ridenoure:
On April 23, 2010, the U. S. Nuclear Regulatory Commission (NRC) completed the onsite
portion of a team inspection at your San Onofre Nuclear Generation Station. Additionally, the
inspectors performed in-office inspections through June 17, 2010. The enclosed report
documents the inspection findings discussed with you and members of your staff during an exit
briefing on June 17, 2010.
The inspection examined activities conducted under your license as they relate to identification
and resolution of problems, safety and compliance with the Commission's rules and regulations
and with the conditions of your operating license. The inspectors reviewed selected procedures
and records, observed activities, and interviewed personnel. The inspectors also interviewed a
representative sample of personnel regarding the condition of your safety conscious work
environment.
When compared with the findings from the previous inspection conducted in September 2008,
the findings from this inspection indicate that the corrective action program effectiveness has
declined. As previously discussed in the past 5 NRC assessment letters your staff's ability to
thoroughly evaluate problems such that the resolutions effectively address the causes and
extent of conditions is of concern. Your efforts to reverse the trend of substantive crosscutting
issues in both the human performance and problem identification and resolution areas have not
sho'lm to be effective.
The inspection identified a number of issues that your staff had previous opportunities to
identify. The Inspectors noted that even after issues were discussed with your staff thorough
evaluations were not consistently completed. We noted examples where your staff's
evaluations for deficient components failed to fully address component safety functions for all
applicable design basis accident scenarios.
Southern California Edison Company -2-
The inspectors reviewed the status of site corrective actions related to the areas of human
performance and problem identification and resolution described in your letters to the NRC
dated April 21, October 29, and October 30, 2009.
The inspectors noted examples where due dates were exceeded and different actions were
performed from those specified in the plan. As a result, the NRC identified a finding related to
your failures to meet the actions discussed in the above referenced letters. During the next
public meeting, that is currently being scheduled, you should address the status of your site
corrective actions and additional controls put in place to effectively monitor their execution. You
should also plan to address the causes for the inability to reverse the poor human performance
and problem identification and resolution trends.
This report documents ten NRC identified noncited violations, one NRC identified cited violation,
one self-revealing violation, and one finding, all of very low safety significance (Green).
Additionally, one licensee-identified violation is also discussed in this report. Because of the
very low safety significance of the violations and because they were entered into your corrective
action program, the NRC is treating these violations as noncited violations consistent with
Section VI.A.1 of the NRC Enforcement Policy. If you contest these noncited violations, you
should provide a response within 30 days of the date of this inspection report, with the basis for
your denial, to the Nuclear Regulatory Commission, ATTN: Document Control Desk,
Washington DC 20555-0001; with copies to the Regional Administrator, U.S. Nuclear
Regulatory Commission, Region IV, 612 E. Lamar Blvd., Suite 400, Arlington, Texas, 76011-
4125; the Director, Office of Enforcement, United States Nuclear Regulatory Commission,
Washington DC 20555-0001; and the NRC Resident Inspector at the San Onofre Nuclear
Generating Station.
The NRC-identified violation is cited in the enclosed Notice of Violation (Enclosure 1). The
violation involved the failure to revise and maintain in effect adequate procedures following plant
modifications. Although determined to be of very low safety significance (Green), this violation
is being cited in the Notice of Violation because not all of the criteria specified in Section VI.A.i
of the NRC Enforcement Policy for a non cited violation were satisfied. Specifically, San Onofre
Nuclear Generating Station failed to restore compliance within a reasonable time after
previously-identified noncited violations were identified in NRC Inspection Report 05000361;05000362/2009003-02 and 05000361;05000362/2009009-02. You are required to respond to
this letter and should follow the instructions specified in the enclosed Notice when preparing
your response. The NRC will use your response, in part, to determine whether further
enforcement action is necessary to ensure compliance with regulatory requirements.
If you disagree with the crosscutting aspect assigned to any finding in this report, you should
provide a response within 30 days of the date of this inspection report, with the basis for your
disagreement, to the Regional Administrator, Region IV, and the NRC Resident Inspector at
San Onofre Nuclear Generating Station.
Southern California Edison Company -3-
In accordance with 10 CFR 2.390 of the NRC's "Rules of Practice," a copy of this letter, its
enclosure, and your response (if any) will be available electronically for public inspection in the
NRC Public Document Room or from the Publicly Available Records (PARS) component of
NRC's document system (ADAMS). ADAMS is accessible from the NRC Web-site at
www.nrc.gov/reading-rm/adams.html(the Public Electronic Reading Room). To the extent
possible, your response should not include any personal privacy, proprietary, or safeguards
information so that it can be made available to the Public without redaction.
Sincerely,
Michael C. Hay, Chief
Technical Support Branch
Division of Reactor Safety
Dockets: 50-361; 50-362
Enclosures:
1. Notice of Violation
2. NRC Inspection Report 05000361/2010006; 05000362/2010006
w/Attachment: Supplemental Information
cc (w/Enclosures):
Chairman, Board of Supervisors
County of San Diego
1600 Pacific Highway, Room 335
San Diego, CA 92101
Gary L. Nolff
Assistant Director-Resources
City of Riverside
3900 Main Street
Riverside, CA 92522
Mark L. Parsons
Deputy City Attorney
City of Riverside
3900 Main Street
Riverside, CA 92522
Gary H. Yamamoto, P.E., Chief
Division of Drinking Water and
Environmental Management
1616 Capito! Avenue, MS 7400
P.O. Box 997377
Sacramento, CA 95899-7377
Southern California Edison Company -4-
Michael L. DeMarco
San Onofre Liaison
San Diego Gas & Electric Company
8315 Century Park Ct. CP21C
San Diego, CA 92123-1548
Director, Radiological Health Branch
State Department of Health Services
P.O. Box 997414 (MS 7610)
Sacramento, CA 95899-7414
The Mayor of the City of San Clemente
100 Avenida Presidio
San Clemente, CA 92672
James D. Boyd, Commissioner
California Energy Commission
1516 Ninth Street (MS 34)
Sacramento, CA 95814
Douglas K. Porter, Esquire
Southern California Edison Company
2244 Walnut Grove Avenue
Rosemead, CA 91770
Albert R. Hochevar
Southern California Edison Company
San Onofre Nuclear Generating Station
P.O. Box 128
San Clemente, CA 92674-0128
Steve Hsu
Department of Health Services
Radiologic Health Branch
MS 7610, P.O. Box 997414
Sacramento, CA 95899-7414
R. St. Onge
Southern California Edison Company
San Onofre Nuclear Generating Station
P.O. Box 128
San Clemente, CA 92674-0128
Chief, Technological Hazards Branch
FEMA Region IX
1111 Broadway, Suite 1200
Oakland, CA 94607-4052
Southern California Edison Company -5-
Institute of Nuclear Power Operations (lNPO)
Records Center
700 Galleria Parkway SE, Suite 10
Atlanta, GA 30339
NOTICE OF VIOLATION
Southern California Edison Company Docket No: 50-361; 50-362
San Onofre Nuclear Generating Station License No: NPF-10; NPF-15
During an NRC inspection, conducted from April 5 to April 23, 2010, a violation of NRC
requirements was identified. In accordance with the NRC Enforcement Policy, the violation is
listed below:
Technical Specification 5.5.1.1.a requires, in part, that written procedures be
established, implemented, and maintained covering the applicable procedures
recommended in Regulatory Guide 1.33, Revision 2, Appendix A, February 1978.
Regulatory Guide 1.33, "Quality Assurance Program Requirements (Operations),"
Appendix A, recommends procedures for the operation of certain plant systems.
Contrary to the above, prior to April 23, 2010, Southern California Edison Company
failed to maintain written procedures as recommended in Regulatory Guide 1.33,
Revision 2, Appendix A, February 1978. Specifically, the licensee failed to ensure that
following modifications made to the instrument air system the affected system
procedures where either suspended, put on administrative hold, or otherwise restricted
from use until the required changes were implemented. As a result, several procedures
with known technical deficiencies were inappropriately available for use following plant
modifications.
This violation is associated with a Green Significance Determination Process finding.
Pursuant to the provisions of 10 CFR 2.201, Southern California Edison Company is hereby
required to submit a written statement or explanation to the U.S. Nuclear Regulatory
Commission, ATTN: Document Control Desk, Washington, DC 20555-0001 with a copy to the
Regional Administrator, Region IV, and a copy to the NRC Resident Inspector San Onofre
Nuclear Generating Station, within 30 days of the date of the letter transmitting this Notice of
Violation (Notice). This reply should be clearly marked as a "Reply to Notice of
Violation EA-09-270," and should include: (1) the reason for the violation, or, if contested, the
basis for disputing the violation or severity level, (2) the corrective steps that have been taken
and the results achieved, (3) the corrective steps that will be taken to avoid further violations,
and (4) the date when full compliance will be achieved. Your response may reference or
include previous docketed correspondence, if the correspondence adequately addresses the
required response. If an adequate reply is not received within the time specified in this Notice,
an order or a Demand for Information may be issued as to why the license should not be
modified, suspended, or revoked, or why such other action as may be proper should not be
taken. Where good cause is shown, consideration will be given to extending the response time.
if you contest this enforcement action, you should also provide a copy of your response, with
the basis for your denial, to the Director, Office of Enforcement, United States Nuclear
Regulatory Commission, Washington, DC 20555-0001.
Because your response will be made available electronically for public inspection in the NRC
Public Document Room or from the NRC's document system (ADAMS), accessible from the
- 1- Enclosure 1
NRC website at www.nrc.gov/reading-rm/pdr.html or www.nrc.gov/reading-rm/adams.html, to
the extent possible, it should not include any personal privacy, proprietary, or safeguards
information so that it can be made available to the public without redaction. If personal privacy
or proprietary information is necessary to provide an acceptable response, then please provide
a bracketed copy of your response that identifies the information that should be protected and a
redacted copy of your response that deletes such information. If you request withholding of
such material, you must specifically identify the portions of your response that you seek to have
withheld and provide in detail the basis for your claim of withholding (e.g., explain why the
disclosure of information will create an unwarranted invasion of personal privacy or provide the
information required by 10 CFR 2.390(b) to support a request for withholding confidential
commercial or financial information).
Dated this 30th day of July 2010.
2- Enclosure 1
U.S. NUCLEAR REGULATORY COMMISSION
REGION IV
Docket: 50-361; 50-362
Report: 05000361/2010006; 05000362/2010006
Licensee: Southern California Edison Co.
Facility: San Onofre Nuclear Generating Station
Location: 5000 So. Pacific Coast Highway
San Clemente, California
Dates: April 5 through June 17, 2010
Team Leader: M. Vasquez, Senior Reactor Inspector, Technical Support Branch, DRS
Inspectors: C. Long, Senior Resident Inspector
R. Smith, Senior Resident Inspector
S. Walker, Senior Reactor Inspector
E. Ruesch, Resident Inspector
S. Matharu, Senior Electrical Engineer
G. Tutak, Project Engineer
Accompanied By: G. Wilson, Chief, Electrical Engineering Branch
S. Marquez, Nuclear Safety Professional Development Program
Approved By: Michael C. Hay, Chief
Technical Support Branch
Division of Reactor Safety
- 1- Enclosure 2
SUMMARY OF FINDiNGS
IR05000361 1201 0006; 05000362/2010006; October 1, 2008, through April 23, 2010:
San Onofre Nuclear Generating Station "Biennial Baseline Inspection of the Identification and
Resolution of Problems."
The report covers a 2-week period of onsite inspection by two senior resident inspectors, a
senior electrical engineer, a senior reactor inspector, a reactor inspector, and a project
engineer. Following the onsite inspection additional in-office reviews were performed through
June 17, 2010. The findings from this inspection include ten Green NRC identified noncited
violations, one Green self revealing violation; one Green cited violation, and one Green finding.
The significance of most findings is indicated by their color (Green, White, Yellow, Red) using
Inspection Manual Chapter 0609, "Significance Determination Process." Findings for which the
significance determination process does not apply may be Green or be assigned a severity level
after NRC management review. The NRC's program for overseeing the safe operation of
commercial nuclear power reactors is described in NUREG 1649, "Reactor Oversight Process,"
Revision 4, dated December 2006.
Identification and Resolution of Problems
The inspectors reviewed approximately 300 condition reports, work orders, engineering
evaluations, root and apparent cause evaluations, and other supporting documentation to
determine if problems were being properly identified, characterized, and entered into the
corrective action program for evaluation and resolution. The inspectors reviewed a sample of
system health reports, self-assessments, trending reports and metrics, and various other
documents related to the corrective action program.
When compared with the findings from the previous inspection conducted in September 2008,
the findings from this inspection indicate that the corrective action program effectiveness has
declined. As previously discussed in the past five NRC assessment letters, the licensee's ability
to thoroughly evaluate problems such that the resolutions effectively address the causes and
extent of conditions is of concern. The licensee's efforts to reverse the trend of substantive
crosscutting issues in both the human performance and problem identification and resolution
areas have not shown to be effective.
Additionally, the inspection identified a number of issues that the licensee's staff had previous
opportunities to identify. The inspectors noted that even after issues were discussed with the
licensees' staff, thorough evaluations were not consistently completed. We noted examples
were the evaluations for deficient components failed to fully address the component safety
functions for all applicable design basis accident scenarios.
The inspectors determined that the licensee adequately evaluated industry operating
experience for relevance to the facility, and entered applicable items in the corrective action
program. The inspectors noted that operating experience was considered in cause evaluations.
The inspectors noted that following the review of operating experience the licensee failed to
consistently incorporate the knowledge into procedural guidance and design calculations.
-2- Enclosure 2
In February 2010, the inspectors found that several work groups at San Onofre did not feel free
to raise safety concerns
without fear of retaliation. This was documented in NRC Inspection Report 050000361;
05000362/2009009 dated March 2, 2010, and in the NRC's Chilling Effect Letter dated March 2,
2010.
A. NRC-Identified and Self-Revealing Findings
Cornerstone: Initiating Events
II Green. The inspectors identified a noncited violation of Technical
Specification 5.5.1.1.a involving the failure of control room operators to follow
San Onofre Procedure S0123-0-A 1, "Conduct of Operations." These included
failures to: implement alarm response procedure place-keeping, announce
alarms to the control room supervisor, stop conversations when an alarm
annunciated and cleared, perform three-way communication during pre-job
briefing, review the summarize, anticipate, foresee, evaluate and review
questions during a pre-job brief, review the prerequisites of a procedure prior to
use, and remain cognitive of the re-activity change evolution by a control room
supervisor. This issue was entered into the licensee's corrective action program
as Nuclear Notification 200871332, and operations management immediately
began actions to institute a recovery plan to improve operator performance.
The finding was more than minor because it was associated with the Initiating
Events Cornerstone attribute of human performance, and it affected the
associated cornerstone objective to limit the likelihood of those events that upset
plant stability and that challenge critical safety functions during shutdown, as well
as during power operations. Using the Inspection Manual Chapter 0609,
"Significance Determination Process," Phase 1 Worksheet, the inspectors
concluded that the transient initiator did not contribute to both the likelihood of a
reactor trip and to the likelihood that mitigation equipment or functions would not
be available. As a result, the issue was of very low safety significance (Green).
The finding has a crosscutting aspect in the area of human performance
associated with the work practices because the licensee did not ensure
supervisory and management oversight of work activities.
H.4(c)(Section 40A2.5e)
II Green. The inspectors reviewed a self-revealing noncited violation of Technical
Specification 5.5.1.1.a involving the failure to maintain adequate instructions in
San Onofre Procedure S023-3-2.4, "RCS Purification and De-borating Ion
Exchanger Operation," Revision 21 to control borating of ion exchangers. The
failure to maintain an adequate procedure resulted in an unplanned power
reduction by control room operators. This issue was entered into the licensee's
corrective action program as Nuclear Notification 200721702. Immediate
corrective actions :ncluded revising the procedure and operator Cff9w training.
-3- Enclosure 2
The finding was more than minor because it was associated with the Initiating
Events Cornerstone attribute of human performance, and it affected the
associated cornerstone objective to limit the likelihood of those events that upset
plant stability and that challenge critical safety functions during shutdown, as well
as during power operations. Using the Inspection Manual Chapter 0609,
"Significance Determination Process," Phase 1 Worksheet, the inspectors
concluded that the transient initiator did not contribute to both the likelihood of a
reactor trip and to the likelihood that mitigation equipment or functions would not
be available. As a result, the issue was of very low safety significance (Green).
The finding has a crosscutting aspect in the area of human performance
associated with the work practices because licensee supervisory personnel did
not ensure activities associated with re-activity control were performed in a
controlled manner such that nuclear safety was assured.
H.4(c)(Section 40A2.5f)
- Green. The inspectors identified a noncitied violation of Technical
Specification 5.5.1.1.a involving the failure to follow procedural guidance of
S0123-XX-11, "Switchyard Work Performance." Specifically, the inspectors
identified temporary equipment stored in the switchyard that was not tethered or
otherwise secured in accordance with the procedure. The licensee entered a
notification in its corrective action program as Nuclear Notification 200870138,
and removed or secured the items.
This finding is more than minor because it impacts the protection against the
external factors attribute of the Initiating Events Cornerstone objective to limit the
likelihood of those events that upset plant stability and challenge critical safety
functions during shutdown and power operations. Using the Inspection Manual
Chapter 0609 "Significance Determination Process," Phase 1 Worksheet, the
inspectors determined that the finding was of very low safety significance (Green)
because it did not contribute to both the likelihood of a reactor trip and the
likelihood that mitigation equipment or functions would not be available. This
finding also has a human performance crosscutting aspect associated with the
work control component in that personnel failed to appropriately plan work
activities involving job site conditions which may impact plant structures, systems
and components. H.3(a) (Section 40A2.5k)
Cornerstone: Mitigating Systems
- Green. The inspectors identified a non cited violation of 10 CFR Part 50,
Appendix B, Criterion V, "Instructions, Procedures, and Drawings,"
involving the failure to follow procedural requirements for performing
operability determinations. Specifically, the licensee's operability
evaluation for a degraded turbine-driven auxiliary feedwater pump steam
admission valve failed to address all the specified safety functions of the
affected component as described in the Final Safety Analysis Report and
design basis documents. For exampie, the operabiiity determination
incorrectly stated that manual closure of the valves was not a credited
-4- Enclosure 2
safety function and incorrectly assumed nonsafety-related instrument air
would always be available to close the valves. This finding was entered
into the licensee's corrective action program as Nuclear Notifications
200869281 and 200887620. The licensee's corrective actions included
re-performing the evaluation and emphasizing with licensee staff the
importance of ensuring ali design basis information is considered in
operability evaluations.
The finding was more than minor because it impacted the Mitigating
Systems Cornerstones and its objective to ensure the availability and
reliability of equipment that responds to initiating events. Using
Inspection Manual Chapter 0609 the issue screened to a Phase 3
analysis because it represented a loss of safety function for greater than
the allowed technical specification allowed outage time and it screened to
greater than Green using the Phase 2 pre-solved worksheet. The senior
reactor analyst determined that this finding was of very low safety
significance (Green) based on a bounding calculation which assumed
inoperability of the component for a year. The senior reactor analyst
determined that the combined significance of these scenarios was a
delta-core damage frequency of 1.3E-7/yr and a delta-large early release
frequency of 4.2E-8/yr. Therefore the violation was determined to be of
very low safety significance (Green). The analyst determined that the
cause of the finding has a crosscutting aspect in the area of human
performance associated with decision making. Specifically, the licensee
utilized unsupportable assumptions in its evaluation that were not
consistent with the Final Safety Analysis Report or the valve vendor
manual. H.1.b](Section 40A2.5a)
- Green. The inspectors identified a noncited violation of 10 CFR Part 50,
Appendix B, Criterion III, "Design Control" in that the licensee failed to translate
design basis information into procedures for the turbine-driven auxiliary
feedwater pump steam admission valves. Specifically, the licensee did not
translate into procedures the design requirements to manually close and gag the
valves within 30 minutes in response to high energy line breaks, a fire in the
auxiliary feedwater pump room, or a steam generator tube rupture event. This
issue was entered into the licensee's corrective action program as Nuclear
Notification 200887620. Immediate actions included posting a leveraging device
for operators to use should it be necessary, training operators, and scheduling
lubrication of the valves.
The finding is more than minor because it impacted the Mitigating Systems
Cornerstones and its objective to ensure the availability and reliability of
equipment that responds to initiating events. The analyst screened the issue to
more than one cornerstone due to its effect on early release (steam generator
tube rupture), fire protection, and mitigating systems (high energy line break).
The analyst performed a Phase 3 analysis that considered the effects of a high
energy line break in the pump room, a steam generator tube rupture, and fires in
-5- Enclosure 2
the pump room and auxiliary feedwater pipe tunnel. The analyst determined that
the combined significance of these scenarios was a delta- core damage
frequency of 5.E-9/yr and a delta- large early release frequency of 1.6E-9/yr.
Therefore, the violation was determined to be of very low safety significance
(Green). The inspectors determined that cause of the finding has a crosscutting
aspect in the area of problem identification and resolution associated with the
corrective action program. Specifically, the licensee had previous opportunities
to identify this problem when the valve was removed from the in-service testing
program and when they evaluated relevant external operating experience.
[P.i (a)j(Section 40A2.5b)
Green. The inspectors identified a noncited violation of Technical Specification 3.7.6, which requires, in part, that Condensate Storage Tank T-120
be operable. Specifically, the tank isolation valve 2HV5715 had been inoperable
for a period greater than the allowed outage time of seven days while Unit 2 was
in Modes 1, 2, and 3. The valve isolates nonseismic piping from the tank and is
required to be manually closed within 90 minutes following a seismic event. The
licensee had not performed preventive maintenance on the valve resulting in the
valve failing to close during an in-service test on January 26, 2010. This finding
was entered into the licensee's corrective action program as Nuclear
Notification 200765235. The licensee's corrective actions included repairing the
isolation valve.
This finding is more than minor because it impacted the Mitigating Systems
Cornerstone objective to ensure the availability, reliability, and capability of
systems that respond to initiating events to prevent undesirable consequences.
Using Inspection Manual Chapter 0609, Phase 1, "Initial Screening and
Characterization of Findings," a Phase 2 analysis was performed because the
condensate storage, Tank T-120, was inoperable greater than that allowed in
technical specifications. Phase 2 analysis resulted in a potential greater than
Green issue therefore, a Phase 3 was performed.
The analyst performed a Phase 3 using San Onofre seismic information and
fragility data associated with the piping that could not be isolated because of the
failed condition of valve 2HV5715. The frequency of a seismic event that would
cause a pipe break and drain tank T -120 was estimated to be 2.7E-5/yr. Given a
seismic event that causes a loss of offsite power (nearly 100 percent of seismic
events that rupture the piping would also cause a loss of offsite power), operators
are compelled by procedure to cool down and initiate shutdown cooling. The
amount of water that is protected with valve 2HV5715 failed to open, which
includes inventory from tank T-121 and water below the break line in tank T-120,
given that operators close the working manual isolation valve within 30 minutes,
is more than what is needed to get to shutdown cooling in natural circulation with
only 1 of 2 steam generator atmospheric dump valves in operation, even if there
is a 4-hour hold time at hot standby. The analyst estimated that the failure
probability of operators to coo! down and initiate shutdown coo!!ng is 1.0E-2.
Therefore, assuming a zero base case, the estimated delta- core damage
-6- Enclosure 2
frequency of the finding is 2.7E-5/yr. (1.0E-2) =2.7E-7/yr.
The inspectors also determined that the cause of the finding has a crosscutting
aspect in the area of human performance associated with resources in that the
licensee did not ensure that equipment was available and adequate to assure
nuclear safety by minimization of long-standing equipment issues in that the
valve was not being maintained through a preventive maintenance program.
H.2(a)(Section 40A2.5c)
- Green. The inspectors identified a cited violation of Technical
Specification 5.5.1.1.a, involving the failure to maintain adequate written
procedures. Specifically, as of April 23, 2010, the licensee's controls over
its backlog of procedure change requests associated with plant
modifications were inadequate to prevent licensee personnel from using
outdated procedures with known technical errors in the plant. The
performance deficiency of failing to control the backlog of procedure
changes, such that procedures with known technical errors were in use in
the plant were previously identified by the NRC on two occasions and
were documented as non cited violations 05000361;05000362/2009003-09 and 2009009-02. Because the licensee failed to
restore compliance within a reasonable time after the previous non cited
violations were identified, this violation is being cited in a Notice of
Violation in accordance with Section Vl.a.1 of the NRC's Enforcement
Policy. This finding was entered into the licensee's corrective action
program as Nuclear Notification 200888919. The licensee's corrective
action included immediate actions to administratively suspend these
procedures until they could be revised and to evaluate changes needed
to its program to prevent recurrence.
The failure to maintain procedures covered by Regulatory Guide 1.33 is a
performance deficiency. The finding is of more than minor significance
because, if left uncorrected, the failure to maintain and control procedures
would have the potential to lead to a more significant safety concern.
Using Inspection Manual Chapter 0609, Phase 1,"Initial Screening and
Characterization of Findings," the finding was determined to have a very
low safety significance because the finding did not result in a loss of
system safety function, an actual loss of safety function of a single train
for greater than its technical specification allowed outage time, or screen
as potentially risk significant due to a seismic, flooding, or severe weather
initiating event. The finding has a crosscutting aspect in the area of
problem identification and resolution associated with the corrective action
program component, because problems were not thoroughly evaluated,
such that the resolutions addressed the causes and extents of condition.
This includes properly classifying and prioritizing conditions adverse to
quality. [P.i (c)](Section 40A2.5h)
-7- Enclosure 2
Green. Two examples of a noncited violation of 10 CFR 50.65(a)(1) were
identified involving the failure to monitor the unavailability time associated
with equipment failures which were maintenance induced. The first
example involved maintenance inadvertently bending the fuse holder
contacts such that there was a loose connection on the power supply on
the turbine-driven auxiliary feedwater pump resulting in its failure. The
second example involved the failure to perform maintenance associated
with a condensate storage tank isolation valve resulting in its failure
during in-service testing. In both cases, if the licensee had assessed the
unavailability time due to the maintenance induced failures, the systems
would have exceeded the 10 CFR 50.65(a)(2) monitoring criteria,
necessitating the systems to be placed in 10 CFR 50.65(a)(1) goal
setting. The licensee's corrective actions included evaluating its
procedures to prevent recurrence, and re-evaluating these systems to
determine the impact of accounting for unavailable time.
This finding is more than minor because it affects the equipment
performance attribute of the Mitigating Systems Cornerstone per
Inspection Manual Chapter 612, Appendix 8. Using Inspection
Manual Chapter 0609, Phase 1, "Initial Screening and Characterization of
Findings," the inspectors determined the finding to be of very low safety
significance (Green) because they did not represent the loss of a system
safety function and did not screen as potentially risk significant due to a
seismic, flooding, or severe weather initiating event The cause of the
finding was determined to have a crosscutting aspect in the area of
human performance. Specifically, personnel failed to use a formal
decision making process to determine how to count unavailable hours for
the maintenance rule. [H.i (a)](Section 40A2.5i)
Green. The inspectors identified a noncited violation of 10 CFR Part 50,
Appendix 8, Criterion XVI, "Corrective Action," in that, from October 2008
to April 2010, the licensee failed to promptly identify and correct
potentially degraded motor-driven relays in safety-related systems and
components. Specifically, after identifying a degraded relay affecting an
emergency diesel generator, the licensee replaced all similar relays in the
other diesel generators but failed to evaluate the use of these potentially
degraded relays in other safety-related systems. The licensee entered
this issue into the corrective action program as Nuclear
Notification 200146292, and developed a plan to replace the 62 degraded
relays that were installed in other safety-related equipment.
This finding was more than minor because it impacted the equipment
performance attribute of the Mitigating Systems Cornerstone objective to
ensure the availability, reliability, and capability of systems that respond
to initiating events to prevent undesirable consequences. Using
11""IoC'-l"'\o,...+il"'\.t"\ I\n"'U'"\II,,,1 f"h", ..... +or
II h.:J't-"",,\.*n,lvl I fYIc;..t1 U ...H;;.t! ""'llt,;;.II.tJl. ...... 1
nanQ "A
VVV-..J. V i ,
Dh",t"1"'\. 01
I f IQ""~ I,
"1_i+i_1 C_V'I"'\.I"'\._i_,..,. __ ...J
II IIl1QI V\.II vvlllll~ QIIU
Characterization of Findings," the inspectors determined the finding to be
-8- Enclosure 2
of very low safety significance (Green) because it did not represent the
loss of a system safety function and did not screen as potentially risk
significant due to a seismic, flooding, or severe weather initiating event.
This finding has a crosscutting aspect in the area of human performance
associated with the decision-making component, in that the licensee did
not use conservative assumptions in making decisions about the extent of
condition [H.1 (b)J(Section 40A2.Sj)
Green. The inspectors identified a noncited violation of 10 CFR Part SO,
Appendix B, Criterion III, "Design Control," involving the failure to translate
nonconservative errors in calculations and procedures identified during review of
external operating experiences. The first example involved the sizing calculation
for the condensate storage tank failing to account for effects of auxiliary
feedwater pump heat during recirculation. The second example involved the
failure to update procedural guidance concerning the adverse effects of placing
the low pressure safety injection system into operation following use of the
residual heat removal system in the shutdown cooling mode of operation above
200°F. This issue was entered into the licensee's corrective action program as
Nuclear Notification 20088626S. The licensee initiated actions to correct its
procedure and calculation for each instance.
The finding is of more than minor significance because it adversely affects the
design control attribute of the mitigating systems cornerstone objective. Using
Inspection Manual Chapter 0609.04, Phase 1, "Initial Screening and
Characterization of Findings," the finding was determined to have a very low
safety significance (Green) because the finding did not result in a loss of system
safety function, an actual loss of safety function of a single train for greater than
its technical specification allowed outage time, or screen as potentially risk
significant due to a seismic, flooding, or severe weather initiating event. The
finding has a crosscutting aspect in the area of problem identification and
resolution associated with the operating experience component because the
licensee failed to implement and institutionalize operating experience information,
including vendor recommendations, through changes to plant processes,
procedures, equipment, and training programs. P.2(b)(Section 40A2.SI)
Cornerstone: Public Radiation Safety
- Green. The inspectors identified a noncited violation of Technical
Specification S.S.1.1.a, "Scope," involving the failure to establish
procedures for component cooling water system alignments such that
leakage of radionuclides to the environment would be monitored during all
operational alignments of component cooling water. Specifically,
radiation monitors could be aligned to only one train of component cooling
water at a time and the licensee's procedures had no provision for
monitoring the second train when both trains were in-service. This finding
was entered into the licensee's corrective action program as Nuc!ear
- 9- Enclosure 2
It Notification 200871387, and actions were implemented to require periodic
grab sampling of the train which was not being monitored.
The inspectors determined that this finding was more than minor because
this issue impacted the Public Radiation Protection Cornerstone and its
objective to ensure adequate protection of public health and safety from
exposure to radioactive materials released into the public domain as a
result of routine civilian nuclear reactor operation. Specifically, the
radiation monitors for component cooling water were not sufficient to
ensure adequate release measurements. The inspectors evaluated the
significance of this finding using Phase 1 of Inspection Manual Chapter 0609.04 and determined that the finding screened to Inspection
Manual Chapter 0609, Appendix D, "Public Radiation Safety Significance
Determination Process." The inspectors evaluated the significance of this
finding using Inspection Manual Chapter 0609, Appendix D, and
determined that the finding was of very low safety significance (Green)
because dose did not exceed Appendix I criteria. This finding was
determined to have a crosscutting aspect in the area of problem
identification and resolution associated with the corrective action program
in that the plant operators did not have a low threshold for identifying
deficiencies in procedures. [P.i (c)](Section 40A2.Sg)
Cornerstone: Miscellaneous
It Severity Level IV. The inspectors identified a Severity Level IV noncited
violation of 10 CFR SO.73, "Licensee Event Report System," in which the
licensee failed to submit a licensee event report within 60 days following
discovery of an event meeting the reportability criteria. On
January 26, 2010, the valve which isolates nonseismic piping from
condensate storage tank T -120 failed its in-service test when the hand
wheel stem snapped after a leveraging device was used in an attempt to
close the valve. This isolation valve, 2HVS71S, must be closed within 90
minutes of an operating basis earthquake in order to prevent the loss of
condensate storage tank T-120 water inventory from a line break in the
nonseismic portion of the condensate system. The failure of this valve
resulted in a condition prohibited by Technical Specification 3.7.6 and
therefore was reportable. This finding was entered into the licensee's
corrective action program as Nuclear Notification 200888616, and the
licensee was taking actions to send a licensee event report to the NRC
for this event.
The inspectors determined that traditional enforcement was applicable to
this issue because the NRC's regulatory ability was affected. Specifically,
the NRC reiies on the iicensee to identify and report conditions or events
meeting the criteria specified in regulations in order to perform its
regulatory function. The inspectors determined that this finding was not
suitable for evaluation using the significance determination process, and
- 10 - Enclosure 2
as such, was evaluated in accordance with the NRC Enforcement Policy.
The finding was reviewed by NRC management, and because the
violation was determined to be of very low safety significance, was not
repetitive or willful, and was entered into the corrective action program,
this violation is being treated as a Severity Level IV noncited violation
consistent with the NRC Enforcement Policy. This finding was
determined to have a crosscutting aspect in the area of problem
identification and resolution associated with the corrective action program
in that the licensee failed to appropriately evaluate corrective
maintenance as a basis for past operability. [P.1 (c)](Section 40A2.5d)
- Green. The inspectors identified a Green finding associated with the
licensee's failure to meet the actions described to the NRC in letters
dated April 21,2009, and October 29 and 30, 2009, addressing corrective
actions to improve site performance in the areas of human performance
and problem identification and resolution. Specifically, 16 actions were
not implemented on time and a number of actions were modified from
what was previously described, all prior to informing the NRC. These
findings were documented in Nuclear Notification 200848923.
The inspectors determined that the licensee's failure to perform actions
as documented in its plan to the NRC was more than minor because if left
uncorrected could result in a more significant safety concern. Using
Inspection Manual Chapter 0609, Appendix M, this finding was reviewed
by NRC management and was determined to be of very low safety
significance (Green). This finding has a crosscutting aspect in the areas
of human performance. (Section 40A2.5m)
B. Licensee-Identified Violations
A violation of very low safety significance, which was identified by the licensee, has been
reviewed by the inspectors. Corrective actions taken or planned by the licensee have
been entered into the licensee's corrective action program. This violation and corrective
action tracking numbers (condition report numbers) are listed in Section 40A7.
- 11 - Enclosure 2
REPORT DETAilS
4. OTHER ACTIVITIES
40A2 Problem Identification and Resolution (71152)
The inspectors based the following conclusions on the sample of corrective action
documents that were initiated in the assessment period, which ranged from October 1,
2008, to the end of the on-site portion of this inspection on April 23, 2010 .
.1 Assessment of the Corrective Action Program Effectiveness
a. Inspection Scope
Approach and Scope: The inspectors visited San Onofre Nuclear Generating
Station from December 14 through 17, 2009, to review the sites corrective action
and maintenance backlogs. The backlog review included corrective actions,
maintenance actions and administrative actions involving pending procedure
changes.
The results of these reviews were used to select issues involving risk important
systems and operator actions that would be reviewed during future inspections.
The following areas were identified for future inspection:
.. Agastat relay failures
.. Medium voltage breakers conditions (safety related and nonsafety
related)
- Switchyard breaker maintenance practices
- Switch yard transformers conditions
.. Backlog of pending procedure changes
.. Component power supplies problems
.. Aged electrolytic capacitors
- High relay and breaker auxiliary contact resistance
e Electrical grounds
- Boric acid leaks
.. Emergency core cooling system voids
- 12 - Enclosure 2
It Reactivity control (chemical and volume control system)
.. Mitigating systems performance indicator trending
.. Component cooling water system voids
- Component cooling water pump in runout conditions
- Drifting undervoltage relays setpoint
- Auxiliary feedwater pump problems
- Battery room hydrogen monitors
- Discolored 4kV and 480V Cables (and thermography results)
- DC Bus 301 low voltage
- High pressure safety injection swing pump logic problems
- Charging pump oil leaks
- Emergency diesel generators degraded conditions
- Pending plant modifications
.. Control room annunciator problems
.. Operator workarounds!operator burdens
The inspectors reviewed approximately 300 condition reports, including
associated root cause, apparent cause, and direct cause evaluations, that were
initiated between October 1, 2008, and April 5, 2010, to determine if problems
were being properly identified, characterized, and entered into the corrective
action program for evaluation and resolution. The inspectors also reviewed
system health reports, operability determinations, self assessments, trending
reports, metrics, and various other documents related to the corrective action
program. The inspectors reviewed work requests and attended the licensee's
corrective action review board and closure review board meetings to assess the
reporting threshold and prioritization processes. The inspectors' review included
verifying that the licensee considered the full extent of cause and extent of
condition for problems, as well as how the licensee assessed generic
implications and previous occurrences. The inspectors assessed the timeliness
and effectiveness of corrective actions, completed or planned, and looked for
additional examples of similar problems.
addressed past NRC-identified violations to ensure that the corrective actions
- 13- Enclosure 2
addressed the issues as described in the inspection reports. The inspectors
reviewed a sampie of corrective actions closed to other corrective action
documents to verify that corrective actions were appropriate and timely.
The inspectors considered risk insights to focus the sample selection and plant
tours on risk significant systems and components. Based on this review, the
samples reviewed by the inspectors focused on, but were not limited to, these
systems. The inspectors also expanded its review to include five years of
evaluations involving the salt water cooling system and various electrical
components to determine whether problems were being effectively addressed.
The inspectors conducted a walkdown of these systems to assess whether
problems were identified and entered into the corrective action program.
b. Assessments
i. Assessment - Effectiveness of Problem Identification
In general, the inspectors found that the licensee has been identifying
problems and entering them into their corrective action program at
appropriately low thresholds. For example, San Onofre Nuclear
Generating Station personnel had identified and initiated over 20,000
nuclear notifications into the corrective action process in 2009. The
inspectors identified many examples of failures to document problems
into the corrective action program resulting in missed opportunities for the
licensee to identify problems and adverse trends. In addition, there were
several issues that took significant NRC interaction with site staff in order
for them to recognize the problem. Examples of ineffective identification
of issues include the following:
- The licensee failed to identify design basis information regarding
the steam admission valves to the turbine auxiliary feedwater
pump. On April 5, 2010, inspectors identified a concern that the
valves might not be able to be manually closed due to the
apparent lack of lubrication and rust on the Unit 3 valve stems
(3HV8200 and 3HV8201). These valves are normally held open
under spring pressure and are normally closed with nonsafety-
related instrument air. In cases where instrument air is not
available, the valve may be closed manually by rotating a hand
wheel approximately 24-25 rotations. The inspectors reviewed
design basis documents and the Final Safety Analysis Report and
found that the valves must be manually closed within 30 minutes
for certain accident sequences where instrument air is not
available. Based on the inspectors' concern that manually closing
the valve would be challenged with the lack of lubrication, San
Onofre Nuclear Generating Station conducted an operability
determination on April 10, 2010. Hovvever, the inspectors round
that the operability evaluation was inadequate and did not
- 14 - Enclosure 2
consider design basis information. After significant NRC
interaction, the licensee consulted with the vendor and found that
the valves could not be manually closed even under ideal
lubrication conditions because the force required to manually turn
the hand wheel exceeded the licensee's guideline for the amount
of force an individual could be expected to exert. Prior to the
inspectors' questioning, the licensee had failed to identify the force
needed to manually close the valve as well as other design basis
information. (Section 40A2.5b)
- San Onofre Nuclear Generating Station failed to identify that the
failure of isolation valve 2HV5715 was reportable to the NRC.
This valve isolates nonseismic piping from the seismic piping on
condensate storage tank T-120. This valve must be closed within
90 minutes of an operating basis earthquake to prevent the tank
from draining its water through a postulated break in the
nonseismic piping. On January 26, 2010, an operator attempted
to perform the 2-year in-service test to manually stroke the valve
by rotating its hand wheel. When the hand wheel would not turn,
the operator followed procedure and contacted the control room to
obtain permission to use a leveraging device to turn the hand
wheel. When the operator used the leveraging device, the hand
wheel sheared off. San Onofre Nuclear Generator Station
reportability determination concluded the event was not reportable
because a mechanic could be called to disassemble the valve
actuator and manually close the valve with a wrench. During the
weeks of April 5 and April 19, the inspectors informed San Onofre
Nuclear Generator Station staff that this use of corrective
maintenance was inappropriate to consider for reportability
determination. The licensee maintained this position through a
"white paper" developed on May 7,2010. Subsequently, the
inspectors contacted the licensee and referred the licensee to the
specific guidance in NUREG 1022, whereby the licensee changed
its position. (Section 40A2.5d)
- The licensee failed to identify that a nuclear notification had not
been written, as required by procedure, to document that a
leveraging device had been used when an operator sheared the
hand wheel off of the isolation valve (2HV5715) which isolates
nonseismic piping from the seismic piping on condensate storage
tank T-120. (Section 40A2.5c)
- The inspectors questioned the ability of plant equipment operators
to identify plant problems during plant tours as a result of
knowledge deficiencies identified by the inspectors. On
AprH 7, 2010, inspectois obseived an experienced piant
equipment operator performing his daily rounds for several hours.
- 15 - Enclosure 2
The inspectors found that the nonlicensed operator did not
demonstrate fundamental knowledge regarding such items as
separation distances between scaffolding and safety-related
equipment, expected panel configurations, and requirements for
standard items like chocking carts. As a result, the inspectors
determined that given these knowledge weaknesses exhibited by
an experienced equipment operator they were limited in their
ability to identify plant problems.
- The inspectors identified that some plant personnel appeared to
accept degraded or unacceptable conditions rather than
identifying the condition through the corrective action process and
getting them corrected. Examples included: (1) the common use
of leveraging devices which can mask degraded conditions;
(2) there were a number of control room alarms that had not been
cleared in preparation for the Unit 2 startup from the steam
generator replacement outage; (3) the inspectors identified that
one control room alarm had been locked in for four days because
data on a computer card needed to be downloaded; (4) after
inspectors questioned control room operators about the vibration
and loose parts monitor alarm, control room staff realized that they
were in day 5 of a 30-day action statement required by licensee
controlled specifications; and (5) the inspectors identified that
unsecured equipment in the switchyard that had been there for
months in violation of licensee procedures even though operators
had been performing routine rounds and others had been going in
and out of the area.
The inspectors noted that operators were not sensitive to a
condition involving the failure to have adequate procedures to
ensure that for all operational alignments of the component
cooling water system radiation monitoring would be in effect to
detect system leakage. (Section 40A2.5g)
.ii Assessment - Effectiveness of Prioritization and Evaluation of Issues
The inspectors found many instances where the licensee had correctly
prioritized and evaluated issues. In fact, there was objective evidence
that the quality of cause evaluations had improved during this inspection
period. However, the inspectors also found that San Onofre Nuclear
Generating Station continued to have significant challenges performing
these actions consistently. While most initial operability determinations
were appropriate, the inspectors identified several examples where poor
evaluations were performed. The following are examples of ineffective or
inadequate evaluation of issues:
- 16 - Enclosure 2
San Onofre Nuclear Generating Station staff performed an
inadequate evaluation of the reportability of the failure of the
isolation valve for condensate storage tank T-120.
(Section 40A2.5d)
San Onofre Nuclear Generating Station staff performed an
inadequate operability determination of the steam admission
valves to the turbine-driven auxiliary feedwater pumps after the
inspectors raised concerns about lack of stem lubrication.
(Section 40A2.5a)
- San Onofre Nuclear Generating Station staff performed an
inadequate extent of condition evaluation involving potentially
degraded Potter & Brumfield motor driven rotary relays.
(Section 40A2.5j)
- An operability determination was inadequate evaluating a loose
electrical connection in high pressure safety injection motor
cubicle 2A0608. Specifically, the method used to evaluate the
circuit continuity did not properly take into account the circuit
operation. The licensee initiated Nuclear Notification 200871532
on April 9, 2010, to evaluate the inspector's concern.
- The inspectors reviewed a root cause, three apparent causes, and
one common cause evaluation dealing with operators failing to
properly make correct operability determinations. In one example,
operators failed to declare an atmospheric dump valve inoperable
based on testing results and failed to write a nuclear notification
when the degraded conditions changed. Additionally, the
inspectors identified that the licensee had also failed to implement
all its corrective actions associated with this example.
iii. Assessment - Effectiveness of Corrective Action Program
The inspectors concluded that actions to correct conditions adverse to
quality were generally adequate; however, there were notable examples
where the licensee had not implemented effective corrective actions.
Some examples included:
- Licensee actions to correct substantive crosscutting issues have
not been effective. Despite actions to reverse the trend, San
Onofre Nuclear Generating Station has experienced five
consecutive assessment cycles with an increasing number of
sUbstantive crosscutting issues.
.. San Onofie Nuclear Generating Station actions have not been
effective in responding to two previously issued NCVs dealing with
- 17 - Enclosure 2
prioritizing the large backlog of procedure change requests.
During this inspection, the inspectors found that procedure
changes were not implemented following modifications to the
instrument air system. The inspectors concluded that San Onofre
Nuclear Generator Station corrective action to two previously
issued noncited violations for the same issue were not fully
effective. This violation is being cited as a Notice of Violation.
(Section 40A2.Sh)
- The licensee's actions to improve the conduct of operations in the
control room have not been effective based on control room
observations, which identified ineffective use of place keeping,
use of 3-way communications, announcing alarms to the control
room supervisor, and review of prerequisites prior to procedures
being implemented. (Section 40A2.Se)
.2 Assessment of the Use of Operating Experience
a. Inspection Scope
The inspectors examined the licensee's program for reviewing industry operating
experience, including reviewing the governing procedure and self-assessments.
A sample of operating experience notification documents that had been issued
during the assessment period were reviewed to assess whether the licensee had
appropriately evaluated the notification for relevance to the facility. The
inspectors also examined whether the licensee had entered those items into their
corrective action program and assigned actions to address the issues. The
inspectors reviewed a sample of root cause evaluations and significant condition
reports to verify if the licensee had appropriately included industry operating
experience.
b. Assessment
Overall, the inspectors determined that the licensee had appropriately evaluated
industry operating experience for relevance to the facility, and had entered
applicable items in the corrective action program. The inspectors noted that
operating experience was considered in cause evaluations. The licensee failed
to incorporate two of the four operating experience evaluation results into plant
operating procedures and design documents. This is documented as a violation
of 10 CFR Part SO, Appendix B, Criterion III. (Section 40A2.SI)
.3 Assessment of Self-Assessments and Audits
a. Inspection Scope
Th""
- _ inc:n""f"'tnrc:
........ ,.... _ _ ........ _ *r"""j""\AI",,rl
_ ... _ ~_
.... _ _ _ C::::Imnl""
_ * * * ,....._ nf
...... lif"'""nc:"""" C:Qlf_""C:C:Qc:c:mQntc:
0' _ _ ' * _ _ _ _ _ ., ""nrl
_ _ _ _ _ _ 111_.1"' _ _ """nit",
,. _ _ _ _ I\.V f,.,.
\.v
assess whether the licensee was regularly identifying performance trends and
- 18 - Enclosure 2
effectively addressing them. The inspectors also reviewed audit reports to
assess the effectiveness of assessments in specific areas. The specific self-
assessment documents and audits reviewed are listed in the attachment.
b. Assessment
The inspectors concluded that the licensee had an effective self-assessment
process. Licensee management was involved in developing the topics and
objectives of self-assessments. Attention was given to assigning inspectors
members with the proper skills and experience to do an effective
self-assessment and to include people from outside organizations. Audits were
self-critical and identified deficiencies in various programs such as the corrective
action program and the equipment reliability program .
.4 Assessment of Safety-Conscious Work Environment
a. Inspection Scope
From February 1-10, 2010, a inspectors conducted 40 focus group sessions
consisting of approximately 8-10 individuals each. The focus groups were
conducted to assess the safety-conscious work environment at the San Onofre
Nuclear Generating Station. The results of the focus groups were documented in
NRC Inspection Report 05000361 ;05000362/2009009 dated March 2, 2010, and
in the NRC's Chilling Effect Letter issued to San Onofre dated March 2, 2010.
b. Assessment
As documented in the NRC's March 2, 2010, Chilling Effect Letter, the NRC
concluded that some employees in multiple workgroups at San Onofre Nuclear
Generating Station have the perception that they are not free to raise safety
concerns using all available avenues, and that management has not been
effective in encouraging employees to use all available avenues without fear of
retaliation. This conclusion resulted from numerous observations, including:
(1) employees expressing difficulty or inability to use the corrective action
program; (2) a lack of knowledge or mistrust of the Nuclear Safety Concerns
Program (NSCP); (3) a substantiated case of a supervisor creating a chilled work
environment in his/her work group; and (4) a perceived fear of retaliation for
raising safety concerns. The licensee replied by letter dated March 31, 2010.
Further actions by the NRC are discussed in the March 2 letter.
.5 Specific Issues Identified During This Inspection
a. Inadequate Operability Determination for Turbine-Driven Auxiliarv Feedwater
Pump Steam Admission Valves
Intrnc!uctinn. A Green noncited violation of 10 CFR Part 50, Appendix 8,
Criterion V, "Instructions, Procedures, and Drawings," was identified involving the
failure to perform an adequate operability determination as required by
- 19 - Enclosure 2
procedure. Specifically, the licensee's operability evaluation for a degraded
turbine driven auxiliary feedwater pump steam admission valve failed to address
all the specified safety functions of the affected component as described in the
final safety analysis report and design basis documents.
Description. On April 7, 2010, inspectors noted what appeared to be
unlubricated valve stems on the Unit 3 steam admission valves to the turbine-
driven auxiliary feedwater pump, which are designated as 3HV8200 and
3HV8201. These valves are normally held open by spring pressure and are
normaiiy ciosed with nonsafety-related instrument air. The design basis requires
that for certain accident sequences in which the nonsafety-related instrument air
system is unavailable, these valves must be manually closed within 30 minutes.
The valves are manually closed by turning their respective hand wheel about
25 rotations. The valves are provided with manual gagging (locking) devices to
force the valves closed without instrument air and to lock the valves closed, such
that they won't inadvertently re-open. The inspectors were concerned that
increased friction from an unlubricated valve stem would make turning the hand
wheel against the spring force more difficult during manual operation.
The inspectors identified the issue to the licensee and noted that design basis
documents required the valves be manually closed and "gagged" or locked in the
following accident scenarios: (1) a high energy line break in the auxiliary
feedwater pump room; (2) a steam generator tube rupture; and (3) a fire in the
auxiliary feedwater pump room. The inspectors also discussed the design bases
with the licensee. As a result of the inspectors' concern, the licensee initiated
Nuclear Notification 200869281, and on April 8, 2010, commenced an operability
determination. On April 10, 2010, San Onofre personnel completed the
operability determination and concluded that the unlubricated gagging devices
were operable. However, the inspectors found that the operability determination
was inadequate.
The operability determination concluded that the valves were used in the postfire
safe shutdown analysis which was addressed by the notification, but did not
address the impact on technical specification operability. The operability
determination stated that the valves could be manually closed and gagged but it
provided no technical basis for the statement. Inspectors reviewed San Onofre
Procedure S0123-XV-52, "Functionality Assessments and Operability
Determinations," Revision 15. Step 6.5.1 required that the immediate operability
determination identify the specified safety function of the affected system,
structure or component. Step 6.5.1.3.2 stated that the operability determination
must identify the performance parameter used to determine operability.
Inspectors found that the April 10, 2010, operability determination was
inadequate because it failed to identify the performance parameters used to
determine operability; specifically, the design basis for these valves.
Inadequacies included:
- 20 - Enclosure 2
i. The determination incorrectly stated, "Manual closure of 2HV8200 and
2HV8201 is not a credited safety function of these valves for emergency
operating events." This was contrary to Final Safety Analysis Report
Table 10.4-7, which described use of the valves during a high energy line
break without the use of instrument air. Final Safety Analysis Report
Section 15.6.3, described valve closure and release termination within
30 minutes of a steam generator tube rupture. Design Basis Document
SD-S023-780 also described the manual action to gag the valves closed.
ii. The determination incorrectly assumed that nonsafety-related instrument
air would always be available to stroke the valves closed from the control
room.
iii. The determination incorrectly assumed the valves are closed against zero
opposing force to prevent them from re-opening on an auxiliary feedwater
start signai.
iv. The determination cited a procedure that only stated to close the valve,
but the procedure did not state during which events the valves should be
closed. This instruction was apparently only used during maintenance of
the valves or terry turbine.
Based on interviews, operations and engineering were not specifically aware that
the valves needed to be manually gagged closed even though the inspectors
discussed the design basis with other licensee personnel. After additional
inspectors' questioning and re-review of the design basis and the gag operation
with San Onofre Nuclear Generating Station personnel, San Onofre Nuclear
Generating Station re-performed the operability determination under Nuclear
Notification 20088760 and completed it on April 21, 2010. Based on the second
operability determination, which included contacting the vendor, licensee
personnel informed the inspectors on April 22, 2010, that the valves were
declared inoperable and that the licensee was taking interim compensatory
corrective actions. Thus, the licensee's initial operability determination on April
10, 2010, had been inadequate even after the inspectors had discussed the
design basis with licensee personnel prior to the licensee's evaluation.
The licensee documented this violation in Nuclear Notification 20088760, and its
short term corrective actions included required training and the staging of a
leveraging device in the vicinity of the valves to assist operators in closing and/or
gagging the valves, as required.
Analysis. Inspectors found that the failure to perform an adequate operability
determination and to identify the degraded condition was a performance
deficiency. The deficiency was more than minor because it impacted the
Mitigating Systems Cornerstones and its objective to ensure the availability and
reliability of equipment that responds to initiating events. Using Inspection
Manual Chapter 0609, the issue screened to Phase 3 because it represented a
loss of safety function for approximately two weeks and it screened to greater
- 21 - Enclosure 2
than Green using the Phase 2 pre-solved worksheet. The inspectors determined
that the finding was Green based on the bounding analyses discussed in the
analysis section of 40A2.5b. Specifically, this vulnerability existed for
approximately two weeks (the time between the inadequate evaluation and the
correct evaluation), which is considerably less than the one year vulnerability
discussed in the analysis section of 40A2.5b. The inspectors determined that the
cause of the finding has a crosscutting aspect in the area of human performance
associated with decision making. Specifically, San Onofre Nuclear Generating
Station utilized unsupportable assumptions in its evaluation that were not
consistent with the Final Safety Analysis Report or the valve vendor manual.
[H.i.b]
Enforcement. Title 10 CFR Part 50, Appendix B, Criterion V, requires, in part,
that activities affecting quality be prescribed by procedures and be accomplished
in accordance with those procedures. San Onofre Procedure S0123-XV-52,
"Functionality Assessments and Operability Determinations," Revision 15,
Step 6.5.1 requires, in part, that the immediate operability determination identify
the specified safety function of the affected system, structure or component. San
Onofre Procedure S0123-XV-52 Step 6.5.3.1 requires, in part, that the
operability determination must identify the performance parameter used to
determine operability. Contrary to the above, from April 10 to April 22, 2010, San
Onofre Nuclear Generating Station performed an inadequate operability
determination required by San Onofre Procedure S0123-XV-52. Specifically,
San Onofre Nuclear Generating Station failed to identify the design basis
parameters for the steam admission valves for the turbine-driven auxiliary
feedwater pumps as described in the Final Safety Analysis Report and design
basis documents. In accordance with the NRC's Enforcement Policy, because
the violation was of very low safety significance, and was entered into the
corrective action program as Nuclear Notification 20088760, this violation is
being treated as noncited violation, consistent with the NRC Enforcement
Policy VI.A: NCV 05000361/2010006-01, "Inadequate Operability Determination
for turbine-driven auxiliary feedwater pump steam admission valves."
b. Failure to Translate Design Basis Information for Closure of Turbine-Driven
Auxiliary Steam Admission Valves
Introduction. On April 7, 2010, inspectors identified a Green noncited violation of
10 CFR Part 50, Appendix B, Criterion III, "Design Control," for steam admission
valves to the turbine-driven auxiliary feedwater pumps that could not be closed
within 30 minutes per the design basis.
Description. As discussed in the previous section, on April 7, 2010, inspectors
found apparently unlubricated valve stems on the Unit 3 steam admission valves
to the turbine-driven auxiliary feedwater pump, which are designated as
3HV8200 and 3HV8201. The inspectors identified a Green noncited violation
related to the inadeauate ooerabilitv determination that lir:en!=:eA OAr!=:()nnAI
I I J ~"" -- ---,-_._-"""--
performed on April 10, 2010.
- 22- Enclosure 2
On April 22, 2010, after performing a second operability determination, the
licensee representatives informed the inspectors that they had contacted the
vendor and found the valves were inoperable because a person would not be
able to manually close and gag the valves under ideal lubrication conditions. The
licensee's standard was that a person would be able to apply a force of up to
100 pounds. However, ideally lubricated valves would require 132 pounds of
force on the hand wheel, and the hand wheel would have to be turned
approximately 25 times in order to close and gag the valve. The increased
friction from lack of lubrication and disc-in-seat forces could exceed 200 pounds
of force on the hand wheel. As a result of this information, the licensee began
taking corrective actions by posting leveraging devices for operators to use in the
event manual closure of the valves was needed. On April 22, 2010, required
reading on valve operation was implemented to train all licensed and non licensed
operators on the valve operation.
For postfire safe shutdown procedures, damage to the 2(3)HY8200 and
2(3)HY8201 solenoid valves associated circuit cables routed to auxiliary relay
cabinet L071 could cause a loss of the ability to close the air-
operated 2(3)HV8200 and 2(3)HV8201 from the control room. San Onofre
Procedure S023-13-21 "Fire" provided instructions for operators to mitigate the
effects of fire damage to safe shutdown equipment in plant areas. The steam
admission valves are required to be closed within 30 minutes of a fire by the
postfire safe shutdown analysis. Based on the April 22, 2010, operability
determination, the licensee added steps to San Onofre Procedure S023-13-21 to
use a crescent wrench and leverage device. These tools were locally staged to
back off the hand wheel stem nut and then use the leverage device on the hand
wheel to force the gag to shut the valve against its opening spring. The
inspectors concluded that prior to April 22, 2010; manual actions could not have
been taken within the 30-minute period because of the lack of tools and the
operator's lack of familiarity with San Onofre Procedure S023-13-21 which
identified key manual actions needed.
The inspectors noted the following prior opportunities the licensee had to identify
this deficiency:
i. In 2004, Action Request 040700869 erroneously stated that the safety
function to close the 8200 valves was not required in the design basis
document.
ii. In 2005, Action Request 050700659 was written to request that design
engineering delete the manual closure of the valves from the ASME O&M
Code in-service testing based on an incorrect evaluation which stated that
the valves were not part of the accident analysis. The action request also
erroneously stated the valves would not impact other programs such as
fire protection.
iii. On November 19, 2009, the licensee failed to identify this issue during its
review of Operating Experience 30062, "Locally Operated Time Critical
- 23 - Enclosure 2
Valves May be Difficult to Operate Under Accident Conditions" which
dealt with the possibility that the expected differential pressure across
locally operated valves must be considered when evaluating the ability of
operators to change valve positions in accident conditions. The operating
experience stated that this evaluation should be similar to the review
required by Generic Letter 89-10 for valves locally operated under high
differential pressure.
On April 22, 2010, San Onofre Nuclear Generating Station's corrective
action was to post leveraging devices and to schedule lubrication of the
valves for August 2010. The NRC considered immediate lubrication to be
an important corrective action that the licensee had not adequately
addressed while the inspectors were onsite. In addition, because a dry
lubricant was used on the valve (in accordance with the manufacturer's
recommendations) and the valve was exposed to the weather, the
inspectors also questioned the 10 year frequency for lubrication. Based
on further questioning from the inspectors, on May 25, 2010, the licensee
wrote Nuclear Notification 200937258 to address the inspectors' concern
about the adequacy of lubrication of the valve stem as well as the
frequency of lubrication.
The inspectors concluded that prior to April 22, 2010; the 8200 series
valves had been inoperable because the licensee had not translated the
design basis into procedures. The licensee did not translate into its
procedures the design bases requirements to manually close the valves
within 30 minutes of the required accident scenarios and did not consider
the force needed to manually close and gag the valves. Inspectors also
found that the licensee was not meeting Licensee Controlled Specification
Surveillance Requirement 3.7.113.1.12 to manually stroke the valve every
24 months to ensure compliance with the fire protection program. In
addition, simulated operator actions during a walkthrough of San Onofre
Procedure S023-13-21, "Fire," could not be performed in the time
specified in engineering calculations, nor were all appropriate steps
specified. The licensee was also evaluating necessary actions for a
permanent corrective action to this issue.
Analysis. The inspectors found that the failure to translate design basis
information regarding the 2(3)HV8200 and 2(3)HV8201 valves into procedures
was a performance deficiency. The deficiency was more than minor because it
impacted the Mitigating Systems Cornerstones and its objective to ensure the
availability and reliability of equipment that responds to initiating events. The
inspectors screened the issue to more than one cornerstone due to its affect on
early release (steam generator tube rupture), fire protection, and mitigating
systems (high energy line break).
Appendix H, because the finding represents an actual open pathway in the
physical integrity of reactor containment during a steam generator tube rupture
- 24- Enclosure 2
accident scenario. In Inspection Manual Chapter 0609 Appendix H, Step 4.1, the
inspectors screened this as a Type B finding (affects large early release fraction
but not core damage frequency) needing a Phase 2 evaluation. Inspectors used
Table 4.1 and found that the finding involved a large release path from the
reactor coolant system to the environment. Using Table 6.2, inspectors screened
the Phase 2 to greater than Green because the condition existed for greater than
one year and the volume of steam released would be larger than the free volume
of containment.
The inspectors screened the issue to Phase 2 for at-power inspection findings
using Inspection Manual Chapter 0609 because the turbine-driven auxiliary
feedwater valves could not be closed within 30 minutes after a high energy line
break to prevent failure of the two remaining auxiliary feedwater pumps. This
represented the potential loss of a safety function.
Inspectors screened the issue to Phase 2 for Appendix F of Inspection Manual
Chapter 0609 because the valves could not be closed for a fire in the auxiliary
feedwater room.
The senior reactor analyst performed a Phase 3 analysis to determine the risk
significance of the degraded turbine-driven auxiliary feedwater steam admission
valve. The analysis considered the effects of a high energy line break in the
pump room, a steam generator tube rupture, and fires in the pump room and
auxiliary feedwater pipe tunnel. The inspectors determined that the combined
significance of these scenarios was a delta-core damage frequency of 1.3E-7/yr
and a delta-large early release frequency of 4.2E-B/yr. Therefore, the violation
was determined to be of very low significance.
The violation has a crosscutting aspect in the area of problem identification and
resolution associated with the corrective action program. Specifically, the
licensee had multiple opportunities to evaluate this problem but failed to do so.
[P.1 (a)]
Enforcement. Title 10 CFR Part 50, Appendix B, Criterion III, "Design Control"
requires, in part, that the design basis for systems, structures, and components
be correctly translated into specifications, drawings, and procedures. Contrary to
the above, prior to April 22, 2010, the licensee failed to translate the following
design basis information into procedures: (1) the requirements to manually close
and gag within 30 minutes the steam admission valves for the turbine driven
auxiliary feedwater pump in response to high energy line breaks or steam
generator tube rupture; and (2) the failure to determine the forces required to
manually close the valves. Because the violation was of very low safety
significance (Green), and was entered into the corrective action program as
Nuclear Notification 200B70B61 this violation is being treated as a non cited
violation, consistent with the NRC Enforcement Policy section VI.A:
Nr.\I _"______ .. __ . nnnR-n?
... _ .. nt:;nnn~R1/?n1 ______ , "1=::Iill
. _"-"_"_I rIO> tn tr::lnc::l::ItlO> rilO>c::inn
... _ .. _._ .. ___ ..... ;::, .. h::lc::ic:: inff"lr""",tinn ff"lr
___ ._ ""_",,_,,,'_'1 ._,
turbine-driven auxiliary feedwater pump Steam Admission Valves."
- 25 - Enclosure 2
c. Lack of Preventive Maintenance Results in Valve Failure of Condensate Storage
Tank
Introduction. A Green noncited violation of Technical Specification 3.7.6 was
identified which requires, in part, that condensate storage tank T-120 be
operable. Specifically, the tank isolation valve 2HV5715 had been inoperable for
a period greater than the allowed outage time of seven days while Unit 2 was in
Modes 1,2, and 3. The valve isolates nonseismic piping from the tank and is
required to be manually closed within 90 minutes following a seismic event. The
licensee had not performed preventive maintenance on the valves resulting in the
valves failing to close during an in-service test on January 26, 2010.
Description. On January 26, 2010, the hand wheel on the Unit 2 condensate
storage tank manual valve 2HV5715 broke while licensee personnel attempted to
perform an in-service test. This valve isolates nonseismic from seismic piping
supporting condensate storage tank T -120. The design basis for the valve is to
be closed within 90 minutes of an operating basis earthquake in order to
preserve the water inventory in condensate storage tank T -120. The water
inventory in that tank is needed to provide a water source for the auxiliary
feedwater pumps to remove heat from the reactor. A line break in the
nonseismic portion of the condensate system could drain tank T-120 of its water
inventory, which is required to support plant cooldown from Mode 1 to Mode 5.
Final Safety Analysis Report 10.4.9.2.3.4, "Emergency Operation," states that
tank T-121 is the primary source of auxiliary feedwater condensate with tank
T-120 required for backup.
The licensee employee performing the in-service test attempted to cycle the
valve but was not able to rotate the hand wheel. So, in accordance with
procedures, the licensee contacted the control room and obtained permission to
use a leveraging device to turn the valve. When the licensee employee applied
the leveraging device to the hand wheel, it sheared the pin connecting the hand
wheel to the valve manual actuator stem. The valve was repaired the next day.
During the subsequent diagnostics, the actuator stem was found to be heavily
rusted and without lubrication. The licensee employee determined that the valve
had been inadvertently removed from the preventive maintenance program
several years prior.
At the time the valve failed, Unit 2 was in an outage and the valve was not
required to be operable. Nuclear Notification 200765235 stated that the valve
was inoperable and could not fulfill its safety function to preserve the water
inventory in condensate storage tank T-120. However, in determining past
operability, emails were attached to Nuclear Notification 200765235 that stated
that corrective maintenance could be performed to open the valve; specifically,
that a mechanic could have been called upon to disassemble the valve actuator
and manually close the valve. Thus, the licensee concluded the valve was
operable prior to January' 26, 2010, and that the failure vvas not reportable.
- 26- Enclosure 2
The inspectors challenged the licensee in its determination that the valve had
been operable prior to the hand wheel breaking and that the failure was not
reportable to the NRC. The inspectors' position was that it was inappropriate to
consider corrective maintenance in the reportability determinations. The licensee
originally maintained its position asserting that the valve was operable and that
the issue was not reportable, again basing its decision on corrective
maintenance. After the inspectors referred the licensee to appropriate NRC
guidance in NUREG-1022, the licensee determined that the broken valve had not
been operable prior to the event and that the event was reportable.
The inspectors also challenged the use of leveraging devices on isolation
valve 2HV5715 as well as other manually-operated valves. Several other
manual valve hand wheels in the area had markings indicative of extensive use
of leveraging devices. Inspectors were informed that nuclear notifications were
not being written each time leveraging devices were used on manual valves,
which was required in accordance with Procedure S0123-0-A6, "Routine
Equipment Operations," Revision 8. San Onofre Nuclear Generating Station is
re-examining its in-service testing periodicity and preventive maintenance
practices in Nuclear Notification 200952866. The inspectors also noted that a
nuclear notification had not been written, as required, when the leveraging device
was used on isolation valve 2HV5715 on January 26, 2010.
In order to determine whether the licensee could reasonably close the valve
within 90 minutes of an operating basis earthquake, inspectors performed a
walkdown of the actions licensee staff would take following a seismic event. The
inspectors interviewed licensee staff who had not been informed of the
inspectors question prior to the walk down. The inspectors proposed a scenario
to the shift manager that the plant experienced an operating basis earthquake
and assessed the time it would have taken before she/he would have contacted
the maintenance general foreman to fix isolation valve 2HV5715. The inspectors
then interviewed the maintenance general foreman in order to understand what
she/he would do for this situation. The inspectors also interviewed three
mechanics and gave them the scenario conditions. Reviewing the time line
starting 90 minutes after the earthquake, the inspectors determined the total time
to close the valve was approximately 105 minutes. Based on this data, the
inspectors raised the concern to the licensee that its staff would be unable to
meet its design basis for cloSing this valve following an operating basis
Subsequent to the inspection, the licensee ran this scenario in the simulator
(without announcing it to the crew in advance). The results were that it took the
crew an estimated 134 minutes to have a mechanic manually turn the valve.
Therefore, the licensee determined that its staff could not complete manually
closing the valve within the 90 minute time frame required by the design basis,
and began taking actions to review its licensing basis and its procedures, and
-_ _-_.--
~nnrllld4'>rl
.. --_ ..,_ .. _... _......
~rlrlitinn~1 tr~inin('f ;:, ..
tA itc cbff
.~ ~ ~,~".
- 27- Enclosure 2
The isolation valve was last stroked in March 2008, and the licensee could not
determine the exact date the valve became inoperable. Given the failure mode,
the inspectors concluded that the valve had been inoperable for greater than
seven days when the licensee was last in Mode 1,2 or 3, when the valve was
required to be operable.
The licensee documented this deficiency in Nuclear Notification 200765235, and
repaired the valve and placed it into the preventive maintenance program.
Analysis. The inspectors determined that the failure to perform preventive
maintenance, including lubricating the valve actuator's components necessary to
manually close valve 2HV5715, was a performance deficiency. This issue is
more than minor because it impacted the Mitigating Systems Cornerstone
objective to ensure the availability, reliability, and capability of systems that
respond to initiating events to prevent undesirable consequences (Le., core
damage). Specifically, the broken valve impacted the protection against external
events attribute for seismic protection. The inspectors used Inspection Manual
Chapter 0609, "Initial Screening and Characterization of Findings," to analyze the
significance of this finding. The inspectors screened the finding to Phase 2
because the condensate storage tank T -120 was inoperable for a significant
period greater than that allowed in technical specifications (using the time over
two methodologies, the tank was inoperable for approximately a year). This
screened the finding out of Phase 2 to Phase 3 because the closest surrogate for
this deficiency was failure of one of the auxiliary feedwater pumps for one year
which screened to red. A Phase 3 analysis was performed by the senior reactor
analyst. Using San Onofre Nuclear Generating Station's seismic information and
fragility data associated with the piping that could not be isolated because of the
failed condition of valve 2HV5715, the frequency of a seismic event that would
cause a pipe break and drain tank T-120 was estimated to be 2.7E-5/yr. Given a
seismic event that causes a loss of offsite power (nearly 100 percent of seismic
events that rupture the piping would also cause a loss of offsite power), operators
are compelled by procedure to cool down and initiate shutdown cooling. The
amount of water that is protected with valve 2HV5715 failed open, which includes
inventory from tank T-121 and water below the break line in tank T-120, given
that operators close the working manual isolation valve within 30 minutes is more
than what is needed to get to shutdown cooling in natural circulation with only
one of two steam generator atmospheric dump valves in operation, even if there
is a 4-hour hold time at hot standby. The analyst estimated that the failure
probability of operators to cool down and initiate shutdown cooling is 1.0E-2.
Therefore, assuming a zero base case, the estimated delta-core damage
frequency of the finding is 2.7E-5/yr. (1.0E)=2.7E-7/yr.
The inspectors also determined that the cause of the finding has a crosscutting
aspect in the area of human performance associated with resources in that San
Onofre Nuclear Generating Station did not ensure that equipment was available
and adequate to assure nuclear safety by minimization of fong-standing
equipment issues in that the valve was not being maintained through a
preventive maintenance program. H.2(a)
- 28- Enclosure 2
Enforcement. Technical Specification 3.7.6 requires, in part, that tank T-120 to
be operable. Valve 2HV5715 is required for tank operability because it must be
closed after an earthquake to preserve tank inventory. Condition C provides for
a completion time of seven days. Contrary to the above, prior to
January 26, 2010, valve 2HV5715 could not be closed for greater than its
completion time of seven days. The valve was failed in the open position.
Because this violation was of very low safety significance and was entered into
the licensee's corrective action program under Nuclear Notifications 200765235.
This violation is being treated as a noncited violation, consistent with Section
VI.A of the NRC Enforcement Policy: NCV 05000361/2010006-03, "Lack of
preventive maintenance results in valve failure and inoperable condensate
storage tank."
d. Failure to Submit a Licensee Event Report Within 60 Days
Introduction. On April 22, 2010, inspectors identified a Severity Level IV violation
of 10 CFR 50.73, "Licensee Event Report System," in which the licensee failed to
submit a licensee event report within 60 days following failure of condensate
storage tank isolation valve 2HV5715.
Description. As previously discussed, on January 26, 2010, condensate storage
tank T-120 manual isolation valve 2HV5715 failed its in-service stroke test after a
leveraging device was used to turn the hand wheel, at which time it sheared off.
The valve operator stem was heavily rusted and did not move resulting in the
failure. This valve must be closed per San Onofre Procedure AOI S023-13-3,
"Earthquake," Revision 13, Attachment 1, Step 2.3.3 within 90 minutes of an
operating basis earthquake in order to prevent the loss of water inventory from
condensate storage tank T-120 from a line break in the nonseismic portion of the
condensate system.
In determining reportability, an email was attached to Nuclear
Notification 200765235 that stated a mechanic could disassemble the valve
actuator and manually turn the valve. Thus, the licensee concluded the valve
was operable prior to January 26, 2010, and that the failure was not reportable.
The inspectors challenged the use of corrective maintenance to determine that
the valve was previously operable. Originally, licensee representatives informed
the inspectors that its mechanics could pry the position indicator off and close the
valve against the frozen operator using a wrench. The inspectors questioned the
licensee on this position because this action would require turning the stem
against the frozen operator thus potentially damaging the operator.
Following discussions licensee personnel provided a more reasonable position
by stating that the valve operator could be unbolted and removed, and the
butterfly disc stem could then be closed vv;th a ~vvrench. The inspectors
determined this method was plausible, but still required corrective maintenance.
The inspectors noted that the use of corrective maintenance did not meet
- 29- Enclosure 2
NUREG 1022, "Events Reporting Guidelines 10 CFR 50.72 and 50.73," guidance
which states that operability must be ensured and that corrective maintenance is
not an appropriate basis for operability. Tanks T-121 and T-120 are required to
be operable per Technical Specification 3.7.6 to supply 24 hours2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br /> of
demineralized water to the auxiliary feedwater system. Without the ability to
close valve 2HV5715, tank T-120 was not operable. Because the tank was not
operable, it met the conditions of 10 CFR 50.73(a)(2)(v) as an event or condition
that could have prevented the fulfillment of the safety function of structures or
systems that are needed to shutdown the reactor and maintain it in a safe
shutdown condition, remove residual heat, and mitigate the consequences of an
accident. As such, the event was reportable under 10 CFR 50.73(a)(1).
The licensee maintained its position that based on corrective maintenance the
condition was not reportable until the inspectors pointed out the section in
NUREG 1022, at which time, the licensee determined the condition was
reportable.
The licensee documented this violation in Nuclear Notification 200888616, and
the licensee took actions to issue a licensee event report.
Analysis. The failure to submit a licensee event report as required was a
performance deficiency. The inspectors reviewed this issue in accordance with
Inspection Manual Chapter 0612 and the NRC Enforcement Policy. The
inspectors determined that traditional enforcement was applicable to this issue
because the NRC's regulatory process was impacted. Specifically, the NRC
relies on the licensee to identify and report conditions or events meeting the
criteria specified in regulations in order for the NRC to perform its regulatory
function, and when this is not done, the regulatory function is impacted. The
inspectors determined that this finding was not suitable for evaluation using the
significance determination process, and as such, was evaluated in accordance
with the NRC Enforcement Policy. The finding was reviewed by NRC
management, and the significance of the violation was classified at Severity
Level IV and treated as a noncited violation consistent with the NRC
Enforcement Policy. This finding was determined to have a crosscutting aspect
in the area of human performance in the decision-making component in that the
licensee did not make safety-significant decision using a systematic process,
especially when faced with uncertainty. [H.1 (a)]
Enforcement. Title 10 CFR 50.73(a)(1) requires, in part, that licensees shall
submit a licensee event report for any event of the type described in this
paragraph within 60 days after the discovery of the event. Title 10 CFR 50.73(a)(2)(v) identifies a reportable event as, in part, an event or condition
that could have prevented the fu!fillment of the safety function of structures or
systems that are needed to shutdown the reactor and maintain it in a safe
shutdown condition, remove residual heat, or mitigate the consequences of an
accident. Conti8iY to the above, prior to March 27, 20-10, San Onofre Nuciear
Generating Station failed to submit a licensee event report within 60 days for the
failure of valve 2HV5715 which could have prevented the fulfillment of the safety
- 30 - Enclosure 2
functions and was a condition prohibited by Technical Specification 3.7.6.
Technical Specification 3.7.6 requires that tank T-120 be operable in Modes 1,2,
and 3 in order to supply 24 hours2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br /> of demineralized water to the auxiliary
feedwater system. Without the ability to close valve 2HV5715, tank T-120 was
not operable. As a result, valve 2HV5715 is a component that is needed to
remove residual heat and mitigate the consequences of an accident. Ail three
trains of auxiliary feedwater could not perform their design function because
there would be insufficient condensate inventory after an earthquake. In
accordance with the NRC's Enforcement Policy, the finding was reviewed by
NRC management and because the violation was of very low safety significance,
and was entered into the corrective action program as Nuclear
Notification 200888616, this violation is being treated as a Severity Level IV
noncited violation, consistent with the NRC Enforcement Policy:
NCV 05000362/2010006-04, "Failure to report conditions that could have
prevented fulfillment of a safety function."
e. Failure by Control Room Operators to Follow Conduct of Operations Procedure
Introduction. The inspectors identified a Green noncited violation of Technical
Specification 5.5.1.1.a, "Scope" for control room operators' failure to adhere to
conduct of operations procedural requirements.
Description. On April 7, 2010, inspectors performed a detailed observation of
control room activities for Units 2 and 3 at San Onofre Nuclear Generation
Station. Unit 2 was performing a startup from a refueling outage and Unit 3 was
operating at 50 percent rated thermal power. The inspectors observed a shift
turnover from night shift to day shift and attended all turnover meetings. The
inspectors watched Unit 2 operators perform startup activities that included a
dilution to within 200 parts per million estimated critical boron concentration.
Additionally, the inspectors observed Unit 2 operators withdraw control rods from
shutdown bank '8' and partial length control rods and viewed Unit 3 operators
perform a dilution with primary water to maintain reactor power at 50 percent.
The inspectors also monitored various routine control room activities such as
acknowledging alarms, refilling the Unit 2 closed cooling water surge tank, and
controlling pressure in the Unit 2 steam generators. The inspectors observed the
control room operators interacting with other departments such as maintenance,
health physics, engineering, and chemistry.
The inspectors compared actions in the control room with San Onofre
Procedure S0123-0-A 1, "Conduct of Operations," Revision 26, and observed
numerous deficiencies. When Unit 2 alarms were received in the control room,
the inspectors observed the following:
.. Place keeping was not implemented on any unexpected alarms received
in the Unit 2 control room per Section 6.4.3.3 and Guideline 5 of
Section 6.'1.3. ,i\!arm response procedures 'vVere refeired to by the reactor
operators and read but no place keeping occurred.
- 31 - Enclosure 2
The operator announcing the alarm did not always report it to the control
room supervisor as required by Section 6.4.3.3.
When an alarm annunciated, or an alarm condition clears, all
conversations in the control room did not stop until the alarm had been
acknowledged or reset, as required by Section 6.4.3.1.
The following alarms were received in the Unit 2 control room during the
inspectors' observations:
- Reactor Coolant Pump 4 seal pressure HIILO
- Generator potential transformer fuse blown
- Channel 4 startup rate high
- Control Element Assembly Group Deviation (the senior reactor operator
in charge of reactivity instructed the reactor operator to mark steps in
alarm response procedure)
- Other alarms were received during the observation but marking of alarm
response procedures were not normally performed
The inspectors observed a control room supervisor conduct a pre-job briefing at
the beginning of shift. During the briefing, numerous questions were asked of the
control room supervisor by on-shift operators on how the supervisor wanted the
operators to control steam generator pressure. The questions or answers were
not acknowledged using three-way communications to ensure full understanding
took place as required by Section 6.6.4.7 of San Onofre Procedure S0123-0-A 1.
Throughout the inspector's observations, additional examples of missed three
way communications were observed.
In addition, when an operator was performing the filling of the closed cooling
water surge tank, the operator did not verify written instruction prerequisites
before using the procedure as required by San Onofre Procedure S0123-0-A 1.
During the pre-job brief for pulling shutdown group '8' and partial length control
rod groups, the reactivity senior reactor operator did not verbalize the five
summarize, anticipate, foresee, evaluate and review questions as part of the
brief. Only the operating experience question was discussed. Other items not
reviewed as required included: (1) four questions dealing with critical steps,
(2) error-likely situations, (3) how bad can it get, and (4) what defenses are in
place and are they adequate. These were required by San Onofre
Procedure S0123-0-A 1.
During a Unit 3 reactivity change, the inspectors obsented the contre! room
supervisor performing oversight of the activity take a phone call while the
evolution was in progress. The control room supervisor first engaged in
- 32 - Enclosure 2
conversation before informing the person that he would have to call them back
later. This was contrary to Section 6.5.2, Step 6.5.2.8 which states, "All reactivity
changes in the control room require direct senior reactor operator oversight.
Senior reactor operator oversight requires the senior reactor operator be
cognitive of, present for, and approve the reactivity change."
The licensee documented these procedural deficiencies in Nuclear
Notification 200871332 and its short term corrective actions included operation's
management reviewing the observations with the inspectors and then
establishing a recovery plan to improve operator performance.
Analysis. The failure of control room operators to adhere to conduct of
operations procedural requirements is a performance deficiency. The finding
was more than minor because, uncorrected, the failure to follow these procedural
requirements could lead to a significant safety concern due to the potential of
operators making errors while operating safety-related systems. Using the
Inspection Manual Chapter 0609, "Significance Determination Process," Phase 1
Worksheet, the inspectors determined the finding had a very low safety
significance because the finding did not result in a loss of system safety function,
an actual loss of safety function of a single train for greater than its technical
specification allowed outage time, or screen as potentially risk significant due to a
seismic, flooding, or severe weather initiating event. As a result, the issue was of
very low safety significance (Green). The finding has a crosscutting aspect in the
area of human performance associated with the work practices because the
licensee did not ensure supervisory and management oversight of work activities.
Enforcement. Technical Specification 5.5.1.1.a, "Scope" requires, in part, that
written procedures be established, implemented, and maintained covering the
activities specified in Appendix A, "Typical Procedures for Pressurized Water
Reactors and Boiling Water Reactors," of Regulatory Guide 1.33, "Quality
Assurance Program Requirements (Operations)," dated February 1978.
Specifically Regulatory Guide 1.33 Section 1.d "Procedure Adherence," requires
operators to follow their procedures. San Onofre Procedure S0123-0-A 1,
"Conduct of Operations," Revision 26, Sections 6.4.3.1,6.6.4, and 6.5.2 require,
in part, the following: implement alarm response procedure place keeping;
announce alarms to the control room supervisor; stop conversations in the
control room when an alarm annunciates; perform 3-way communications during
pre-job briefing; review the five questions, summarize, anticipate, foresee,
evaluate and review, during a pre-job brief; and review the prerequisites prior to
each use of a procedure; and requires that a senior reactor operator remain
cognitive of the reactivity change evolution.
Contrary to this, on April 7, 2010, control room operators failed to follow San
Onofre Procedure S0123-0-A 1, "Conduct of Operations," Revision 26,
requirements in numerous instances including fai(ui6s to: implenlent c3;arrn
response procedure place keeping; announce alarms to the control room
supervisor; stop conversations in the control room when an alarm annunciated;
- 33- Enclosure 2
perform 3-way communications during a pre-job briefing; review the five
questions, summarize, anticipate, foresee, evaluate and review, during a pre-job
brief; review the prerequisites prior to each use of a procedure; and remain
cognitive of the reactivity change evolution by a control room supervisor.
Because this finding is of very low safety significance and has been entered into
the licensee's corrective action program as Nuclear Notification 200871332, this
violation is being treated as a noncited violation, consistent with Section VI.A of
the NRC Enforcement Policy: 05000362/2010006-05; "Control Room Operators'
Failure to Adhere to Conduct of Operations Procedural Requirements."
f. Failure to Provide Adequate Procedures for Boron Dilution Activities
Introduction. The inspectors reviewed a self-revealing Green noncited violation
of Technical Specification 5.5.1.1.a, "Scope" for the failure of boron saturation
procedure to have adequate direction to prevent an unplanned power transient
Description. On December 25, 2009, the chemistry department requested
operators to perform a reactor coolant system delithiation using ion
exchanger 3ME074 for Unit 3. This ion exchanger was not boron saturated so
the evolution would require diverting to radiological waste while performing a
manual blended makeup.
The operating crew performed a pre-job brief prior to commencing the evolution
where they discussed the procedures, expected plant response, and
compensatory actions for power increase. The crew reviewed the logs and found
the last blended makeup to be light in boron concentration which could result in a
slight power increase. The crew was aware of Unit 3 having a feedwater heater
leak that was identified on the previous shift. The leak was scheduled to be
repaired later that day and would require a slight down power to remove the
feedwater heater from service. The crew was concerned with exceeding the
licensed power limit and therefore set an upper power limit of plus 0.5 percent.
Due to a down power scheduled later that day the crew did not set a lower power
limit nor did they believe it was required.
San Onofre Procedure S023-3-2.4,controlling the evolution, "RCS Purification
and De-borating Ion Exchanger Operation," Revision 21, provided a guideline to
stop the evolution at 10 minutes for the crew to monitor plant response to
determine if it was as expected. The crew believed that the blended makeup
would be light and plant response was known.
The crew commenced the evolution to boron-saturate the ion
exchanger 3ME074. However, the crew did not stop the evolution at 10 minutes
because they did not believe it to be a requirement. As a result, the crew over-
borated the reactor and caused an unplanned down power of 0.74 percent.
Operation management conducted an investigation of the event and initiated a
Nuclear Notification 200721702. The crew members involved in the event were
coached about expected performance during reactivity manipulations.
- 34- Enclosure 2
Operations issued a priority 2 notification to the operations department describing
the event and management's expectations for reactivity activities. San Onofre
Procedure S023-3-2.4 was revised to place procedure requirements in place to
prevent events such as this from occurring again.
Analysis. The failure to have adequate procedural direction to control plant
power changes is a performance deficiency. The finding was more than minor
because it was associated with the initiating events cornerstone attribute of
human performance, and it affected the associated cornerstone objective to limit
the likelihood of those events that upset piant stabiiity and that challenge critical
safety functions during shutdown, as well as during power operations. Using the
Inspection Manual Chapter 0609, "Significance Determination Process," Phase 1
Worksheet, the inspectors concluded that the transient initiator did not contribute
to both the likelihood of a reactor trip and to the likelihood that mitigation
equipment or functions would not be available. As a result, the issue was of very
low safety significance (Green). The finding has a crosscutting aspect in the
area of human performance associated with the work practices because licensee
supervisory personnel did not ensure activities associated with reactivity control
were performed in a controlled manner such that nuclear safety was assured.
Enforcement. Technical Specification 5.5.1.1.a requires, in part, that written
procedures be established, implemented, and maintained covering the activities
specified in Appendix A, "Typical Procedures for Pressurized Water Reactors
and Boiling Water Reactors," of Regulatory Guide 1.33, "Quality Assurance
Program Requirements (Operations)," dated February 1978. Specifically
Regulatory Guide 1.33 section 3.n "Chemical and Volume Control System," shall
have instructions for controlling power changes. Contrary to this, as of
December 25, 2009, San Onofre Procedure S023-3-2.4, "RCS Purification and
De-borating Ion Exchanger Operation," Revision 21, was inadequate in that it
only provided guidelines, not requirements, to control the borating of ion
exchangers. As a result, an operations crew performed the evolution and did not
adhere to guidelines (because they were not required) and over-borated the
reactor, which in turn caused an unplanned down power transient of
0.74 percent. Because this finding is of very low safety significance and has
been entered into the licensee's corrective action program as Nuclear
Notification 200721702, this violation is being treated as a noncited violation,
consistent with Section VI.A of the NRC Enforcement Policy: 05000362/2010006-
06, "Failure to provide adequate procedure for boron dilution activities."
g. Inadequate Procedures for Radiation Monitoring of Component Cooling Water
Introduction. The inspectors identified a noncited violation of Technical
Specification 5.5.1.1.a, "Scope," involving the failure to establish procedures for
component cooling water system alignments such that leakage of radionuclides
to the environment would be monitored during all operational alignments of
component cooling water. Specifically, radiation monitors could be aligned to
only one train of component cooling water at a time and the licensees'
- 35 - Enclosure 2
procedures had no provision for monitoring the second train when both trains
were in-service.
Description. On April 5, 2010, inspectors walked down the component cooling
water system during which San Onofre Nuclear Generating Station personnel
discussed heat exchanger tube leakage in the Unit 2 train B heat exchanger
below the operability limit of 18 gallons per minute. The surge tank level was
decreasing and component cooling water inventory was being lost to the salt
water system. The salt water system is the ultimate heat sink for safety
equipment and it operates at a lower pressure than component cooling water.
Inspectors reviewed the current operability evaluation contained in Nuclear
Notification 200823240 as well as system piping and instrumentation drawings,
and learned that radiation monitor 7819 (Unit 2 and Unit 3) can only be aligned to
one train of component cooling water at a time. That is because it is connected
to the non-critical loop. Noncritical loop loads include the radioactive waste
building and containment loads such as control rod drive mechanism cooling and
reactor coolant pump cooling. Leakage from the shutdown cooling heat
exchangers would be captured by the component cooling water system but the
radioactivity may not be measured depending on which train of component
cooling water is aligned to radiation monitor 7819.
Inspectors reviewed Final Safety Analysis Report, Section 9.2.2.1 and found that
the component cooling water system is designed to be an intermediate barrier
between salt water and contaminated heat loads during non-accident scenarios.
Final Safety Analysis Report Section 11.5.2.1.3.1 describes radiation
monitor 7819 on the non-critical loop: "The component cooling water monitor
samples component cooling water from a noncritical component cooling water
line that may be isolated from the rest of the component cooling water for certain
engineered safety features actuation system conditions. Whenever the
noncritical loop of component cooling water is isolated, the system is not
monitored and in-leakage to the component cooling water from a higher activity
system will not be detected." The Final Safety Analysis Report states that
component cooling water is operated at a higher pressure than salt water. This
also causes a potential release path.
The alignment of the noncritical loop radiation monitor described in
Section 9.2.2.2.1 of the Final Safety Analysis Report was not in accordance with
procedures. San Onofre Procedure S023-2-17, "Component Cooing Water
System Operation," Revision 32, Step 6.7 and Attachment 9 Step 6.2 did not
direct operators to align the letdown heat exchanger to the component cooling
water loop being monitored by radiation monitor 7819 or direct compensatory
radiation monitoring by other means. The steps leave this part of system
alignment to the discretion of the operator. In addition, plant operators did not
question the procedure's adequacy when both trains of component cooling water
were in service.
The inspectors concluded that the licensing basis was not correctly implemented
with this procedure. San Onofre Procedure AOI S023-13-7, "Loss of Component
- 36- Enclosure 2
Cooling Water (CCW)/Saltwater Cooling (SWC)," Revision 14 (EC 14-1),
Step 13.e, directed operators to check that the trend on radiation monitor 7819
was normal when system leakage is detected. Inspectors found that this was not
in accordance with Final Safety Analysis Report, Section 9.2.2.3.2. The steps
contained instructions to check the radiation monitor trend but not to ensure that
it was aligned to the train that was suspected of leakage.
The inspectors found that San Onofre Nuclear Generating Station did not
translate the component cooling water system design into procedures that
ensured that radionuclide releases would not occur without monitoring in all
operational alignments. Radiation monitor 7819 is not in the Offsite Dose
Calculation Manual as a release point. Final Safety Analysis Report
Section 11.5.1.2, Effluent Monitoring Systems, does not describe the component
cooling water system as a monitored release point or radiation monitor 7819 as
an effluent radiation monitor. Plant procedures and sections of the Final Safety
Analysis Report support general design criterion 64 for monitoring of radioactive
releases.
Inspectors concluded that San Onofre Nuclear Generating Station did not
consider the component cooling water system heat exchangers as release paths
in several alignments such as shutdown cooling, emergency core cooling system
sump recirculation, normal chemical and volume control system letdown, and
spent fuel cooling. Plant procedures contained no consideration that component
cooling water radiation monitors 7819 (Unit 2 and Unit 3) could only be aligned to
one train of component cooling water at a time but that in-leakage could
potentially occur in the opposite component cooling water train and be released
to the salt water system. Although monthly grab sample monitor component
cooling water, this frequency is not sufficient to monitor for radionuclides which
could be released into Salt Water Cooling. As a result, existing procedures to
monitor component cooling water leakage while at power were inadequate to
ensure grab sampling of the component cooling water train not aligned to
radiation monitor 7819 (Unit 2 and Unit 3).
The licensee entered this issue into the corrective action program as Nuclear
Notification 200871387, and instituted compensatory actions to routinely sample
the component cooling water train that is not aligned to the radiation monitor and
to perform sampling when the radiation monitor is not in service. These
compensatory measures are to remain in place until San Onofre Nuclear
Generating Station completes its evaluation of the issue.
Analysis. The failure to translate the design bases into procedures that ensure
the radiation monitoring of the safety-related component cooling water system in
all operational alignments is a performance deficiency. The inspectors
determined that this finding was more than minor because this issue impacted
the Public Radiation Protection Cornerstone and its objective to ensure adequate
protection of public health and safety from exposure to radioactive mateiials
released into the public domain as a result of routine civilian nuclear reactor
operation. Specifically, the component cooling water radiation monitors were not
- 37 - Enclosure 2
sufficient to ensure adequate release measurements. The inspectors evaluated
the significance of this finding using Phase 1 of Inspection Manual
Chapter 0609.04 and determined that the finding screened to Inspection Manual
Chapter 0609, Appendix D, Public Radiation Safety Significance Determination
Process. The inspectors evaluated the significance of this finding using
Inspection Manual Chapter 0609, Appendix D, and determined that the finding
was of very low safety significance because dose did not exceed Appendix I
criteria. This finding was determined to have a crosscutting aspect in the area of
problem identification and resolution associated with the corrective action
program in that plant operators did not have a low threshold for identifying
deficiencies in procedures. [P.1 (c)]
Enforcement. Technical Specification 5.5.1.1.a. requires, in part, that written
procedures be established, implemented, and maintained covering the activities
specified in Appendix A, "Typical Procedures for Pressurized Water Reactors
and BOiling Water Reactors," of Regulatory Guide 1.33, "Quality Assurance
Program Requirements (Operations)," dated February 1978; Section 7.g requires
procedures for radiation monitoring operation. Contrary to the above, prior to
April 22, 2010, the licensee failed to establish procedures for component cooling
water system alignments that would prevent unmonitored leakage to the
environment through leakage into the Salt Water Cooling system. Because the
violation was of very low safety significance and was entered into the corrective
action program as Nuclear Notification 200871387, this violation is being treated
as noncited violation, consistent with the NRC Enforcement Policy VI.A:
NCV 05000361/2010006-07, "Failure to Establish Component Cooling Water
Radiation Monitoring Procedures."
h. Failure to Revise Procedures with Known Technical Errors
Introduction. The inspectors identified a cited violation of Technical
Specification 5.5.1.1 a for the failure to maintain written procedures covered in
Regulatory Guide 1.33. Specifically, as of April 2010, the licensee failed to
properly control procedure changes associated with plant modifications resulting
in procedures with known technical deficiencies being used at the facility.
Description. On April 8, 2010, the inspectors reviewed corrective actions from
two previous noncited violations for the licensee's failure to maintain procedures.
The first noncited violation was 05000361 :05000362/2009003-02 and was
associated with the licensee's failure to implement controls over its backlog of
procedure change requests such that procedures with known technical
deficiencies were in use in the field (before being revised). The second noncited
violation was 05000361 :05000362/2009009-02 and also involved the licensee's
failure to implement controls over its backlog of procedure change requests such
that procedures with known technical deficiencies were in use in the field.
During this inspection, the inspectors identified that the backlog of procedure
change requests had increased to 3,389. The inspectors identified that most of
these procedure changes were appropriately classified according to the "TEAM"
- 38- Enclosure 2
method in accordance with San Onofre Procedure S023-XV-1 09.1, "Procedure
Action Request Committee Process," Revision 1. The inspectors approach
classifies procedure changes as technical, enhancement, administrative
correction, or modification. Technical changes were defined for plant impacting
procedures or procedures that must be issued the next business day as changes
that could place a structure system or component in an unevaluated condition;
could cause a plant trip; could cause a loss of megawatts; could degrade nuclear
safety; could cause unexpected reactivity changes; or could cause an immediate
personnel safety issue. However, for procedure changes related to plant
modifications the inspectors identified that there was no procedural direction to
ensure technical procedure changes were incorporated for operating the
equipment following modifications. Additionally, the inspectors identified at least
one procedure change request that had been inappropriately classified as a plant
modification when it was, in fact, a technical procedure change that was
unrelated to a plant modification. (see Section 40A2.51)
The inspectors requested that the licensee review the backlog of modification-
related procedure changes to determine if any were related to modifications that
had already been installed in the plant. Of the 212 modification-related
procedure changes in the backlog, the licensee identified 60 procedure changes
associated with plant modifications that were either installed or partially installed.
These 60 pending changes included changes to 10 procedures, including one
alarm response procedure, associated with a modification to the instrument air
system which had been installed during Unit 2 refueling outage R2C16; these
procedures did not reflect the current plant configuration. Following the
inspectors' identification of these unincorporated technical changes, the licensee
initiated a full review of plant modifications classified procedure change requests.
The licensee identified a total of 18 procedures which required technical changes
as a result of plant modifications. The licensee agreed that these procedure
changes should have been made prior to the associated plant modifications
being turned over to operations. The result had been that procedures with
known technical deficiencies as a result of plant modifications had been in use in
the field.
The inspectors further identified that the process for ensuring modification-related
procedure changes were incorporated prior to the modifications being turned
over to operations was informal and was not controlled by procedure. The
determination of which procedure changes were important and which could be
deferred was left up to the procedure writer; there was no procedural guidance
for making this determination. This finding was entered into the licensee's
corrective action program as Nuclear Notification 200888919, and the licensee
took actions to suspend use of the affected procedures until they could be
revised.
Analysis. The failure to maintain San Onofre Nuclear Generator Station
procedures covered by Regu!atory Guide 1.33 is a performance deficiency. The
finding is of more than minor significance because, if left uncorrected, it would
have the potential to lead to a more significant safety concern by having
- 39- Enclosure 2
technically inaccurate procedures being used on important plant systems. Using
Inspection Manual Chapter 0609.04, Phase 1 "Initial Screening and
Characterization of Findings," the finding was determined to have a very low
safety significance because the finding did not result in a loss of system safety
function, an actual loss of safety function of a single train for greater than its
technical specification allowed outage time, or screen as potentially risk
significant due to a seismic, flooding, or severe weather initiating event. The
finding has a crosscutting aspect in the area of problem identification and
resolution associated with the corrective action program component because
problems were not thoroughly evaluated such that the resolutions addressed the
causes and extents of condition. [P.1 (c)]
Enforcement. Technical Specification 5.5.1.1.a requires, in part, that written
procedures be established, implemented, and maintained covering the applicable
procedures recommended in Regulatory Guide 1.33, Revision 2, Appendix A,
February 1978. Regulatory Guide 1.33, "Quality Assurance Program
Requirements (Operations)," Appendix A, recommends procedures for the
operation of certain plant systems. Contrary to the above, as of April 2009, the
licensee failed to maintain written procedures as recommended in Regulatory
Guide 1.33, Revision 2, Appendix A, February 1978. Specifically, the licensee
did not ensure that following equipment modifications made to the instrument air
system, procedures requiring technical changes were suspended, put on
administrative hold, or otherwise restricted from use until the required changes
were made. As a result, severa! procedures with known technical deficiencies
were available for operator use.
This performance deficiency was previously identified by the NRC on two
occasions and were documented as noncited violations 05000361:
05000362/2009003-09 and 05000361;05000362/2009009-02. The inspectors
determined that the licensee had failed to restore compliance within a reasonable
time following issuance of these noncited violations. Therefore, this violation is
being cited in a Notice of Violation consistent with Section VI.A of the NRC
Enforcement Policy: VIO 05000361;05000362/2010006-08, "Failure to Maintain
Written Procedures Covered in Regulatory Guide 1.33."
i. Failure to Set Goals In Accordance With the Maintenance Rule
Introduction. The inspectors identified two examples of a Green noncited
violation of 10 CFR 50.65(a)(2) for failure to monitor the performance of auxiliary
feedwater system components against established goals in a manner to provide
reasonable assurance that the system was capable of fulfilling designated
auxiliary feedwater maintenance rule functions.
Description. Under the maintenance rule, San Onofre Nuclear Generating
Station defines three separate functions for monitoring the auxiliary feedwater
system. Function 1 has a stated purpose fOi motor-driven Train A to supply
feedwater from the condensate feedwater tanks to steam generator 88 for plant
cool down when main feedwater is unavailable. Function 2 is identical except
- 40- Enclosure 2
that it tracks motor-driven Train B auxiliary feedwater. Function 3 covers the
turbine-driven auxiliary feedwater pump to supply both steam generators. All
three functions stated that they include the water supply piping and valves from
condensate storage Tanks T -120 and T-121. The auxiliary feedwater system
has unavailability goals of 1.2 percent per 12 month period for functions 1 and 2
and 1.1 percent per 12 month period for function 3. This equates to
approximately 79.3 hours3.472222e-5 days <br />8.333333e-4 hours <br />4.960317e-6 weeks <br />1.1415e-6 months <br /> of unavailability for function 3.
On December 9, 2008, San Onofre Nuclear Generating Station performed flawed
maintenance that bent the fuse holder contacts such that there was a loose
electrical connection. On December 19, 2008, the control room received an
annunciator indicating a problem with the Unit 3 turbine driven auxiliary
feedwater pump. The licensee identified the loose electrical connection caused
the alarm and repaired the connection early on December 20, 2008. In Nuclear
Notification 200253911, the licensee counted 9.6 hours6.944444e-5 days <br />0.00167 hours <br />9.920635e-6 weeks <br />2.283e-6 months <br /> of unavailability because
that was the time from control room annunciation of a problem to the completion
of repairs. The maintenance rule evaluation did not elaborate as to why this
amount of time was used. The inspectors noted that the ioose connection
existed for approximately 10 days prior and for approximately 64 hours7.407407e-4 days <br />0.0178 hours <br />1.058201e-4 weeks <br />2.4352e-5 months <br /> while
Unit 3 was in Mode 1.
When questioned by the inspectors, San Onofre Nuclear Generating Station
personnel stated the basis for using 9.6 hours6.944444e-5 days <br />0.00167 hours <br />9.920635e-6 weeks <br />2.283e-6 months <br /> was that the pump was functional
because only its earthquake qualification was in question. Inspectors found that
the evaluation of the loose connection did not consider resistance heating of the
loose connection or that an earthquake was not required for the failure to be
annunciated in the control room. The counting of additional unavailability hours
would have caused the Unit 3 turbine driven auxiliary feedwater pump to exceed
its 10 CFR 50.65 (a)(2) goal and be placed into (a)(1) status. However, since the
pump had been previously placed in (a)(1) status in April 2009 due to functional
failures, the approximately 64 hours7.407407e-4 days <br />0.0178 hours <br />1.058201e-4 weeks <br />2.4352e-5 months <br /> additional hours of unavailability would have
prevented the system from transitioning back to (a)(2) status within 6 months.
When questioned, plant personnel informed the inspectors that it utilized NRC
performance indicator guidance from NEI 99-02 as its evaluation of unavailability.
Combined with other system unavailability, function 3 would exceed its
approximately 80 hour9.259259e-4 days <br />0.0222 hours <br />1.322751e-4 weeks <br />3.044e-5 months <br /> unavailability monitoring goal and (a)(1) monitoring would
have been significantly extended. Because the licensee performed an
inadequate evaluation of unavailability time, the system was returned to (a)(2)
status on August 20, 2009.
The second example of inadequate evaluation of unavailability time involved the
licensee's maintenance rule evaluation of the failure of auxiliary feedwater
condensate isolation valve 2HV5715. On January 26, 2010, valve 2HV5715
failed its in-service stroke test (as described in section 40A2.5c). The hand
wheel stem snapped when a leveraging device was used to attempt to open the
valve. The valve operator stem \lJaS heavily rusted because it had been i6moved
from the preventive maintenance regimen program. This valve must be closed
per procedure within 90 minutes of an Operating Basis Earthquake to prevent the
- 41 - Enclosure 2
loss of water inventory from condensate storage tank T -120 from a line break in
the non-seismic portion of the condensate system. Nuclear
Notification 200765235 was written to evaluate the broken valve.
Inspectors found that the maintenance rule evaluation counted a functional
failure for the valve, but utilized Mitigating Systems Performance Index guidance
from NEI 99-02 for unavailability. Inspectors found that use of this guidance was
inappropriate to the circumstances and that the evaluation was inadequate. The
licensee also utilized Appendix C to NRC Inspection Procedure 71111.13 for
evaluating unavailability time, but only considered limited portions of the
guidance. The licensee's program procedure described availability but did not
provide sufficient guidance for this situation.
The maintenance rule evaluation in Nuclear Notification 200765235 also stated
that since the valve failed its stroke test in Mode 6, that there was no
unavailability impact. The evaluation stated: "The timing of when this valve
would no longer close is unknown and may have been during the required
Mode 1 thru 3." inspectors found that the iicensee had not attempted to perform
an engineering evaluation to determine when the valve failed due to the rust.
Given the as-found condition, the number of unavailability hours was most likely
significantly higher than the 79 hour9.143519e-4 days <br />0.0219 hours <br />1.306217e-4 weeks <br />3.00595e-5 months <br /> monitoring threshold. This long-standing
deficiency was significant because no preventive maintenance had been
performed on the valve resulting in its degradation. This deficiency impacted all
three maintenance rule functions for the auxiliary feedwater system.
Step 6.5.1.7 of San Onofre Procedure S0123-XV-5.3, "Maintenance Rule
Program," required the monitoring of unavailability for these trains. Due to
inadequate tracking and accounting, the licensee failed to identify that it
exceeded the auxiliary feedwater trains' monitoring goal. San Onofre
Procedure S0123-XV-5.3, Step 6.5.1.7, required review of functional
unavailability information from all sources as necessary to ascertain performance
relative to established criteria. Lastly, San Onofre Procedure S0123-XV-5.3,
Step 6.5.1.7, required that when a trend of performance indicates a performance
criterion has been exceeded, the train will be evaluated for goal setting. This did
not occur. As a result, the plant engineering department took action to evaluate
the issues identified by the inspectors and is reviewing existing guidance. These
actions were documented in Nuclear Notification 201001922.
Analysis. Failure to adequately account for unavailability time in the licensee's
maintenance rule evaluation of the auxiliary feedwater system is a performance
deficiency. This finding is more than minor because it affects the equipment
performance attribute of the Mitigating Systems Cornerstone per Inspection
Manual Chapter 612, Appendix B. Specifically, San Onofre Nuclear Generator
Station failed to appropriately account for system unavailability hours which
would have resulted in the moving the system to (a)(1), requiring goals and
........"" ..... i+_r;-...... _ +h ___ r+_ ... ..--. _______ ;_ .... 4- ... L-.. ______ I .... ,c_ .... +L-.. ........ L.. ... __ ..... , ... ..,:I: __ ~. & __ -.1 *** _.&._-
IIIVIII'V'~ l I l v ,",vIIVI' llal ,,",v a~alll;:)l LlIV;:)t; l::Ival;:) IVI LIlt; Llil t;t; ClUAIIiClI Y IttUVVClttl
functions. The inspectors evaluated the significance of this finding using
Inspection Manual Chapter 0609.04, Phase 1 "Initial Screening and
- 42- Enclosure 2
Characterization of Findings," and determined that this finding is of very low
safety significance, Green. Specifically, the maintenance rule is an
administrative activity that could not result in the loss of a system safety function,
an actual loss of safety function of a single train for greater than its technical
specification allowed outage time, or screen as potentially risk significant due to a
seismic, flooding, or severe weather initiating event. The cause of the finding
was determined to have a crosscutting aspect in the area of human performance
in the decision-making component because the licensee did not use a systematic
process when faced with the unexpected unavailability for the latent equipment
deficiencies. H.1.a]
Enforcement. Title 10 CFR 50.65 requires, in part, when performance of
systems, structures, or components cannot be demonstrated per paragraph
(a)(2), that performance goals and corrective action shall be established under
paragraph a(1). San Onofre Nuclear Generating Station's monitors auxiliary
feedwater maintenance rule functions 1, 2, and 3 with an unavailability goal of
approximately 80 hours9.259259e-4 days <br />0.0222 hours <br />1.322751e-4 weeks <br />3.044e-5 months <br /> per rolling 12 month period. Contrary to the above, on
January 26, 2009, and December 9, 2008, San Onofre Nuclear Generator
Station's auxiliary feedwater maintenance rule functions 1, 2, and 3 exceeded
their (a)(2) monitoring goals and San Onofre Nuclear Generator Station failed to
evaluate and establish (a)(1) goals. Specifically, the evaluations discounted
significant unavailability hours from long maintenance induced failures that would
have cause the 80 hour9.259259e-4 days <br />0.0222 hours <br />1.322751e-4 weeks <br />3.044e-5 months <br /> goals to be exceeded. Because this violation was of
very low safety significance and was entered into the licensee's corrective action
program under Nuclear Notification 201001922, this violation is being treated as
a noncited violation in accordance with the NRC Enforcement policy:
NCV 05000361/05000362/2010006-09, "Failure to Establish Goals and Monitor
for a(1) auxiliary feedwater trains."
j. Failure to Identify and Correct the Use of Degraded Relays in Safety-Related
Equipment
Introduction. The inspectors identified a Green noncited violation of
10 CFR Part 50, Appendix B, Criterion XVI, "Corrective Action," for the licensee's
failure to promptly identify and correct conditions adverse to quality. Specifically,
the licensee performed an inadequate extent of condition review and failed to
identify that deficient motor driven rotary relays were installed in various safety-
related applications.
Description. On August 5,2007, the Unit 3 emergency diesel generator 3G002
was taken out of service for preventive maintenance. On August 9,2007, the
licensee performed preventive maintenance in the emergency diesel generator
cabinet 3L 160. The maintenance activity instructs personnel to perform continuity
checks for all associated contacts in the electrical cabinet to ensure they are in
the correct position, and then perform relay checks to ensure the relays and
associated contacts peiform as expected when energized or de-energized.
During performance of the maintenance activity, maintenance personnel reported
(Action Request 070800466) that normally de-energized relay 3L 160-2-K52, a
- 43- Enclosure 2
Potter & Brumfield motor driven relay, was sluggish and would not rotate
completely. The 2008 problem identification and resolution team documented
the deficiency in NCV 05000362/2008012-02, "Failure to Properly Implement
Operability Determination Process" because the licensee did not perform an
operability determination of the sluggish relay. The licensee entered the issue
into the corrective action program as Nuclear Notification 200146292.
The licensee evaluated the motor driven relays in Direct Cause
Evaluation 8001654561, and determined that the cause of the failure was an
oversized coil manufacturing deficiency. The licensee stated that this was a "well
documented failure mechanism for Potter & Brumfield motor driven relays
manufactured between 1989 and 1992." The licensee also stated that "there are
a large number of normally de-energized motor driven relays in the plant from the
manufacturing lots with the oversize coils." The licensee replaced the relays,
whose failure could impact the operability of the emergency diesel generators,
with new relays that were manufactured with a retaining ring around the coil to
prevent oversized coil failures. The licensee generated Nuclear
Notification 200188863 to address the extent of condition. However, the extent
of condition only focused on the motor driven relays installed in the four
The inspectors asked if the deficient motor driven relays, which remained
installed in the plant and were not covered in the scope of the extent of condition
review, were installed in safety-related applications. The licensee found 62
normally de-energized relays whose failure "could impact the performance of a
specified safety function." The licensee generated Nuclear
Notification 200887995 and created maintenance orders to replace the degraded
relays at the next available opportunity.
Analysis. The failure to perform an adequate extent of condition evaluation and
identify and correct a condition adverse to quality was a performance deficiency.
This finding was more than minor because it impacted the equipment
performance attribute of the Mitigating Systems Cornerstone objective to ensure
the availability, reliability, and capability of systems that respond to initiating
events to prevent undesirable consequences. Using Inspection Manual
Chapter 0609.04, Phase 1 ,"Initial Screening and Characterization of Findings,"
the inspectors determined the finding to be of very low safety significance
(Green) because it did not represent the loss of a system safety function and did
not screen as potentially risk significant due to a seismic, flooding, or severe
weather initiating event. This finding has a crosscutting aspect in the area of
human performance associated with the decision-making component in that the
licensee did not use conservative assumptions in making decisions about the
extent of condition. [H.i (b)]
Enforcement. Title 10 CFR Part 50, Appendix B, Criterion XVI, "Corrective
Action/' requires, in part, that measures sha!! be established to assure that
conditions adverse to quality are promptly identified and corrected. Contrary to
the above, from October 2008, to April 2010, the licensee did not promptly
- 44- Enclosure 2
identify and correct the use of deficient motor driven relays in safety-related
systems and components. Because the finding is of very low safety significance
and has been entered into the corrective action program as Nuclear
Notification 200146292, this violation is being treated as a noncited violation
consistent with Section VLA of the NRC Enforcement Policy: NCV 05000361;05000362/2010006-10, "Failure to Identify and Correct Use of Deficient Motor
Driven Relays."
k. Failure to Secure Loose Items in the Switchvard
Introduction: A Green noncited violation of Technical Specification 5.5.1.1.a was
identified involving the failure to follow San Onofre Procedure S0123-XX-11,
"Switchyard Work Performance" Revision 2. Specifically, the inspectors
identified the licensee's faiiure to adequately control loose material within the
Description: On April 7, 2010, inspectors performed a walkdown of the 230kV
switchyard. During the walkdown, inspectors identified several pieces of
temporai)' moveable equipment that were not tethered in the switchyard.
Inspectors determined that loose material in the switchyard could be hazardous
to electrical equipment that could affect the loss of offsite power in the event of
seismic activity, tornados, high winds, or hurricanes. The licensee entered a
Nuclear Notification 200870138 in their corrective action program to evaluate the
condition. The licensee's San Onofre Procedure S0123-XX-11 "Switchyard
Work Performance" Revision 2, under Section 6.12, "Temporary Equipment",
Step 6.12.1, specifically states, "All unattended temporary movable equipment
left in the Switchyard or Relay House SHALL be restrained in such a manner so
as to prevent damage to any installed equipment during a seismic event."
The inspectors interviewed plant personnel and determined that personnel failed
to remove the materials from the switchyard subsequent to completing assigned
work activities in the switchyard. The licensee verified that three of the loose
items found by inspectors had been in the switchyard unrestrained since the first
week of October 2009, three other items had been unrestrained in the switchyard
since March 2, 2010, and two more items had been unrestrained in the
switch yard for the life of the plant. The licensee failed to provide effective
oversight to ensure the loose material was tied down throughout the duration of
work being performed in the switchyard as well as the removal of material
following completion of the respective jobs.
The licensee documented this violation in Nuclear Notification 200870138, and
its short term corrective actions included removing or securing loose items,
evaluating materials in the switchyard for high winds and seismic concerns, and
ensuring operator rounds that included checking for loose material.
Analysis. The failuie to contiOl loose material near risk-significant equipment is a
performance deficiency. This finding is more than minor because it impacts the
protection against the external factors attribute of the Initiating Events
- 45- Enclosure 2
Cornerstone objective to limit the likelihood of those events that upset plant
stability and challenge critical safety functions during shutdown and power
operations. Using the Significance Determination Process Phase 1 worksheets
from Inspection Manual Chapter 0609, the inspectors determined that the finding
was of very low safety significance (Green) because it did not contribute to both
the likelihood of a reactor trip and the likelihood that mitigation equipment or
functions would not be available. This finding also has a human performance
crosscutting aspect associated with the work control component in that personnel
failed to appropriately plan work activities involving job site conditions which may
impact plant structures, systems and components. H.3(a)
Enforcement. Technical Specification 5.5.1.1.a, in part, requires that procedures
be established, implemented, and maintained covering the applicable procedures
in Regulatory Guide 1.33, Appendix A. Regulatory Guide 1.33, Appendix A,
requires in part, written procedures for Acts of Nature (e.g. tornado, flood, dam
failure, earthquakes). Contrary to the above, the licensee failed to follow
procedure as required by Regulatory Guide 1.33, Appendix A. Specifically, the
licensee failed to adequately control loose material in the switchyard as required
by San Onofre Procedure S0123-XX-11, "Switchyard Work Performance,"
Revision 2. The licensee entered a notification in their corrective action program
as Nuclear Notification 200870138. This violation is being treated as a noncited
violation, consistent with Section Vela of the Enforcement Policy:
NCV 05000361;05000362/2010006-11, "Failure to control loose items in the
electrical switchyard."
I. Failure to Translate Design Basis Information into Affected Calculations and
Procedures
Introduction. The inspectors identified two examples of a noncited violation of
10 CFR Part 50, Appendix B, Criterion III, "Design Control," for the failure of the
licensee to ensure that new information affecting the plant design bases was
incorporated into affected procedures, calculations, and drawings. Specifically,
the inspectors identified two instances where the licensee determined, based on
a review of information provided by vendors, that design margins or instructions
in safety-related calculations or procedures were adversely impacted but failed to
revise the calculations or procedures to reflect these non-conservative
assumptions.
Description. On April 9, 2009, the licensee initiated Nuclear
Notification 200385686 to evaluate Westinghouse Technical Bulletin TB-09-4.
This technical bulletin identified that auxiliary feedwater pump heat was not
explicitly considered in the sizing calculation for the condensate storage tank;
addition of this heat could have an effect of approximately 3000 gallons on the
required condensate storage tank volume. On September 25, 2009, the licensee
completed its evaluation of this technical bulletin. The licensee concluded that
\Nhi!e Technical Bulletin T8-09-4 'vvas applicable to San Onofre Nuciear
Generator Station, there was sufficient margin in the existing calculation for the
system to perform according to design requirements; no further action was
- 46- Enclosure 2
necessary. The affected calculation was not updated. The inspectors verified
the licensee's determination that the non-conservatism addressed in the
technical bulletin was bounded by other assumptions in the condensate storage
tank sizing calculation. However, the inspectors determined that the failure of the
licensee to note the neo-conservatism in the calculation could result in the loss of
margin should the bounding assumptions be changed in the future. The licensee
initiated Nuclear Notification 200886265 to address this deficiency.
On November 5, 2009, the licensee initiated Nuclear Notification 200656309 to
evaluate Westinghouse Nuclear Safety Advisory Letter NSAL-09-8. This letter
identified the potential for the presence of vapor in emergency core cooling and
residual heat removal systems during certain modes of operation. The letter
identified the potential that if the residual heat removal system is operated in the
shutdown cooling mode above 200°F, initiation of safety injection following a loss
of coolant accident could result in the injection water flashing to steam, binding
the low pressure safety injection pumps. On January 14, 2010, the licensee
completed its evaluation of this nuclear safety advisory letter and determined that
while it was appiicabie to San Onofre Nuclear Generator Station, the concerns
noted in the letter had already been addressed in San Onofre Nuclear Generator
Station procedures or instructions which contained cautions against operation of
shutdown cooling above 200°F. A task was generated under Nuclear
Notification 200656309 to modify San Onofre Procedure S023-5-1.3, "Plant
Startup from Cold Shutdown to Hot Standby," Revision 35, to note flashing of
injection water as a reason shutdown cooling operation should be secured in
Mode 5 prior to entering Mode 4. This task was improperly characterized as an
plant modifications or modification-related, procedure and assigned a due date of
June 30, 2010. During a review of all plant modification procedure change
requests requested by the inspectors, the licensee determined that the plant
modification classification was inappropriate and changed it to an "E," or
enhancement. The inspectors determined that this procedure change should
have properly been classified as a "T," or technical change, and been
implemented prior to the next use of the procedure during reactor startup.
On March 26, 2010, during reactor startup following Unit 2 outage R2C16, the
licensee was operating in Mode 4 at approximately 270°F while attempting to
restore one train of auxiliary feedwater. When this restoration was delayed, the
licensee's risk analysis group advised the operators to place shutdown cooling in
standby to provide an alternate source of core cooling should the single operable
train of auxiliary feedwater be lost. Because the procedure only noted that
reactor coolant temperature "should" be maintained below 200°F while shutdown
cooling is in operation and did not reference the conclusions drawn from the
licensee's analysis of NSAL-09-8, operations personnel failed to recognize the
vulnerability of the system to flashing and vapor binding the pump on initiation of
low pressure safety injection. Referencing a note contained in the limitations
section of the procedure (Attachment 12, Step 15.1) which states, "When
shutdm,Am cooling is in-service, then [ieactor coolant system] temperature ... shaH
not exceed 340°F ... ," operations personnel began taking steps to place
shutdown cooling in standby with reactor coolant temperature at approximately
- 47- Enclosure 2
270°F. After this course of action was questioned by the NRC resident
inspectors and station management, the licensee identified the operating
experience evaluation performed under Nuclear Notification 200656309 and
determined that placing shutdown cooling in standby at 270°F was inappropriate
with current procedures. The licensee initiated Nuclear Notification 200855352
to identify why this course of action was considered.
The licensee's review of NSAL-09-8 under Nuclear Notification 200656309 also
identified that while cooling down, San Onofre Procedure S023-3-2.6, "Shutdown
Cooling System Operation," Revision 26, contains procedural steps to isolate the
suctions of the low pressure safety injection pumps prior to the initiation of
shutdown cooling. However, the inspectors noted that, similar to the startup
situation, there is no procedural step or limitation to indicate that these valves
must be shut above 200°F to prevent flashing of the fluid should a safety injection
signal be received. Further, the limitations and specifications section of the
procedure (Attachment 16, Step 1.1) only restricts operation to at or below
340°F. In its evaluation of NSAL-09-8, the licensee did not initiate a procedure
change request to address this vulnerability in this procedure. Because
procedural restrictions in both San Onofre Procedures S023-5-1.3 and
S023-3-2.6 permit shutdown cooling operation up to 340°F and because
Section 5.4.7 of the Final Safety Analysis Report specifies that shutdown cooling
is put into service once reactor coolant system temperature has been reduced
below 350°F, the inspectors determined that site procedures and design basis
documentation are inadequate to ensure that operators do not place shutdown
cooling in service above 200°F.
Analysis. The failure of the licensee to maintain plant design basis
specifications, drawings, procedures, and instructions up-to-date is a
performance deficiency. The finding is of more than minor significance because
it adversely affects the design control attribute of the Mitigating Systems
Cornerstone objective. Using Inspection Manual Chapter 0609.04, Phase 1,
"Initial Screening and Characterization of Findings," the finding was determined
to have a very low safety significance because the finding did not result in a loss
of system safety function, an actual loss of safety function of a single train for
greater than its technical specification allowed outage time, or screen as
potentially risk significant due to a seismic, flooding, or severe weather initiating
event. The finding has a crosscutting aspect in the area of problem identification
and resolution associated with the operating experience component because the
licensee failed to implement and institutionalize operating experience information,
including vendor recommendations, through changes to plant processes,
procedures, equipment, and training programs. P.2(b)
Enforcement. Title 10 CFR Part 50, Appendix B, Criterion III, "Design Control,"
requires, in part, that measures be established to assure that the design basis for
safety-related structures, systems, and components are correctly translated into
specifications, dravvings, procedureS, and instructions. Contrary to tJlis
requirement, on June 27,2009, September 25,2009, and January 14, 2010, the
licensee failed to assure that the design basis for safety-related structures,
- 48- Enclosure 2
systems, and components was correctly translated into specifications, drawings,
procedures, and instructions. Specifically, the licensee identified
nonconservative errors in calculations and procedures but failed to incorporate
this new information into the affected calculations and procedures. Because this
finding was of very low safety significance, was not repetitive or willful, and was
entered into the corrective action program, this violation is being treated as a
noncited violation, consistent with Section VI.A of the NRC Enforcement Policy:
NCV 05000361 ;05000362/2010006-12, "Failure to Maintain Design Basis
Information."
m. Failure to Meet Action Plan for Substantive Crosscutting Issues
Introduction. The inspectors identified a Green finding involving examples of the
licensee's failure to meet its action plan as described in letters to the NRC
documenting actions San Onofre Nuclear Generator Station would take to correct
the third and fourth consecutive assessment cycles of substantive crosscutting
issues in the areas of human performance and problem identification and
resolution.
Description. The NRC's annual assessment letter dated March 4,2009, was the
third cycle where substantive crosscutting issues were identified in human
performance and problem identification and resolution. San Onofre Nuclear
Generator Station responded to the open substantive crosscutting issues in a
letter titled, "Response to Annual Assessment Letter Inspection
Report 05000361/2009001, 05000362/2009001," dated April 21,2009, with the
status of corrective actions planned to address the human performance and
problem identification and resolution crosscutting issues, including schedules,
milestones, and performance monitoring metrics. San Onofre Nuclear Generator
Station committed to completing six initiatives to improve its human performance
and eight initiatives to improve its process for problem identification and
resolution. The licensee committed to completing specific actions to improve
performance in these areas.
The status of these commitments was provided to the NRC in an
October 30, 2009, letter. As of that date:
- Of the 48 commitments made to improve performance in the human
performance area, 28 were complete. Of the 20 remaining open, 4
(20 percent) were past due.
- Of the 36 commitments made to improve performance in the problem
identification and resolution area, 21 were complete. Of the 15 remaining
open, 3 (20 percent) were past due.
Several of the specific actions to which the licensee committed were not
compieted by theii specified due dates and/or were not compieted as specified,
as evidenced by the following examples:
- 49- Enclosure 2
i. The licensee committed to establishing response inspectors training and
providing this training to selected personnel by December 31,2009. As
of March 31,2010, this training had not been completed.
ii. The licensee committed that divisions that were not meeting apparent
cause evaluation timeliness goals would develop action plans to improve
apparent cause evaluation timeliness to less than or equal to 40 days by
December 10, 2009. As of March 31, 2010, these divisions had not
developed action plans. In a letter dated March 31, 2010, the licensee
revised the language of this commitment to reflect actions taken.
iii. The licensee committed to establishing a specific work down curve and/or
schedule for backlog of actions requiring closure review boards by
February 20, 2010, so that by March 2010, closure review boards were
normally completed within 30 days of action completion. As of
March 31,2010, the licensee had failed to establish work down curves or
schedules and closure review boards were not being completed in a
timely fashion.
On October 29, 2009, after completing an independent safety culture survey in
which it noted several areas requiring improvement, the licensee sent another
letter to the NRC committing to 56 specific actions to resolve these issues. Five
areas requiring action to preserve and improve safety culture were identified.
The licensee committed to completing specific actions in each of these areas by
specified due dates. Several of the specific actions to which the licensee
committed were not completed by their specified due dates and/or were not
completed as specified, as evidenced by the following examples:
i. The licensee committed to establishing a specific work down curve andlor
schedule for backlog of actions requiring closure review boards and
implement by February 20, 2010, so that by March 2010, closure review
boards were normally completed within 30 days of action completion. As
of March 31, 2010, the licensee had failed to establish work down curves
or schedules and closure review boards were not being completed in a
timely fashion.
ii. The licensee committed to establishing a project plan and schedule for
resolving SAP issues by February 15, 2010, that included: mechanisms
for employee input on problems and solutions; definition of end-state
desired performance; implementation of improvements; and evaluation of
effectiveness. As of March 31, 2010, the licensee had failed to establish
a project plan as specified.
On March 25, 2010, the licensee initiated Nuclear Notification 200848923, noting
that "a number of the commitments to the NRC iTrade in the October 29, 2009,
and October 30, 2009, letters ... were completed with inadequate initial quality or
were completed late." On March 31,2010, the licensee submitted a letter to the
- 50- Enclosure 2
NRC modifying some of these commitments. Specifically, the licensee changed
the wording for 19 committed actions to reflect the actions taken, which did not
align with the actions initially committed, including due date changes for nine
past-due commitments for which the licensee changed the due date to a future
date. As a result, the licensee failed to satisfy several commitments and due
dates made to the NRC related to corrective actions to correct the substantive
crosscutting areas. In addition, the licensee modified commitments without first
discussing the changes with the Nuclear Regulatory Commission.
Also, the licensee's letters dated April 21, 2009, identified the metrics by which
the licensee would assess the state of its corrective action program. The
inspectors reviewed the metrics and identified several questions regarding the
data the licensee was evaluating for its metrics. Examples included: 1
- The metric for measuring the time to perform root cause evaluations has
been relatively flat over the monitoring period; the metric for measuring
the time to perform apparent cause evaluations has exhibited a
downward (improving) trend. However, the inspectors found that these
metrics are tracked from the assignment date to the "Evaluation Complete
Date." As discussed in Nuclear Notification 200886035, the assignment
date can be weeks or months after the issue/event was discovered before
San Onofre Nuclear Generator Station begins counting time against the
metric; and San Onofre Nuclear Generator Station stops counting time
against the metric after the divisional corrective action program
coordinator review which can be weeks or months before final approval
by the corrective action review board. Thus, the data for the time to
perform cause evaluations does not reflect the true time it takes the
licensee to assign and complete the cause evaluation until the time the
corrective actions are identified and approved.
For example:
o At the corrective action review board on April 19, 2010, an
apparent cause evaluation charter was approved for a notification
that was originally written on December 19, 2009. As of
April 20, 2010, the apparent cause evaluation had not been
assigned; therefore, the clock had not started to track the metric.
Thus, the metric for this evaluation did not account for about four
months of time.
o At the corrective action review board on April 19, 2010, an
apparent cause evaluation was approved that had been started on
November 15, 2009, for a notification generated on
November 13, 2009. San Onofre Nuclear Generator Station
stopped counting time for purposes of the metric when the
I Unless otherwise mentioned, all examples cover metrics tracked from July 2009 through February 20 I o.
- 51 - Enclosure 2
divisional corrective action program coordinator approved the
corrective action on January 22, 2010; yet final approval did not
actually occur until the corrective action review board on
April 19, 2010. Thus, the metric for this evaluation did not account
for about three months of time.
.. Licensee management had explained to NRC inspectors that their
upward trend in the number of nuclear notifications written demonstrates
an improvement in the corrective action program in that more people are
using it. However, this data only goes back through July 2009. While
there was a marked increase in the number of nuclear notifications
generated over the first few months of the period, the number has since
been constant. 2 The overall increase in nuclear notifications did not
account for the expected increase in nuclear notifications from a larger
number of personnel on site and the larger workload during the recent
extended outage.
- Similarly, licensee management has cited the declining average age of
open actions as an indicatOi of improvement. However, while the
average age of corrective actions related to cause evaluation has been
trending steadily downward, this appears to be largely due to a concerted
effort by the licensee to work off the oldest corrective actions rather than
a true overall reduction in the age of corrective actions. Further, this
metric does not track the average age of corrective actions to prevent
recurrence, which has been trending sharply upward.
.. The number of nuclear notifications open has demonstrated a significant
upward trend since November 2009. In its April 21, 2009, letter to the
NRC, the licensee committed to reducing the number of open nuclear
notifications, in part by developing actions to reduce backlog for each
division not meeting its divisional metric. On April 14, 2010, the closure
review board package related to this commitment was closed, with the
statement that the metric had been met for 2009. However, this
commitment was modified by the licensee's March 31, 2010, letter which
stated that the 2009 goals had been met; that the licensee was now
focusing on 2010 goals. The commitment was closed as having been
accomplished; however, this metric has been red and trending upward
since January 2010.
Analysis. The inspectors determined that the licensee's failure to perform actions
as documented in its plan to the NRC was more than minor because if left
uncorrected could result in a more significant safety concern. Using Inspection
Manual Chapter 0609, Appendix M, this finding was reviewed by NRC
management and was determined to be of very low safety significance (Green).
2 Significance level 5 and lower exhibit a slight upward trend over the past three months; significance level 1-4
nuclear notification generation has been trending slightly downward over the same period.
- 52 - Enclosure 2
The finding has a crosscutting aspect in the area of human performance
associated with the work practices because the licensee did not ensure
management oversight of work activities. H.4(c)
Enforcement. The finding does not involve an enforcement action because no
violation of regulatory requirements was identified. Because the finding does not
involve a violation and it has very low safety significance, it is identified as
FIN 05000361 ;05000362/201 0006-13, "Failure to meet action plan for
substantive crosscutting issues."
40A6 Meetings
Exit Meeting Summary
On April 23,2010, the inspectors conducted a briefing of the status of potential findings before
concluding the onsite portion of the inspection. This briefing was presented to
Mr. R. Ridenoure, Senior Vice President and Chief Nuclear Officer, and other members of the
licensee staff. At the conclusion of the inspection on June 17, 2010, the inspectors conducted
an exit briefing with Mr. Ridenoure and other members of the licensee staff. The licensee
acknowledged the issues presented. The inspectors asked the licensee whether any materials
examined during the inspection should be considered proprietary. No proprietary information
was identified.
40A6 licensee-Identified Violations
The following violations of very low safety significance (Green) were identified by the licensee
and are violations of NRC requirements which meet the criteria of Section VI of the NRC
Enforcement Policy, or being dispositioned as NCVs.
Green. The inspectors reviewed a licensee-identified finding involving the failure to follow San
Onofre Procedure S0123-1-1.3, "Work Activity Guidelines," Revision 26, Step 6.18.2.2, which
allows the licensee to skip a preventive maintenance work order step as long as an evaluation is
documented that identifies why it is okay not to perform the step. Specifically, in March 2006,
the licensee identified that cubicle for breaker 2A0807 had not been cleaned prior to Cycle 7
due to an energized reserve auxiliary transformer and generated Action Request 060300521 to
document the deficiency. The action request further indicated that this issue was applicable to
all of San Onofre Nuclear Generator Station 4.16 kV switchgear. All of the switchgears (A03,
A04, A05, A06, A07, A08, and A09 for both units) have feeders from both the reserve auxiliary
and unit auxiliary transformers (the GDC 17 off-site source of power). The licensee stated that it
was not possible to clean every cubicle in a given bus within a single work window. The
manufacturer recommended a cleaning frequency of five years of 1000 cycles of operation;
however, cubicle for breaker 2A0807 had not been cleaned in over 14 years without an
evaluation documenting a basis for postponing the preventive maintenance. Licensee
personnel entered this issue into their corrective action program as Nuclear
Notifications 200876216 and 200880374.
This finding was more than minor because it impacted the human performance attribute of the
Initiating Events Cornerstone objective to limit the likelihood of those events that upset plant
- 53- Enclosure 2
stability and challenge critical safety functions during shutdown as well as power operations.
Using Inspection Manual Chapter 0609.04, Phase 1, "Initial Screening and Characterization of
Findings," the inspectors determined the finding to be of very low safety significance (Green)
because it did not contribute to both the likelihood of a reactor trip and the likelihood that
mitigation equipment or functions would not be available. This finding was determined not to
have a crosscutting aspect because it is a latent condition.
- 54- Enclosure 2
KEY POINTS OF CONTACT
Licensee Personnel
C. Amundson, Maintenance Engineer
V. Barone, Design Engineer
R. Battish, System Engineer
G. Becker, Operations Procedures
S. Chun, Maintenance Engineering Manager
S. Gardner, Electrical Supervisor, System Engineering
J. Jay, Site Procedures Manager
J. Madigan, Health Physics Manager
A. Martinez, Corrective Action Program Manager
A. Matheny, System Engineer
M. McBrearty, Licensing Engineer
C. Mitchell, Operations Procedures
J. Osborne, Project Manager
T. Remick, Engineer, Nuclear Fuel Management
R. Sandstrom, Manager, CAP Project
A. Shean, Nuclear Oversight Manager
NRC personnel
R. Caniano, Director, Division of Reactor Safety
M. Hay, Chief, Technical Support Branch
M. Shannon, Chief, Plant Support Branch 1
LIST OF ITEMS OPENED, CLOSED, AND DISCUSSED
Opened and Closed
05000361/2010006-01 NCV Inadequate Operability Determination for Turbine-Driven
Auxiliary Feedwater Pump Steam Admission Valves
(Section 40A2.5a)05000361/2010006-02 NCV Failure to Translate Design Basis Information for Turbine-
Driven Auxiliary Feedwater Pump Steam Admission
Valves (Section 40A2.5b)05000361/2010006-03 NCV Lack of Preventive Maintenance Results in Valve Failure
and Inoperable Condensate Storage Tank
(Section 40A2.5c)05000362/2010006-04 NCV Failure to Report Conditions That Could of Prevented
Fulfillment of Safety Function (Section 40A2.5d)
A-1 Attachment
Opened and Closed
05000362/2010006-05 NCV Control Room Operators' Failure to Adhere to Conduct of
Operations Procedural Requirements (Section 40A2.5e)05000361/2010006-06 NCV Failure to Provide Adequate Procedure for Boron Dilution
Activities (Section 40A2.5f)05000361/2010006-07 NCV Failure to Establish Component Cooling Water Radiation
ivionitoring Procedures (Section 40A2.5g)05000361/2010006-08 NOV Failure to Maintain Written Procedures Covered In
05000362/2010006-08 Regulatory Guide 1.33 (Section 40A2.5h)05000361/2010006-09 NCV Failure to Establish Goals And Monitor for A(A) Auxiliary
05000362/2010006-09 FeedlJJater Trains (Section 40A2.5i)05000361/2010006-10 NCV Failure to Identify and Correct Use of Deficient
05000362/2010006-10 Relays (Section 40A2.5j )05000361/2010006-11 NCV Failure to Secure Loose Items in the Electrical Switchyard
05000362/2010006-11 (Section 40A2.5k)05000361/2010006-12 NCV Failure to Maintain Design Basis Information
05000362/2010006-12 (Section 40A2.5!)05000361/2010006-13 FIN Failure to Meet Action Plan for Substantive Crosscutting
05000362/2010006-13 Issues (Section 40A2.5m)
Discussed
None
LIST OF DOCUMENTS REVIEWED
NUCLEAR NOTIFICATIONS
051001450 200000500 200002210 200002831 200003235
200005532 200005669 200006247 200006366 200006369
200006446 200038227 200047962 200047966 200051692
200052533 200057409 200057494 200057495 200059017
200059581 200060319 200062659 200063244 200080798
200081823 200085457 200095432 200096864 200105838
200112302 200114904 200145364 200146292 200149442
200161642 200166828 200173442 200177549 200177574
200179975 200182897 200184754 200184925 200185228
200187140 200187174 200187386 200188818 200188819
200188863 200189008 200191575 200191643 200191644
A-2 Attachment
NUCLEAR NOTIFICATIONS
200191645 200193463 200194565 200196248 200198876
200199177 200199779 200199803 200200494 200200611
200202392 200202393 200204501 200207687 200209764
200210468 200214923 200216417 200216513 200216785
200217658 200220855 200220901 200224995 200226676
200226851 200229880 200231408 200232002 200237510
200240476 200243930 200244824 200244829 200245222
200249395 200253140 200253911 200253923 200256206
200256262 200258836 200262707 200273137 200281150
200283647 200289984 200301597 200304171 200305694
200309516 200310250 200318226 200319240 200321468
200323460 200323662 200327156 200329766 200337121
200339686 200346192 200347912 200348622 200348676
200350707 200351309 200353559 200353830 200354725
200356209 200357930 200358255 200360012 200362207
200362248 200366460 200375226 200375263 200375271
200375476 200378003 200378783 200383586 200383717
200385686 200385833 200388215 200388299 200389219
200389465 200389602 200391307 200396072 200396074
200396078 200396106 200397538 200402044 200402733
200403327 200403903 200403904 200403907 200403931
200403942 200404016 200407083 200407263 200407581
200408677 200408745 200411720 200413389 200413417
200414063 200414385 200416902 200417206 200420952
200423048 200424908 200425771 200427466 200427700
200439005 200442871 200445728 200449046 200450694
200453351 200454549 200454708 200454875 200456738
200457151 200458808 200461737 200462842 200463613
200469510 200476904 200481911 200493704 200495283
200496192 200498067 200498776 200501125 200505402
200507991 200509834 200511477 200514597 200518579
200545007 200545500 200550606 200550985 200553431
200554449 200554503 200554762 200556120 200559128
200564587 200569111 200572704 200581670 200585309
200591743 200596242 200596804 200599691 200599743
200600926 200604461 200607694 200611851 200613666
200613716 200614081 200625389 200628825 200631222
200631367 200635119 200636471 200636549 200638562
200638824 200640096 200647126 200656309 200657895
200663614 200663620 200663692 200664434 200666345
200666345 200666778 200667666 200668488 200670338
200683591 200684138 200685073 200688490 200688648
200689102 200689282 200689526 200689551 200689650
"'U("".""\.",,,...,...
')()()aC()A()O
&-vvv.;../v-rvu
....,f'\n~f'\no-,n-
&:.VVV;::JVOIO
1""\ 1"\
.c.UUO::1U::1UU 20069097-i 200691209
200691226 200691370 200691516 200692319 200692334
200692335 200692347 200692815 200692819 200694409
A-3 Attachment
NUCLEAR NOTIFICATIONS
200698869 200699499 200703718 200703793 200704636
200704875 200710313 200711245 200711324 200711339
200711991 200712412 200715724 200718801 200721702
200722117 200727789 200728270 200728441 200737719
200743785 200743785 200745033 200746950 200752137
200760309 200761459 200769308 200778595 200778598
200780929 200781022 200791845 200792682 200801929
200803364 200804931 200805827 200809842 200814132
200832315 200834923 200835619 200836042 200841643
200847163 200848923 200853352 200858260 200866485
200866488 200866490 200867104 200870138 200871526
200871527 200874078 200876130 200876216 200877698
200877796 200877799 200877834 200880374 200882433
200886035 200887746 200887995 200888919
ORDERS
800011270 800049251 800073513 800073728 800076896
800076907 800081649 800164561 800183273 800185541
800192268 800216674 800216676 800216677 800216678
800269843 800275473 800289258 800314547 800354225
CONDITION REPORTS/OTHER
AR 020201440 AR 020201440 AR 020201440 AR 020801305 AR 020801305
AR 020801305 AR 020801305 AR 030100348 AR 030100348 AR 030100348
AR 030100348 AR 030401460 AR 030401460 AR 030401460 AR 041200133
AR 041200133 AR 041200133 AR 050401537 AR 050401537 AR 050401537
AR 060300521 AR 060300521 AR 060300521 AR 060301666 AR 060301666
AR 060301666 AR 061200817 AR 061200817 AR 061200817 AR 070500851
AR 070500851 AR 070500851 AR 070700345 AR 070700345 AR 070700345
AR 070700366 AR 070700366 AR 070700366 AR 070800283 AR 070800283
AR 070800283 AR 070800284 AR 070800284 AR 070800284 AR 070800285
AR 070800285 AR 070800285 AR 070800286 AR 070800286 AR 070800286
AR 070800287 AR 070800287 AR 070800287 AR 070800288 AR 070800288
AR 070800288 AR 070800289 AR 070800289 AR 070800289 AR 070800993
AR 070800993 AR 070800993 AR 071000901 AR 071000901 AR 071000901
AR 071200416 AR 071200416 AR 071200416 AR 071201393 AR 071201393
AR 071201393 AR 071201417 AR 071201417 AR 071201417 AR 080101417
AR 080101417 AR 080101417 AR 080200546 AR 080200546 AR 080200546
AR 080300666 AR 080300666 AR 080300666 AR 080301122 AR 080301122
AR 080301122 AR 080301404 AR 080301404 AR 080301404 AR 080400545
AR 080400545 AR 080400545 AR 080401137 AR 080401137 AR 080401137
AR 080401137 AR 080401144 AR 080401144 AR
- ....
()~()L!()11AA
..... _v lV f I,-r
I\DI"IOI"IAI"I-1-1AA
I\I'\. uuv*"tV I f"t
AR 080500972 AR 080500972 AR 080500972 AR 080600104 AR 080600104
AR 080600104 AR 080600212 AR 080600212 AR 080600212 RCE 93-004
A-4 Attachment
ENGINEERING DOCUMENTS
NECP 800071431 NECP 800071494 NECP 800071495 NECP 800071764
NECP 800071869 NECP 800074314 NECP 800074316 NECP 800074486
NECP 800129634
MAINTENANCE ORDER
06101428 0412153600 05101896000
PROCEDURES
NUMBER REVISION / DATE
Replacement of Foxboro CVCS Boric Acid Makeup 0
System Controls with Ovation Distributed Control
System (DCS)
Shutdown Nuclear Safety 24
2-10-010 Operating Instruction Attachment 7 Boron Saturating January 26, 2010
2(3)ME-074, CVCS Ion Exchanger
A610 Operation of Manual (Gearbox) Butterfly Valves 21
Attachment 29
LCS 3.3.108 Vibration and Loose-Parts Monitoring System
M-0050-017 BTB RSB 5-1 Condensate Inventory Calculation 4
N/A SONGS System Health Report, 4KVS 4th Quarter, 2009
SCES-004-08 Corrective Action Program Audit May 16, 2008
SCES-012-09 Equipment Reliability Audit October 17, 2009
SCES-014-09 Corrective Action and Self-Assessment Program March 5, 2010
Audit
SD-S023-110 220 kV Switch yard System 19
SD-S023-120 6.9 kV, 4.16 kV, and 480 V Electrical Distribution 19
Systems
S0123-0-A6 Operations Division Procedure (Precautions) 8
S0123-!-1.28.1 Electric Distribution Grounding Guide
A-5 Attachment
PROCEDURES
NUMBER REVISION I DATE
S0123-0-A6 Operations Division Procedure (Precautions) 8
S0123-1-1.28.1 Electric Distribution Grounding Guide 4
S0123-1-1.3 Work Activity Guidelines 26
S0123-1-1.3 Work Activity Guidelines 26
S0123-1-1.34 Scaffolding Erection 27
S0123-1-4.13 Megger Testing 6
S0123-1-9.9 Square "0" and Westinghouse Type OS Circuit 4
Breakers Inspection and Testing
SO 123-11-9.48 Magnetrol and Other Miscellaneous Liquid Level 6
Switches Calibration
S0123-MA-1 Maintenance and Construction Services Division 7
S0123-MA-1 Maintenance and Construction Services Division 7
S0123-0R-1 Operating Experience Program 9
S0123-RX-1 Reactivity Management Program 4
S0123-VI-1 Review/Approval Process for Orders, Procedures, 28
and Instructions
S0123-XV-1.20 Seismic Controls o
S0123-XV-109 Procedure and Instruction Format and Content 1
S0123-XV-109.1 Procedure Action Request Committee (PARC)
Process
S0123-XV-3.3 NRC Reporting Requirements and Assessments 15 EC 15-1
S0123-XV-303 Closure Review Process o
S0123-XV-50 Corrective Action Program 15
S0123-XV-50 Corrective Action Program 16
Functionality Assessments and Operabiiity 14
Determinations
A-6 Attachment
PROCEDURES
NUMBER TITLE REVISION I DATE
S0123-XV-91 Reactivity Management Implementation 4
SO 123-XV-H U-1 Human Performance Program 6
S0123-XV-HU-4 Human Performance Roles and Responsibilities 1
S0123-XX-11 Switchyard Work Performance 2
S0123-XX-11 Switch yard Work Performance 2
S0123-XX-6 Operator Work Around Program 7
S0123-XXIV-5.1 Engineering & Technical Services Software Quality 6
Assurance
S0123-XXX-3.5 Evaluation and Reporting of Problems to the NRC 3
Pursuant to 10 CFR Part 21
S0123-XXX-3.5 Evaluation and Reporting of Problems to the NRC 3
Pursuant to 10 CFR Part 21
S0123-XXXVI-1 Nuclear Fuel Management (NFM) Quality Program 6
S023-10-9 Turbine Lube Oil System Operation 19
S023-13-8 Severe Weather 8
S023-13-8 Severe Weather 8
S023-15-53.A CIRC Water Box Cathodic Protection Sys Trouble 20
S023-15-53B Condensate Pump P050 Flow Lo 18
S023-15-63.o Annunciator Panel 63D, Switchyard/Penetration 12
Switchgear
S023-15-63.E Annunciator Panel 63E, Switchyard 8
S023-3-2.4 Operating Instruction Attachment 7 Boron Saturating 21
2(3)ME-074, CVCS Ion Exchanger
S023-3-2.4 Operating Instruction Attachment 7 Boron Saturating 22
2(3)ME-074, CVCS Ion Exchanger
S023-3-2.6 Shutdown Cooling System Operation 26
A-7 Attachment
PROCEDURES
NUMBER TITLE REVISION I DATE
S023-3-3.30A Main Steam System Online Valve Test 12
S023-3-3.S Safety Injection Monthly Tests 25
S023-5-1.3 Plant Startup from Cold Shutdown to Hot Standby 35
S023-6-25 Generator Stator Cooling Water System Operation 23
S023-6-30 230kV Switch yard Rounds and Inspections 26
S023-6-30 Switch yard Inspection and Operation 26
S023-6-30 Switchyard Inspection and Operation 27
Main and Auxiliary Transformer Operation 20
S023-6-6 Reserve Auxiliary Transformer Operation 15
S023-IV-6.3.2 Security Intrusion Detection System Probability 7
Testing
S023-V-16 Emergency Core Cooling System (ECCS) Piping Gas 0
Void Calculation
S023-V-2.14 Thermal Inspection of Plant Components 9
S023-V-2.14 Thermal Inspection of Plant Components 9
S023-V-4AO Electrical Equipment Monitoring Program 4
S023-V-4.40 Electrical Equipment Monitoring Program 3 TCN 3-1
S023-V-4AO Electrical Equipment Monitoring Program 3
S023-XV-2 Troubleshooting Plant Equipment and Systems 6
S023-XV-50.CAP-1 Writing Nuclear Notification for Problem Identification 3
and Resolution
S023-XV-50.CAP-2 SONGS Nuclear Notification Screening 5
S023-XV-50.CAP-4 Implementing Corrective Actions 3
S023-XV-S5 Boric Acid Corrosion Control Program (BACCP) Top 5
Risk Significant Systems
A-S Attachment
PROCEDURES
NUMBER TITLE REVISION I DATE
S023-XX-10 Maintenance Rule Risk Management Program 3
S023-XX-10 Maintenance Rule Risk Management Program 3
Implementation
On-Line itVork Management Process 3
S023-XX-29.1 Seasonal Readiness 0
S023-XX-30 Nuclear Maintenance Order (NMO) Generation, 1
Screening, and Classification
S023-XX-30 Nuclear Maintenance Order (NMO) Generation, 2
Screening and Classification
S023-XX-8 Integrated Risk Management 5
S023-XXXVI-1.4 Documentation of Reload Fuel Cycle Analysis 6
SOS-025-09 Surveillance Report: Corrective Action Program May 26, 2009
Implementation
SOS-040-09 Surveillance Report: Station Integrated Business October 09, 2009
Plan Closure Review Process
SY -S023-G-2 Systems Engineering Handbook 4
SY-S023-G-2 Systems Engineering Handbook 3
SY-S023-G-2 Walkdown Standard-Stainless Steel Schedule 10S 4
Pipe
TM-2791A SONGS Air Management From RWST and CES July 2008
Licensee Meetings Attended
Production/Ops Focus Meeting (2)
Closure Review Board (CRB) (3)
Corrective Action Review Board (CARB) (2)
SONGS Switchyard Oversight Committee (SSOC) (1)
A-9 Attachment
OTHER MISCELLANEOUS DOCUMENTS
TITLE
Leadership Engagement Trending System Engagement Summary November 2009 to
Report March 2010
Unit 2/3 Operations Leadership Observation RAA-Plant Monitor 1st Quarter, 2010
Reactivity Affecting Activiiy LOP 14
Fundamental LOP 14 RAA-Plant Monitor Reactivity Affecting Activity April 6, 2010
From 10/01/2009-10/31/2009
Monthly Meeting Reactivity Oversight Group (ROG) March 30, 2010
ROG Report-Meeting 03/30/2010 March 30, 2010
List of Operator Workarounds July 2009 to
November 2009
List of Operator Burdens December 2009 to
January 2010
List of Control Room Issues July 2009 to
January 2010
List of Temp Mods October 2009 to
March 2010
List of Control Room Deficiencies April 2009 to
February 2010
SONGS Operational Focus Index April 6, 2010
Operations Division Corrective Action Burndown Plan November 6,2009
Leadership Engagement Trending System 10.14 RAA-PLANT Monitor February 2010
Reactivity Affecting Activity (LOP 14)
Impaired Alarm Record Unit 2
Priority 1 HPSI Pump Control Circuits (Implementation of ECP that June 5, 2009
corrects problem)
Top Ten Equipment Issues at SONGS
Leadership Engagement Trending System 10.19 Verification Practices
(LOP 19)
A-10 Attachment
OTHER MISCELLANEOUS DOCUMENTS
TITLE
Site Plan Status Control Misposition Events
Mission Times for Operability Determinations Rev A
February 24, 2010
March 2010
Gas Void Trend in 8" LPSI Header (U2 Loop 2A) March 2009 to
September 2009
Root Cause Directed Maintenance Evaluation dated September 1995 "RCDM 95-02 "Design
Life of Normally Energized Agastat E700017000 Series Time Delay Relays" lAW NA TS No.
9509010". (This study found that the relays have a design life much longer than the 10 years
specified by the manufacturer and that the actual design life was greater than 40 years)
CALCULATIONS
NUMBER SUBJECT
M-0013-005 Safety Injection Tank Fluid Nitrogen Evolution
ACTION REQUESTS
050100895 050101779 050200370 050200417 050201676
050300921 050600035 050600474 050800143 050800896
051101380 051200838 051200895 060100358 060100673
060101480 060101481 060200732 060300521 060401277
060501595 060600040 060700177 060700430 060701103
060701128 060800908 060800909 060800962 060900185
060900824 060900981 061000969 061001517 061001528
061100693 061101379 061101448 061101460 061101464
061101478 061101632 070300300 070300991 070301057
070500395 070500760 070801108 071100540 071100697
071101406 071101426 071101427 071101428 071101429
071101431 071101432 071200061 071200215 071200331
071200546 071200614 071200621 071200927 071201038
071201039 071201169 071201203 071201205 071201245
071201468 071201558 071201814 080100008 080100688
080100844 080200815 080200903 080201125 080300231
080300994 080400273 080400955 080500248 080500381
080501351 080600108 080600397
A-11 Attachment
DRAW INGS
NUMBER TITLE REVISION
30342 Elementary Diagram Diesel Generator 2G002 Control DC System
11
30344 Elementary Diagram Diesel Generator 2G002 Excitation
14
A-12 Attachment