0CAN111407, Supplemental Reply to Notice of Violation EA-14-008
| ML14324A783 | |
| Person / Time | |
|---|---|
| Site: | Arkansas Nuclear  |
| Issue date: | 11/20/2014 |
| From: | David Bice Entergy Operations |
| To: | Document Control Desk, NRC/NMSS/SFPO |
| References | |
| 0CAN111407, EA-14-008 | |
| Download: ML14324A783 (10) | |
Text
0CAN111407 November 20, 2014 ATTN: Document Control Desk Director, Spent Fuel Project Office Office of Nuclear Material Safety and Safeguards U.S. Nuclear Regulatory Commission Washington, DC 20555-0001
SUBJECT:
Supplemental Reply to Notice of Violation EA-14-008 Arkansas Nuclear One - Units 1 and 2 Docket Nos. 50-313 and 50-368 License Nos. DPR-51 and NPF-6
REFERENCES:
- 1.
NRC letter dated June 23, 2014, Final Significance Determination of Two Yellow Findings and Notice of Violation, NRC Inspection Report 05000313/2014008 and 05000368/2014008 (0CNA061403)
- 2.
Entergy letter dated July 18, 2014, Reply to Notice of Violation; EA-14-008 (0CAN071402) (ML14199A412)
Dear Sir or Madam:
In Entergy Operations, Inc. (Entergy) letter dated July 18, 2014 (Reference 2), a response was provided to the violation transmitted in Reference 1. This response was based on the root cause investigation conducted associated with Arkansas Nuclear One (ANO) condition report CR-ANO-C-2013-00888. This condition report examined the cause of the temporary lift assembly failure that occurred at ANO the morning of March 31, 2013.
While the conclusions in this root cause were valid and applicable to the violation response, the root cause evaluation was primarily focused on the cause of the temporary lift assembly failure as opposed to ANOs adherence to the procedural guidance associated with the review of design and testing information related to temporary lift assemblies, which was the basis for the NRC violation. To further evaluate the cause of the lack of adherence to the applicable procedure guidance a second root cause evaluation was performed (CR-ANO-C-2014-02318).
The purpose of this letter is to supplement the Reference 2 response to include additional insights gained from this second root cause evaluation, including additional corrective actions.
The attached information supplements the violation response provided in Reference 2.
Entergy Operations, Inc.
1448 S.R. 333 Russellville, AR 72802 Tel 479-858-4704 David B. Bice Acting Manager, Regulatory Assurance Arkansas Nuclear One
0CAN111407 Page 2 of 3 This letter contains no new commitments.
Should you have any questions concerning this submittal, please contact me.
Sincerely, ORIGINAL SIGNED BY DAVID B BICE DBB/dej
Attachment:
Supplemental Reply to Notice of Violation EA-14-008 cc:
Mr. Marc L. Dapas Regional Administrator U. S. Nuclear Regulatory Commission Region IV 1600 East Lamar Boulevard Arlington, TX 76011-4511 NRC Senior Resident Inspector Arkansas Nuclear One P. O. Box 310 London, AR 72847
Attachment to 0CAN111407 Supplemental Reply to Notice of Violation EA-14-008
Attachment to 0CAN111407 Page 1 of 7 Supplemental Reply to Notice of Violation EA-14-008
=
Background===
In the notice of violation (EA-14-008), the NRC cited Arkansas Nuclear One (ANO) for the following violation of10 CFR Part 50, Appendix B, Criterion V, Instructions, Procedures and Drawings. The notice stated, in part:
10 CFR Part 50, Appendix B, Criterion V, Instructions, Procedures and Drawings states, in part, that activities affecting quality shall be prescribed by documented instructions, procedures, or drawings of a type appropriate to the circumstances and shall be accomplished in accordance with these instructions, procedures, or drawings.
Quality Procedure EN-MA-119, Material Handling Program, Section 5.2[7], Temporary Hoisting Assemblies, Step (a) states, in part, that vendor supplied temporary overhead cranes or supports, winch-driven hoisting or swing equipment, and other assemblies are required to be designed or approved by engineering support personnel. The design is required to be supported by detailed drawings, specifications, evaluations, and/or certifications.
Quality Procedure EN-MA-119, Material Handling Program, Section 5.2[7], "Temporary Hoisting Assemblies," Step (b) states, in part, that the assembly shall be designed for at least 125 percent of the projected hook load and should be load tested and held for at least 5 minutes at 125 percent of the actual load rating before initial use. The assembly shall be load tested in all configurations for which it will be used.
Contrary to the above, on March 31, 2013, ANO did not accomplish the Unit 1 main turbine generator stator lift and move, an activity affecting quality, as prescribed by documented instructions and procedures. Specifically:
A.
The licensee approved a design for the temporary hoisting assembly that was not supported by detailed drawings, specifications, evaluations, and/or certifications. The licensee failed to identify the load deficiencies in vendor Calculation 27619-C1, "Heavy Lift Gantry Calculation," and the incorrectly sized component in the north tower structure of the temporary hoisting assembly. In addition, the temporary hoisting assembly was not designed for at least 125 percent of the projected hook load.
B.
The licensee failed to perform a load test in all configurations for which the temporary hoisting assembly would be used.
As a result, on March 31, 2013, while lifting and transferring the Unit 1 main turbine generator stator, the temporary overhead crane collapsed causing the 525 ton stator to fall on and extensively damage portions of the plant, affecting safety-related equipment.
The original root cause report (CR-ANO-C-2013-00888) identified the direct cause of the temporary lift assembly failure to be the buckling of the northwest tower assembly lower column.
The root cause of the temporary lift assembly collapse was determined to be that the subcontractor did not ensure the lift assembly north tower was designed adequately to support the loads anticipated for the lift. A second root cause was the subcontractors failure to perform
Attachment to 0CAN111407 Page 2 of 7 a required load test of the modified temporary lift assembly prior to use at ANO in accordance with Occupational Safety and Health Administration (OSHA) regulations. The following four contributing causes were also identified during the original root cause investigation: 1) the contractor and subcontractors inaccurate representation that the temporary lift assembly had been used at other electric power stations to lift components that exceeded the anticipated weight of the ANO, Unit 1 (ANO-1) stator, the safe lifting capacity of the assembly, and their compliance with Entergy Operations, Inc. (Entergy) procedure EN-MA-119, 2) the contractor failed to provide proper oversight of the subcontractors performance, 3) Entergys material handling procedure EN-MA-119 did not provide clear guidance regarding independent reviews of special lift equipment, and 4) supplemental Project personnel lacked sufficient knowledge of OSHA and ASME NQA-1 code application to temporary lift assemblies and accepted the subcontractors assertion that load testing was not required based on a combination of engineering analysis and previous use.
Entergys initial response to the violation was based on the facts and conclusions identified in CR-ANO-C-2013-00888, particularly the third and fourth contributing causes listed above.
Among other things, the response noted that the procedure section that contained the steps referenced in the violation was modified by a NOTE which permitted: Specially designed lifting devices, for specific application, may be designed and tested to other approved standards. Per Entergy Procedure Writer Manual (EN-AD-101-1), NOTE statements apply to the step/paragraph/section directly following the note. Relying on this note, Entergy supplemental personnel accepted the sub-contractors professional engineer (PE) certification of the adequacy of the lift rig design for the stator lift, including certification (subsequently determined to be inaccurate) that the hoist assembly had been used at other electric power stations to lift components that exceed the weight of the Unit 1 stator.
To address these deficiencies, as described in Reference 2, Entergy has completed a number of corrective actions. For example, procedure EN-MA-119 has been revised to require substantial review and documentation associated with the use of temporary lifting devices. The guidance in Section 4.0[8](f) of EN-DC-114, Project Management, was revised to include in the consideration of project team composition experience necessary to assess adequacy of associated decisions related to high consequence evolutions. The standards for establishing vendor oversight plans were reinforced to the Strategic Capital Project Managers. Additionally, a vendor oversight What It Looks Like (WILL) sheet was developed for use during one cycle period to monitor and provide feedback to ensure Project Managers are rigorously implementing the vendor oversight process.
Supplemental Information To gain further insights into why Entergy accepted the subcontractors PE certification in lieu of a load test and why Engineering Support Personnel failed to perform an adequate review of the subcontractors design as required by EN-MA-119, a second root cause evaluation has been performed. The second root cause (CR-ANO-C-2014-02318) examined why the defective design prepared by the lift assembly vendor and lack of a load test were not recognized as deficient by station personnel. The failure to recognize the vendor deficient design and lack of load test did not cause the vendor failures that were determined to be the root causes of the lift assembly collapse; however, this failure was determined to have contributed. This second root cause evaluation is the basis for the supplemental information provided below in response to the violation.
Attachment to 0CAN111407 Page 3 of 7 (1)
The reason for the violation (supplement)
This second root cause evaluation identified two root causes for the violation cited by the NRC.
The first root cause was that the Stator Rewind Project was not organized or managed in a manner that provided sufficient oversight of the vendors design and testing for the temporary lift assembly. This root cause resulted in insufficient supervision and technical oversight of personnel assigned to evaluate and monitor the sub-contractors work, and insufficient engagement with Engineering and the Material Handling process owner to ensure detection of flaws in the sub-contractors design and prior-use certification.
The second root cause was that procedure EN-DC-114, Project Management, provided insufficient guidance to identify and manage risk items with high consequence, particularly for cases where the probability of the event is judged to be very low. This root cause resulted in failure to identify the level of risk associated with structural design of the stator lift assembly, and, consequently, failure to identify an appropriate risk mitigation strategy.
Four contributing causes of the violation were also identified during the second root cause evaluation:
- 1.
Weak implementation of administrative controls applicable to the stator refurbishment project. This cause contributed to the failure to adequately implement a number of administrative requirements, which resulted in weaknesses in oversight and risk management.
- 2.
EN-MA-119, Material Handling Program, did not provide clear guidance regarding the level of review required to approve the design and testing of vendor-supplied special lift equipment, including how an alternate standard should be identified and approved for use.
- 3.
ANO placed undue confidence in the primary contractor and sub-contractors capabilities. This cause contributed to responsible ANO personnel perceiving the risk of structural failure of the lift assembly as low because an expert vendor had certified the assembly, similar lifts had been made before, and the vendor asserted that the lift assembly had been used for heavier lifts in the past. This cause contributed to a lack of verification of vendor representations by supplemental personnel, leading ANO personnel to believe compliance with EN-MA-119 was met, when it was not.
- 4.
The corrective action plan developed and implemented for CR-ANO-C-2012-0596, Conservative Assumptions in Decision Making (H.1.b) Substantive Cross-Cutting Issue, was ineffective in ensuring the Project Management decision to not test the temporary lift assembly was technically sound and met the requirements of EN-MA-119. This cause contributed to poor risk recognition and faulty decision-making contributing to the failure to detect the deficient vendor design and justification for not load testing.
Attachment to 0CAN111407 Page 4 of 7 (2)
The corrective steps that have been taken and the results achieved (supplement)
A number of the actions described in Entergys initial response to EA-14-008 serve to address the causes of the violation described above. In addition to those actions, the following actions have been completed to further address the causes of the violation identified in the second root cause evaluation:
A.
Entergy procedure EN-HU-104, Engineering Task Risk and Rigor, has been revised to implement portions of Institute of Nuclear Power Operations Event Report (IER)
L1 14-20, Integrated Risk - Healthy Technical Conscience. The focus of this IER includes project risks that involve activities with a potential to introduce an underlying enterprise risk to the degree increased scrutiny and technical / executive review and challenge may be required. Specifically, EN-HU-104 was revised in Revision 5 to align the procedure with IER 14-20 specifying additional actions and oversight for high consequence risks. This revision included changes to:
Strengthen and clarify process to specifically identify the type of 3rd party review required, qualification of the reviewers, and documentation of the review.
Add requirement for responsible manager to concur with risk determination.
Add requirement for station senior management to be notified when products are determined to be high consequence or risk level 1 and 2.
Revise the list of high, medium, and low risk consequence activities.
Add a requirement to retain completed attachments for risk rank determination.
B.
Procedure EN-DC-114, Project Management, has been revised to provide guidance in specifying contract language to ensure detailed engineering calculations, quality requirements, and standards are provided for internal and third party review, in accordance with revised EN-MA-119, Material Handling Program, when specially designed temporary lift assembles are to be used.
(3)
The corrective steps that will be taken (supplement)
Entergy is taking further steps to address the root and contributing causes identified in the second root cause evaluation. These include corrective actions to prevent recurrence (CAPRs), interim actions, and additional actions to address contributing causes.
Corrective Actions to Prevent Recurrence A.
Project Management procedures EN-FAP-PM-003 and EN-FAP-PM-004 will be revised to ensure projects are organized and managed with (1) effective support by subject matter experts and (2) effective vendor and technical oversight.
Specifically, the review and approval requirements will be formalized for (1) initial project organization and staffing, and (2) changes to project staffing or reporting relationships.
Attachment to 0CAN111407 Page 5 of 7 The review and approval process shall delineate the following responsibilities for those approving the project organization and staffing or changes thereto:
- 1.
The team includes station and external Subject Matter Experts necessary to support project design, development of technical requirements, risk assessment and mitigation, and project execution.
- 2.
Personnel qualifications, experience and proficiency satisfy the qualifications and experience requirements for the project assignment.
- 3.
Supervisory span of control considers the type of work involved (craft and/or technical), qualifications and experience of those to be supervised, and experience and qualification of the supervisor. Supervisory span of control considers collateral duties of the supervisor.
- 4.
Project Team members that are not a qualified Supervisor for the discipline in question (i.e. Project Managers, Project Oversight, etc.) should not provide supervisory direction to personnel belonging to a discipline that is covered by an INPO accredited program or NRC licensed program (i.e. Engineering, Operations, etc.).
- 5.
Supervision of supplemental personnel is established in accordance with EN-MA-126.
Consistent with a graded approach, some minor changes may warrant a lower level of documented approval. For example, replacement of personnel with an equally qualified individual in the same role and reporting relationship may not require the same level of formal approval. Additionally, criteria may be defined to address short term or emergent project impacts.
Formalize the review and approval requirements for changes in project scope or approach (changes impacting the departmental interfaces associated with review, approval and/or implementation, changes in method of implementation, or changes in plant conditions assumed in the installation plan - outage, non-outage, including partial implementation plans during plant operation).
For initial project staffing and major changes in project scope or approach, line management approval shall be documented.
A graded approach based on risk may be utilized provided the process ensures the approach is comprehensive in identifying the risk (i.e. the right expertise is involved in the risk assessment).
B.
Project Management Procedures EN-FAP-PM-003 and EN-FAP-PM-004 will be revised to ensure high consequence risks are properly identified and eliminated/mitigated through a structured risk management process (Reference INPO 12-008, Excellence in Integrated Risk Management, and IER 14-20, Integrated Risk - Healthy Technical Conscience).
Attachment to 0CAN111407 Page 6 of 7 Specifically, these Project Management procedures will be revised to address the gaps as documented in the root cause evaluation associated with EN-DC-114, Revision 14, and include the following:
Guidance for considering the potential for latent risks associated with vendor technical services (e.g. design and testing activities).
Risk management guidance to specifically identify credible (though potentially low probability) items of high consequence.
Focus attention on risk management of nuclear and other safety risks, as distinct from cost and schedule risks.
Methods for involvement of engineering and line management in the formal approval and focused oversight of potential high consequence projects.
The intent of this action is to assure project risks with potentially high safety consequences are identified as early as possible and managed in a manner commensurate with the risk.
Interim Actions A.
ANO Project Management will review current and scheduled projects through 1R25 outage to identify any gaps in Team Composition or Project Oversight. Multiple considerations for this review were provided in this action.
B.
Review current active ANO projects for effective risk management to ensure credible high consequence (though potentially low probability) risk items including the potential for latent errors in vendor technical services (e.g. design and testing activities) are identified. For those identified projects, ensure credible high consequence items are formally tracked and sound plans exist to eliminate/mitigate the associated risks.
C. For each active or new project develop an interim process for the review and approve of changes in scope, project staffing, or reporting relationships consistent with the guidance outlined in the CAPRs described above.
Additional Actions A.
Issue EN-OM-126, Management and Oversight of Supplemental Personnel, (supersedes EN-MA-126) to include the following provisions:
Ensures a Supplemental Oversight Plan is developed prior to supplemental personnel commencing work.
Ensure the Supplemental Oversight Plan includes an assessment of the risk associated with the job.
Attachment to 0CAN111407 Page 7 of 7 Define responsibility for development and review of the Oversight Plan.
Define responsibility for ensuring appropriate resources are available and to provide adequate oversight of Supplemental Workers per the approved Oversight Plan.
Ensure Entergy personnel are responsible for ensuring oversight of supplemental personnel either directly or through oversight of appropriately qualified supplemental personnel responsible for the oversight.
When supplemental personnel are utilized to provide oversight of the performance of other supplemental personnel, ensure appropriate qualifications prior to performing oversight activities.
Ensure each Supplemental Worker and Supplemental Supervisor receives an expectations Briefing prior to commencement of work.
Ensure Supplemental Worker oversight is available whenever work is being conducted.
Ensure the Oversight Plan is revised if the scope of work changes.
Include in the expectations briefing for Supplemental Personnel in Professional, Technical or Supervisory roles a discussion and definition of the limitations of authority associated with their task, responsibility for contacting supervision when boundaries of authority are challenged, and expectations for risk management.
The intent of this action is to assure oversight activities and opportunities are defined, proceduralized, and managed.
B.
Multiple additional actions, such as presentations to site personnel and training, are being initiated to reinforce desired behaviors relative to oversight of supplemental personnel, Engineering roles and responsibility, use of human performance tools, strict application of procedures, validation of assumptions, involvement of others in decisions when unsure, and overconfidence in experts. Additionally, actions were issued to communicate lessons learned from the event focused on Project Management, Engineering, and station supervision.
(4)
The date when full compliance will be achieved (supplement)
Based on the actions previously implemented and described in the original violation response, ANO remains in full compliance. The additional actions outlined above will ensure projects are organized and managed with (1) effective support by subject matter experts and (2) effective vendor and technical oversight and that high consequence risks are properly identified and eliminated/mitigated through a structured risk management process outlined in project management procedures.