ML20150C954
| ML20150C954 | |
| Person / Time | |
|---|---|
| Site: | Comanche Peak  |
| Issue date: | 03/16/1988 |
| From: | Beck J TEXAS UTILITIES ELECTRIC CO. (TU ELECTRIC) |
| To: | NRC OFFICE OF ADMINISTRATION & RESOURCES MANAGEMENT (ARM) |
| References | |
| TXX-88254, NUDOCS 8803220460 | |
| Download: ML20150C954 (11) | |
Text
_ _ _ _ _ _ _ _ _ _ _
- "llll' =
Log # TXX-88254
. ~ - -
File # 10010 r
r 10068 itIELECTRIC March 16, 1988
% miam G. cournil Euaane the FrenJens U. S. fluclear Regulatory Commission Attn:
Document Control Desk Washington, D.C.
20555
SUBJECT:
COMANCHE PEAK STEAM ELECTRIC STATION (CPSES)
DOCKET NOS. 50-445 AND 50-446 DESIGN DEFICIENCIES IDENTIFIED IN THE CORRECTIVE ACTION PROGRAM REF:
Letter from W. G. Counsii to the NRC, logged TXX-88135 dated February 1, 1988 Gentlemen:
During the public meeting between the NRC staff and TV Electric on December 9, 1987, and in the letter from S. Ebneter of the NRC staff to W. G. Counsil of TV Electric dated January 22, 1988, the NRC staff requested that 10 Electric identify the nature (root causes) of the deficiencies and weaknesses that have occurred in the design process in the past.
We received an extension in submitting this report from the Office of Special Projects and have kept them informed of the status of this respense. While agreeing that generic implication evaluations are not necessary when a complete design validation is performed, the staff requested root cause identification in order to evaluate the adequacy and sufficiency of the corrective and preventive actions for current and future CPSES design activities.
Pursuant to that request and our last response to the NRC (see referenced letter), the attached assessment has been prepared.
The assessment was prepared by 10 Electric and engineering contractor personnel who have extensive experience in the engineering and design of nuclear power plants.
This experience includes knowledge of current day technical methods and industry design control and documentation practices.
The 10 Electric / engineering contractor personnel have applied their current day knowledge and expertise to identify the potential root causes.
The assessment was developed by first performing a collective evaluation of the issues addressed in Appendices A and B of the Project Status Reports (PSRs) and defining broad, programmatic categories of potential root causes based upon the TV Electric / engineering contractor personnel familiarity with the PSR issues.
The causative nature of each issue was then reviewed to identify the applicable category or categories of root causes. At the same time, each issue was reviewed to assure that its nature would not indicate a category of root cause not encompassed by those specifically identified.
Additionally, the corrective and preventive actions that had been taken were reviewed to verify that the potential root causes were adequately addressed.
28 2880ee 8888h
" " " " " " " ' ' " ' " " " - =
S2au
+
A DCD
}
I
4
'TXX-88254 March 16,1988 Page 2 of 2 In addition to this assessment, the PSR Appendices A and B identify-Significant Deficiency Analysis Reports (SDARs) and their corresponding TXX letter from TV Electric to the NRC for those issues determined to be
~
reportable under the provisions of 10CFR50.55(e).
These SDARs include a description of the deficiency, safety implications, and corrective action, and typically identify the cause for the specific issue.
The attached assessment conservatively identifies broad, programmatic root cause categories to assure that the corrective and preventive actions implemented are likewise broad and programmatic. We believe that TV Electric's corrective and preventive actions fully address the identified root causes, are being properly implemented in current design activities and will be appropriately incorporated into future design activities.
Very truly yours, W. G. Counsil By:
2 Jobh W. Beck Vice President, Nuclear Engineering JCil/grr Attachment c - Mr. R. D. Martin, Region IV Resident inspectors, CPSES (3) 2 1
Attachment to TXX-88254 March 16, 1988 Page 1 of 10 ROOT CAUSE ASSESSMENT During the public meeting between the NRC Staff and TV Electric on December 9,1987, and in the letter from S. Ebneter of the NRC Staff to W. G Counsil of TV Electric dated January 22, 1988, the NRC Staff requested that TV Electric identify the nature (root causes) of the deficiencies and weaknesses that have occurred in the design process in the past. While agreeing that generic implication evaluations are not necessary when a complete design validation is performed, the Staff requested root cause identification in order to evaluate the adequacy and sufficiency of the corrective and preventive actions for current and future design activities.
This assessment has been prepared in response to that request.
As is discussed more fully below, root causes have been identified in the following broad, programmatic categories:
o Translation of licensing commitments into design criteria, o Design document development, control and verification, o Transidion of design criteria into engineering installation and procurement documents (specifications and detail drawings).
o Interface control.
o Audits.
o Other.
The design validation portion of the TV Electri Corrective Action Program (CAP) is described in detail in several lettersi from TV Electric to the Nuclear Regulatory Commission (NRC).
As described in those letters, the design validation is a complete review of the safety-related aspects of CPSES design performed by experienced architect / engineer firms (engineering contractors) that were rot involved previously in the portions of the design they reviewed.
The design validation program was subdivided into 11 disciplines and the results reported in 11 Project Status Reports (PSRs).
Within each PSR, Appendix A summarizes the resolution of the Comanche Peak Response Team (CPRT) and external issues related to the particular discipline, and Appendix B summarizes the resolution of issues identified during the performance of the CAP design validation that were determined to be reportable under the provisions of 10CFR50.55(e).
The corrective and preventive actions established in response to each of these issues are also described in these Appendices.
l The PSR Appendices A and B issues were reviewed by TV Electric and engineering contractor personnel.
These personnel have extensive experience in the engineering and design of nuclear power plants, including current industry design control and documentation practices.
Based on this experience and a detailed understanding of the issues, they have reviewed the nature of the issues and assessed their root causes to provide a basis for evaluo+ing the adequacy of the corrective and preventive actions implemented by TV Eicctric.
Attachment to,TXX-88254 March 16, 1988 Page 2 of 10 The engineering process is com3rised of many complex and interrelated activities, including establisiment of design criteria, design control, implementation of criteria, iteration of engineering analyses, control of interfaces and interpretation of codes, standards and regulations. As a result, identification of rcot cause is a highly judgmental process.
Nevertheless, a review of the issues in Appendices A and B collectively, without regard to their significance, can provide a conservative, programmatic understanding of their root causes.
Such a collective evaluation has been performed by first defining broad programmatic categories of potential root causes based upon familiarity with the PSR issues.
The causative nature of each issue was then reviewed to identify the applicable category or categories of applicable root causes. At the same time,.each issue was reviewed to assure that its nature would not indicate a category of root cause not encompassed by those specifically identified.
Additionally, the corrective and preventive actions that had been taken were reviewed to verify that the potential evet causes were adequately addressed.
Both primary and contributory root causes have been identified.
For example, in some instances involving calculations that did not provide a complete identifiable basis for validation of the design, the primary root cause was determined to be incomplete documentation of inputs or assumptions.
In addition, it was judged possible that additional or contributory cause(s) may have existed, such as incomplete design verification, weakness in technical auditing, personnel training or incomplete design.;riteria.
Accordingly, these have been identified as root causes as well. While the PSRs identify the correc;ive and preventive actions specific to the nature of a particular issue, the cverall corrective and preventive actions implemented by TV Electric envelope all potential root causes.
This root cause assessment is based on an "engineering and design perspective" of the issues.
That is, even where an issue was caused, at least in part, by the action of another organization (e.g., Construction, Quality Control or Operations /Startup), a root cause was attributed to Engineering if it appeared that an action by Engineering might have prevented or ameliorated the issue.
For example, where such other organizations had not fully implemented design requirements, the issue was deemed attributable to the engineering documents (specifications, calculations or drawings) not specifying the pertinent engineering and design requirements with the clarity and detail expected under current practice.
Such an "engineering and design perspective" can result in overemphasis of the responsibility of engineering and design activities for some of the issues discussed in the PSRs.
However, this perspective results in the identification of potential root causes even remotely relating to engineering and design and thereby provides a conservative basis for assessing the adequacy and sufficiency of the corrective and preventive actions.
In evaluating the issues identified in the PSRs and assessing root causes, the TV Electric / engineering contractor personnel have applied their current day knowledge and expertise.
Factors such as regulatory requirements, technical methods and the inter 3retation of design control requirements have evolved significantly since t1e inception of the CPSES project.
This evolution affects perception of the issues and the formulation of corrective and preventive actions.
Attachment to TXX-88254 March 16, 1988 Page 3 of 10 for example, evolving regulatory requirements and guidelines frequently lead to changes in the understanding or interpretation of preexisting regulatory requirements or standards for regulatory acceptance, and thus to dif ferent or additional actions to satisfy existing licensing connitments.
This evolution can manifest itself in changes to the actual design of the p? ant, such as the set of electrical equipment requiring environmental qualification or the provisions for fire protection, or in the extent of as-built design documentation required.
Application of new technical methods to designs developed using earlier methods can lead to identification of apparent design inconsistencies which are more efficiently resolved by plant modification than justified through detailed analysis.
For example, in the case of pipe support design at CPSES, it was decided that the design of the pipe support system should be validated using current technical methods.
The original analyses had been performed a number of years before and the design had progressed through a number of iterations using the original designer's technical methods of that earlier time.
It would have been a complex, time-consuming task to attempt to reconstruct those original analyses.
It was judged to be not only technically correct but also more efficient to apply current technical methodology during design validation and to install any indicated modifications, it was neither necessary nor practical to detennine which modifications, if any, might be required to assure support function and which resulted merely from the use of different technical methods.
As the duration and complexity of nuclear projects have increased, the accepted design control practices and documentation of the design process have similarly expanded and become more complex.
i The approach used in the PSRs and this assessment yields root causes and a
corrective and preventive actions which are based on current accepted practice.
This assessment does not make judgments as to whether an identified issue reflected a condition that would have been regarded as a deficiency when the original design was performed.
i The following paragraphs discuss the root causes identified for the PSR i
A)pendices A and B issues using the approach and perspectives described above.
i T1e root causes are presented in broad programmatic categories and, for each category, the corrective and preventive actions are sunnarized.
1.
TRANSLATION OF LICENSING COMMITMENTS INTO DESIGN CRITERIA Design criteria comprise the basic input from which the design is implemented (e.g., through calculations and drawings) and translated into instructions (e.g., detail drawings and installation specifications) for subsequent activities such as procurement, construction and quality control inspection. Complete, well documented design criteria significantly enhance the ability to implement the design uniformly, thus assuring licensing connitments are met.
Conversely, weaknesses in the documentation of design criteria were judged to be the cause of a number of the specific issues resolved by the CAP design validation process, c
i Attachment to TXX-88254 March 16 1988 Page4of10 Examples within this category include instances where design criteria required to meet licensing commitnents were either not specifically defined, not correct or not completely documented to assure that the design uniformly complied with the licensing commitments.
In other instances, design criteria were defined and documented but with insuf ficient clarity to assure unifonn application.
In a few instances, issues resulted from design criteria having been documented in multiple locations inconsistently.
The corrective and preventive actions taken by TV Electric resolve this root cause category.
Engineering and Construction-Engineering (ECE) procedures require the development of Design Basis Documents (DBDs) specifying the design criteria necessary to assure that the design of safety related structures, systems and components complies with the licensing commitments.
For each discipline, licensing documentation was reviewed to identify applicable regulatory requirements and licensing commitments.
Design criteria were established from these requirements and commitments and consolidated in the Design Basis Documents (DBDs).
ECE procedures are in place to control the DBDs in the future and assure they remain consistent with the licensing documents.
The design was validated using these criteria, and uas redone or modified where necessary to correct or resolve specific issues, in accordance with ECE procedures, current and future engineering and design activities must be based on the criteria specified in the DBDs.
2.
DESIGN DOCUMENT DEVELOPMENT, CONTROL AND VERIFICATION Weaknesses in design document development, control and verification contributed to a number of the issues reported in the PSRs.
These issues can be grouped into a number of subcategories.
For example, some issues 4
relate to calculations which did not provide the complete basis for.a system, structure or component design.
These issues, while not necessarily resulting in deficient designs, manifested thernselves in an inability to readily validate or reproduce the results of' original calculations.
The causes in these instances generally related to inadequate or incomplete documentation of design input, assumptions, methodology or engineering judgment.
A number of PSR issues were related to the implementation of design changes.
Design changes were not always identified fully on all affected documents and thus inconsistencies existed between design documents.
Contributing to this may have been instances of design changes being made without following approved procedures thus resulting in lack of appropriate engineering review. Controls also did not assure consideration of changes for their effect on previously completed work.
That is, Engineering did not always identify the need to backfit new design requirements to previously completei construction activities or quality control inspections.
A contributory root cause related to design document development'and control is that design verification activities were not always sufficient to detect inconsistencies or errors in design documents.
Attachment ~to TXX-88254 March 16, 1988
=Page 5 of 10 The corrective and preventive actions taken by TV Electric resolve this root cause category.
TU Electric's Comanche Peak Engineering (CPE) thoroughly reviewed and revised the ECE procedures not only to assure consistency with the requirements of 10CfR50 Appendix B and ANSI N45.2.ll, "Quality Assurance Requiiements for the Design of Nuclear Power Plants",
but also to enhance their effectiveness.in assuring compliance.
These ECE procedures govern design document development, control and verification activities for both CPE personnel and engineering contractors.
Performance of the CAP design validation progran, in accordance with these procedures resolved the issues in this category.
'1 Engineering contractors performing design activities at CPSES are required by the ECE procedures to utilize the ECE procedures directly or to develop their own procedures which are consistent with the requirements of the ECE procedures. CPE's Engineering Assurance Group reviews and concurs with the engineering contractors' procadures for design and design control to assure that the essential requirements of the ECE procedures are implemented.
i The ECE procedures require that calculations identify inputs and describe and justify assumptions, methods and eng~ineering judgments... Safety-related calculations, drawings a.nd specifications are requireu to be design verified.
The issuance and documentation of safety-related calculations, drawings and specifications cre controlled.
1
.The ECE procedures clearly specify the approved methods for design change j
(Design Change Authorization-DCA, Non-Confonr.ance Report-NCR or document revision) and require that changes to design documents be reviewed,.
<erified and controlled in a manner similar to the requirements for the original design document.
They also require that the changes be transmitted to all affected organizations.
The ECE procedures require Engineering to Montify any requirements for backfit. ting construction work or quality control inspection when developing changes to design documents. Conformance of previously installed hardware to final acceptance attributes of the validated design is assured through the Post-Construction Hardware Validation Program (PCHVP).
1 l
3.
TRANSLATION OF DESIGN CRITERIA INTO ENGINEERING INSTALLATION AND PROCUREMENT 00CUMENTTT5PECIFICAT10NS AND DETAll DRAWIf(GS)
Installation specifications and detail drawings are the primary means by which Engineering transmits the derign to Construction for installation and to Quality Control (QC) for inspection.
Similarly, procurement
'l specifications are the means by which Engineering requirements are provided to equipment vendors. A number of the PSR issues involve situations in which instelled hardware did not satisfy desian requirements.
Similarly, some issues related to vendor qualification or
)
testing of procured items which did not satisfy design requirements.
From l
the viewpoint of the "engineering and design perspective" utilized in this j
)
root cause assessment, such issues were attributed to incompletely or unclearly s)ecified technical requirements or inspection acceptance i
attributes ay Engineering.
1 1
i i
I
]
Attachment to TXX-88254 March 16, 1988 Page 6 of 10 The corrective and preventive actions taken by TV Electric resolve this root cause category.
The safety-related installation specifications and detail drawings were reviewed and revised to comply with the validated design documents and to clearly and completely define the technical requirements for hardware installation at CPSES.
The installation specifications contain not only the technical requirements but also the QC inspection acceptance attributes.
The construction procedures and QC inspection procedures were also revised to be consistent with the revised installation specifications and detail drawings.
Current and future construction work and QC inspections are performed in accordance with the revised installation specifications, detail drawings, construction procedures and QC inspection procedures.
Previously procured safety-related equipment is validated during the CAP to cssure compliance with design interface requirements.
The present and future procurement of equipment is controlled by revised ECE procedures which assure compliance with the design requirements, including required qualification and testing.
These ECE procedures identify the required technical interface between CPE/ engineering contractors and equipment vendors. This interface control assures that the technical requirements identified in procurement documents are understood and complied with by equipment vendors.
The ECE procedures also identify technical reviews which Engineering must perform to assore that supplied equipment complies with design requirements.
For example, Engineering is required to review the documentation of testing or analysis supplied by equipment vendors to assure that equipment is environmentally or seismically qualified.
4.
INTERFACE C0f1 TROL Some of the PSR issues were attributed to the fact that interface control did not always prevent inconsistencies from occurring in design or in design change activities.
It appeared that Engineering did not always assure that design documents and design changes were transmitted to and fully reviewed by all af fected organizations.
)
The corrective and preventive actions taken by TV Electric resolve this j
category of root cause.
A revised ECE procedure specifically requires engineering contractors and the NSSS vendor to develop procedures to define and control their interfaces.
In addition, revised ECE procedures require that safety-related design documents and changes (e.g., DCAs and applicable NCRs) receive an interdisciplinary review prior to approval to assure proper consideration of the impact of the design or design change on design activities of interfacing engineering disciplines / groups.
Revised ECE procedures also provide for appropriate interfaces between Engineering and other TV Electric organizations.
For example, they sequire that Quality Control review and concur with all changes to the installation specifications.
This review and concurrence identifies required QC inspection procedure revisions and QC inspector training.
Installation specification changes (DCAs or NCRs) are transmitted to
m.
Attachment to.TXX-88?54 March-16, 1988 Page 7 of 10 Construction, which detennines if changes are required to its procedures to assure compliance with Engineering requirements.
Procedures require that Startup review changes to design documents that are required to develop system and equipment test procedures and perform the test.
Engineering reviews test procedures prior to test performance and test -
results for acceptability.
ECE procedures require that Operations review changes to specifications and design changes which affect system operation.
5.
AUDITS Although it was not necessary to review the adequacy of audits of the original engineering and design for purposes of conducting the design validation program, it is recognized that a contributory root cause for some PSR issues may have been a weakness in the emphasis on technical,1y oriented, performance based audits by TV Electric Quality Assurance.or the original engineering organization's Quality Assurance group.
That is, while audits do not cause problems, it is possible that more technically oriented audits might have detected and corrected some issues.
The corrective and preventive actions taken by TV Electric resolve this root cause category.
10 Electric Quality Assurance audits (as part of its Technical Audit Program-TAP) performs technical and programmatic audils of the engineering contractors.
TV Electric Quality Assurance performs programmatic audits of CPE activities and will' also perform-technical audits when CPE assumes responsibility for safety-related design j
activities. The CPE Engineering Assurance group performs technical and programmatic surveillances of CPE and engineering contractor design activities to assure that the requirements of the ECE procedures for d: sign and design control are satisfied and that the design documentation is technically correct.- The implementation of i.CE procedures for design and design control is aswred by audits and surveillances.
6.
OTHER l
Several root causes were identified that are not included in the above categories.
Some instances may be attributable to the training of engineering personnel or to isolated engineering errors.
In addition, f
j there was a limited number of instances which even with the "engineering and design perspective" did not have a root cause related to engineering at CPSES.
The corrective and preventive actions taken by TV Electric resolve these other root causes.
The CPE Engineering Assurance organization is responsible for training CPE personnel to the ECE design and design control procedures. Records of this training are maintained.
In addition, the engineering contractors are required to train their j
personnel to their design and design control procedures and to maintain
]
records of the training.
TV Electric and the engineering :ontractors have requirements to confirm the qualifications of their engineers.
4 i
I
Attachcent to TXX-88254 March 16, 1988 Page 8 of 10 The results of technical and programmatic audits and surveillances 3
perfonned by TV Electric Quality Assurance and CPE Engineering Assurance I
are trended to, identify recurring deficiencies or errors.
When appropriate, changes to'the design process are identified and implemented t.o prevent similar types of deficiencies or errors.
The corrective and preventive actions for issues that did not have an engineering root cause are described in the PSRs.
The root causes identified in this assessment have been compared to the following potential root causgs related to design identified by the Senior Review leam (SRT) in a lettere to TV Electric in February 1987:
1.
Design Commitment Control 2.
Code Compliance Procedures 3.
Organizational Interface Procedures (ii,ternal and external) 4.
Design Change Control Procedures 5.
Design Verification Procedures 6.
Procedure implementation 7.
Technical Audits of Design Control 8.
Technical Audit Corrective Actions 9.
Training and Personnel Qualification As shown in Table 1, although the characterizations differ slightly, the root causes identified by this assessment encompass those preliminarily identified by the SRT.
As described above, a conservative root cause assessment has been completed, from an engineering and design perspective, for the issues presented in the 11 PSRs.
The results, similar to potential root causes for design issues suggested by the SRT, are presented in broad programmatic categories.
This assures that corrective and preventive actions are likewise considered in a broad programmatic context.
Extensive programmatic corrective and preventive actions have been implemented by 10 Electric and their ef fectiveness demonstrated in the perfonnance of the CAP.
The effectiveness of these actions is further assured by the detailed audit, surveillance, review and overview activities applicable to that program. These activities include the audits and/or surveillances performed by TV Electric Quality Assurance (QA), Comanche Peak Engineering (CPE)
Engineering Assurance, and the QA organizations of the engineering contractors; as well as the reviews and/or overviews performed by the Comanche Peak Response Team (CPRT), the TV-Electric Quality Assurance Technical Audit Program (TAP), the Engineering Functional Evaluation (EFE) and Cygna.
The improvements demonstrated in current design activities will continue to be utilized for future design activities.
Accordingly, TV Electric's corrective and preventive actions fully address the identified root causes, are being properly implemented in current design activities and will be appropriately incorporated into future design activities.
Attachment to.TXX-88254 March 16, 1988 Page 9 of 10 TABLE 1 COMPARISON OF ROOT CAUSE ASSESSMENT 10 SRT IDENTIFIED ROOT CAUSES CAP DEFINED ROOT CAUSES*
SRT IDENTIF!ED ROOT _CAUSES 1-2 3
4 5
6 Design Commitment Control x
x Code Compliance Procedures x
x Organizational Interface Procedures x
Design Change Control Procedures x
Desigri Verification Procedures x
Procedure implementation x
x x
Technical Audit of Design Control x
Technical Audit Corrective Actions x
Training and Personnel Qualification x
- Root cause categories:
1.
Translation of licensing commitments into design criteria.
2.
Design document development, control and verification.
3.
Translation of design criteria into engineering installation and procurement documents (specification; and detail drawings).
4.
Interface control.
5.
Audits.
6.
Other.
l l
A 4
- )-
Attachment to.TXX-88254 March 16,1988 Page 10 of 10
-REFERENCES ~
i 1.
Letter No. TXX-6500, "Response to Request for Additional'information in Conjunction with Program Plan' Update," dated June,25, 1987, W. G. Counsil to V. S. Nuclear Regulatory' Commission.
~
Letter No. TXX-6631, "Comanche Peak," dated August 20, 1987, W. G. Counsil
-c, to U. S. Nuclear Regulatory Commission, o Letter No. TXX-6712 "Post Construction Hardware Validation Program (PCHVP) Engineering Evaluation Methodology,". dated September 8,1987, W. G. Counsil te,'J. S. Nuclear Regulatory Commission.
a Letter No. TXX-6676, "Technical Audit Program and Engineering functional-Evaluation," dated September 8,1987,- W.< G. Counsil to V. S. Nuclear r
Regulatory Commission.
Letter No. TXX-6961, "Comanche Peak Programs Description Clarification,"
dated November 25, 1987, from W. G. Counsil to U. S. Nuclear Regulatory Commission.
2.
Letter J. W. Beck to W. G. Counsil, dated February 4,1987.
-1
- t t
k e
u rw y w
y y---e-
- m 1r-
,i-y-
y y