ML20198T104
| ML20198T104 | |
| Person / Time | |
|---|---|
| Site: | Comanche Peak  |
| Issue date: | 04/18/1986 |
| From: | Beck J TEXAS UTILITIES ELECTRIC CO. (TU ELECTRIC) |
| To: | |
| Shared Package | |
| ML20198T043 | List: |
| References | |
| PROC-860418, NUDOCS 8606110315 | |
| Download: ML20198T104 (54) | |
Text
r
'l r
e.
O COMANCHE PEAK RESPONSE TEAM RESULTS REPORT o
ISAP: VII.a.4
Title:
Audit Program and Auditor Qualification REVISION 1 O
k
'f 11 f5 I
Coordinator /
f Date Qsla YY
%Y/7 f4 Revi
~ Team Leader' Date/
/
~
K A). S_J=
+//tltL John
. Beck, Chairman CPRT-SRT Date 8606110315 060606 PDH ADOCK 05000445 A
ppR
s.
Revisient 1
Page 1 of 53 RESULTS REPORT b
ISAP VII.a.4 Audit Program and Auditor Qualification
1.0 DESCRIPTION
OF ISSUE The Comanche Peak SSER 11 Appendix P, Section 4.7. pages P-31 through P-34, describes the NRC concerns in the areas of Quality Assurance auditing and audit reporting. The concerns pertaining to the audit program and audit personnel qualification have been extracted and are presented here
" Region IVefound that TUEC's audit procedures did not comply with NRC requirements, and that the program was not implemented in accordance with procedures. The lack of an established audit program was also substantiated by Region IV.
For example, Region IV Report No. 50-445/84-32 cited TUEC for failure to establish and taplement a comprehensive system of' planned and periodic audits. Non-compliances identified were:
annual audits were not adequately addressed by audit implementation procedures; planning and staffing to perform 1983 audits were inadequate; the Westinghouse site organization performing Nuclear Steam Supply System (NSSS) engineering services was not audited by TUEC from 1977 through 1981; and audits of vendors that manufacture or fabricate parts, components, and equipment for safety-related systems were not conducted in compliance with annual or other applicable requirements dating back to August 1978.
Assessments by the Miscellaneous and Mechanical and Piping Groups concurred with the QA/QC Group that the audit frequency of vendors did not comply with ANSI N45.2.12 requirements.
Review of past administration of the audit program disclosed that during 1981 and 1982, the height of construction, the audit staff consisted of four auditors.
From 1982 to 1984, the audit staff has increased from 4 to 12. Also, on occasions, individuals participating on the audit teams were not QA auditors. As such, a potential existed to compromise their independence. The TRT reviewed the technical background, experience, and training of auditors, as well as the quality of audit reports. The TRT determined auditor staffing and qualifications to be questionable, which rendered the audit results for 1981 through 1983 potentially ineffective.
The TRT and Region IV reviewed the scope of the QA Program audited during 1983. Of approximately 650 safety-related procedures, 165 (25% overall) were audited.
In looking at quality procedures. TUEC audited 24% of TUCCO's implementing proceduren and 39% of Brown & Root procedures for a composite 32% audit rate. Although audits on a sampling basis are
~
i
' s
\\
s Rsvision:
1 o
Page 2 of 53 f'~'s RESULTS REPORT
'-')
l
\\
ISAP VII.a.4 (Cont'd)
1.0 DESCRIPTION
OF ISSUE (Cont'd) acceptable, there was no evidence that all safety-related areas were audited. The audits did not encompass all aspects of the QA Program in order to determine effectiveness.
With respect to audit corrective action follow-up, it was learned that TUEC QA had nbt been verifying that corrective action on previous audit findings was accomplished. For example, audit TCP-111, initiated to verify corrective actions on previous audit findings, was started prior to the TRT's review. TUEC. emphasized that TCP-111 be considered a " Punch List of Completion Tasks" to verify that corrective action had been implemented and not an attempt to rewrite or change
~
previous audit findings. Another specific example of ineffective follow-up action was found that pertained to a deficiency identified in audit TCP-23, performed in September, i
1981. Audit TCP-68, conducted in March, 1983', attempted to verify corrective action of TCP-23's audit finding, but logs (s
that would document the corrective action had been destroyed.
A new deficiency was written at that time and the response was
\\-
accepted, but the corrective action implementation is still unverified.
Following the Lobbin Report, the NRC performed a CAT inspection (IR 445/83-18;446/83-12, dated April 11, 1983) and included a review of the TUCCO audit program at the corporate offices.
The inspection included a review of 18 audits (conducted between 1978 and early 1983), auditor l
qualifications, audit planning and scheduling, audit reporting and follow-up, and audit program effectiveness. The report concluded that weaknesses existed in the established QA audit program which included the scheduling and frequency of audits, the lack of effective monitoring of the construction program, and the lack of effective resolution of certain audit findings. The inspection also indicated that the QA Program should have been more effective.
During the TRT's evaluation of allegations and concerns, it was observed thct the audit function did not always identify QA Program breakdowns, or if reported, effective corrective action was not instituted to prevent recurrence.
Based on its findings and observations, the TRT concludes that the QA audit.... program has had and continues to exhibit deficiencies. Over a significant period of time, recurring deficiencies include inadequate staffing during peak periods;
.. procedural and implementation inadequacies; questionable 1
.-, ~., -
r-
s Rsvision:
1 9
Page 3 of 53
/'~}
RESULTS REPORT
\\
)
\\/
ISAP VII.a.4 (Cont'd)
1.0 DESCRIPTION
OF ISSUE (Cont'd) qualifications and capabilities; incomplete assessment of the QA Program on an annual basis;
. In summation, the QA/QC Group finds the past audit... system less than adequate, and the audit
.. program at the time of the TRT review was questionable."
2.0 ACTION IDENTIFIED Evaluate the TRT findings and consider the implications of these findings on construction quality. "... examination of the potential safety implications should include, but not be limited to the creas or activities selected by the TRT."
" Address the root cause of each finding and its generic implications..."
G'i
" Address the collective afgnificance of these deficiencies..."
" Propose an action plan... that will ensure that such problems do not occur in the future."
3.0 BACKGROUND
In addition to the TRT issue and SSER 11, the NRC issued the following notice of violation (445/8432-03; 4t,6/8411-03) on February 15, 1985:
" Contrary to requirements, the following examples were identified which demonstrate the failure to establish and implement a comprehensive system of planned and periodic audits of safety-related activities as required, as noted below:
Annual audits were not adequately addressed by the audit implementation procedures.
TUCCO Procedure DQP-CS-4, Revision 0, dated August 9, 1978, only required two audits of vendors fabricating reactor coolant pressure boundary components, parts, and equipment; one audit 'of vendors f abricating engineered safeguards components, parts, and equipment; and audits of balance of plant (safety-related) as
/
required by the quality assurance manager.
x/
R1visien:
1 3
Page 4 of 53 f'~
RESULTS REPORT
\\
ISAP VII.a.4 (Cont'd)
3.0 BACKGROUND
(Cont'd)
(
TUGC0 Procedure DQP-CS-4, Revision 2, dated April 16, 1981, required only that organizations will be audited on a regularly scheduled basis.
TUGC0 Procedure DQP-CS-4, Revisions 2 and 10, did not specify auditing fr'equencies for design, procurement, construction, and' operations activities.
TUGC0 Procedure DQP'-CS-4, Revision 10, based audit requirements on Regulatory Guide 1.33, Revision 2
~
February, 1978. This commitment did not fully address the requirements of the Construction Quality Assurance Program.
The above procedure and subsequent revisions failed to describe and require annual audits in accordance with commitments and requirements. Earlier audit procedures were J
not available to determine if they met requirements.
Planning and staffing to perform 1983 audits was inadequate to assure that a comprehensive system of audits was established and implemented to verify compliance with all aspects of the Quality Assurance Program, in that, of 656 safety-related procedures (which control safety-related activities) the NRC review revealed that the applicant sampled enly 165, or 25 percent, during the 1983 audit program. Consequently, significant aspects of the safety-related activities were not adequately audited.
The Westinghouse site organization, established in 1977 to i
perform Nuclear Steam System Supply (NSSS) engineering services, was not audited by TUGC0 during the years of 1977, 1978, 1979, 1980, and 1981.
i Audits of vendors that manufacture or fabricate parts, i
components, and equipment for reactor coolant pressure boundary and engineered safeguards systems have not been conducted annually dating back to August 9, 1978."
e k
I I
I Rsvision:
1 Paga 5 of 53 RESULTS REPORT l
ISAP VII.a.4 j
(Cont'd) 4.0 CPRT ACTION PLAN l
4.1 Scope and Methodology 4.1.1 The scope of this action plan was to evaluate the adequacy of the TUGC0 QA Audit Program from its inception to the present, determine the effect of any identified inadequacies on the Quality Assurance 4
Program and/or the physical plant and to recommend appropriate corrections and/or improvements to the current program. This evaluation was accomplished
+
I through a review which addressed audit planning and i
scheduling, preparation, performance, reporting, follow-up and closeout and audit personnel qualification.
~
i l
4.1.2 The specific methodology is described below.
l 4.1.2.1 Available revisions of the program and j
procedures pertaining to the QA Audit Program for CPSES were reviewed to identify licensing commitments and the degree of conformance to these commitments in the written program.
Included in this review were the CPSES l
PSAR/FSAR (Appendixes 1A(N) and 1A(b),
Chapter 17.1, and QA Branch questions and answers), TUCCO Corporate Quality Assurance
/
Program, CPSES Project Quality Assurance Plan (Design and Construction) and Dallas Quality 1
Procedures / Instructions Manual.
4.1.2.2 Reports, documentation and data generated during the implementation of the QA audit program were selected for review on the basis I
of concerns identified by the NRC; j
significant revisions to commitments, program i
description and/or organization; and to i
pursue questionable areas identified during the review. Specific topics addressed i
include the following:
Audit Planning Criteria, j
Published and As-run Schedules, Audit Plans.and Checklists,
)
e f
1 i
l
., - - - - -, ~,
Rsvision:
1 Pags 6 of 53 RESULTS REPORT O
ISAP VII.a.4 (Cont'd) 4.0 CPRT' ACTION PLAN (Cont'd)
Audit Reports, Audit Deficiency Follow-up, Audit Team Members, Including Qualifications and Staffing Levels, Organizations Performing Audit Activities and 4
Application of Audit Activities to Hardware vs Program / Procedures.
l
{
From this evaluation, conclusions were drawn concerning the adequacy of program assessment j
provided by the audit program. This information was used as input for the recommendations for revisions to the current program.
4.1.2.3 Should audit program deficiencies or weaknesses related to construction activities be identified, they will be evaluated to determine whether action beyond that specified in ISAP VII.c is required to i
identify potential areas of concern regarding i
construction quality. Should such actions be required, a detailed plan will be developed i
and this ISAP revised to describe the methodology.
(This conditional step was determined to be not required and therefore j
was not performed.
See Section 6.0 for discussion.)
In addition, should any identified audit program deficiencies apply also to off-site TUGC0 suppliers, a program will be developed to determine the acceptability of the suppliers' quality assurance programs for the applicable +
equipment and services during the period in
}
question.
This program, if required, will l
utilize external sources of information such as other utility or architect-engineer i
audits, the Coordinated Agency for Supplier
< Evaluation and the NRC " White Book."
If suppliers are identified for which the I
f
Ravision:
1 Page 7 of 53 s
g-,
RESULTS REPORT (Cont'd) 4.0 CPRT ACTION PLAN (Cont'd) adequacy of the QA program cannot be determined, a detailed plan will be developed i
to resolve the concern and this ISAP revised to describe the methodology.
(This conditional step was determined to be not required and therefore was not i
performed.
See Section 6.0 for discussion.)
.4.l.2.4 The current audit program, including auditor qualification requirements, was evaluated against licensing commitments contained in the FSAR.
The qualifications of the current audit staff were evaluated. A determination was made concerning the adequacy of the TUGC0 written program and the qualifications of the audit staff for the remaining construction phase of Unit 2 and for the operations phase.
4.1.2.5 QA/QC Program deviations were identified for correction and suggested program improvements are presented.
4.1.2.6 Copies of the results report for this ISAP will be provided to TUCCO for their consideration in responding to the NRC Notice of Violation (445/8432-03; 446/8411-03).
4.2 Par'cicipants Roles and Responsibilities 4.2.1 TUGC0 4.2.1.1 Scope TUGC0 assisted in identifying and locating applicable information and documentation to support the Review Team activities.
)
4.2.1.2 Personnel Mr. D. McAfee, Dallas QA Manager, provided x
coordination between the Review Team and TUGC0 QA staff personnel.
A f
'N y
~
Revision:
1 e
a Page 8 of 53 fN RESULTS REPORT G
ISAP VII.a.4 (Cont'd) 4.0 CPRT ACTION PLAN (Cont'd) 4.2.2 ERC 4.2.2.1 Scope ERC was responsible for reviewing applicable TUGC0 documentation to determine the adequacy of the program.
4.2.2.2 Personnel Mr. J. Hansel Review Team Leader Mr. J. Gelzer Issue Coordinator
~
4.3 Qualifications of Personnel Participants were qualified to the specific requirements of the CPRT Program Plan.
4.4 Standards / Acceptance Criteria Audit activities shall be in compliance with 10CFR50, Appendix B, Criterion XVIII and the applicable codes and standards relating to the CPSES PSAR/FSAR sections pertaining to the QA audit program as interpreted in Section 7.1 of the NRC Standard Review Plan (NUREG 0800). Specifically such activities arc in compliance if (quoting from NUREG 0800):
" Audits to assure that procedures and activities comply with the overall QA program are performed by:
~
The QA organization to provide a comprehensive independent verification and evaluation of quality-related procedures and activities.
The applicant to verify and evaluate the QA programs, procedures, and activities of suppliers.
An audit plan is prepared identifying audits to be performed, their frequencies, and schedules. Audits
('"N should be regularly scheduled based upon the status and safety importance of the activities being-performed and are initiated early enough to assure effective QA during design, procurement, manufacturing, construction, installation, inspection, and testing.
Rsvision:
1 Page 9 of 53 O
RESULTS REPORT ISAP VII.a.4 (Cont'd) 4.0 CPRT ACTION PLAN (Cont'd)
Audits include an objective evaluation of quality-related practices, procedures, instructions; activities and items; and review of documents and records to ensure that the QA program is effective and properly implemented.
Provisions are established requiring that audits be performed in all arhas where the requirements of Appendix B to 10CFR Part 50 are applicable. Areas which are often neglected but should be included are activities associated with:
Indoctrination and training programs.
~
Interface control among the applicant and the principal contractors.
O Corrective action, calibration, and non-conformance control systems.
SAR commitments.
Audit data are analyzed and the resulting reports indicating any quality problems and the effectiveness of the QA program, including the need for reaudit of deficient areas, are reported to management for review and assessment.
Audits are performed in accordance with pre-established written procedures or checklists and conducted by trained personnel having no direct responsibilities in the areas being audited."
4.5 Decision Criteria Decisions concerning the number of individual reports, records and files to be reviewed, and the level of detail to which they were reviewed, were based on the quantity and quality of data obtained as implementation of the ISAP proceeded.
Sufficient data were evaluated to support the conclusions reached as to the acceptability of the program against the acceptance criteria contained in Paragraph 4.4.
O
i Ravision:
1 Page 10 of 53 RESULTS REPORT
\\~ /
ISAP VII.a.4 (Cont'd) 5.0 IMPLEMENTATION OF ACTION PLAN AND DISCUSSION OF RESULTS The TUCCO QA Audit Program adequacy was evaluated from its inception to the present. The specific TRT findings in the SSER 11 and the February 15, 1983, Notice of Violation are addressed in this section.
The implementation of this action plan was accomplished in two phases: Audit Program (Section 5.1) and Audit Personnel Qualification (Section 5.2).
These sections are sub-divided as follows to address the various elements of the Audit Program and the Audit Personnel-Qualification Program:
Section 5.1 Audit Program 5.1.1_
Organization 5.1.2 Staffing fsg 5.1.3 Audit Planning and Scheduling 5.1.4 Audit Preparation s-5.1.5 Audit Performance 5.1.6 Audit Reporting 5.1.7 Follow-up and Closeout Section 5.2 Audit Personnel Qualification 5.2.
Qualification Requirements
- 5. 2. :!
Training / Qualification Records Implementation consisted of a review of program definition and implementation documents, a review of records and other related documents and interviews with personnel involved in the program.
The historical PSAR/FSAR was reviewed to identify the standards and regulatory guidance to which TUGC0 was committed, pertaining to the audit program and qualification of audit personnel. This review determined that:
The basis for the audit program from December 1973 (Amendment ps
(
j
- 2 of the PSAR) through the present is ANSI N45.2.12-1973
\\d (Draft 3, Revision 0, May 2, 1973) as contained in " Guidance on Quality Assurance Requirements During Design and
- Revision:
1 Page 11 of 53 s
RESULTS REPORT ISAP VII.a.4 s_-
(Cont'd) 5.0 IMPLEMENTATION OF ACTION PLAN AND DISCUSSION OF RESULTS (Cont'd)
Procurement Phase of Nuclear Power Plants" (Gray Book), dated June 7, 1973, and as amplified by the " Regulatory Staff Comments and Supplementary Guidance on N45.2 Standards" contained in Section D.2.b of the Gray Bock.
The only exception to this commitment is that since August 1984 (Amendment #52 of the FSAR) TUGC0 elected to adopt, as an alternative for scheduling external audits, the guidance provided by Regulatory Guide 1.144, Revision 1, September 1980, Paragraph C.3(b).
The basis for audit personnel qualifications was ANSI N45.2.12-1973 until June 1, 1983, which was the effective date of TUGC0 commitment to Regulatory Guide 1.146, August 1980, which endorses ANSI N45.2.23-1978.
The documents and record files reviewed included the following:
f, s
(
I
- CPSES FSAR, TUCCO Corporate Quality Assurance Program, CPSES Project Quality Assurance Plan for Design and Construction, Dallas Quality Procedures / Instructions Manual.
- Audit Report Files,
- Audit Report Notes Files,
- Equipment Release Files (Vendor Compliance Release Inspections),
Vendor Correspondence Files,
- Vendor Reevaluation Files,
- Vendor Performance Evaluation Files and
- Audit Personnel Training / Qualification Files.
Rsvision:
1 Page 12 of 53 RESULTS REPORT ISAP VII.a.4 (Cont'd) 5.0 IMPLEMENTATION OF ACTION PLAN AND DISCUSSION OF RESULTS (Cont'd)
Findings pertinent to each topic will be discussed within the appropriate sub-section. At the end of each sub-section a statement regarding possible adverse effects of the findings on the audit program will be made. There is no direct connection between the audit program and the quality of installed hardware; therefore, no conclusions regarding the st~atus of the physical plant have been made.
Recommended corrective action for QA/QC Program Deviations identified in the body of this report are summarized in section 5.3, "Evalration of Findings".
5.1 Audit Program
~
Documents and files were reviewed to evaluate the adequacy and effectiveness of the TUGC0 audit program including organization, staffing, planning and scheduling, preparation, performance, reporting and deficiency follow-up and closecut.
Initially, 69 audit files were reviewed to assist in reaching a conclusion pertaining to the performance of the TUGC0 QA audit personnel.
Internal audits were selected to provide data from the inception of the program to the present, as well as a sampling of the various disciplines and major topics of interest such as document control and nonconformance/ corrective action systems. Vendor audits were selected primarily to review TUGC0 activities pertaining to prime contractors (B&R, W, G&H) as well as on-site contractors. This review was not intended to be a j
comprehensive evaluation of the scope, content, conclusions, etc., of the audit; rather, it was intended to determine if the required steps in the audit process had been properly performed. The review addressed the following items:
I l
Audit Preparation - To determine if the audit scope was documented, if an audit team leader had been designated, if checklists had been prepared and if they appeared to contain an appropriate level of detail.
Audit Performance - To identify the audit team, to determine if pre-audit and post-audit conferences were conducted, if it appeared that objective evidence had been examined and if audit. personnel appeared to a
! t evaluate program elements to some depth.
Rsvision:
1 Page 13 of 53
's RESULTS REPORT ISAP VII.a.4 (Cont'd) 5.0 IMPLEMENTATION OF ACTION PLAN AND DISCUSSION OF RESULTS (Cont'd) 1 Audit Reporting - To determine if audit reports had been prepared that would provide appropriate management with sufficient information concerning the adequacy and effectiveness of the audited QA program elements.
Audit Deficiency Follow-up and Closeout - To determine v if responses to audit deficiencies were evaluated, if an acceptable course and schedule for corrective action had been agreed on hnd if implementation of corrective action was verified.
Later, as the result of input received from other QA/QC Review l
Team Issue Coordinators, nine (9) audit files were reviewed in depth in an attempt to reach a conclusion as to why particular programmatic problem areas (e.g., non-conformance control and corrective action systems, inspector training, qualification, and certification programs), which were identified by External Sources had not been identified and corrected through implementation of the TUGC0 QA audit program. The review of these nine (9) files included a re-review of four (4) of the sixty-nine (69) previously reviewed and were selected on the basis of their titles in the audit index as being the total availtble that addressed these subjects. This review included the following activities:
l A detailed review of the audit checklist for content 1
and to attempt to determine if the checklist was l
developed from requirements documents or from the existing written program, A review of auditor field notes to attempt to determine the depth of examination, sample size (when applicable), type of objective evidence evaluated,
[
whether program documents were evaluated against j
requirements, and if apparent deviations had been flagged to be included in the audit report, T
j A review of the audit report to determine if the i
findings documented in the auditor field notes were accurately reflected in the report and A review of documentation subsequent to the audit report in an attempt to determine the adequacy of auditor evaluation of responses to audit deficiencies, dialogue leading to agreement on proposed corrective action and verification of completion of corrective i
j action.
i
Ravision:
1 Page 14 of 53
g RESULTS REPORT ISAP VII.a.4 (Cont'd) 5.0 IMPLEMENTATION OF ACTION PLAN AND DISCUSSION OF RESULTS (Cont'd) i In response to the NRC Notice of Violation (445/8432; 446/8411), the audit files representing the 29 internal audits of the design / construction phase performed during 1983 were reviewed to identify the major activities, organizations, and Appendix B criteria which were addressed, to enable an evaluation of the audit program coverage for the year. These 29 files included five which had been previously reviewed.
The Design Adequacy Review Team provided a list of design specifications which they plan to evaluate for design adequacy and Architect-Engineer / Vendor interface. From the vendors identified, twelve were selected for this review based on considerations such as frequency of audits performed, awarenees of problems with a particular vendor, apparent time frame of vendor activity and type of product. Additional rationale for selection is contained in the working files for this ISAP.
s A review was performed of TUGC0 QA Audit and Vendor Compliance files (audit reports, audit notes, equipment releases, vendor correspondence, vendor performance evaluation and vendor reevaluation files) for these twelve vendors to evaluate the degree of awareness by TUGC0 QA of the adequacy, effectiveness and status of the QA programs of their vendors despite the fact that annual vendor audits were not scheduled.
The results of these reviews are summarized in the following paragraphs.
5.1.1 Organization ANST N45.2.12-1973, Section 1.4, " Organization",
states, "The organizational structure, functional responsibilities, levels of authority, and lines of internal and external communication for management direction of audits of the quality assurance program shall be documented".
Although Section 17.1.18 (Audits) of the CPSES FSAR states that TUGC0 will perform audits, nowhere in Section 17.1.1 (Organization) is the responsibility for the audit program delineated.
Ravision:
1 Page 15 of 53 RESULTS REPORT
\\s ISAP VII.a.4 (Cont'd) 5.0 IMPLEMENTATION OF ACTION PLAN AND DISCUSSION OF RESULTS (Cont'd)
Within section 17.1.1.1, the list of principal duties and responsibilities of the Director, Quality Assurance includes " monitoring and surveillance of the quality assurance activities conducted by TUGCO, prime contractors, sub-contractors and vendors." However there is no specific mention (nor has there been since the original docketing of the PSAR) of responsibility fo'r the audit program.
During the review of the QA program and records, it was determined that the QA organization was in fact responsible for the audit program and that TUGC0 management recognized and supported this
~
responsibility.
It was noted that on December 30, 1985, TUCCO QA initiated a request for an FSAR revision to include the On' audit program among the principal duties and responsibilities of the Director, Quality Assurance.
The TUGC0 QA Audit Program is administered and implemented by the Dallas-based Quality Assurance organization. The implementing procedures for the audit program are contained in the Dallas Quality Procedures / Instructions Manual. Since 1982, procedures j
in this manual have assigned responsibilities for execution of portions of the QA audit program to the position of Supervisor, QA Audits. This position was not officially recognized in published organizational descriptions and organization charts until the issue of procedure DQP-QA-16, " Dallas Quality Assurance Organization" in September 1985. During the review of the audit program proceduresfand records, it was j
determined that the position was filled during this period and was recognized and supported by QA i
management.
Although ANSI N45.2.12-1973 and the accompanying staff comments and guidance pertain primarily to the auditing organization, the requirements of the ANSI standard and the associated supplementary guidance listed below apply as well to the audited organization as an
Revision:
1 Page 16 of 53 s
RESULTS REPORT ISAP VII.a.4 (Cont'd) 5.0 IMPLEMENTATION OF ACTION PLAN AND DISCUSSION OF RESULTS (Cont'd) integral part of the overall audit program. Therefore, these requirements should be included in QA program documents applicable to the audited organization as well as the auditing organization (e.g., Corporate QA Program, Project QA Plan) to ensure compliance by all responsible organizations.
An effective audit system shall include provisions for reasonable and timely access of audit personnel to facilities, documents, and personnel necessary in the planning and performance of the audits.
At the post audit conference an effort shall be made to establish a tentative course and schedule of corrective action for non-
)
conformances. Where it is not possible to provide such information at the post audit conference, the management of the audited organization should commit to a specific date for the determination of the course of corrective action and the schedule for implementation.
Management of the audited organization or activity shall review and investigate the audit findings to determine and schedule appropriate corrective action. They shall i
respond to the report in writing, within 30 days after receipt.
The review of these files indicated that these requirements had been met 13 that:
There was no evidence to indicate that audit personnel had been denied reasonable and timely access as required to perform their duties.
There was no evidence to indicate that problems had been encountered pertaining to the establishment of the tentative course and schedule of corrective actions and schedule i
for implementation.
Ravision:
1 Page 17 of 53 RESULTS REPORT ISAP VII.a.4 (Cont'd) 5.0 IMPLEMENTATION OF ACTION PLAN AND DISCUSSION OF RESULTS (Cont'd)
There was evidence that management of the audited organization reviewed and investigated audit findings and determined and scheduled corrective action. Written responses were received by the auditing i
organization within a reasonable time of the regt.ested date (on time, or within one week) i in approximately eighty (80) percent of the files reviewed. This ratio is not unusual in the experience of the RTL.
The first two of these requirements listed above are not addressed in the TUGC0 QA program documents. The*
third is addressed to the audited organization only through the transmittal of the audit report.
Conclusions It is concluded that the failure to formally document the responsibility for the audit program in the 4
PSAR/FSAR resulted in no adverse effect on th" audit program because sub-tier program documents were written and management acted as if the responsibility had been formally documented.
It is concluded that the failure to formally describe the position and responsibilities of the Supervisor, QA I
Audits, in organization descriptions resulted in no adverse effect on the audit program because audit program documents were written and QA management acted as if the position had been formally documented.
It is concluded that the fai ure to include the responsibilitiesoftheaudi}tedorganizationin appropriate program documents resulted in no adverse i
effect on the audit program because review of the audit files revealed that the requirements had been met by the audited organization.
t l
5.1.2 Staffing l
The adequacy of the TUGC0 audit staff (number and l
qualifications) has been questioned by the NRC. When considering the adequacy of an audit staff, one must j
include not only personnel assigned full time to the audit function, but also the qualified personnel available on call to the audit organization.
__m
g
--,-r
vy-wy 7----
7---w---r-+-'
Rsvision:
1 Paga 18 of 53 RESULTS REPORT ISAP VII.a.4 (Cont'd)
- 54. 0 IMPLEMENTATION OF ACTION PLAN AND DISCUSSION OF RESULTS (Cont'd) j The applicable standards and regulations contain no y
requirement for an audit staff, per se; only that management shall establish an audit system that f
includes manpower, funding and facilities to implement the system. This allows management maximum flexibility to allocate resources, even to the extreme of assigning a full-time audit administrator responsible for planning and scheduling of audits and tracking open items, who has free access to qualified audit personnel as required for individual audit ureparation, conduct, reporting and follow-up, but who?has no permanently assigned audit staff.
Therefore, judgement on the adequacy of the size of an 1
audit staff must be based on factors such as assigned duties and responsibilities, cutside resources available on a timely basis and the ability to accomplish the scheduled audit activities (providing schedules are not dictated by manpower availability).
The years 1981 - 1982 were cited by the TRT as a period when the numbers and qualifications of the TUGC0 audit staff were questionable.
In addition, Region IV Inspection Report 445/84-32 cited 1983 as a period when j
the audit staff was inadequate. A review was performed of training and qualification files for TUGC0 Quality Assurance personnel (designated audit staff and others) l who participated in the audit program as Lead. Auditors or auditors during these periods. The following tabulation summarizes the characteristics of these 4
l personnel.
l l
i.
+
Rsvision:
1 Page 19 of 53 RESULTS REPORT ISAP VII.a.4 (Cont'd)
I 5.0 IMPLEMENTATION OF ACTION PLAN AND DISCUSSION OF RESULTS (Cont'd) 1981 1982 1983 1
Designated Audit Staff 5
11 14 l
(Including Supervisor)
Total Participants 13 20 25 Qualified Lead Auditors 10 16 18 l
Education
- Masters Degree 0
1 1
i Bachelors (Engineering) 7 6
8 Bachelors (Other) 2 2
5 Associates (Technical) 0 1
2-Associates (Other) 0 1
1 Some College 1
2 3
NOTE:
Four of the engineers were registered O
Professional Engineers, and one was a t
certified Quality Engineer. One holder of an Industrial Management degree was also a certified Quality Engineer.
1 Since the formation of a designated audit staff in 1979, through the present, it has been the practice to j
supplement the staff as needed with other qualified members of the QA organization to function as auditors l
and lead auditors.
It is therefore concluded that, based on the evaluation of the years 1981 through 1983 and the other files reviewed, the formally designated i
audit staff could be considered deficient in numbers and technical qualifications, but that the effective audit staff of Quality Assurance personnel was, and i
continues to be, adequate in both number and l
qualifications.
l Conclusions i
l Based on the reviews performed, it is concluded that l
individuals did not audit activities which they were responsible for performing.
It is concluded that audit staffing was adequate to implement the audit program and schedules during the periods of interest.
Personnel holding multiple degrees are listed only once, in the highest applicable category.
Ravision:
1 Page 20 of 53
/'~'
RESULTS REPORT ISAP VII.a.4 (Cont'd) 5.0 IMPLEMENTATION OF ACTION PLAN AND DISCUSSION OF RESULTS (Cont'd) 5.1.3 Audit Planning and Scheduling This section addresses the following topics:
Scheduling requirements,
- Vendor audit sche,duling and performance, NRC Notice of Violation pertaining to Westinghouse site organization.
Planning and scheduling process for internal audits and J
NRC Notice of Violation pertaining to 1983 audit program coverage.
fs The requirements of ANSI N45.2.12-1973 pertaining to
(
audit planning and scheduling can be paraphrased by saying that audits are performed by or for an organization participating in activities affecting quality and that elements of the Quality Assurance program affecting these activities shall be audited at least annually or at least once within the life of the activity, whichever is shorter. Within this context, the following terms are defined:
Organization - Plant owner, contractor, etc.,
Activities - As listed in the last sentence of ANSI N45.2.12-1973, Paragraph 1.2, and Elements - The 18 criteria of 10CFR50, Appendix B.
The TUGC0 written program (i.e., PSAR/FSAR, QA Program, QA Plan and implementing procedures) reflected the requirement for scheduling internal audits on an annual basis only from August 1973 through April 1981. The annual requirement was not reintroduced until November 1984. Additionally, the program never has required that vendors be audited on an annual basis.
\\
Rsvision:
1 Page 21 of 53 4
RESULTS REPORT ISAP VII.a.4 (Cont'd) 5.0 IMPLEMENTATION OF ACTION PLAN AND DISCUSSION OF RESULTS (Cont'd)
Not until the issuance of DQP-CS-4, Revision 11, in November 1984, was the TUGC0 written program in full compliance with applicable planning and scheduling commitments. This revision reflected the options permitted by Regulatory Guide 1.144 to audit vendors on a triennial basis, provided annual assessments are performed. This revision also contained the requirement to audit other applicable organizations (e.g., TUGCO, site contractors) at least annually.
This lack of requirements in the written program for scheduling annual internal audit and supplier audits constituted a noncompliance with licensing commitments.
Vendor Audit Program In evaluating the effect of this noncompliance on the vendor audit program, the following evolution of TUGC0 QA audit program responsibilities was identified:
From 1973 to 1977. TUGC0 was responsible for auditing only the major contractors: Gibbs &
Hill (G&H), Brown & Root (B&R), and Westinghcuse (H). During this same time period G&H, B&R, and y had prime responsibility for vendor preaward surveys, audits, and source inspections. However, TUCCO did participate in these activities.
Early in 1978, although E retained its original role concerning NSSS vendors and B&R retained full responsibility for its ASME suppliers TUGC0 assumed more direct control in the other vendor preaward survey, audit, and source inspection activities previously delegated to C&H and B&R. This included balance of plant vendors (safety-related and non-safety-related), on-site contractors and non-ASME material suppliers. Additional control was exercised through increased involvement in the scheduling process and the conduct of more joint audits.
O
Rsvision:
1 4
Page 22 of 53
/N RESULTS REPORT
\\
ISAP VII.a.4 (Cont'd) 5.0 IMPLEMENTATION OF ACTION PLAN AND DISCUSSION OF RESULTS (Cont'd)
Early in 1981 TUGC0 began performing vendor-related activities without G&H involvement. Also during 1981, TUGC0 assumed full responsibility for non-ASME material suppliers from B&R. This level of responsibility has continued to the present.
Based on the review'of the audit index, it was determined that TUGC0 QA performed audits of the major contractors during the period from 1973 to 1977 as follows:
1973 1974 1975 1976 1977 G&H N/A 4
1 1
2 W
1 0
1 6
4 B&R (Houston) N/A 1
0 2
1 B&R (Site)
N/A N/A 2
1 1
The PSAR was docketed in 1973 and the Construction Permit was issued in December 1974. The tabulation indicates that with two exceptions, required audits were performed during this period in accordance with the annual requirements of ANSI N45.2.12-1973.
From 1978 (when TUCCO became more involved in the administration of the audit program) through 1984, the performance of contractor / vendor audits was not in accordance with applicable requirements for timeliness or frequency.- For example, for five of the twelve vendors reviewed, the initial audit was performed after the first release inspection (in three cases more than a year afterward) which was not in compliance with the ANSI requirement that auditing be initiated as early in the life of the activity as practicable to assure timely implementation of quality assurance requirements.
It was also noted that anywhere from four months to six years elapsed between audits of these vendors. No single vendor was audited on an annual basis throughout this time period, which constituted a violation of the ANSI requirement for annual audits.
O The foregoing raises a concern that the TUCCO QA organization was not cognizant of the effectiveness of the vendor's QA programs.
In assessing this concern, the following was disclosed:
l Rsvision:
l' Page 23 of 53 N
)
RESULTS REPORT
\\'- '/
ISAP VII.a.4 (Cont'd) 5.0 IMPLEMENTATION OF ACTION PLAN AND' DISCUSSION OF RESULTS (Cont'd)
In October 1978, TUGC0 QA instituted a Vendor Rating program whereby numerical ratings were assigned to vendors based on type, severity, and quantity of defects identified during source inspections. These ratings were utilized to adjust the scheduling of inspections and audits.
In January 1981, a Vendor Performance Evaluation system was instituted which enhanced the Vendor Rating system by also considering the number of 4
items submitted for inspection in calculating the rating.
It was noted during the review that various audit notifications and/or reports documented that some of these audits had been initiated as a result of unsatisfactory source inspection reports.
's In May 1979, a Joint Inspection / Audit program 4
was instituted whereby auditors would
,\\
accompany inspectors on release inspection trips to investigate specific program areas based on problems identified during previous t
source inspections.
TUGCO QA also utilized " Requests for Corrective Action" which were correspondence with the vendors requesting corrective action for specific, identified problems when it was concluded that the problems did not warrant an audit investigation or did not appear to l
indicate an adverse trend.
Internal documents known as " Yellow Flag Sheets" were also utilized by the TUCCO QA staff to ensure that applicable QA personnel were aware of status and/or problems with particular vendors, e.g., a hold placed on shipments from a vendor until after an audit was performed.
While it is recognized that source inspections and limited-scope audits performed in response to l
identified problems might not be considered adequate j
substitutes for regularly scheduled program audits, they did provide data which were utilized to some extent by TUGC0 to evaluate the effectiveness of vendor QA programs.
l Rsvision:
1 Pags 24 of 53 RESULTS REPORT O
4 ISAP VII.a.4 (Cont'd) 5.0 IMPLDfENTATION OF ACTION PLAN AND DISCUSSION OF RESULTS (Cont'd)
It is concluded that during the period prior to 1978, the frequency of performance of contractor / vendor audits, with the exception of the two cases noted, was satisfactory, but for the period from 1978 through i
November 1984 the frequency was not satisfactory.
However, it is further concluded that as a result of the source inspection activities, the limited audit activity, vendor ratings, corrective action requests and internal QA communications, that sufficient information was available to, and used to some extent by, the QA organization to evaluate the effectiveness of vendor QA programs even though audits of the vendors were not performed on an annual basis.
l The NRC notice of violation (445/8432-03; 446/8411-03) identified that the Westinghouse site organization had not been audited from its inception in 1977 through e-~s 1981. To provide background, the description of the Westinghouse site organization in the NSSS contract between TUGC0 and Westinghouse includes the following in Section 5.0 (paragraphs A and B are not applicable to this discussion):
"C.
Construction Site Services - Beginning 21 months prior to the arrival of the first of Vendor's equipment at the job site (or such later date as may be mutually agreed upon),
vendor shall establish an organization with representation at the job site to the extent necessary in the judgement of Purchaser to coordinate construction site activities for Vendor's equipment. This organization will be retained with representation at the job site until fuel loading of the second NSSS is j
completed.
D.
Field Testing and Startup Services - Vendor shall establish an organization to provide advice and consultation on testing and startup. Special startup and testing equipment (or its then current equivalent) identified in Exhibit A shall be provided at the job site by Vendor on a cost-free loan
- basis. Testing and startup procedure j
preparation assistance shall also be provided."
Rsvision:
1 Page 25 of 53 x
g RESULTS REPORT ISAP VII.a.4 (Cont'd) 1 5.0 IMPLEMENTATION OF ACTION PLAN AND DISCUSSION OF RESULTS (Cont'd)
Further, the safety-related activities relating to erecting, installing, inspecting, testing, etc., of the NSSS were the responsibility of TUGC0 and were performed by TUGC0 or Brown & Root organizational elements. The activities performed by the Westinghouse i
site organization were primarily liaison and did not include'the categories of safety-related activities requiring periodic audits as listed in Section 1.2 of ANSI N45.2.12-1973.
This understanding of the involvement of the Westinghouse site organization in the site safety-related activities was confirmed by TUGCO QA during the performance of an audit of Westinghouse site activities pertaining to work on the reactor internals (audit i
TWH-23) in March 1982. This audit confirmed that the development of work travelers was performed by Texas Utilities Services, Inc. engineering and that construction and inspection activities were performed by Brown & Root. The audit also revealed that j
Westinghouse-Pensacola supplied drawings and performed overview surveillance of TUGC0 and Brown & Root activities related to the reactor internals. After concerns were expressed by the NRC, an audit of Westinghouse site activities was scheduled and performed in November 1984 (audit TWH-30). This audit confirmed the TUGC0 understanding of the Westinghouse site organization scope of work.
It was determined that the W site organization scope of work did not include the categories of safety-related activities requiring periodic audits per ANSI N45.2.12-1973. This position was confirmed by TUGC0 through the performance of the above-mentioned audits.
It is therefore concluded that there was no requirement to audit the Westinghouse site organization.
j Internal Audit Program As previously discussed, ANSI N45.2.12-1973 requires that applicable program elements affecting an activity be audited annually. This means that the activities to i
be audited must be identified along.with the program elements affecting these activities. Audits of these
R3 vision:
1 Pags 26 of 53 RESULTS REPORT ISAP VII.a.4 (Cont'd) 3 1
5.0 IMPLEMENTATION OF ACTION PLAN AND DISCUSSION OF RESULTS (Cont'd) activities are to be scheduled on the basis of their status and safety importance and performed to evaluate the adequacy and effectiveness of the program as developed and implemented by the participating organizations.
Evaluating the adequacy of the TUGC0 QA internal audit
]
program at the construction site is complex. Prior to 1978, site audits were scheduled and conducted as vendor audits. Since that time, except for contractors operating under their own QA programs, site audits have been scheduled and conducted as internal audits.
i j
The following paragraphs describe the evolution of the l
audit planning process during the Design / Construction phase of CPSES.
I When the TUCCO internal audit program was first i
implemented at the construction site in 1978, audit planning and scheduling appeared to be primarily activity-oriented. However, by 1982 audit planning appeared to have become primarily organization-oriented. During the review of the audit files.
l observations related to the structure of checklists, task assignments of team members during the audit, structure and content of audit reports, as well as audit titles indicates that audit planning and scheduling continued to be organization-oriented.
This approach to audit planning evolved to accommodate the organizational structure and procedure systems that developed as TUGC0 assumed additional responsibility for site activities.
The written program has always required that audit i
planning and scheduling activities be performed, and l
the reviews of the files revealed that audit schedules i
had been prepared. However, it was determined that l
until the issuance of procedure DQP-AG-4, "CPSES Construction Phase Audit Program," Revision 0, on September 4, 1985, there was no procedure that detailed i
the audit planning and scheduling process. A review of this procedure revealed that although audits were to be 1
l 1
R3 vision:
1 Page 27 of 53 RESULTS REPORT l
ISAP VII.a.4
)
(Cont'd) j 5.0 IMPLEMENTATION OF ACTION PLAN AND DISCUSSION OF RESULTS (Cont'd) l
\\
performed annually, activities to be audited would be determined from existing programs and procedures, rather than from an analysis of site activities affecting quality and their importance to safety. The 2
I procedure also implied that audit planning was organization-oriented.
However, as a result of discussions with TUGC0 QA management and a review of audit schedules, it is l
concluded that the audit planning and scheduling process has generally considered project activities as an input. Procedure DQP-AG-4 has since been revised (Revision 1 dated January 27, 1986), and now describes an adequate audit planning and scheduling process.
Procedure DQP-AG-4 also provides for the development of i
an annual audit schedule which is subject to periodic review and adjustment based on project status, program i
changes, outside reviews, etc.
As-run schedules for past years have been prepared to document audits scheduled but not performed (and the reasons why they were not performed), and additional audits which had been added to the schedule.
It was neced that the proposed 1986 audit schedule includes a short scoping statement for each scheduled audit. This enhancement to the schedule provides a l
more complete description of the planned audit program coverage.
1 Also related to audit planning, the NRC Notice of Violation (445/8432-03; 446/8411-03) stated that TUGC0 audit planning for 1983 was inadequate because only 25 percent of the safety-related procedures were audited.
This use of procedures as an audit planning baseline is appropriate only when the procedures are pre-defined as l
the program elements. Accountability for audit coverage would then be simplified because a procedure listing is very specific as opposed to developing a list of activities, applicable criteria, and participating organizations. Attractive as it may appear, this method of audit planning contains weaknesses and is not recommended because auditing strictly by the procedures could result in overlooking an activity for which a procedure was required but had not been developed.
i
Ravision:
1 1
5 Page 28 of 53 i
RESULTS REPORT a
ISAP VII.a.4 i
(Cont'd) i 4
l 5.0 IMPLEMENTATION OF ACTION PLAN AND DISCUSSION OF RESULTS (Cont'd)
By calculating the percentage of procedures addressed by the audit program, an inference could be drawn as to the adequacy of audit program coverage. However, unless this information is correlated with actual j
project activity, previous audit and/or surveillance results, importance of the activities represented by procedures not audited, e.tc., no judgement should be made.~of the^ adequacy of audit program coverage based solely on percentage of procedures audited.
l l
A review of the 1983 Audit Program revealed that TUGC0 4
performed twenty-nine (29) internal audits of design / construction phase activities as well as eight-i onsite audits of construction contractors.
From data gathered during the review of the twenty-nine (29) internal audit files, a matrix was developed to j
evaluate the activities audited, the program elements 1
(Appendix B criteria) addressed, and the organizations j
through which they were addressed.
In addition, the t
j list of activities audited (contractor and internal) 1 was compared to the anjor site activities in process I
l during the yest as identified by project control i
personnel. These evaluations revealed that the 1983 audit program addressed the major on-going site l
activities, that all site organizations were audited, and that all program elements (Appendix B criteria)
}
were addressed. Therefore, it is concluded that the audit program coverage of the Design / Construction Phase of CPSES during 1983 was adequate.
It was also noted j
that the audit schedule was adjusted throughout the year in response to organizational and program changes, 1
to address problems identified by outside organizations and to reflect changes in the project schedules.
I Conclusions l
It is concluded that the failure to perform vendor j
audits on an annual frequency resulted in no adverse j
effect on the audit program because of the additional i
i activities implemented by the QA organization to supplement the audit activity.
l i
I i
l 1
I l
Revision:
1 Pega 29 of 53 l
RESULTS REPORT
)
ISAP VII.a.4 (Cont'd) 3 i
i 5.0 IMPLEMENTATION OF ACTION PLAN AND DISCUSSION OF RESULTS (Cont'd)
It is concluded that there was no requirement to audit i
the Westinghouse site organization.
i i
It is concluded that the lack of a formalized
{
methodology for internal audit planning and scheduling l
resulted in no adverse effect on the audit program j
because it was established that audit planning was accomplished and audit schedules were prepared to implement the audit program.
l 5.1.4 Audit Preparation i
The audit preparation phase consists primarily of j
accumulating pertinent documents and data from internal and external sources from which the Audit Team Leaders 1
would develop the final detailed audit scopes and audit i
checklists.
3 1
j The reviews of the audit files revealed that while
(
checklists had been prepared for the conduct of the l
audits and that sources such as TUCCO site
,l surveillance, INPO evaluations, CAT reports, etc., had i
been utilized to address topics of special interest, j
the majority of the checklists reviewed in detail appeared to have been developed from existing
)
procedures rather than from governing requirements. As j
discussed previously, utilizing procedures as the i
baseline could preclude verification thst the procedures, individually or collectively, adequately implement the requirements.
I l
It can be an acceptable practice to perform only compliance audits for those portions of a program which remain basically unchanged. However, there was no i
evidence in the TUCCO QA audit files reviewed to i
indicate that a comprehensive assessment of the written 4
l program had previously been performed in the areas of i
Non-conformance Control / Corrective Action or QC
}
Training / Qualification / Certification to verify that i
the program procedures were in compliance with
)
governing requirements.
1 In addition, the checklists reviewed did not provide for verification that procedures in different CPSES procedure systems that address the same or similar j
activities were consistent in approach, level of detail j
or control of the activity.
i I
RSvision:
1 Paga 30 of 53
()
~'g RESULTS REPORT h
\\w /
ISAP VII.a.4 (Cont'd) 5.0 IMPLEMENTATION OF ACTION PLAN AND DISCUSSION OF RESULTS (Cont'd)
During 1985, the QA Audit Group initiated the use of an
" Appendix B Checklist." Applicable portions of this standard checklist are now utilized by the audit team to verify that the procedures within the scope of the sudit are in compliance with the applicable criteria of 10CFR50 Appendix B.
Conclusions It.is concluded that the audit preparation activities were not adequate. This resulted in an adverse effect on the audit program, in that most audits reviewed did~
not evaluate the adequacy of the written program to implement the governing requirements.
5.1.5 Audit Performance ANSI N45.2.12-1973, to which TUGC0 was committed i
through May 1983, contains no specific criteria for an
\\s -
Audit Team Leader.
From February 1982 through February 1985, as a portion of on-the-job training, the TUGC0 program contained provisions for a lead auditor candidate to be designated as the acting Audit Team Leader, to perform under the direct supervision of a qualified Lead Auditor.
This practice did not constitute a noncompliance prior to the June 1983 commitment to ANSI N45.2.23-1978.
However, from that time until the issue of procedure DQP-CS-3, IUGC0 QA Audit Program," Revision 12, dated February 15, 1985, the practice did constitute a technical noncompliance with the requirements of ANSI N45.2.23-1978.
It is not uncommon for an audit team leader to delegate functions to team members during the course of an audit, for training or other reasons, while retaining ultimate responsibility therefore. However, in the TUCCO program, full responsibility for an audit was assigned to other than a qualified Lead Auditor, which was contrary to. the requirements of ANSI N45.2.23-1978.
O l
Ravision:
1 rs
=
Page 31 of 53
]
RESULTS REPORT l
ISAp VII.a.4 l
(Cont'd) 1 5.0 IMPLEMENTATION OF ACTION PLAN AND DISCUSSION OF RESULTS (Cont'd)
ANSI N45.2.12-1973 requires that auditors be l
independent of any direct responsibility for the activities they will audit. The reporting level for the Director, Quality Assurance provides adequate I
independence of the QA staff from the design and construction activities.
The NRC raised a question in SSER 11 concerning the l
~
objectivity of surveillance personnel performing audits l
of activities over which they had surveillance responsibility. This practice is not uncommon in the industry and, in fact, can be beneficial to the audit i
program in that the knowledge and experience gained by j
personnel from surveillance activities can be utilized as input to the audit planning process and through j
participation on the audit team. The only time this situation might constitute a conflict would be the case where the purpose of the audit was to evaluate the adequacy and effectiveness of the st.rveillance program itself. No audits of this nate ? r 'e identified during this review.
1 It was determined that pre-audit and post-audit conferences were conducted by the audit team and that appropriate personnel were in attendance.
The review of the audit files revealed that objective evidence had been examined when appropriate to support l
the audit activity as defined by the audit checklists, j
It appears that the auditors evaluated program elements i
to an appropriate depth to support the audit checklists.
l Conclusions i
It is concluded that the practice of designating a lead auditor candidate as an acting team leader resulted in l
no adverse effect on the audit program because the 4
acting team leader performed the activities under the direct supervision of a qualified lead auditor who was i
a member of the audit team.
i.
It is concluded that the practice of utilizing site l
surveillance personnel as audit team members had no adverse effect on the audit program because these personnel did not perform audits of activities which they were responsible for performing.
Revision:
1 l
Page 32 of 53 e
RESULTS REPORT s g
\\s-ISAP VII.a.4 (Cont'd) i 5.0 IMPLEMENTATION OF ACTION PLAN AND DISCUSSION OF RESULTS (Cont'd) 5.1.6 Audit Reporting The review of audit files revealed that audit reporting has not always complied with the requirements of ANSI N45.2.12-1973. Early in the program, audit reports did not consistently include items such as a description of the audit scope, persons contacted, a summary of audit results and &n evaluation statement regarding the effectiveness of the QA Program elements which were audited. However, the information not included in the reports, with the exception of the summary and the evaluation statement, can be found in the audit files.
The content and quality of audit reports have gradually improved over time. Currently, the audit reports provide the reader with an adequate understanding of the scope and results of the audit. The current
~s procedures adequately reflect the requirements of ANSI N45.2.12-1973.
During the detailed review of the nine (9) audit files in the training and nonconformance control areas (see Page 12), several deviations were observed that indicate weaknesses in the implementation of the audit program related to audit reporting:
4 There were approximately twenty instances where the published audit reports did not appear to accurately reflect the auditors' field notes and/or checklists.
For example, items in the notes that appeared to be l
deficiencies were reported as concerns or comments, and in one case, an item identified i
as "unsat" on the audit checklist was not i
reflected in the audit report at all. The l
items identified did not pertain directly to any items of hardware.
(It is recognized by the RTL that additional data could have been obtained which could have altered the conclusions and was not reflected in the notes / checklist).
i
Ravision:
1 4h Page 33 of 53 s
RESULTS REPORT ISAP VII.a.4 (Cont'd) 5.0 IMPLEMENTATION OF ACTION PLAN AND DISCUSSION OF RESULTS (Cont'd)
In many of these cases, the accompanying notes seemed to indicate that the auditor either judged the "importance" of the finding in the decision to document, or may have had additional information which was not included in the audit files. There was no discernible pattern for not documenting identified unsatisfactory conditions.
It was noted that the category of " concern" was deleted from audit reports in April 1904 with the issue of revision 7 of DQI-CS-4.6.
All discrepancies are now categorized as deficiencies in the TUGC0 audit program.
It is the responsibility of the Audit Team Leader to ensure that the audit report
\\
accurately reflects the audit findings as
,j/
documented by the audit team, and that s
pertinent backup data is included in the audit files.
There were occasions when major sections of checklists were not completed which, in effect, altered the scope of the audit. This was not documented in the audit report. This could have provided an inaccurate view of audit program coverage.
In some cases, the reason for the omission was noted on the checklist (e.g., " time constraints", " covered by another checklist",
"would have required verification of other procedures", etc.), but in some cases was not.
There was no discernible pattern in the stated reasons for not completing checklists.
In March 1985, the QA Audit Supervisor i
instituted a " Checklist Review Verification Form" (not procedurally defined) which is used by the Audit Team Leader to document the i
extent of completion of the audit checklist.
f\\
This form is now utilized in the audit
,)
planning process to ensure that incomplete
\\
s items are rescheduled as required.
i
Revision:
1 Page 34 of 53 RESULTS REPORT fs s
s_s ISAP VII.a.4 (Cont'd) 5.0 IMPLEMENTATION OF ACTION PLAN AND DISCUSSION OF RESULTS (Cont'd)
It was noted that the TUGC0 audit reports do not include a formal Recommended Corrective Action for identified deficiencies. The language of ANSI N45.2.12-1973 can be interpreted either way as to whether or not this is a requirement. The RTL supports the TUGC0 position, although it is contrary to experience, because the responsible organization is better able to determine the proper course of corrective action and is ultimately responsible for the final determination and implementation. The QA organization remains responsible for evaluation and approval of the course of corrective action as well as verification of effective implementation.
O Audit report distribution did not appear to be consistent in that the reports in some cases were addressed to the supervisor responsible for the activity requiring corrective action rather than to project management, although project management did receive copies. Although this is technically not a violation of requirements, it can cause project management to be omitted from direct participation in the reporting, investigation, and corrective action activities of the audit process. In addition, original recipients did not always receive subsequent correspondence related to follow-up and closecut.
Conclusions It is concluded that the lack of information in early audit reports (e.g., audit scope, summary of audit results, evaluation statement) could have resulted in an adverse effect on the audit program in that it would have contributed to a lack of full understanding by the reader of the adequacy and effectiveness of the QA program as evaluated by the QA organization.
O
^
]
Revision:
1 o7 Page 35 of 53
('~
RESULTS REPORT t
a ISAP VII.a.4 (Cont'd) 5.0 IMPLEMENTATION OF ACTION PLAN AND DISCUSSION OF RESULTS (Cont'd)
It'is concluded that the failure to properly document audit deficiencies could have resulted in an adverse effect on che adequacy of the audit program in that appropriate management may not be made aware of all items requiring corrective action or of their importance.
It is concluded that the failure to document deviations from-the audit. checklists could have resulted in an adverse effact on the audit program in that appropriate
- ~
management migh'c not be made aware of program activities / elements which were not evaluated.
'J It is concluded that the failure of the auditor to y
recoumend effective corrective action for audit dificiencies resulted in no adverse effect en the
/g audit program because the organization responsible for the activity has the ultimate responsibility for (v) determination of corrective action.
It is concluded that the lack of consistent
- t distribution of audit reports and subsequent correspondence could have resulted in an adverse effect r
on the adequacy of the audit program in that project management would not be involved in the activities pertaining to investigation, corrective action and cloceout of audit deficiencies.
5.1.7 Follow-up and Closcout The final major step in the audit process is timely follow-up, by both the audited' organization and the auditing organization.
The review of the audit report files indicated that, throughout the program, the ectly steps of the follow-up process appeared to have been completed in a timely manner by both the audited organization and the auditing organization. This included determination of, l
and schedule for completion of corrective action by the audited organization, evaluation of the proposed corrective action by the auditing organization and additional correspondence between the two, as required, until resolution was reached.
g
-,---._.r
R* vision:
1 o-
+
Page 36 of 53 RESULTS REPORT ISAP VII.a.4 (Cont'd) 5.0 IMPLEMENTATION OF ACTION PLAN AND DISCUSSION OF RESULTS (Cont'd)
The timeliness of implementation of corrective action by the audited organization cannot be ascertained from the audit files for reasons that are discussed below.
Interviews with the Assistant QA Manager indicate that the general, feeling within the auditing organization was that implementation of corrective action was timely. However, the timeliness of verification of implementation of corrective action by the auditing c.
organization, as well as closeout of the audit deficiencies and audit reports was often unsatisfactory.
This situation was eventually recognized and addressed by TUGC0 QA through the performance of audit TCP-ill,
" Closeout of Open Consttuction Audit Findings" in August 1984. This audit was performed to verify implementation of corrective action for 83 deficiencies O
and 22 concerns, some of which had been open for over six years. During the audit, all but four (4) items (2 deficiencies and 2 concerns) were found to have been completed. These four items were reissued as three deficiencies against audit TCP-lli, which were then closed in a follow-up two months later.
The NEC nddressed the inability of TUGC0 QA to close an audit finding from TCP-23 during the conduct of audit TCP-68.
The deficiency in question was subsequently identified as audit TCP-40, Deficiency 2.
It was determined that this deficiency was closed during the conduct of audit TCP-99 in March 1984.
Contributing causes to the lack of timely follow-up were a combination of (1) the practice of each lead auditor being responsible for scheduling applicable follow-ups during subsequent audits, (2) related audits being scheduled up to a year later, and (3) lack of a formal tracking and status system for audit deficiencies. The apparent lack of urgency for follow-up may have been influenced in part by the general feeling that implementation was indeed timely, and therefore additional action was not required to expedite implementation.
)
Procedure D0?
-1.8, " Internal Audit Deficiency Follow-up/'
3eout" issued in June 1985,. addressed
-these problems by instituting a tracking system for internal audit deficiencies. The tracking system described in the procedure is adequate.
s
~
Rsvision:
1 F
Page 37 of 53 w
A
/' \\
RESUL?G REPORT ISAP VII.a.4 (Cont'd) 5.0 IMPLEMENTATION OF ACTION PLAN fND DISCUSSION OF RESULTS (Cont'd)
Conclusions It is concluded that the failure to perform timely verification of implementation of corrective action for audit deficiencies could have resulted in an adverse effect on the adequacy of the audit program in that one of the major goals of an audit program is to identify and ensure corrective action for program deficiencies.
5.2 Audit Personnel Qualification Program documents and audit personnel training / qualification files were reviewed to evaluate the adequacy and effectiveness of the TUCCO program for qualification and certification of audit personnel. The results of these reviews are summarized in the succeeding paragraphs.
/N 5.2.1 Qualification Requirements (v)
Prior to June 1983, TUGC0 was committed to the requirements of ANSI N45.2.12-1973 for audit personnel training and qualification. The TUGC0 written program for this period was in compliance with the ANSI requirements.
It was noted from reviewing the training and qualification files for this period that, beginning in 1977, a form similar to the form contained in ANSI N45.2.23 was being used to document certifications for lead auditor. The use of the modified form was l
proceduralized in 1980.
It was noted that in the use of this form (for which there was yet no commitment) i t
TUGCO was awarding qualification credits for attributes in addition to those specizied in ANSI N45.2.23 and was not requiring the same number of qualification credits as was the ANSI.
(
Revision 5 of CQI-QS-2.1, " Qualification of Audit Personnel," issued June 1, 1983, resulted in the written program being in compliance with ANSI N45.2.23-1978. At this time, audit personnel who had been qualified and certified under the previous procedure revision were reevaluated and new
'j certifications issued.
This procedure has since been cancelled and the appropriate material incorporated I
into DQP-QA-2, Revision 11.
The current procedures are in compliance with governing requirements.
L
Revision:
1 Paga 38 of 53 RESULTS REPORT
\\
ISAP VII.a.4 (Cont'd) f 5.0 IMPLEMENTATION OF ACTION PLAN AND DISCUSSION OF RESULTS (Cont'd) i Conclusions It is concluded that the audit personnel qualification program adequately reflected the requirements of the appropriate governing standards and regulatory guidance and therefore resulted in no adverse effect on the audit program.
4 5.2.2 Training / Qualification Records t
Of approximately fifty (50) personnel identified as having performed as auditors or lead auditors since 1978, the personnel qualification files of 41 of these individuals were reviewed for evidence of education, training and experience used as a basis for qualification and/or certification. The personnel reviewed were all members of the TUGC0 QA organization, though not necessarily assigned to the audit staff.
Included in this total are all who performed as auditors or lead auditors since 1981, as well as some who had participated earlier.
All files, except as discussed below, contained adequate documentation (e.g., resume, training records, 2
i reading lists, certifications, etc.)
appropriate to the time of certification, including evaluation and recertification of personnel on-board on June 1, 1983 (Date of Commitment to ANSI N45.2.23-1978).
i Files for six personnel contained no resume or other independent supporting documentation for lead auditor certifications issued in 1977, 1978, and 1981. None of
}
these personnel have participated in the program since 1982. Although no judgement of competency of these personnel can be made from the records reviewed, others certified during this period were determined to be adequately qualified, based on their records.
Review of Lead Auditor qualification files indicates that all personnel currently certified as Lead Auditor, except as noted below, have satisfied the requirements of ANSI N45.2.23-1978.
\\s_,f Although there is no regulatory requirement to verify the education, experience, etc., for audit personnel, cit was noted that TUGC0 QA is in the process of performing these verifications for the current staff and has incorporated such a requirement in DQP-QA-2.
Ravision:
1 e'
Page 39 of 53
')
(J
~~
RESULTS REPORT w
ISAP VII.a.4 (Cont'd) 5.0 IMPLEMENTATION OF ACTION PLAN AND DISCUSSION OF RESULTS (Cont'd)
ANSI N45.2.23-1978, Section 4.2 states that the development and administration of the examination for lead auditor is the responsibility of the employer. It also states that this activity may be delegated to an independent certifying agency but that the employer shall retain responsibility for conformance of the J
examination and its administration to the standard.
The applicable TUGCO procedure does not specifically reflect this requirema.nt, although it could be interpreted to do so.
During the review of qualification files of current staff members, it was observed that at least three personnel had been granted certification as lead auditor who had no record of being administered an examination by or for TUGCO. They had been given credit for examinations administered by previous
[
employers.
It is believed that this action has not
\\
resulted in the certification of unqualified personnel, but it constitutes a violation of the requirements of 1
Based on the review of audit personnel training and qualification files, it was concluded that with the exception pertaining to the examination of Lead Auditors, the TUGC0 program for qualification and certification of audit personnel was satisfactorily fmplemented.
Conclusions It is concluded that independent backup information not being in the files for auditors in the past resulted in no adverse effect on the audit program in that the files did contain evidence that QA management had performed an evaluation of the personnel and found them to be qualified.
l It is concluded that the failure of TUGC0 to administer l
lead auditor examinations to particular individuals l
resulted in no adverse effect on the audit program in l
that the personnel are otherwise qualified and are l
believed capable of satisfactorily performing as lead auditors.
l l
l
i Rtvision:
1 i
Page 40 of 53 j
RESULTS REPORT
-~
k ISAP VII.a.4 (Cont'd) 5.0 IMPLEMENTATION OF ACTION PLAN AND DISCUSSION OF RESULTS (Cont'd) 5.3 Evaluation of Findings This section summarizes the deviations and weaknesses identified throughout this report. Each item includes the determination of whether or not it was considered to be a QA/QC program deviation, states a recommended corrective action if appropriate, or describes corrective action which has aTready been taken.
The corrective actions describe below are intended to preclude recurrence of the identified deviations.
It has been determined that remedial corrective action would provide no additional benefit to the QA Audit Program.
Audit Program 10CFR50, Appendix B, Criterion I, states in part, the authority and duties of persons and organizations performing activities affecting the N
safety-related functions of structures, systems, and m
i components shall be clearly established and delineated j
in writing..." Criterion XVIII states in part, "A comprehensive system of planned anc periodic audits shall be carried out.
Contrary to the above, the CPSES FSAR does not address the organizational responsibility for the definition l
and implementation of the audit program.
l l
This is a QA/QC Program Deviation.
l l
Therefore, it is recommended that the CPSES FSAR be revised to define the responsibilities for the audit program. A proposed change to the FSAR has been submitted to TUGC0 licensing, has been reviewed by the RTL and found to be acceptable.
I 10CFR50, Appendix B, Criterion I, states in part, the authority and duties of persons and organizations performing activities affecting the safety-related functions of structures, systems, and components shall be clearly established and delineated in writing..." Criterion XVIII states in part, "A comprehensive system of planned and periodic audits shall be carried out.
s Contrary to the above, the QA program documents did not delineate responsibilities for the Supervisor, QA Audits.
I Revision:
1 3
Page 41 of 53 RESULTS REPORT l
ISAP VII.a.4 (Cont'd) i 5.0 IMPLEMENTATION OF ACTION PLAN AND DISCUSSION OF RESULTS (Cont'd) i This is a QA/QC Program Deviation.
?
j Corrective action was taken in DQP-QA-16, Revision 0, issued in September 1985. This has been reviewed and found acceptable.
10CFR50, Appendix B, Criterion V, states in part,
" Activities affecting quality shall be prescribed by documented instructions, procedures.
. of a type appropriate to the circumstances..."
ANSI F45.2.12-1973, paragraph 3.2, states in part, "An effective audit system...shall include.
~
provisions for reasonable and timely access of audit personnel to facilities, documents, and personnel necessary in the planning and performance of the audits.
ANSI N45.2.12-1973, paragraph 4.5.1, states in part,
"... management of the audited organization or l
activity shall review and investigate the audit i
findings to determine and schedule appropriate corrective action. They shall respond to the report in writing, within 30 days after receipt."
The " Gray Book", paragraph D.2.b. states in part, j
.during the [ post-audit] conference an effort shall be made to establish a tentative course and schedule of corrective action for non-conformances.
Where it is not possible to provide such information at the post-audit conference, the management of the audited organization should commit to a specific date for determination of the course of corrective action and the schedule for impicmentation."
Contrary to the above, naither the TUCCO QA Program nor the CPSES QA Plan, the documents in the written program which would provide direction to the management of the audited organizations, addresses the requirements.
(
This is a QA/QC Program Deviation.
It is recommended that appropriate QA program documenta (e.g., Corporate QA Program, Site QA Plan) be revised to include these requirements.
The TRT concluded that the QA audit program had inadequate staffing during peak periods.
t m.
m.
' - " - - - - - - - - - " ' - ' - - - ' ' ~
Rsvision:
1 g
Page 42 of 53 RESULTS REPORT ISAP VII.a.4 (Cont'd) 5.0 IMPLEMENTATION OF ACTION PLAN AND DISCUSSION OF RESULTS (Cont'd)
This NRC finding was evaluated by considering both the designated audit staff and other members of the QA organization who performed as auditors. The total number of participants in audits shows that the audit staff was adequate in numbers and qualifications.
l Accordingly, this NRC Finding was not substantiated.
ANSI N45.2.12-1973, paragraph 3.4.2, states in part,
" Applicable elements of the Quality Assurance Program shall be audited at least annually or at least once within the life of the activity, whichever is shorter."
Contrary to the above, internal annual audits were not specified in procedure from 1981 to 1984.
This is a QA/QC Program Deviation. The NRC finding in this area was substantiated.
Corrective action was taken in DQP-CS-4, Revision 11 issued in November, 1984. This has been reviewed and found acceptable.
ANSI N45.2.12-1973, paragraph 3.4.2, states in part.
" Applicable elements of the Quality Assurance Program i
shall*be audited at least annually or at least once within the life of the activity, whichever is shorter."
Contrary to the above, vendor annual audits were not specifdad in procedures.
i This is a QA/QC Program Deviation. The NRC finding in this area was substantiated.
i
{
Corrective action was taken in FSAR Amendment #52 issued in August 1984 and DQP-CS-4, Revision 11 issued in November, 1984. This has been reviewed and found acceptable.
ANSI N45.2.12-1973, paragraph 3.4.2, states in part,
+
" Applicable elements of the Quality Assurance Program shall be audited at least annually or at least once within the life of the activity, whichever is shorter."
9 e
\\ f\\
Contrary to the above, annual audits were not performed for contractor organization in two (2) instances.out of, 9
sixteen (16) required audits from 1973 to 1977.
_______m
i' Rsvision:
1 Page 43 of 53 RESULTS REPORT
!k ISAP VII.a.4 f
(Cont'd) i i
j 5.0 IMPLEMENTATION OF ACTION PLAN AND DISCUSSION OF RESULTS (Cont'd) l This is a QA/QC program deviation. The NRC finding in i
this area was substantiated.
4 Corrective action was taken by QA management in that i
for the remainder of that phase of the TUGC0 audit program, audits of major contractors were performed at least annually.
9 ANSI N45.2.12-1973, paragraph 3.4.2, states in part, j
" Applicable elements of the Quality Assurance Program shall be audited at least annually or at least once within the life of the activity, whichever is shorter."
Contrary to the above, annual vendor audits were not performed in that initiation was not timely and/or i
intervals were too large in all twelve (12) organizations examined between 1976 and 1984.
This is a QA/QC program deviation. The NRC finding in this area was substantiated.
Corrective action was taken in DAP-CS-4, Revision 11, i
issued in November 1984. This has been reviewed and j
found acceptable.
NRC Notice of Violation (445/8432-03, 446/8411-03) stated that the W site organization was not audited i
i between 1971 and 1981.
It was determined that the nature of the work activities for the W site organization did not involve i
control of safety-related activities and therefore is not required to be included in the audit schedule.
i Accordingly, this NRC Finding was not substantiated.
i ANSI N45.2.12-1973, paragraph 5.3, states in part,
" Planning of the audit system shall be conducted to assure coverage of the applicable quality assurance program... and in paragraph 3.4.2, " Audits'shall
. assure conformance with the developed and implemented program."
l Contrary to the above, audit checklists were developed from existing procedures rather than governing requirements for the majority of audits reviewed.
For i
example, no assessment of the written program adequacy 1
waa performed in areas of non-conformance or corrective action or QC Training / Qualification / Certification.
. -.., _., -, __,,_, _., _.. - - _,-_m.
m_..__,,__
i Rsvision:
1 Paga 44 of 53
]
RESULTS REPORT j
ISAP VII.a.4 (Cont'd) l 5.0 IMPLEMENTATION OF ACTION PLAN AND DISCUSSION OF RESULTS (Cont'd)
This is a QA/QC Program Deviation.
Corrective action consists of use of an " Appendix B checklist". This has been reviewed and found acceptable. However, for program enhancement, DQP-AG-2 should be revised to reflect use of this checklist.
' Checklists did'not provide verification that procedures in different systems were consistent for the same activities performed by different organizations.
This is not a QA/QC Program Deviation.
ANSI N45.2.12-1973, paragraph 1.5.1, states in part,
"[An] Audit [is] a formal documented activity performed in accordance with written procedures or checklists to verify, by evaluation of objective evidence that a p
quality assurance program has been.
. implemented in accordance with applicable requirements of ANSI N45.2.
Contrary to the above, the audit preparation phase was less than adequate because audits did not provide sufficient checking of program adequacy for the activities being audited and resulted in incomplete evaluation of the adequacy of the overall QA/QC program.
This is a QA/QC Program Deviation.
1 Corrective action consists of the use of an " Appendix B Checklist." This has been reviewed and found acceptable. However, for program enhancement, it is recommended that procedure DQP-AG-2 be revised to reflect the use of this checklist.
ANSI N45.2.12-1973, paragraph 4.2.2, states in part, responsibilities (of the audit team leader] include orientation of the team,..., assuring communications within the team and with the organization being audited, participation in the audit performance, and coordinating the preparation and issuance of reports."
Revision:
1 Page 45 of 53 i
RESULTS REPORT ISAP VII.a.4 (Cont'd) 5.0 IMPLEMENTATION OF ACTION PLAN AND DISCUSSION OF RESULTS (Cont'd)
ANSI N45.2.23-1978, paragraph 1.1, states in part
" requirements... for the qualification of audit team leaders, henceforth identified as a " Lead Auditors", who organizes and directs audits, report audit findings, and evaluates corrective action."
Contrary to the above, Lead Auditor candidates were designated as acting team leaders (with signature authority).
This is a QA/QC Program Deviation.
Corrective action was taken in DQP-CS-3, Revision 12, -
issued in February 1985. This has been reviewed and found acceptable.
ANSI N45.2.12-1973, paragraph 4.4, states in part "An O
audit report shall be written and signed by the audit team leader which provides:
Description of the audit scope A summary of audit results including an evaluation statement regarding the effectiveness of the QA program elements which were audited."
Contrary to the above, audit reports did not include scope description and evaluation statements.
This is a QA/QC Program Deviation.
Corrective action was taken in CQI-CS-4.6, Revision 0, issued in July 1981. This has been reviewed and found acceptable.
Audit files contain instances of:
Field notes on identified conditions.that are not properly addressed in the final report.
Checklists that are not completed.
Ravision:
1 3=
Page 46 of 53 IO RESULTS REPORT ISAP VII a.4 1
(Cont'd) 5.0 IMPLEMENTATION OF ACTION PLAN AND DISCUSSION OF RESULTS (Cont'd)
This is not a QA/QC Program deviation, but indicates i
implementation weaknesses in that the documented record j
does not reflect all actions of the audit team.
ANS1 N45.2.12-1973, paragraph 4.4, states in part,
" Distribution of the [ audit] report shall include responsible management."
Contrary to the above, original recipients did not 4
always receive subsequent correspondence related to follow-up and closeout of audit reports.
This is a QA/QC Program Deviation.
Corrective actio'n was taken in DQP-AG-5, Revision 0, dated September 9, 1985. This has been reviewed and found acceptable. However, for program enhancement, applicable procedures should be revised to define the
)
addressees and minimum distribution for audit reports.
ANSI N45.2.12-1973, paragraph 4.5, states in part,
" Follow-up action shall be performed by the audit team leader or management of the auditing organization to:
.. confirm that corrective action is accomplished as scheduled."
2 Contrary to the above, QA verification of timeliness of closeout was unsatisfactory although audit personnel believed that actual implementation of corrective action was timely, i
This is a QA/QC Program Deviation. The NRC finding in i
this area was substantiated.
i Corrective action was taken in DQI-AG-1.8, Revision 0, issued in June 1985. This has been reviewed and found acceptable.
Audit Personnel Qualifications The procedure for qualification of cudit personnel contained weaknesses in that, for example, OJT could be performed under qualified s
auditors in addition to Lead Auditors.
i
Rsvision:
1 Pags 47 of 53 1
p
)
RESULTS REPORT C
ISAP VII.a.4 (Cont'd) 1 5.0 IMPLEMENTATION OF ACTION PLAN AND DISCUSSION OF RESULTS (Cont'd) 1 The required reading list was incomplete.
There was no stated requirement to retain i
information pertaining to examinations.
I This is not a QA/QC Program Deviation.
When procedure DQP-QA-2, Revision 11 was issued in December 1985, the weaknesses were eliminated.
i l
Six (6) audit personnel qualification files in the Il period prior to June 1983 are incomplete.
This is not a QA/QC Program Deviation.
ANSI N45.2.23-1978 Paragraph 4.2, states in part, "The development and administration of the examination for lead auditor... is the responsibility of the employer..."
i Contrary to the above, at least three members of the i
current audit staff have been certified as Lead Auditors by TUGCO, but have not been administered an examination by, or for TUGCO.
4 This is a QA/QC Program Deviation.
]
l It is recommended that these personnel, and any others currently certified who did not meet the requirements.
be administered an examination in accordance with the requirements of the standard.
5.4 Trend Analysis A total of fourteen (14) QA/QC Program Deviations were l
identified. These findings are characterized as failures to fully translate audit program requirements into the procedures j
used to implement the program. This can also be characterized as a lack of specificity in the CPSES implementing procedures. This therefore resulted in a failure to properly implement the audit program.
Because each of the deviations can be traced to inadequacy of procedures, this has been determined to constitute a trend.
i Therefore a trend analysis has been performed in accordance with Appendix E of the CPRT Program Plan.
l I..
~
Rsvision 1
Page 48 of 53
.s f
RESULTS REPORT s
s_-
ISAP VII.a.4 (Cont'd) 5.0 IMPLEMENTATION OF ACTION PLAN AND DISCUSSION OF RESULTS (Cont'd)
None of the deviations described in Section 5.3, taken individually, is believed to represent a significant breakdown in the audit program. However, taken collectively, there was a potential for breakdown in the audit program. Therefore this is considered to be a trend adverse to the audit program.
~
As noted previously, there is no direct connection between an audit program and the quality of installed hardware. For this reason, it cannot be concluded that this trend is an " adverse trend" as defined by Section 5 of Appendix E of the CPRT Program Plan. However, an inadequate audit program can fail to identify weaknesses in the prescribed QA/QC programs that in turn could allow hardware problems to go undetected. For' this reason the RTL has determined that a root cause and i
generic implications analysis should be performed.
5.5 Root Cause and Generic Implication Evaluation The NRC findings led to two hypothesized root causes which s.
were:
- 1) TUEC's audit procedures did not comply with NRC requirements and 2) the program was not implemented in accordance with procedures. The CPRT evaluation also found instances of failure to implement governing requirements properly, but in each case this was traced to a lack of specificity in the implementing procedures for the CPSES Quality Assurance Audit Program. Therefore, all substantiated findings can be traced to the first of NRC's hypothesized root causes. As discussed below, CPRT investigation hrs shown that the lack of procedural compliance with NRC requf*.ements was caused by lack of a management system which wonid provide for overall coordination and control of project retivities and i
provide a consistent mechanism for translation of governing requirements into implementing procedur:s and by failure to periodically check program procedures against program j
requirements and commitments. This lack of procedural compliance remained uncorrected because of the apparent lack of full appreciation by previous TUEC management of the role of an effective QA Audit Program in ensuring the overall effectiveness of the CPSES QA Program and therefore is considered to be the root cause.
l The generic implications of this root cause will be evaluated l
during the conduct of other ISAPs and will be considered i
during the collective evaluation phase of the QA/QC Program.
l l
l l
l L
~
R:violen:
1 Page 49 of 53 t
(N RESULTS REPORT ISAP VII.a.4 (Cont'd) 4 5.0 IMPLEMENTATION OF ACTION PLAN AND DISCUSSION OF RESULTS (Cont'd) i A substantive contributing cause for the lack of a full assessment of the completeness of QA Audit Program implementing procedures is determined to be the lack of a procedural hierarchy for CPSES. A procedural hierarchy would provide for at least one additional tier of procedures which would provide a bridge from the requirements documents to the implementing procedures, and in addition would provide a vehicle for management to describe the overall coordination and control of project activities and organizations for topics 7
such as the following:
Interpretation and application of commitments to codes, standards and regulations, Definition and application of self-imposed requirements.
Standard eation of terminology and definitions.
Delineation of responsibilities, Description and responsibility of organizationni interfaces and j
Structure and content of sub-tier procedures.
A potential generic implication is that the types of procedural inadequacies identified in the audit program may exist in other procedure systems at CPSES. The lack of an upper tier of procedures covering the topics identified above leads to procedure development by each organization without formal corporate guidance or formal coordination between organizations. Further investigation is required to determine if this lack of standardization contributed to similar procedure inadequacies in other areas or organizations.
Further investigation is beyond the scope of this action plan and will be addressed as part of Collective Evaluation of the Construction QA/QC Program using information from this and other ISAPs.
Corrective actions related to the adequac) of the current CPSES QA Audit Program have been identifis.d. Additi mal actions to prevent future occurrence of similar deviations or I
programmatic weaknesses are discussed in Section 8.0 below.
i 1
R3vicion:
1 Pags 50 of 53 RESULTS REPORT ISAP VII.a.4 (Cont'd)
6.0 CONCLUSION
S Section 4.1.2.3 of ISAP VII.a.4 describes conditional steps that were to be performed depending on the results of reviews and evaluations. Although audit program deviations and weaknesses related to construction activities were identified, it has been determined that no action is required beyond that specified in ISAP VII.c, which addresses all areas of safety-related hardware and is intended to provide confidence that any currently unidentified concerns related to the quality of construction of the CPSES hardware will have been identified, evaluated and resolved.
Therefore, no additional programa or plans were developed to address this area. Audit program deviations applicable to off-site TUGC0 suppliers were also identified. During the investigation and evaluation of the impact of these deviations, sufficient information was obtained to provide confidence that the acceptability of the suppliers' quality assurance programa for the applicable equipment and services could be, and was, evaluated by TUGC0 during the periods in question. Therefore, no additional programs or plans were developed to determine the acceptability of the suppliers' quality assurance programs based on the efforts of others.
As a result of this review of the TUGC0 QA audit program, it has been determined that at no time has the written program been in full compliance with governing standards and regulatory guidance, and in addition has exhibited other weaknesses as described in this report. This determination substantiated the TRT finding that
"... TUEC's audit procedures did not comply with NRC requirements."
The overall effectiveness of the audit program has been less than fully adequate. Specific examples are it has failed to identify l
and cause corrective action of inadequacies in such areas as QC l
inspector training, qualification, and certification; and control of non-conforming items and corrective action.
It is concluded, i
from the evidence observed, that the failure to identify and cause corrective action in these two areas was due primarily to the practice of auditing to existing procedures while not performing verification of the adequacy of existing procedures to implement program requirements.
It is further concluded that the cause of the deviations and weaknesses in the QA audit program which have j
been identified in this report are the result of inadequate i
procedures. This resulted from the lack of a procedure hierarchy which would have provided for overall coordination and control of project activities and organizations by senior management for i
topics such as the following t
L
a Ravision:
1 ga Page 51 of 53 RESULTS REPORT ISAP VII.a.4 (Cont'd)
6.0 CONCLUSION
S (Cont'd)
Interpretation and application of commitments to codes, standards and regulations.
Definition and application of self-imposed requirements, Standardization of terminology and definitions, Delineation of responsibilities.
Description and responsibility of organizational interfaces and Structure and content of sub-tier procedures.
Based on the written record the overall competency of the audit personnel appears to be adequate to perform the required audits for the design, conutruction, and operation phases of CPSES. Except as previously noted, the program fer qualification of audit personnel is considered to have been adequate, although TUGC0 was slow to recognize and incorporate applicable standards and guidance available to the nuclear industry.
The NRC TRT and Region IV findings concerning seditor staffing and qualifications during 1981, 1982, and 1983 couli be considered valid if the program had been totally dependent on the personnel assigned to the permanent audit staff. However, in light of the total number and qualifications of personnel available to, and utilized in, the audit program, the audit staff is considered both adequate and qualified.
Audit planning and scheduling, though in the past not in compliance regarding frequency and not formally systematized until recently, did appear to be well thought out in the context of the TUGC0 concept of the audit program requirements at any particular time.
However, due to the lack of a procedure hierarchy which would provide a firm basis for the development of implementing procedures, the effectiveness of the audit program to determine the adequacy of the construction QA program was less than desirable.
There is evidence that appropriate consideration has been given to input from sources such as NRC inspections, previous audits, and surveillance.
Except as noted, the findings pertaining to scheduling of audit frequency are valid. Appropriate procedures have been revised, which now properly address scheduling of audit frequency.
6 R& vision:
1 9...
Page 52 of 53 g'~N RESULTS REPORT
\\
ISAP VII.a.4 (Cont'd)
6.0 CONCLUSION
S (Cont'd)
The current written program for auditing and audit personnel qualification is in compliance with applicable requirements except in the following areas:
Definition of responsibility for the audit program in the FSAR Definition of responsibilities of the audited organization In addition, at least three currently certified lead auditors have not been administered an examination by or for TUGCO.
Appropriate corrective action should be taken in these areas.
1 7.0 ONGOING ACTIVITIES
^
The Audit Program and procedures discussed in this report are applicable to the remaining construction phase on Unit 2 and for
)
the operating phase of the plant.
s j
8.0 ACTION TO PRECLUDE OCCURRENCE IN THE FUTURE To prevent future occurrence of the same or similar deviations and weaknesses in the QA audit program (or other portions of the CPSES program) management, particularly senior management (Vice President and above), must actively participate in the program and be kept appraised of audit program results.
This may be accomplished through development of a hierarchy of procedures as described in Section 6.0 of this report. This would provide the necessary standards and management controls under which the overall program can be effectively defined and implemented.
In addition, the procedures should provide for direct senior management participation in the audit program in areas such as audit planning and scheduling, and resolution of programmatic problems.
Management at all levels must utilize the output of the audit program (audit reports, summaries, trend analyses, etc.)
to evaluate the adequacy and effectiveness of their portion of the QA program. This should be accomplished by developing requirements I
for appropriate participation in the development of corrective
)
action for identified deficiencies, and follow-up to ensure that j
'V the corrective action has been effectively implemented.
I L
Ravision:
I g,,.
Page 53 of 53
<s RESULTS REPORT ISAP VII.a.4 (Cont'd)
6.0 CONCLUSION
S (Cont'd)
In addition, an irlependent, periodic assessment of the audit program is essential to provide Senior management with timely identification and corrective action of deficiencies or weaknesses in the audit program. Such a program will be defined in ISAP VII.a.5.
'l 1
4 i
1 i
e 1
i i
l