ML20215E991
| ML20215E991 | |
| Person / Time | |
|---|---|
| Site: | Comanche Peak  |
| Issue date: | 07/06/1979 |
| From: | Norris J MANAGEMENT ANALYSIS CO. |
| To: | |
| Shared Package | |
| ML20214X072 | List: |
| References | |
| MAC-79-G021, MAC-79-G21, NUDOCS 8610160044 | |
| Download: ML20215E991 (7) | |
Text
-
_ ._ _.__..._..--m .
- - " Nca~-
- ' ~
' ~
, ' k MAC-79-G021 CRITIQUE OF THE FOURTHANNUAi.MANAGEMENTf.UDIT CONDUCTED FOR THE -
BROWN & ROOT, INC.
QUALITY ASSURANCE MANAGEMENT REVIEW BOARD J. M. Norris
~
r July 6 J979 MANAGEMENT ANALYSIS CCMcANY 1}100 ROSE 1.LE STREET f : G,'.
SAN DIEGO. CALJFORNIA 92121 Su 1882f 81838 6 - - -
A PDR
- ,,-e- - -- - ,-. - - - . . - -- -,-------..-..-.r,ww.
.w .v .+. w ' w.1
.wvNw- -
N.:n'i m w w M2 & N & W " " '
[
~
.{.
~
SUMMARY
The fourth annual management audit of the Brosn & Root Power Division Quality @
Assurance Program was conducted during a three week period starting June 4, 1979. The team was led by Peter F. Foscola, Brown & Root Construction. -
Except for one person, J. M. Norris, Management Analysis Company..who served as an outside auditor' and an audit team' advisor, all of the team members were ,
Brom & Root employees. Half the team members had previous experience in conducting audits. This was consistant with a recommendation made following -
the third QAMRS audit, that a higher percentage of the audit team have prior auditing experience. -
Preparation for the audit started May .18,1979, although there were audit team meetings prior to that date. The preparation included an auditer training program conducted by Brown & Root, review of Brown & Root Quality Assur.ance Manual and related pro,fect documents and implementing procedures, and a reference documents such as NRC Requirements and American National Institute Consensus Standards. .
GE.':
b .;-
The method of using Brown & Root personnel as team members and as a team '~
leader continued to show merit' as a means of familiarizing Brown & Root personnel with the requirements for the quality as'urance s program and means of its implementation. ,While the team leader for the 1979 QAMRB audit is to be complimented for his performance during the audit, the efficiency and effeitivness of the audit can be improved by using team leaders with prior experience in planning, scheduling, conducting and in reporting audits.
Because of its industry acceptance, it is again reconnended that team leaders be qualified in accordance with American National Standards In*,titute Con-sensus Standards ANSI N45.2.23.
I The audit team members participated diligently in the preparation of audit t
questions and in the conduct of the audit. More than ample time was allowed for preparing for the audit; it is reccmmended that a higher proportion of such time be spent in training auditors in the proper formation of audit questions, in documenting findings succinctly and incontrovertibly, and 'n i
. y;.
Qt -
l .
- - - - --* ._---.--__n____ .,----.,-------,,.-,..-..-,..-,.-...--,.,,,-,,_n. . - - . . - , - - , - , , _ , ,,n, - , - - - . - - - , - , ,
- - - . . c . .. .
.i'=:71 _t qv.*v 3 :y, sg, eg% g yo g.g y,3Rgig, go 'gy;gss'g;, ;,Q
, gs:((,'
, i
_ conducting interviews, researching files and preparation for and participation
- - i in pre and post audit meetings. Future audits should provide more time for audit performance, particularly on the STp project. The plan for preparing for the audit should .be made more definite, and participation of the person who performs as an advisor should be coordinated well in advance of'the audit schedule to assure availability during the preparations for the audit as well as during its performance. ' The team leader and the. advisor should work as a team in planning the audit, training personnel, making assignments and review-ing performance.
As previously recommended, participation in the audit by higher levels of Brown & Root management should have a beneficial effect in making clear the importance placed on the use of the audit as a management tool, on the importance of audit findings, and on the importance of documented and effective corrective action'. '
The scope this year was expanded to include design' and procurement activities of two fossil plants and site audits of the construction of two such plants in addition to audits of the,STp nuclear ' site and of the related design and pro-curement activities.
The QARM8 audit could be considered as four to six
[Q individual. audits, depending on whether the audit of tihe sites are considered separate from design and procurement. The cooperation of audited organiza-tions was good durin~g the audit. It'-W36t'TTEaM9h65eElhat .the imped- '
ance of g ap gdskWUT J.co JEs een:.estabMshed or is und'erstood. bgsome7,of- thee 7ahdy,[,ofgaMahdd$~.'.Ihisc fs: represented' by' the attitude of some' organizations ' and ,by, thir. persfstanca'. of pi oblems identified in ear +1et-'hadfts. *
- REPORT ~
The content of this report follows the structure of ANSI N45'.2.12 " Require-
.ments for Auditing of Quality Assurance Programs for Nuclear power Plants".
Personnel ._
The technical experience of personnel selected for the audit was excellent.
In acccrdance with recommendations made in 1978, half of the audit team had previous audit experience. The inexperienced auditors performed well.
?
D -
fiEiiii : .
- e =*
,,L'. ..+ 2 L 2 _:: ..
.. . ;..u_,6 a -:- a
,, ,, .. c -
!. ..g{- '
3 '
Team Leader
8 Peter F. Foscola performed diligently as team leader. His effectivene'ss could have been improved if he had had the benefit of prior auditing experience, and in an audit as important as this one, if he had been fully qualified in accordan'ce with the requirements of ANSI N45'.2.23. It
,is difficult to simultaneously weld eight divergent personalities into an offective team, learn auditing principles and direct their application. i Mr. Foscola deserves credit for his' accomplishments in this regard during -
this audit. The efficiency of the audit could 'have been improved if he
' had had the benefit of prior training and experience. The efficiency of
~
the next audit he conducts will be improved.
Traing
- The audit team members performed well. The diligence of all team members is to be complimented. Future training should include more practice sessions in forming audit questions and in documenting succinct responses to such questions.
$.?G.~
Audit System Obfectives
.y*
'It is not clear that the objectives of the QAMR8 audit are fully under-
, stood by the audited organizations. While the general cooperation among audited organizations was good, responses of some organizations did not
. indicate that the evaluation of their performance by this method was of sufficient importance to merit th'etr interest or concern. While the scope of the audit was good, more, attention should be given to the results of prior audits and the implementation and effectiveness of corrective actions fran such prior audit. Considering'the scope of the fourth annual QARMS audit, more time should be allotted for auditing; a tradeoff of shorter preparation time would be justified.
Audit Planning .
i The audit plan developed, as in the past, was satisfactory for informa-tion to the audited organization and for guidance of the audit team members. The team leader and advisor should review plans of individual
$.'-)
.zy
! l Q ,
_ _ _ _ _ _ s-.a - = - - - -- - -- -
l w_ ..,. .., -.me- _
_ r _ ___._,u.,,mmm ~
( ,.
~ ~
4
.. team members to assure that they are prepared to maintain the pace of the audit and cover the entire audit scope. Auditoc performance sliculd be monitored against such' plans.
l Audit Schedulinq _ _ - .
A prublem encountered in the third'QARMB audit and repeated this year is the conflict with cIther audits conducted by the aliant, by the NRC, or by the Brown & Root Quality Assurance during the same timeframe as the QARMB audit. An excessive number of audits reduces their effectiveness. This
- is particularly true where two or more occur simiultaneously or in quick l succession. If Brown & Root chose to integrate its QARMB audit into its overall audit schedule, internal quality assurance audits could be i
suspended during .its performance. It is likely also that arrangements could be made to eliminate client and/cr NRC audit conflicts during the QARMB audit.
Audit Imolementation .
- Except as previously stated, the conduct of the fourth annual QARMS audit e.;#
t.
was satisfactory and the results indicate a valid audit; however, a dis-
, proportionate amount of time was spent. in preparing for the audit as compared to performing tt or preparing its report. The following addi-
, tional recomendations are made: -
.- a. The checklist format used by Brown & Root was commented on last year as to being too constraining. It still is. Further, the formation
. of questions by the -audit team should clearly reference the source of the question and provide adequate definition of the. required result to guide the auditor in his friterview and in documenting his' findings,
- b. Audit findings should be prepared daily and reviewed by the team leader and advisor with the team members. The purpose of such activity is to maintain the schedule, cross pollinate results with
' .y, -
EiiE i I
,--s - - - - - , .-
_ "- 2 .' ,w '.. L * '
._2 ?:GM'-Y ' W D- YQ .x.5-S=5.0h."--f the full audit team, . weigh the importance of findings and of con- '
N.
carns, and to consolidate and. strengthen the results of the audit. .
- c. Exit interviews are a problem area. The purpose of the exit inter-view should be to inform the audited organization of the,iesults of the audit, and to obtain concurrence -in the correctness of such
, results or identify and hopefully resolve, arty.differe.nces r:esulting to the findings. Several barriers exist to accomplishing this end. .
Generally, insufffeient time is allotted by the audit team or by the audited participants 'to permit identification and resolution of .
problem areas. -At m-.STP c3gagager: beokei. oft discussion on tWbasis that,there:' would,be ampWtime to; respond-td the written ; report and .
that..
w --ha wanc.e.t-To%
_ . _ . . . homeA., At . Mot.is.tca anuchec.. manager ob.vicusly
- teW.tlikt th,el;y%tntergegwasqaq., unnecessary .tnterruptf on {n his
.schedu.la..and.,he ;"gagg,,back to work". It ts apparent that naitner or .cness..managar.s.gieregived. the..QARMS. aud.1t as a. manageme,nt level appratsal .of. tris-:performancee,or"partigipation in the audit a part of his . assigned.w. .m. responsib.111.ty..... ..m-
.m ._.... Its. does not. appear that. either ;'-j ,,
4.
n +..
. perceive tNe -QARMS..aud5t.as, a ===g==ne.r.. tooi. he. can .use to improve i
the..peEfcreenca.-leibtheforgarrtrarferrt.fordich het is respons.ibla.. It is-notiTMTrisrFtTrah,sNerdNOl[idN'7Es fdentified as prot >1e getE3&M& arm @ide'a'Efhedd.s.'pinob'lem- areas in 1979.
It was recommended last year that the team leader prese,nt the audit findings and to team members' supplement his comments. To some degree that was followed in the exit interviews for QMm8 4; however, it was not as successful as desired. Some of the audited organizations did not desire any dissertation in the areas where. they had been found satisfactory and only wanted tc~ hear the negatives. Some wanted no
. discussion or disagreement to be aired at the meeting. Some wanted no presentation of auditors concerns on the basis that such concerns were matters of opinion. It is evident that both the audit team and the audited organizations must more fully understand the objectives, the importance and the usefulness of the QNRMB audit. More time e
i
_ ._ - A. J. ._. 4 3 :.' - i 2 m ,m
. _ #G .:.wmmermo-r:c1
\ r s, should be spent by the audit team in preparing the presentation of its findings. More time should be allotted by the audited organiza-tion t,o receive, understand and to resolve disagreements relative to such findings.
Reporting _ _ , , '
.It appears to be desirable to develop a format for the QAMRB report to I
assist the team leader and the team members in its preparation. This is -
particularly true where team members and team l'eadership change from year to year. Consideration should also be given to use of handwritten ADRs and " concerns" as attachments to the report as long as such are legible.-
Retyping fcr cosmetic effect is of little value. The complexity of this year's audit of three sites and three engineering and procurement func-tiens, the inclusion of all audit deficiency reports and auditors con-carns, precaring summaries of findings for each location and project audited, and inclusion of all checklists makes for, a very large and complex report. An attempt has been made, in a synopsis, to identify those areas where the QARMS should direct its greatest attention, however h(e[fff .
.as presented, there is some confusion as to the relative importance of audit , deficiencies and auditor concerns. Emphasis should be clearly on
'the deficienci,es. The synopsis should provide enough substance for executive action and should project the possible consequences of failure i to effect corrective measures. In an audit as complex as this one, consideration might be given. to issuing a separate and consise report for each of the projects audited. 'While the current report is arranged by projecc and location, it would probably be more effective as a tool for -
action if it were presented in a more concise manner to the individual project. The QARMB and the audited organizations must recognize that audits are a very superficial sample; thus it should require resolution of deficiencies on a generic basis, not just on the items reported, and should require comment by the audited organization on the validity and resolution of expressed auditor concerns.
1 l
Q '
E # -
71 .