ML20206M608
| ML20206M608 | |
| Person / Time | |
|---|---|
| Issue date: | 06/06/1986 |
| From: | Linehan J NRC OFFICE OF NUCLEAR MATERIAL SAFETY & SAFEGUARDS (NMSS) |
| To: | Purcell W ENERGY, DEPT. OF |
| Shared Package | |
| ML20206M614 | List: |
| References | |
| REF-WM-1 NUDOCS 8607010236 | |
| Download: ML20206M608 (1) | |
Text
_
Dirtribution:
g S/F: 3405 JDonn:lly(IE) '
WMRP r/f Malloy(IE)
NMSS PDR M $ $6 CF RShipman (0)
REBrowning 405/2 /^'/^~/19 M MJBell
_1- BCs(4)
On-Site Reps (3)
Mr. William J. Purcell, Director JGiarratana Office of Geologic Repositories DHedges & r/f Office of Civilian Radioactive Waste FCameron Management JTrapp U. S. Department of Energy MNataraja RW-20 DMattson Washington, DC 20585 BHill(IE)
SBilhorn
Dear Mr. Purcell:
TAnkrum(IE)
The purpose of this letter is to transmit Nuclear Regulatory Commission (NRC) staff reports on observations of recent Department of Energy (DOE) audits of contractors' quality assurance programs. The audits observed involved the QA programs of Raymond Kaiser Engineers / Parsons, Brinkerhoff, Quade and Douglas, Inc. (RKE/PB) and the U.S. Geological survey (USGS), and were conducted by Management Analysis Corp. (MAC) and Science Applications Inc.
(SAIC) respectively on behalf of the DOE project offices. The NRC staff members observed a number of areas needing improvement and we expect to be meeting with your staff in the near future to discuss them in greater detail.
If you have any questions, please contact Dale Hedges of my staff at 427-4491.
Sincerely, M If*'
John J. Linehan, Acting Chief Repository Projects Branch Division of Waste Management Office of Nuclear Material Safety and Safeguards
Enclosures:
Memo to G. Ted Ankrum from Craig Walenga dated May 9, 1986
[p 1. Memo to J. J. Linehan from Susan Bilhaggi.it SW W ,,,j
,o* y f-2. dated June 2, 1986 Og g g.
Yb .d$tS '/, _ _ _
" ~ " '
\ cc:
L. Olson, BWIP - ~ -
4 P. Saget, BWIP D. Vieth, NNWSI M8Ni - "- ""
J. Blaylock, NNWSI .m - . . . .
, bfD = - -
j Met _uKieXOYNV
~
T. Y t
OFC : :1 R :
n , h k 860606 _
NAME:D,. .
,. . . . . . k,,Jirht g ,....._.
eY , >, . .:JKenne$,U .
t____,____________.________ __'_:_ __________:____________:____________:____________:-__________ J
i Ee Ytanett. 1 ttg y
[fMG {gg o UNITED STATES NUCLEAR REGULATORY COMMISSION g 'j W ASHINGTO N, D. C. 20555
\ *****
/
MM 0 o m3 MEMORANDUM FOR: G. Ted Ankrum, Chief Quality Assurance Branch Division of Quality Assurance, Vendor, and Technical Training Center Programs Office of Inspection and Enforcement FROM: Craig G. Walenga Quality Assurance Branch Division of Quality Assurance, Vendor, and Technical Training Center Programs Office of Inspection and Enforcement
SUBJECT:
REPORT OF OBSERVATIONS MADE DURING A DOE-SPONSORED AUDIT OF THE KAISER ENGINEERS / PARSONS, BRINKERHOFF QUALITY
, PROGRAM.
From April 15-17, 1986, I observed an audit conducted bv Management Analysis Corporation (MAC) on behalf of the Department of Energy's Basalt Waste Isolation Project Office (BWIP) on the implementation of Raymond Kaiser Engineers / Parsons, Brinckerhoff, Quade and Douglas, Inc. (RKE/PB) quality assurance program.
RKE/PB is designing the BWIP exploratory shaft and working on a repository con-ceptual design.
On March 19, 1986, DOE's BWIP project office at Richland informed RKE/PB that an audit of RKE/PB's " quality program as it relates to exploratory shaft design and repository design" would be conducted on April 15-18, 1986. In accordance with the NRC agreement with 00E/HQ, NMSS chose to observe the conduct of this audit and ask specifically for IE participation. This three-day audit was con-ducted by two MAC employees, a Rockwell quality engineer, and a 00E/HQ QA con-tractor.
Two objectives were identified for this observation. The first objective was to determine if the DOE-sponsored audit was capable of detecting whether or not the RKE/PB QA program was being effectively implemented. The second objective was to form an independent opinion about whether or not RKE/PB's quality program is being effectively implemented. As to the first objective, the observed audit was not capable of detecting whether or not the RKE/PB QA program was being effectively implemented. The second objective was more.
difficult to achieve because the NRC observer's role was simply that--observa-tion, and there was little information from the DOE-sponsored aud;it from which a conclusion could Le based. However, there appears to be the potential that RKE/PB may be unable to ensure that its engineering products can be demon-strated to have the required quality. Appendix A is a listing of observations and associated conclusions concerning the quality of the audit. Appendix B is a listing of observations and associated conclusions concerning the RKE/PB QA program. Appendix C is a listing of other observations that are pertinent to this audit.
Yl5Db Y
s p G. Ted Ankrum By the time of the exit meeting, it appeared that a single quality audit finding would be issued concerning the lack of an approved procedure for the collection, storage, and maintenance of records and lack of a proper record storage facility. While portions of the RKE/PB QA program may be effectively implemented, the observed audit could not have substantiated this.
During the period August-November 1985, Rockwell allowed work to proceed using a " Work Around Plan," which was audited by Rockwell at the end of August 1985.
Rockwell reviewed the new NQA-1-based QA program and allowed RKE/PB to implement the QA plan and procedures in November 1985. The observed audit apparently was the first DOE sponsored audit of the NQA-1-based QA plan since its inception.
Given the potential for weaknesses shown in the RKE/PB implementation of the QA program and the weakness in the DOE-sponsored audit, the ultimate usefulness of RKE/PB's work for licensing purposes is in question and will require further review.
Craig Walenga ~da Quality Assurance Branch Division of Quality Assurance, Vendor, _
and Technical Training Center Programs Office of Inspection and Enforcement
Enclosure:
Appendices A, B, C cc: B. K. Grimes, IE R. Browning, NMSS
. J. J. Linehan, NMSS J. Kennedy, NMSS
- 0. Hedges, NMSS o
. . . - . m - - .
- 4 Appendix A Observations and associated conclusions concerning the quality of the audit.
- 1. The audit was poorly planned, did not appear to address itself to the stated objectives, and showed poor use of available time.
- The audit checklists were finalized on the day prior to the audit entrance meeting.
RKE/PB contract requirements for quality assurance were not reviewed prior to the audit or used in the preparation of audit checklists. It was not apparent that the auditors were aware of RKE/PB's contractural commitments for quality assurance.
While the audit team leader indicated that he had reviewed a 1985 audit report on RKE/PB, he did not have the audit report with him during the current audit, had not shown it to the other team members, nor was it apparent that the prior audit had been used in the pre-paration of the audit checklist.
The audit plan was so brief as to be almost unusable as a planning document.
The audit team was unprepared for its role. Apparently, the primary basis of the audit was to be the Management Analysis Corporation's (MAC's) " Auditing for Effectiveness" program. The lead auditor was a MAC employee under contract to the BWIP prime contractor (Rockwell).
No familiarization training was given to two of the four auditors who were not MAC employees. Neither of the two non-MAC team members were listed on the audit plan as auditors. Both expected to observe the audit, but found on the day of the audit, that one was to be an auditor and the other a technical specialist. None of the team members were
aware of the scope of RKE/PB's current efforts and the audit team leader
- requested that RKE/PB present a 20-minute overview of BWIP work history and the RKE/PB QA program during the entrance meeting. Even after the presentation, the auditors required further briefing during the audit to clarify what work was being performed and under what QA program the work was being accomplished.
The audit scope was not adequate to achieve the stated objectives of the audit. The March 19, 1986 letter from DGE (RL) management stated that the " audit scope is implementation of RKE/PB quality program as it
- - relates to Exploratory Shaft design and Repository design..." The audit o
plan appears to narrow the scope to the " audit of quality program 1 elements for design control..." The audit team leader indicated at the entrance meeting that the scope of the audit was to be limited to the following program areas:
Organization DocumentContdol Design Control Test Control -
Procurement Document Control Corrective Action Instructions, Procedures, and Drawings Records Management A-1 t
- b Based on discussions with the audit team leader, the decision to perform a more limited scope audit was made by a MAC employee and it was not apparent that this reduction was known to, reviewed, or
- approved by DOE- particularly since the audit checklists were not finalized until the day prior to the audit and no approvals were noted on the checklists. :
The audit was conducted over a three-day period. The first two days were not full workdays, and the third day was spent reviewing the previous two days' activities in preparation for a 3:00 PM exit meeting. A significant portion of the productive hours available was spent with the audit team being familiarized with RKE/PB QA program and procedures and in the annotation of their audit checklists.
- 2. The audit concentrated'on superficial paperwork issues.
There were no members of the team who were capable of evaluating the technical adequacy of work performed. The team member (Rockwell quality engineer), who was drafted by the audit team leader for the technical review of RKE/PB's activities, said that he was not technically com-petent to assess the calculation and design efforts, and indicated that his review would be limited to RKE/P8's procedural compliance.
The lack of technical review capability was readily apparent during an evaluation of design control. One checkprint being reviewed had numerous comments with apparent resolutions. One resolution docu-mented on the checkprint for a number of individual comments was "I'm doing it to Rockwell input." When I questioned the auditor as to the adequacy of that response, or of any of trie comment resolutions, he stated that the answer to that question "goes beyond the scope of a lowly auditor." His review was strictly for procedural compliance as he made no attempt to assess the adequacy or technical correctness of any comment resolution on the checkprint.
When some closed RKE/P8 surveillance deficiency reports were found to have an incomplete " verification acceptable" block, the auditor randomly sampled the verification of five surveillance reports. The auditor then elected to verify that corrective action had been taken for deficiencies identified in the surveillance reports. The deficiencies for which cor-rective action was verified were:
(1) pagination errors in a calculation file.
(2) failure to have " findings and conclusions" section of a calcu-
_, lation file properly titled.
(3) failure to deliver back up documents to the project files.
Despite the fact that vendor-supplied services and supplies are a recurrent QA problem area in NRC-licensed activities and that the
' RKE/PB employee who was handling procurement activities bas newly assigned, the audit was accomplished with only an intervi w. There was no indepth guidance on the audit checklist and no attempt was made to review any RKE/PB surveillance reports or audits of their subcontractors.
A-2
- 6
. ~
Despite the fact that RKE/PB had performed a corporate audit of its newly implemented NQA-1-based QA program in January 1986, the 00E-sponsored audit did not evaluate the corporate audit or check to see whether necessary corrective action was being taken on any corporate-identified deficiencies.
Training was reviewed by discussing the training progra$ with manage-ment, reviewing individual training course file folders, and reviewing the basic training form used to identify and document an individual's training. It was not apparent that any review of the adequacy, appropri-ateness or effectiveness of training was performed.
The checklists were apparently drawn from MAC's bank of prep'ared check-lists for an NQA-1 QA program, with no tailoring for the situation.
e e
we 6
A-3 i
e n
Appendix B I'
Observations and associated conclusions concerning the RKE/PB QA program.
- 1. It appears that RKE/PB may not have sufficient staff with appropriate QA expertise to assure that its engineering products can be demon-strated to have the required quality.
- 2. RKE/PB produced a letter showing that they had requested money from Rockwell to implement the records man ~gement a and document control portion of RKE/PB's QA program. No response to their request had been given. It appears that the RKE/PB QA program is being imple-mented on a task-by-task basis with RKE/PB able to proceed with QA program implementation only when funds are provided by Rockwell.
- 3. RKE/PB QA apparently does not conduct audits. However, since the audit function was not within the truncated scope of the DOE-sponsored audit, this issue was not pursued by the audit team.
- 4. RKE/PB operates on a task-by-task basis with the prime contractor (Rockwell) and thus, is not able to plan on a long-range basis.
RKE/PB individuals noted that very little work remains on current tasks, and that many engineers had to be laid off until future tasks are approved for them.
It was noted that at one time there were four individuals on the QA staff and now there is only one in addition to the QA manager. Continuit for the design engineers,ymay on the HLW have repository significant project,.
effects on the especially quality -
of RKE/PB. activities.
4 I
1' B-1
e t Appendix C Observations pertinent to the conduct of the audit.
- 1. The audit team leader stated that the audit's preparation was rushed due to time pressures. The audit plan was specifically idenuified by the audit team leader as being below normal standards due to time pressures. However, the audit plan was dated March 4, 1986, at least five weeks prior to the audit.
- 2. The MAC " audit for effectiveness" program has merit especially if it is used to evaluate a wr.itten QA program for compliance to NQA-1.
The use of the MAC program for implementation effectiveness reviews appears to be dependent on a successful combination of training, sampling, auditing techniques, and auditor qualification and experience which was not demonstrated by the audit team during this audit.
1
- 3. This audit especially makes it apparent that the NRC observer should pend one or two days prior to the audit's conduct at the 00E/ contractor site where the audit preparation is taking place to allow for a more productive observation concerning the project office's management of the audit / overview function.
e me i
l i
I
& -
- c. WiE-1 - I g .% .,
5I l