ML16340D910
| ML16340D910 | |
| Person / Time | |
|---|---|
| Site: | 05000000, Diablo Canyon |
| Issue date: | 09/19/1983 |
| From: | NRC |
| To: | NRC |
| Shared Package | |
| ML16340D758 | List: |
| References | |
| NUDOCS 8310180521 | |
| Download: ML16340D910 (84) | |
Text
DRAFT )IORKINO PAPER QUILlTY A<eUPPNCE CASE STUDY 'IRKING PAPER CASE C
PREPARED FOR NUCLEAR REGULATORY COMMISSION MAS8INGTON, D, C,
200j.Q September 19, 1983 sssaisos2i ssioi4 REVISCO P
PDR ADOCK 05000275 DRAFT l)0P}(lN PDR
DRAFT. MORKING PAPER TABLE OF CONTENTS I.
SUMMARY
OF FINDINGS A.
Introduction B.
Background
C.
Summary II.
ROOT CAUSES OF THE LICENSEE'S PROBLEMS WITH QUALITY IN DESIGN A.
Primary Root Cause B.
Secondary Root Cause III.
REMEDIAL ACTIONS TAKEN TO CORRECT QUALITY PROBLEMS IV.
GENERIC IMPLICATIONS V.
IMPLICATIONS OF THE CASE STUDY FOR NRC QA INITIATIVES A.
Measures for Near-Term Operating Licenses (NTOL)
B.
Industry Initiatives C.
NRC Construction Inspection Program D.
Designated Representatives E.
Management Initiatives 13 15 17 22 A3 24 24 25 26 VI.
IMPLICATIONS OF THIS CASE STUDY FOR THE FORD AMENDMENT ALTERNATIVES 26 A.
More Prescriptive Architectural and Engineering Criteria B.
Conditioning the Construction Permit on the Applicant's Demonstration of Its Ability to Manage an Effective Quality Assurance Program C.
Audits, Inspections, or Evaluations by Associations of Professionals Having Expertise in Appropriate Areas-Management Audits D.
Improvement of NRC's QA Program E.
Conditioning the Construction Permit on the Applicant's Commitments to Submit to Third-Party Audits of Its QA Program Appendix A Appendix B
27 27 28 28 29 DRAFT MORKING PAPER
DRAFT WORKING PAPER QUALITY ASSURANCE CASE STUDY WORKING PAPER CASE C
I.
SUNfQRY OF FINDINGS A.
Introduction The Nuclear Regulatory Commission (NRC) has undertaken a study of selected nuclear reactor construction projects to determine the important factors or root causes that underlie effective and ineffective assurance-of-quality programs.
Several nuclear projects which have experienced major quality-related problems and several which have not will comprise the study popula-tion.
Data and findings from these site-specific studies will be used by the NRC in the formulation of generic policies and programs related to assurance of quality and in responding to the Congressional Amendment to the FY 1983 Authorization Bill.
This working paper summarizes the findings from the third case study.
B.
Background
The Licensee of the Case C Study had established its own in-house engineering and construction management capability in the 1930s.
During the late 1940s and early 1950s, outside architect-engineer (A-E) firms were utilized because of unusually large (post-WMII) system expansion requirements.
In the mid-1950s, the 'Licensee's earlier practice of'oing its own engineering and construction management was resumed.
During the late 1950s and early 1960's, the Licensee planned an ambitious program to construct several nuclear power stations.
Nuclear power was recognized as a new technology and the Licensee took actions to prepare itself for entry into this field, including having observers at the construction sites of some early nuclear power plants, participating ;n the design of a test reactor, and studying A-E's designs of proposed nuclear plants.
The Licensee decided to build its first nuclear plant a small
((100fMe) power reactor through a "turn-key" contract for design and construction.
The plant was completed in the early 1960s, and the Licensee operated it successfully for about 15 years until it was retired.
The DRAFT WORKING PAPER
DRAFT HORKING PAPER Licensee capitalized on the turn-key design and construction activity to familiarize its staff with nuclear activities to enable it to engineer and construct subsequent nuclear plants.
The Licensee had been successful in engineering and construction activities on 'a variety of generating technologies and related electrical transmission systems.
Ouring the early and mid-l960s, the Licensee announced plans for several nuclear plants.
Environmental and/or seismic problems, coupled with intense intervention, political factors, load growth changes, and other considerations, resulted in all but the Case C nuclear station being cancelled.
Many of these factors were also present in the Case C project, resulting in significant delays and cost increases, The Case C nuclear station is comprised of two large
()1000NWe) units.
The Licensee announced Units 1
and 2 in 1966 and 1968, respectively.
Construction permits were issued in 1968 and 1970.
Unit 1 of the nuclear station was largely completed by the mid-1970s and fuel was received onsite for both units in 1975 and 1976.
Then occurred a series of required modifications to the nuclear station which delayed its completion.
Included in these were NRC regulations related to pipe-break-outside-containment which necessitated, among other
'things, relocation of a number of conduits (1973-75); identification and/or reconsideration of a seismic fault which required such modifications as column stiffening, tank bracing, revising piping hangers and equipment
- supports, diaphragm st'iffening, buttressinq and foundation chanqes (1978-79);
the 8rown's Ferry incident which required modifications related to cable spreading, inerting atmosphere, new decking, and extensive concrete anchor bolt installation (1980); the TMI accident which required installation.'f extensive additional wiring, sub-cooled monitors, hydrogen recombiners, and other modifications (1981).
DRAFT WORKING PAPER
DRAFT GORKI"IG PAPER It is important to note that, over the time span of about eight years, at least one of the two units had been within a few months of being completed on a number of occasions.
Thus far, Unit 1
has undergone three hot functional tests and three containment leak tests'.
Unit 2 has undergone one containment leak test.
In September 1981 the Licensee received operating licenses for its two units.
These were suspended two months later following notification by the Licensee to HRC that the diagrams used to locate the vertical seismic floor response spectra in the Unit 1 containment annulus area were in error.
Briefly,- the
~
error occurred as follows:
the Licensee had transmitted to its seismic consultant a sketch of the vertical loadings from which the consul'tant was to determine the seismic response spectra.
There was no indication on the sketch which unit the loadings applied to, though the consultant understood (correctly) that they were for Unit 2, The consultant thought that Unit 1
was a slidealong uni t (instead of a mirror-image unit) and performed the analysis on Unit 1
based on that assumption.
The information returned to the Licensee was marked as "Unit 1" (in fact, the analysis applied to Unit 2, not Unit 1).
The Licensee accepted the data at face valve as being for Unit 1
- and, because it knew the plants to be mirror-image plants, flipped the data so as to be applicable to Unit 2 (in fact, the data in the flipped condition were correct for Unit 1, not Unit 2).
The seismic response spectra were now incorrect for both Units 1 and 2.
Upon confirmation that wrong diagrams were used in the development of Unit 1
design requirements, the Licensee reanalyzed the design requirements for Unit 1 using the appropriate containment annulus frame orientation diagrams and determined that, as a result of the error, modifications were required to be made on 31 Unit 1 pipe supports.
These modifications involved such actions as adding snubbers, changing the snubber size, adding braces, replacing structural
- members, and stiffening base plates.
DRAFT MORKING PAPER
DRAFT MORKING PAPER In an inspection report of seismic-related
- errors, the Nuclear. Regulatory Commission stated that the basic cause of this problem appeared to be the informal manner in which the subject data were developed by the Licensee and transmitted to its seismic consultant, and the lack of independent review of the data within the Licensee's organization prior to submittal to that consultant.
The Licensee had been the architect-engineer/construction manager for the Case C nuclear power station.
One of the major actions that the Licensee took as a result.of the aforementioned error was ihe formation of a Project.
Completion Team comprised of the Licensee's engineering/construction personnel and personnel from a newly hired architect-engineering firm.
An extensive Independent Oesign Verification Program (IDYP) was initiated in early 1982 in response to the seismic errors discovered in 1981.
The Project Completion Team is also conducting a concurrent design verification program.
As of January 1983, it was reported that an estimated 90Ã of the design and 405 of the construction required for modifications as a result of a wide range of reviews spawned by discovery of the seismic diagram error had been completed.
The Licensee has applied for reinstatement of the operating licenses.
At the time of the case study visit, neither the Independent Design Verifica-tion Program nor the L'icensee's design verification program had revealed significant further deficiencies in the design or construction of the nuclear station.
The design errors which were identified were not considered to have prevented the affected systems from performing their functions satisfactorily.
DRAFT MORKING PAPER
DRAFT HORKING PAPER The objective of this case study was to determine the underlying root, causes for the above design problem and to determine the generic implica-tions this experience may have for the nuclear industry.
The Case C study team was comprised of six personnel; tycho assigned to concentrate on the project engineering/design
- aspects, two on construction, and two on quality assurance programs.
Unlike other case
- studies, the case study team was unable to divide into sub-teams to pursue these separate topics.
All interviews of licensee and contractor personnel were conducted by the entire team.
- Thus, these working papers do not contain separate subteam inputs.
Prior.to, during, and following the site visit, the team reviewed several dozen documents and reports related to this plant and its history, including licensing correspondence, inspection investigation reports, and third-party reviews of the Licensee's gA, design, and construction programs.
The team spent three and one-half days with the Licensee, including a one-day plant visit.
Prior to the Licensee discussions, two of the team spent one day with the HRC regional staff, and during the Licensee discussions, the entire team spent a day with the regional staff.
The site visit culminated
(.at a
later time) in a briefing for Licensee staff by the team leader in which the findings of the team were reviewed and the Licensee staff had an opportunity to comment on them.
C.
Sumary Based on review of the background documentation and the interviews described
- above, the Case C Study Team identified the following factors which it'onsidered significant in contributing to the quality problem experienced by the Licensee:
1.
The primary root cause of the desi n-related uality roblem was the Licensee's failure to plan, establish, and effectivel im lement a
management s stem which rovided ade uate control and oversi ht over all aspects of the project.
The Licensee failed to fully control the flow of information across all the interfaces inherent in the engineer-ing/design process and to provide apprbpriate reviews of the information transmitted.
DRAFT MORKING PAPER
DRAFT NORKING PAPER There appear to be several factors which contributed to this failure.
Using the experience gained from their earlier turn-key plant and part'icipation of the staff in other nuclear projects, the Licensee after considerable evaluation assumed the 'role of architect-engineer for this nuclear project.
As previously stated, the Licensee had good success with various types of generating projects it had engineered and managed over the years.
The nuclear project was fitted into a design, engineering, and management system that may not have been adequately modified.o handle all aspects of nuclear work, including the control of quality at design interfaces.
As a general rule, it has been more difficult to apply gA to the engineering process than to the construction
- process, and the Licensee found this to be the case.
Even though gA was apparently rigorously applied to the construction of the project in question (and growing in strength as NRC requirements and guidance evolved) the Licensee did not implement NRC quality requirements for engineering as intensely as they did for construction.
Their attitude seemed to be that the engineering organization was comprised of professionals capable of doing what is right without overlaying a
stringent formal quality assurance program beyond the normal controls considered part of good engineering practice.
Another factor in the problem of assuring quality in engineering related to changes in NRC requirements that occurred between the late 1960s and late 1970s.
It appears that the Licensee did not completely understand the implications of the changes as they occurred;
- hence, a
gA program for engineering that the AEC might have found acceptable early in the project might not pass NRC scrutiny in the late 1970s.
DRAFT MORKING PAPER
r
DRAFT ItllORKIN6 PAPER 2.
Secondary root causes included the following:
a.
Failure to understand and appreciate the potential merit of a formal institutionalize'd gA program.
This is suggested by the fact that the Project Completion Team adopted the A-E's quality assurance
- program, even though they were concerned about imposing a
new system on the project at a late date (the Licensee's engineering procedures were maintained, however).
Examples of program deficiencies (drawn from various reports on the project and discussions with NRC inspectors) which had occurred during the project and the key indications of these deficiencies were as follows:
Design Control
~
The Licensee's engineering staff did not always documeht important data transmitted to subcontractors Verbal transfer of design informat'.on to subcontractors occurred
~
Assigned cognizant engineers were sometimes bypassed in the information ol approval processes Adequate internal communications among the disciplines did not always exist within the Licensee's organization
~
Requirements for independent reviews were not always followed Control of Instructions, Procedures, and Drawings/Document Control The Licensee's engineering did not develop and/or implement formalized procedures to comply with early gA program requirements In some cases, outdated drawings were used to establish seismic criteria In some cases, diagrams in lieu of release drawings were used --
a contributing factor to the seismic problem DRAFT MORKING PAPER
~
r
DRAFT NORKING PAPER
~
Control of Service Contracts Proceduralized activities for services contracts were lacking to control all interfaces with some subcontractors
~
Informal "letter.-type" contracts and documents were used
~
Service contracts were not treated as formally as hardware contracts
~
Formal quality requirements were not placed on some subcontractors until the late 1970s b.
NRC's failure to Sell QA as a 11anagement Tool The NRC requirement for quality assurance seemed to come across as just another requirement.
The emphasis from NRC seemed to be on externals; the trappings of a QA program, rather than its substance
-- develop a
QA manual, set up a
QA organization, make the QA manager report high in the organization, etc.
NRC tended to lose sight of what it was trying to achieve and failed to provide adequate guidance on what a quality assurance program should be.
NRC failed to inspect against QA requirements in the engineering area to the extent they inspected against QA requirements. for construction.
c.
Long Period of Time Between Inception of the Project and Operation As previously stated, the period of time between the issuance of a construction permit and the present has been about 15 years.
This long period of time greatly increased the exposure to changes in technology, to changing regulatory requirements, and to changing state of the art in technical matters with the attendant opportunities for quality failure.
DRAFT WORKING PAPER
r
DRAFT. NORKIN(i PAPER II.
ROOT CAUSES OF THE LICENSEE"S PROBLEMS WITH QUALITY IN DESIGN Based on the study team's evaluation of NRC's files and other documentation regarding the Case C Project, discussions with cognizant NRC personnel and Licensee and contractor personnel, the fol'lowing are proposed as the primary and secondary root causes of the Licensee's quality problems in the design and construction of its nuclear station:
A.
Primary Root Cause The primary root cause of the design-related quality problem was the Licensee's failure to plan, establish, and effectively implement a
management system which provided adequate control and oversight over all aspects of the project.
The primary root cause emerged out of several factors which, taken together, increased the likelihood of a design error.
These factors include the pressure (whether real or felt) to complete, the nuclear station, informal communications across important interfaces, an inadequate application of quality assurance/quality control to all aspects of the design process, and the resistance by-engineering of the application of formal quality assurance procedures.
Reviews and audits of the project indicate that the station, as, it existed in 1975, had been properly and correctly designed and constructed.
The aeismic and other analyses (at the then state-of-the-art) had been performed to the correct configurations and bases.
It was in the reanalysis after the essential completion of Unit 1 in 1975, prompted by new seismic assumptions, that the design error previously described occurred.
From the issuance of construction permits for the Licensee's nuclear station to the present time was approximately 13-15 years, making this station one of the longest if not the longest -- in the construction process.
The large amount of rework resulting from changing regulatory requirements, coupled with turnover in personnel and increased facility costs lengthened the construction period and increased the real (or felt) pressure to complete the facility.
As a facility nears completion or is in a prestartup condition (as the Licensee's station was in the mid-1970s) and new ov changed requirements
- arise, there is a tendency to ac'complish the activity and to formalize action later.
Such conditions, coupled with informal interface procedures, increase DRAFT WORKING PAPER
DRAFT. WORKING PAPER 10 possibility of error.
Another factor which contributed to the problem was the need for additional expertise, especially in the seismic area, resulting in greater use of consultants and engineering service contractors than had been customary on the Licensee's other generating projects.
This increased the possibility of interface problems and required changes from the customary operation of the Licensee's engineering staff.
There seems to have been a tendency to extend the informality common in close-knit engineering organizations to some of these outside groups.
Customary controls and review processes for dealing, with them were not always effectively applied.
These interfacing problems were increased by the proximity of the consultants and engineering service contractors to the Licensee (a greater distance might have required more formalization of communications).
A well-developed engineering team which relies heavily on informal communication among its members has both advantages and disadvantages from a quality assurance standpoint.
Such close contact generally contributes greatly to the quality of the engineering'ork.
At the same time, it can create practices which are not appropriate in dealing outside the organization.
Geographical separation generally requires a higher degree of formalization in communication.
Geographical proximity (working.
in the same office or building) can result in items being discussed sufficiently that a
common understanding is reached between the parties involved.
In the Case C project, the practice of utilizing informal communications with key consultants located in the immediate area (city and suburbs) developed; however, in some cases, the distance was probably great enough that the level of communication required to reach full understanding of key points was probably not achieved.
DRAFT WORKING PAPER
DRAFT WORKING PAPER One of the comments made by the Licensee's staff was "engineering viewed their consultants as an extension of themselves."
(It should be noted that the Licensee's staff assigned to the Project Completion Team has been physically relocated to the A-E's facility).
While the problem of inter-face control cannot be considered the primary cause of the diagram error that occurred, it was a contributing factor.
The error is indicative of less than 'a'dequate procedures for design reviews and communications.
During much of the project, the application of quality assurance/quality control (QA/QC) to the design process was not well understood by the Licensee or emphasized by NRC as much as construction QC.
Changing require-ments resulting in redesign, coupled with turnover in design personnel not familiar with all the ramifications of the original design, make the application of QA/QC to the design process increasingly important.
The matter was further complicated by the developing nature of the 10CFR50 Appendix B criteria and their implementation.
During the meetings with the Licensee, its A-E, and the regional NRC staff, there were repeated comments with respect to ongoing problems in interpreting 10CFR50 Appendix B criteria and their application to the engineering
- process, aq well as concerns about infringing on "professionalism" and "creativity."
These considerations, plus the fact that the Licensee's engineering organization was very independent, contributed to their resisting application and/or understanding of formal quality assurance procedures; There also seems to have been a
tendency to require, or at least receive, more stringent quality assurance from contractors than was applied to in-house efforts.
NRC investigations in late 1981 and early 1982 found that design and engineering QA practices in consulting contractors'rganization's were better than those in the Licensee s engineering organization.
It is not entirely clear whether this reflected a difference in the Licensee's requirements or a difference in practices.
NRC inspectors made the observation that the Licensee is "tougher on its contractors than on itself."
This attitude may have contributed to the apparent success in assuring quality in the construction efforts; since essentially all of that work was done by contractors.
DRAFT MORKING PAPER
0
DRAFT. WORKING PAPER 12 Th'e significance attached to this finding is the possible reflection of an attitude in engineering -- an attitude of reflecting some degree of professional arrogance that "we do no wrong, but we sure have to watch out for thoseother guys."
The major quality probl'ems identified to date have been within that organization.
It appears that the application of rudimentary quality assurance practices for design document control should have prevented the error that occurred.
The engineering deficiencies discovered in September 1981 occurred during a time, ironically, when quality assurance appeared to be undergoing significant strengthening within the Licensee's organization.
In September
- 1976, the Licensee hired a
new corporate director of gA who was qualified, knowledgeable, and aggressive.
During late 1976 and
- 1977, the gA program was restructured and a
new gA program was established in 1978.
The case study team was unable to establish the attitudes and relationships between engineering and the new quality assurance director during those years.
To summarize, the primary root cause was the failure to manage completely a
project that is large and complex, and the failure tq plan and effectively implement a management system embodying all of the controls necessary to ensure correct completion of such a project.
There were several factors that contributed to this primary root cause.
The Licensee had a high degree of confidence with respect to its engineering capability.
The Licensee had been successful with various types of generating projects.
What had worked for those projects was assumed to work for its first (in-house) nuclear project and, thus, the'project was fitted into an existing structure which probably carried with it practices not appropriate to nuclear work.
DRAFT WORKING PAPER
DRAFT NORKIN6 PAPER 13 The enaineering function in the Licensee's organization was very strong; strong enough politically to resist the imposition of management controls that were required elsewhere in the company or for contractors.
Many management, personnel had come from the, engineering function; the appreciated its capability, had been part of its aood performance, and had not seen a
need to enforce additional, more stringent quality controls over ii.
Concurrently, the atmosphere and regulations for construction of a nuclear power plant were changing significantly.
The Licensee may not have completely understood the implications of the changes as they were occurring.
B.
Secondary Root Causes Based on a review of referenced materials, discussions and interviews with the Licensee, the Regional NRC office, and analysis, the study team has identified three secondary root causes of the design problems experienced at the Licensee's plant.
They are:
- 1) failure to understand and appreciate the potential merit of a formal institutionalized gA program,
- 2) NRC's failure to sell quality assurance as a management tool, and 3) the long period of time between inception of the project and completion.
Each is discussed in more detail:
1.
Failure to understand and a
reciate the otential merit of a formal A
ro ram.
The Licensee had a highly capable organization and had successfully completed many projects.
It had started to organize a
quality assurance program before the requirements of 10CFR50 Appendix B
became mandatory.
It is believed that the Licensee's perception was that good quality was achieved in their projects and that, while the new requirements might change some things, it would not affect the under-lying bases for their good quality performance.
Consequently, the early program could be characterized as a documented or proceduralized process of meeting the requirements.
It did not significantly affect the way that the Licensee had been doing its engineering/design work.
When seismic (or other) problems arose, the Licensee reacted as any concerned or conscientious organization would. If a mistake had been made, it was
~ totally willing to make it right.
DRAFT MORKING PAPER
DRAFT. WORKING PAPER 14 A member of the Project Completion Team (an A-E employee) said that he had reviewed the Licensee's quality assurance program in great, depth prior to formation of the Project Completion Team.
He noted that the Licensee's early program had weaknesses, but had improved greatly during the project.
He was concerned about imposing a
new quality assurance system (the A-E's) on the project at such a late date.
In the end;
- however, the A-E's quality assurance program was adopted, even though the Licensee's engineering procedures were maintained--
perhaps implying that the Licensee's QA program did not fully satisfy what the A-E considered necessary for a nuclear plant.
2.
NRC's failure to sell A as a
mana ement tool.
As far as the Licensee was concerned, the requirement for quality assurance came across as just another NRC requirement.
The emphasis from NRC seemed to be on the trappings of a QA program -- develop a
QA manual, set up a
QA organization rather than its substance.
NRC appeared to lose sight of what it was trying to achieve.
NRC inspection emphasis seemed to focus first on operations, then construction.
The message conveyed was that the most important area was not design and engineering, because NRC did not effectively provide guidance for regulations or inspect extensively in the design and engineering area.
NRC failed to provide guidance on what constituted a design quality assurance program.
It did not have sufficient technical strength to provide effective inspection oversight of design and engineering QA programs.
3.
Len th of time between ince tion of the ro ect and o eration.
The period of time between the issuance of a construction permit and the case study totals about 15 years for Unit 1.
This long period of time greatly increased the project's exposure to the normal occurrence of
- events, to changing regulatory requirements, to changing state-of-the-art in technical matters, and to changing political climates and public perception.
Some of these factors required portions of the facility to be redesigned.
Redesigns may not be subject to as thorough analysis as original design efforts because the personnel have changed (employees retire or are moved to new jobs) and the scope of review may be less.
DRAFT WORKING PAPER
DRAFT. WORKING PAPER 15 One of the major causes of redesign on this project was the changing seismic picture.
Initially, experts with impressive geological and seismological experience postulated the kinds of earthquakes that might occur, That information was given tb the Licensee's consultants to describe the loads and seismic response criteria.
The seismic field was developing very rapidly during this period and new data were dev'eloped on faults in the plant area.
Data from a seismic event in the region resulted in the Licensee's plant being designed to two different types of earthquakes; a design earthquake and a double-design earthquake.
Other major causes of redesign were the Brown's Ferry fire; the TMI accident, and other changes in NRC requirements.
This frequent retrofitting affected morale and contributed to a climate conducive to errors and quality failures.
III.
REMEDIAL ACTIONS TAKEN TO CORRECT (TURN AROUND) QUALITY PROBLEMS The major remedial action taken by the Licensee was the formation of the Project Completion Team.
Approximately 275 of the Licensee's staff was merged with about 600 of the A-E's staff to form a new project engineering organization.
The design and licensing unctions were merged into the Project Completion Team.
The project engineer for Unit 1 is a licensee staff member; the project engineer for Unit 2 is an A-E staff member.
The Licensee's chief engineer's stamp still appears on drawings and 'its discipline engineers can ask for documents to approve, though the discipline engineers appear to be involved in an overview function.
It was understood that as the plants become operational, the A-E's staff will phase out and the Licensee's staff will again resume responsibility for engineering.
DRAFT MORKING PAPER
DRAFT MORKING PAPER 16 The Licensee's personnel commented that the transition to the Project Completion Team was difficult but that it was a good learning experience for his staff which had "become kind of hide bound.">
Now that they are working with the A-E's sta f, they see a much more alive and respohsive organization, one in which decisions are made at lower levels, reviewed by appropriate management, and work is pushed forward with considerable aggressivness.
He said these things were good for his staff to experience and it will be better for it when the project is completed.
I The Licensee has also learned that it is important to review consultants'ork.
The Licensee had retained a large number of consultants perhaps 100 or more--
and the Licensee did not have adequate manpower to totally review all of the work done by consultants.
The need for the Licensee (or Project Completion Team) to carefully scrutinize consultants'ork has now been clearly established.
The Licensee now realizes that quality assurance is a total envelope of management-controlled procedures.
If they were to start a
new nuclear plant, they would ensure that the entire quality system was in place before starting.
All organizations involved in the project would have similar..quality systems.
~quotations are not verbatim, but they are believed to convey the meaning intended.
DRAFT WORKING PAPER
DRAFT WORKING PAPER 17 IV.
GENERIC IMPLICATIONS Based on the information reviewed and analyzed by the Case C Study Team, several possible generic implications, or lessons, emerge.
These are high-I lighted for each case study to provide input and to help form overall con-
'clusions concerning factors which constitute important elements in nuclear plant construction quality.
The first four address licensee implications; the last three NRC implications:
A.
Nuclear ower plants are complex facilities and licensee mana ement must a
reciate that fact.
Desi n and construction ractices normall a
lied to fossil fueled plants are not ade uate to assure ualit in nuclear 21ants.
Licensees which have designed and constructed fossil fueled power plants only should not expect experience and technology alone to be adequate for undertaking nuclear plant construction under the present regulatory climate.
One difference is that the licensee's management must be knowledg-able about how to achieve quality in nuclear plant design and construction.
In this Case, the Licensee seemed to lack a full understanding of how to institute a quality assurance program to adequately control the design
- process, even though (or perhaps because) much of tAe management came out of the engineering organization.
There is no question that the Licensee's management wanted a quality facility.
At the time of the Case C visit, all
.indications were that it was achieved as far as construction was concerned
- and, apart from the seismic design error, it appears to have been substantially achieved in the design.>
All this seems to have been achieved more by previously learned good practices than by the application of a formalized
~It was reported by the Project Completion Team staff that the Independent Design Verification Program (IDVP) which has involved about 50 personnel, had examined the containment and other systems in considerable detail.
Perhaps 40,000-50,000 different items had been looked at.
Only 63 needed a more detailed analysis
- and, of that number, only eight to ten were classified as legitimate design errors.
Of the legitimate design errors, none were considered to have prevented the affected systems from performing their functions satisfactorily.
DRAFT MORKING PAPER
DRAFT. WORKING PAPER 18 approach to quality, The HRC Regional Office also indicated that in the early 1970s there had been'roblem in interpreting 10CFR50 Appendix B in its application to the dhsign pr'ocess.
Licensee personnel noted that it was not unti 1 the 1973-74 period that quality assurance was actively considered for application to the design process and, by,that time, much of the design was completed.
Since quality was already thought to be part of the design (and apparently it was) it was considered unnecessary to put in a more substantive quality assurance program for the remaining design work (which proved to be far more extensive than thought at the time).
The fact that the Project Completion Team adopted the A-E's quality assurance program may be indicative of the judgment that the Licensee's methods of applying gA to the design process for nuclear plants needed improvement.
B.
A licensee needs to understand its own cor orate limitations as it under-takes a nuclear power ro ect, and set u
a ro ect mana ement structure in which its role is consistent with its capabilities and complements the roles of its contractors.
The capabilities of its contractors must augment the licensee's lack of e'xperience or expertise in engineering, procurement, construction, and management.
The Licensee recognized its limitations in certain areas and made use of a large number of consultants (Section IIA).
What apparently went unrecognized were the evolving requirements for engineer-ing support over the life of the project, necessitated in part by changing regulatory requirements.
A-E personnel stated that in earlier nuclear
- projects, engineering staffs (A-E or licensee) did not document the design process in the manner done today, and that the Licensee's practice was typical of the earlier practices.
- Further, the types of problems experienced in the design of the Licensee's nuclear station have also occurred to some extent in some other plants in which the A-E has been involved.
A licensee's engineering staff involved in a single plant may have difficulty staying current with the state of the art in nuclear technology and regulation, and is not as likely to assimilate advanced procedures that the industry as a
whole has developed.
DRAFT MORKING PAPER
DRAFT NORKIN6 PAPER 19 Mhereas the A-E's staff was able to gain appropriate experience because of involvement in a variety of plants, the Licensee's engineering staff was not.
Those organizations which were involved in several plants were able to staff appropriately.
Those with single planks had greater difficulty in doing so and had to rely on consultants or other contractors.
Another facet of the same problem is the evolution of the understanding necessary to incorporate new criteria.
As an example, it was pointed out that when introduced, 10CFR50 Appendix B was a
new language to many engineers.
Later projects were able to apply Appendix 8 requirements to the design process more readily because personnel had a better understanding of the requirements and process.
A-E personnel also stated that the length of the project can have detrimental effects due to the turnover of'ersonnel over a long period of time.
A-E personnel said that the length of a project is a
common thread in all projects that have gotten into trouble, as'ar as quality assurance is concerned.
Long exposure opens the project to many potential changes and delays.
Successful design activities require experienced personnel, but doe to promotions, retirements, etc., over'the course of the project, the possibility for error is introduced, because newer employees are often unaware of all of the earlier considerations that had gone into a decision.
C.
A licensee needs to mana e the nuclear ro'ect and ensure that interfaces between the ro 'ect artici ants A-E, construction contractors, etc.
are ro erl maintained and monitoried.
A total ro'ect s stem that imooses effective controls and checks over all ke as ects of the ro 'ect is re uired, including records management and document control, as well as design, construc-tion, procurement,
- cost, schedule, etc.
The system must also be able to accommodate change; for example, the changing regulatory environment has presented the Licensee's engineering staff with moving targets that required
- change, but which may not always have been fully recognized or quickly accomnodated by them.
The large number of consultants used for this project was different from previous projects;
- however, the consulting DRAFT NORKING PAPER
DRAFT NORKING PAPER 20 ro'les were of limited scope and, in some cases, a collegial relationship developed.
There was a lack of formality in the processes for passing information across some interfaces.
This occurred in part because some of the consultants also consulted for the, Atomic,Energy Commission and it was assumed that they knew what the requirements were.
'I The geographical proximity of a number of the consultants also helped erode a
ormal interface control system.
It was noted that procedural matters would not have been handled with the same informality if the subcontractors had been 50 miles away, rather than across town.
For example, it was stated that there was much more formality in procedures with a seismic consultant located about 40 miles from the Licensee's offices than with one in the same city.
Mhen the project started, there were no require-ments or regvlations for control of contractors which would provide for an auditable trail.
As the project developed, regulations became more detailed and complex, bvt in-house relationships and procedures did not evolve rapdily enoucn to fully accommodate all changes.
The Licensee did not have adequate manpower or expertise to fully review all of the work done by its consultants/contractors.
The interfaces between engineering functions or operations mvst be minimized and carefully monitored.
That the Licensee recognized this problem was apparent from a senior staff's comment that moving the Project Completion Team together on three floors in the A-E's building was immensely helpful in the comnunication process.
He also stated that there was no substitute for good procedures to monitor interfaces.
D.
The licensee must be committed to ualit from to mana ement down, and it must be effec ivel communicated b
to mana ement and manifested in rocedures and controls.
It is helpful when the licensee recognizes that an assurance of quality program properly conceptvalized, structured, and implemented can be an effective management tool that can be cost effective.
If management attempts to implement a "canned gA" program rather than an assurance-of-quality program, it can be seen as threatening to some, and as an artificial "laying on of another system by others -- a system with which one must contend, but one which has no useful purpose or function.
DRAFT WORKING PAPER
DRAFT MORKING PAPER 21 E,
NRC needs to treat A as a mana ement tool, not as ust another reauirement.
As another requirement, the concept of quality assurance is treated as just another system laid on the licensee.
As a management
- tool, the concept of gA assumes a much more importan't and useful role in the eyes of management.
It tells them something about the amount of rework and project cost, about the projected reliability and safety of the operating plant.
NRC needs to understand and stress this aspect to gain better acceptance of its gA programs.
F.
NRC needs to a
more attention to ensurin qualit in the desi n
rocess
~
During the Case C project, there was no effective in-depth evaluation by NRC of the Licensee's design process.
The Licensee had nearly completed the engineering work in the early 1970s when the quality assurance require-ments of 10CFR50 Appendix B were brought into the picture.
Part of the problem was the imprecise nature of 10CFRSO Appendix B, and this factor did not encourage the Licensee to install a gA program to handle the remaining design work to be done.
The NRC did not insist on it, either, perhaps because Unit 1
was already in process and 10CFR50 Appendix B
requirements were to be applied "as practicable
" as..far as Unit 1
was concerned.
It can be expected that engineering organizations in general will resist the introduction of quality assurance into the design process.
There is little acknowledgment from the Licensee's enqineerinq that, had better'A procedures been adopted, it would have avoided the design diagram error (this attitude does not apply to the Project Completion Team).
DRAFT WORKING PAPER
22 G.
NRC needs to focus more on the effectiveness of im lementation of the ualit assurance ro ram and less on the trap in s of licensee ro rams; e.g.,
less on the QA manual, organization charts, where the QA manager I
- reports, and paperwork per se.
There appears to be a lack of understanding of how to effectively apply quality assurance to the day-to-day design
- process, and additional guidelines are needed for application of QA to design.
The design process may inherently contain a high degree of assurance of quality.
Perhaps for this reason, it has been difficult to formalize an acceptable QA program for design.
Licensee personnel stated that the early implementation of 10CFR50 Appendix 8 was manufacturing oriented.
That orientation, together with the perception that QA can' be applied to the engineering
- process, are barriers to its adoption.
NRC needs to address the issue of assurance of quality in the design/engineering process.
V.
IMPLICATIONS OF THE CASE STUDY FOR NRC QA INITIATIVES NRC has underway or under study a number of initiatives which are designed to establish additional confidence in ihe quality of design and construction activities, to improve the management control of quality, and/or to improve the NRC capability to evaluate the implementation of licensee programs.
The initiatives are described in the NRC Staff Paper SECY 82-352, "Assurance of Qual-ity," and subsequent correspondence between the Commission and the NRC staff.
One of the purposes of this Case Study is to provide feedback regarding the relevance of the various initiatives to the Case C Licensee's nuclear construction project.
Subsequent paragraphs take each initiative in turn and discuss whether the initiative, had it been an ongoing activity at the time of the Licensee's design error, would have made a difference; i.e., would the initiative have prevented or at least mitigated the design error that has been discussed earlier.
A more complete discussion of the scope and details of the various NRC QA initiatives may be found in SECY 82-352 and SECY 83-32, "First Quarterly Report on Implementation of the Quality Assurance Initiative."
Most of these initiatives were discussed with the senior management of the Licensee.
II DRAFT WORKING PAPER
DRAFT WORKING PAPER 23 A.
Measures for Near-Term Operatin Licenses NTOL) 1.
Licensee sel f evaluation - maybe This initiative applies to action that would take place when the licensee is in the process of receiving its operating license.
It requires that the licensee examine selected portions of the engineering design or construction.
A licensee self evaluation permits an evaluation of the project from beginning to end and would permit the Chief Executive Officer to state that the station had been built according to its commitments.
Had this been a requirement, it is quite possible that one of the design areas audited would have related to seismic considerations, since that has been such a major considera-tion in the design and construction of this particular station.
Although it is unlikely that the review would have identified the error that actually occurred, it should have identified the problem of design.
document control.
2.
Regional evaluation - no The licensee regional evaluation is an action that would take place when the licensee is in the process of receiving its operating license.
The effect of the regional evaluation could be similar to that described in (1) above.
The scope of the regional evaluation would have to be expanded to include detailed design review for it to be applicable to the problem in Case C.
3.
Independent Design'Verification Program (IDVP) - yes The licensee IDVP is an action that takes place when the licensee is in the process of receiving its operating license.
The IDVP would have applied in the case of this Licensee's plant in which the design and construction are essentially completed.
Design verifications can be performed at any stage in the design, but the most productive period is when the design is essentially completed.
DRAFT WORKING PAPER
DRAFT WORKING PAPER
~
I
~
It is likely that an IDYP would address one or more of the. sensitive issues relating to the plant under review.
This would have included the seismic problem as stated under (1) above and, since an IDYP should be more thorough in the des'ign arel than either of the evalua-tions in (1) or (2) above, there is an increased probability that the diagram error and design document control deficiencies would have been found.
B.
Industr Initiatives l.
INPO "Construction" audits - maybe This initiative is applicable because phase 2 part of INPO "construction" audits now considers design.
It is possible that an INPO "construction" audit would have detected the problem that occurred in this licensee's plant.
However great the scope of these audits, it is not likely-<hat the specific error would have been detected; it would have probably identified the design document control deficiency.
2.
Utility Evaluation Using INPO Method - not applicable; not being done now.
C.
NRC Construction Ins ection Pro ram C
1.
Revised procedures and increased resources
- yes (if included design)
This particular initiative applies to the construction program.
The deficiency found in the Licensee's plant related to design and not to construction.
If this initiative were expanded to include design, then it would probably have detected the design document control deficiency.
2.
Construction Appraisal Team (CAT) Inspection - no This initiative applies to the construction phase; the Licensee's quality problems occurred in the design phase.
DRAFT WORKING PAPER
DRAFT NORKING PAPER 25 3.
Integrated Design Inspection - maybe The integrated design inspection is an action that would take place when the licensee is in the process of receiving its operating
- license, though it could be done before.
For the same reasons given for the effectiveness of measures for Near-Term Operating Licenses, ihe integrated design inspection would likely have uncovered the design document deficiency.
It is possible, but unlikely, that it would have detected the error.
4.
Evaluation of Reported Information - not likely This initiative would computerize 10CFR50.55e and Part 21 reports, facilitating trend and other analyses of these 'event reports.
This analysis provides an additional cross-check on the quality and operations at a licensee's site.
The type of quality ailure that occurred at the Licensee's site is not unlike other errors that result from lack of interface control.
- Possibly, the reporting of similar problems in other plants would have been useful to either the NRC Inspection and Enforcement staff or to the Licensee's engineering staff in looking for errors of this nature.
D.
Desi nated Re resentatives
- maybe At the time of this case study, it was unclear how the designated repre-sentative system might be implemented by the NRC.
Generally, it has b'een considered to apply to the construction
- process, and not to the design process.
However, the FAA uses designated engineering representatives (DER) who are employees of manufacturers, but are deputized by the FAA to review and verify certain elements of design.
(There are also designated manufacturing representatives (DME) who verify that the assembly or fabrica-
'ion process is acceptable).
The DER could be used to spot check the design or design process.
However, this initiative, had it been in effect, may have uncovered the design document control deficiency that the Licensee experienced, and possibly the error itself, had seismic analyses been subject to DER review.
DRAFT NORKING PAPER
r
DRAFT WORKING PAPER 26 E.
Mana ement Initiatives l.
Seminars
- yes Seminars similar to those that the, NRC Commissioners conducted in years past, as well as seminars by utility executives who had design-related problems would probably have been helpful in bringing the Licensee's management to an increased awareness of the importance of quality control measures in the design process.
I 2.
Qualifications/Certifications of Quality Assurance/Quality Control Personnel
- no The problem that the Licensee experienced did not relate to the qualifica-tions of the quality assurance/quality control personnel.
It related to the institution of adequate quality assurance control procedures in the design process.
3.
Craftsmanship
- no The quality problem experienced by the Licensee had nothing to do with the training or skill level of craftsmen.
VI.
IMPLICATIONS OF THIS CASE STUDY FOR THE CONGRESSIONAL AMENDMENT ALTERNATIVES Section 13 to NRC's FY 1983 Authorization bill requires NRC to conduct a
study of existing and alternative programs for improving quality assurance and quality control at nuclear power plants under construction.
This Section, called the Congressional Amendment, requires NRC to look in particular at the feasibility and efficacy of five specific alternative program concepts.
As a
part of this analysis, each alternative concept was evaluated with respect to whether it would have made a difference in the Licensee's construction
- program, had it been in place at the time of the Licensee's construction permit.
Each of the alternatives was discussed with senior utility personnel.
DRAFT MORKING PAPER
DRAFT WORKING PAPER 27
~
~
A.
More Prescri tive Architectural and En ineerin Criteria - maybe The Authorization Act requires NRC to evaluate the following alternative:
13(b)1 - adopting a more prescriptive approach to defining principal I
architectural and engineering criteria for the construction of commercial nuclear power plants that would serve as a basis for quality assurance and quality control inspection and enforcement actions.
In the case of the Licensee's design error, more prescriptive architec-tural and engineering criteria by itself would not have affected the Licensee's problem.
No one could have foreseen the seismic complications.
However, if more stringent criteria were expanded to cover the design process itself, then the design document control deficiency might not have occurred.
B.
Conditionin the Construction Permit on the Applicant's Demonstration of Its Abilit to Mana e an Effective ualit Assurance Pro ram - maybe The Authorization Act requires NRC to evaluate the following alternative:
13(b)2 - requiring as a condition of the issuance of construction permits for commercial nuclear plants that the Licensee demonstrate the capability of independently managing the effective performance of all quality assurance and quality control responsibilities for the plant.
It should be noted that, at the time the Licensee received its construction permit, it was among the better qualified utilities for undertaking nuclear plant construction.
At the time the construction permit was issued, 10CFR50 Appendix B was not a requirement.
At that time, the Licensee could have pointed to its excellent record in the construction of other types of power plants.
It could have also pointed to its performance in the operation of a small nuclear plant.
It is unlikely that the Licensee would have changed its procedures sufficiently to prevent, almost a decade later, the type of design error from occurring that resulted in the withdrawal of its operating license;
- however, the requirement for a demonstration of ability, if done
- today, would presumably evaluate the procedures for the transfer of informa-tion across interfaces between an applicant and its cohtractors.
DRAFT WORKING PAPER
~
~.
4
~
DRAFi MORKING PAPER 28 C.
Audits, Inspections, or Evaluations b
Associations of Professionals Havin Expertise in A ro riate Areas - Management Audits - yes The Authorization Act requires NRC to evaluate the following alternative:
13(b)3 - encouraging and obtaining more effective evaluations, inspections, or audits of commercial nuclear power plant construction by independent industry or'nstitutional organizations based on best experience and practices.
It is likely that audits by independent or professional organizations look-ing at the design process would have identified the quality assurance deficiency that was inherent in the transmittal of information between the engineering organization and its consultants that led to the design error.
If the design process was not audited, it is unlikely that the deficiency would have been discovered.
D.
Improvement of NRC's A Program - yes The Authorization Act requires NRC to evaluate the following activities:
13(b)4 - reexamining the Comnission's organization and method for quality assurance development, review, and inspection with the objective of deriving improvements in the Agency's program.
It is clear from previous sections of this report that NRC was part of the problem.
The following changes to NRC's programs would have mitigated and possibly prevented the development of the design quality problems discussed earlier:
a.
Modify the licensing review process for a construction permit to cover the applicant's ability to effectively manage a project as complex and technically demanding as the construction of a nuclear reactor in accordance with NRC requirements.
The construction permit review would need to have included a review of the design procedures that the applicant proposed to use and its relationship with its consultants.
DRAFT HORKING PAPER
~
L k
~
DRAFT WORKING PAPER 29 A thorough audit of the applicant's implementation of its proposed procedures might have been sufficient.
The focus of this type of review would be for the applicant to demonstrate
'its capability to effectively manage and/or overview'll aspects of the project, including quality assurance and control of design documents.
b.
Revise the NRC inspection program to 1) focus more on the design and engineering aspect of nuclear plant construction, and
- 2) increase NRC presence and capability in the regional offices to review and overview design practices and the design process.
Part of the reason for HRC's failing to recognize the problem was the lack of NRC inspection effort in the design process.
This was due largely to inspection resources that were limited in both number and technical expertise to overview the design process.
E.
Conditionin the Construction Permit on the A licant's Commitments to Submit to Third-Part Audits of His ualit Assurance Pro ram - yes The Authorization Act requires HRC to evaluate the following alternative:
13(b)5 - requiring as a condition of the issuance oi'onstruction permits for commercial nuclear power plants that the applicant enter into contracts or make other arrangements with an independent inspector for auditing quality assurance responsibilities for the purposes of verifying quality assurance performance.
An independent inspector is a third party who has no respon-sibilities for the design or construction of the plant.
This alternative, as it applies to this case
- study, was discussed under Alternative C above.
It is believed that, had this initiative been in place, it could well have prevented the Licensee from making the error that occurred.
A comprehensive review of the implementation of the quality assurance procedures that existed within the design organization should have revealed the design document control deficiency that resulted in the design problem.
DRAFT WORKING PAPER
L
~
~
REVISED DRAFT WORKING PAPER APPENDIX A EVALUATION OF GENERIC KEY INDICATORS REVISED DRAFT WORKING PAPER
r
DRAFT WORKING PAPER APPENDIX A EVALUATION OF GENERIC KEY INDICATORS FOR CASE C STUDY 1.0 Licensee is full corenitted to a
ro ram for assurance of 'alit A.
Project management appears to firmly believe'that their plant has been and is being built with adequate quality -- maybe excessive quality.
Certainly it is their intent to build a quality facility.
This Licensee, through its construction organization, appears to have supported a good QA/QC effort in the on-site construction activities; however, this diligence has not, in all cases, extended to service contractors and materials suppliers.
Prior to 1982, an equal commitment was lacking with respect to the engineering activi ties on the project.
This is reflected in the deficiency in management follow-up that allowed the violations of procedures and inadequate management reviews in 1977 to remain undetected for four years.
B.
Corporate QA audits construction activities on a periodic basis, but there did not appear to be the same attention given to engineering activities.
The Licensee has taken considerable care to separate quality assurance from quality control.
The QC function is the responsibility of the construction organization.
This was supplemented in many instances by the engineers who did the design overseeing construction and/or resolving construction problems.
At one time, QA appeared to be a term used to describe an organization required by regulations; now the Licensee has an appreciation of its importance and the cost of failing to.adequately document engineering actions.
The utilization of the A-E's QA program for the IDVP gives evidence that the Licensee now acknowledges the need to apply QA controls to the engineering process.
DRAFT WORKING PAPER
(
I
DRAFT RORKING PAPER 2.0 Res onsibilit and authorit are clearl defined and roperlv im lemented A.
At present, there appears to be clearly defined policy guides with respect to responsibilities and authorities for nuclear power plant construction and operational quality.
Apparently, good experience in designing, constructing, and operating other types of power generating facilities. led the Licensee to assume that similar procedures would be adequate for this nuclear station.
This project -has been a long time in the design and construction phase.
Contractor responsibilities and authorities and changing interfaces failed to keep up with the formaliza-tion required by NRC regulations over the past decade.
This failure
- occurred, at least in part, because the project has been on the verge of completion for about eight years, and the need to change was probably not pressing nor thought necessary.
The Project Completion Team members interviewed seemed clear as to their understandings of responsibilities and authorities.
There was some uncertainty as to the role of the chief engineers in the utility relative to the project.
The Corporate Manager of gA expressed a clear understanding of his responsibilities and authority.
B.
There are no observations for the present organization; the Licensee is aware that engineering gA should have been more formal in the early program.
DRAFT MORKING PAPER
l
DRAFT l<ORKlNG PAPER s
i 1
~
3.0 Oualified work force is utilized A.
Overall, the work force employed on the Licensee's project appear well qualified.
The Licensee's engineerin~ staff had limited nuclear experience entering into the project, and the staff was not large as measured-by present-day standards.
Where the engineering work force was not qualified, liberal use of consultants or contractors was employed, apparently for the most part, quite successfully:
To effectively manage them requires that adeauate qual:ty assurance procedures be in place and followed carefully.
This does not appear to have been the case with seismic consultants and other early contractors.
The construction work forces which were employed appear to have adhered to good construction practices.
Some reservations evolved relative to the corporate gA staff.
These
- came, in part, from impressions in one of the group meetings and, in
- part, from opinions expressed by a regional inspector B.
The Licensee apparently did not fully appreciate the importance of staffing with experienced gA personnel in the beginning.
4.0 Instructions, rocedures, and drawin s are clear and ade uate I
A.
Instructions, procedures, and drawings were not reviewed in detail apart from those associated with loadings for the seismic consultant's analysis.
While the drawings. were inadequate for this one case, there was no evidence of pervasive inadequacies in design drawings.
It should be noted that the engineering work currently being done by the Project Completion Team is guided by the utility's engineering procedures and the A-E's gA manual.
Since this hybrid. team has existed for less than one year, one would be surprised, indeed, if there have not been communica-tions problems.
It was not possible to probe deeply enough to identify any specific problems, however.
DRAFT WORKING PAPER
t t
I g
r DRAFT HORKINC PAPER B.
Presently, the quality assurance organization reviews drawings, but does not sign.them.
This is consistent with the Licensee's philosophy of engineering maintaining sole responsibility for design.
5.0 Qualit /OA ro ram deficiencies are sou ht out and reported rom tl A.
This factor seems to be strongly and effectively supported at the construction site.
The "mirror image" problem indicates a weakness in this regard in the engineering activities.
It must be noted, however, that deficiencies once discovered, have been promptly reported and addressed.
6.0 Corrective action ro ram is effective A.
- Good, once a problem had been identified.
The Licensee has been ver~
responsove to identified needs for corrective action;
- however, gA program deficiencies in engineering may not have been regarded with the same intensity as gA deficiencies in construction as far as corrective action was concerned.
B.
Ho observations made.
7.0 Desi n review activities detect and resolve desi n deficiencies A.
The procedures for design review appeared to be similar to those success-fully used by the L'.censee in the design and construction of other types of generating facilities.
Among other things, the overall designs were reviewed by chief (discipline) engineers.
No data were obtained on the numbers and types of field changes.
The "mirror image" and related problems represent a unique deficiency in the area of design review.
Although the problem was eventually discovered and is being resolved, the discovery was more fortuitous than the result of an orderly process.
At the present time, of course, very extensive reviews are in progress.
DRAFT MORKING PAPER
r
DRAFT WORKING PAPER 0
8.0 Design in ut data is ade uatel controlled A.
The look-back reviews have not discovered major design problems wi th the original plant.
Performance in this I
satisfactory during the original plant design process.
breakdown in 1977 in the handling of seismic diagrams.
to review design input data control are currently being in the jDVP.
or construction area was probably There was a major Hajor efforts applied, especially B.
No observations made.
9.0 Or anizational structure is conducive to attainment of ualit A.
No fault was identified with the formal organization structure;
- however, the (informal) position of power on the part of engineering is suspected to have been a factor in resisting the application of an effective design gA process.
This level of informal power now appears to have been significantly eroded.
B.
gC functions are performed by the departments responsible for the task.
10.0 Plannina, schedulin, and bud etin rovide the resources to do the
'ob A.
The engineering problems which have been so costly are suspected to have resulted, at least in part, from very heavy schedule pressures.
Whether these pressure were real or felt was not, established.
There was no indication of lack of resources applied to the project.
B.
No observations made.
DRAFT WORKING PAPER
~,
DRAFT WORKING PAPER 11.0 Design control rocess A.
As identified in numerous earlier indicators, this is the major breakdown which occurred on this project.
During the past year, this has been I
corrected with a very extensive IDVP and an internal "look-back" program initiated by the Project Completion Team.
B.
In the early days of the project, there was not a formal design control process which was independent of the engineering organization.
12.0 Work acka e develo ment and control A.
Hot investigated.
8.
No observations made.
1.3.0 Procurement control A.
A satisfactory evaluation of this indicator is difficult because of the time period of this project.
The IDVP consultant auditing the gA program has conceded that the numerous, significant deficiencies that have been identified are deficient by today's standards, but not by the standards existing at the time the procurements were made.
8.
The original seismic contract did not specify a gA program.
In fact, the contractor was not required to explain its gA program until 1977 '--
long after much work was completed.
14.0 Nonconformance control A.
Not investigated.
B.
No observations made.
DRAFT MORKItIG PAPER
r V
~.
c
)
h DRAFT NORKIiNG PAPER 15.0 Special rocess control A.
No significant QA/QC problems have occurred at the construction site.
I B.
Welders are qualified today and all indications are that construction practices have always required welder qualification.
No observations made on other processes.
16.0 Examination, test, and ins ection control A.
This point was not probed in depth, but NRC regional staff and expressions at the site indicated that the construction site efforts were excellent.
B.
No observations made.
17.0 Calibration control A.
Not investigated.
B.
No observations made.
18.0 Records A.
Although major problems have resulted from weak documentation practices in engineering, the available records led to discovering the error about four years after it occurred.
- Further, we were advised by an NRC regional inspector that the records and traceability relative to construction (materials,
- heats, location, etc.)
were unusually extensive and accurate.
DRAFT MORKING PAPER
P
~,
DRAFT WORKING PAPER 19.0 Audits A.
The use of audits in the early years of the project appears to have been limited to those typically done in projects involving other types of I
generating facilities.
The audit of the design process was probably not a strong emphasis or the design control procedure deficiency would have been noted, The audit program has been very extensively strengthened during the past year.
B.
The present program includes audit activities; however, they were not verified.
The Licensee had a
QA/QC program, but the problems they have experienced would indicate that they did not have an aggressive system to verify implementation in the design control area.
NRC audit reports gave the Licensee good reports on construction quality program implementa-tion.
20.0 Identification and control of material items A.
This was only superficially investigated;
- however, a
HRC regional inspector specifically commented that the utility had been far above average in this regard.
DRAFT MORKING PAPER
<<I
~ w 0
+ ll
DRAFT NORKIN6 PAPER APPENDIX 8 DEFINITION OF LEVELS OF OUALITY FA.ILURE REVISED DRAFT WORKING PAPER
~
P RIe
-I
~
I P.
DRAFi j'jORKING PAPER APPENDIX B
DEFINITION OF LEYELS OF QUALITY FAILURE CAUSES 1.
The Deepest Sense of ualit Failure There are basic underlying causes of quality failure, which clearly transcend QA and QA programs.
They can be characterized aC broadly philosophical.
They are at the extremity of the chain of causes (e.g., building a nuclear power plant without knowing how -- which has as necessary conditions
- 1) the, licensee does not know how, and 2)
NRC permits them to build, even though they don' know how).
It is usually very difficult, if not impractical, to develop recommendations that address such philosophical issues.
These are, nonetheless, root causes.
For our purposes, we are defining root causes at the following, more operative level.
2.
The Operative Sense of ualit Failure There are basic underlying causes of quality failure, which frequently transcend QA and QA programs, but not necessarily.
They can be characterized as general.
They are near the end of the chain of causes, but are limited to where it is practical to bring about corrective action (e.g.,
lack of manage-ment commitment).
It is at this level that corrective actions often treat many symptoms of poor quality. It is in this sense that the term "root cause" applies in this report.
There is yet another level which we have defined as symptomatic/procedural.
3.
The S
ptomatic/Procedural Sense of ualit Failure These are often the immediate causes of quality failures.
These can transcend QA and QA programs, but it is unlikely.
They are characterized as detailed and specific.
They are intermediate in the chain of causes
- and, as such, are subcauses of (2) above.
Recommendations for corrective actions at this level are relatively easy, but are likely to treat without addressing underlying causes.
DRAFT WORKING PAPER
aW
~
~$
4 C~
<c f