ML20082A459

Final Rept of Level 3 Review Board on Millstone Point, Unit 3 Probabilistic Safety Study

ML20082A459
Person / Time
Site:	Millstone
Issue date:	08/31/1983
From:	Levine S, Rasmussen N, Wood P AFFILIATION NOT ASSIGNED
To:
Shared Package
ML20082A440	List: ML20082A436 ML20082A459
References
NUDOCS 8311180150
Download: ML20082A459 (17)

Text

.

1 I

1 FINAL REPORT OF THE LEVEL 3 REVIEW BOARD ON THE MILLSTONE POINT UNIT 3 PROBABILISTIC SAFETY STUDY NORMAN C. RASMUSSEN, CHAIRMAN SAUL LEVINE PAUL J. WOOD O

e AUGUST 1983 O

A

-i

1.0 INTRODUCTION

Following a formal request by Harold Denton of the U. S. Nuclear ,

Regulatory Commission (NRC) on September 21, 1983, Northeast Utilities Service Company (NUSCo) undertook a probabilistic safety study (PSS) on millstone Point '

Unit 3 (MP3). The two primary objectives of that study were:

~

1) To characterize the public risk associated with the opera-tion of MP3 resulting from both intern.a1 and external events, and to compare internal risks to that predicted in .

the Reactor Safety Study (RSS) as being representative of Pressurized Water Reactors (PWR's);

2) To develop a set of technical tools to support management decision-making in a continuing program designed to assure the effectiveness of future plant betterment projects aimed at improving safety.

In an effort to assure the production of a high-quality study to satisfy these objectives, NUSCo undertook.a three-level review process. The first two levels of review have been described in the MP3 PSS Sumary Report.

The third level of review involved the commissioning of a review board with two principal responsibilities:

1) To assess the precess employed to perform the PSS to assure that the methocology being employed was consistent with the study objectives and with the state-of-the-art;

2) To assess the quality of the product of the PSS both by evaltating the consistency between the study as implemented and the defined methodology, and by reviewing the study results in light of the experience of the reviewers.

A copy of the Charter of this Level 3 Review Board is provided as Attachment 1. The purpose of this report is to documert the opinions of the Review Board as well as to surmarize the process which led to these opinions.

As noted in the attached Charter, the Review Board was not expected -

to perform a detailed technical review of all . facets of the PSS. Rather, we were expected to support completion of the study by providing technical comments extracted from two types of review: ,

l

1) A%tendance a2 a series of six reviec board mee%ings carried out during the approxirately one year prior to publication of the study. These meetings were typically one- and one-half days in duration and involved preser.tations by the technical people, both NUSCo and outside contractor, who were performing the study;

2) Review at a somewhat more detailed level of special topics -

which suggested themselves at project review meetings as being potentially important to the achievement of project objectives.

• _ The total effort devoted to the Level 3 review process was insuffi-cient for a final definitive statement to be nade by the Board regarding the absolute " correctness" of the study and its results. However, sufficient.

effort was devoted to the review that solidly based opinions on the validity of the methods, the quality of the analytical process, and the reasonableness of the results can be made. This report represents the collective best evaluation of the members of the Review Board on these subjects.

In keeping with the Review Board Charter, the report is segregated into sections which summarize findings (Section 2) and elaborate on specific issues (Section 3). These sections follow.

e l

l h

}

0 2

2'. 0 SUP.i.F.Y OF FINDItGS 2.1 COMPETENCE OF PROJECT IMPLEMENTATION The prima ry measures of competent PSS project implementations include: *

1) Assuring that project objectives and scope are compatible; i

2) Monitoring progress closely enough to assure that resources are not wasted in characterizing technical issues of little importance to risk;

3) Promoting the integration of the various segments of the PSS to assure product quality in areas such as:

a) Compatibility between failure data and failure modes being quantified; b) Compatibility between success criteria called for in event trees and top events in fault trees; c) Consistency between containment failure modes and accident sequences producing them.

4) Identifying and acting on information relating to poten-tially deficient methods or technical work; ,

5) Presenting the study in a manner that communicates its strength and lim.itations relative to its potential uses.

With a modest number of understandable exceptions, which are noted in Section 3, the NUSCo team was quite successful in demonstrating technical and management ccmpetence, as assessed against the five measures above, in complet- l ing the study.

Perhaps the most noteworthy observation which can be made regarding the competence with which the pro. ject was implemented relates to the breadth and depth of the involvement by NUSCo. This involvetent began four months prior to selection of a set of contractors with development of a detailed engineering specification for the study. This specification was prepared by a group of NUSCo analysts who had been assembled' to provide expertise in vir-tually every facet of a PSS. Several of these analysts had participated intimately in the completion of the . Interim Reliability Eva10ation Project 3

(.KEP) study carried out on Millstone Foint Unit 1. Evidence of the effective-ness of the NUSCo team in managing and supporting the progress of the study was abundant.

The degree of competence evidenced by the various contractors on the project was not uniform. Occasionally, too stringent adherence to a procedure .

or methodology was observed. Infrequently, this behavior seemed to persist

• even after it was pointed out and critiqued ,at review board meetings. In

, general, however, the NUSCo technical and management team was quite effective ,

in identifying and correcting contractor performance 'iapses.

2.2 COMPARABILITY OF METH0nCLOGY TO STATE-0F-THE-ART The state-of-the-art in risk assessment technology is gradually evolving. Although the basic methods established in the RSS have not changed dramatically since the publication of that study, each new risk assessment has claimed responsibility for at least a modes.t advance in the technology. In the conduct of the MP3 PSS, NUSCo and its contractors have utilized the proven nethodology described in the PRA Procedures Guide (NUREG/CR-2300) in most situations. Some examples of modest deviations from the existing state-of-the-art are evident in the study. These are sumarized briefly below:

Source-Term Development During the past three to four years, a number of advances have been made in our understanding of the chemical and physical phenomena which signif-icantly affect the magnitude of the radionuclide source term predicted to be released from containment in core melt accidents. As in the Zion study, a strong effort was made in the MP3 PSS to-incorporate this evolving understand-ing in the characterization of source terms. To this end, new analysis con-sistent with that performed by Westinghouse in the Sizewell B risk assessment was utilized as the basis for defining uncertainties in the accident source terms. The source-term reduction resulting from this analysis has contributed significantly to the reduction of the predicted median CCDF's for analyzed health consequences. It should be noted that the approach utilized in defining accident source terms has not reduced the peak health consequences predicted 4

,-,r- w

for MP3, but that er apparent error in the CRAC II code analysis has led to a significant underprediction of this parameter. This error is discussed in Section 3.2.

Common Cause Failure Analysis The connon cause failure analysis (CCFA) performed in the MP3 PSS .

represented a new approach to dealing with an old problem. The contributors to CCF were divided into assignable (i.e., known sources) and unassignable (i.e.,

unknown sources) components. The unassignable component was characterized using a binomial failure rate (BFR) model together with a data base developed by Atwocd at INEL. The data base was analyzed and augmented to assure that.

assignable components of CCF were deleted. This approach is conceptually correct and represer.ts a different view of an important problem.

Recovery Analysis The MP3 PSS was begun with the idea that significant credit might be taken for recovery from accidents leading to core degradation prior to full core melt and penetration of the primary system boundary. This question was investigated and tirre windows for recovery action defined. Although these windows were too narrow for significant probabilistic credit to be taken for recovery actions subsequent to the onset of core degradation, the analysis was of value in providing good estimates of the time available for recovery prior to the onset of core degradation. This infonnation was used in evaluating the probability of recovery prior to the onset of core degradation. This recovery analysis was separated from the report sections in which accident sequences were developed and probabilistically quantified.

l ~

Treatment of Uncertainties l

t IS the MP3 PSS, extensive use was made of discrete probability distributions (DPD's) in the characterization and propagation of uncertainties.

This approach represents a conceptually appealing mechanism for describing.

l analyst judgment and opinion on difficult-to-characterize uncertainties such as centainment failure pressure, radionuclide source terms, and ana)ysis of public

'5

?

N

i hcalth consequences. The difficulty in the use of DPD's arises in the dif- 1 ficulty with which the reviewer, and ultimately the utility user, is able to understand both the important specific accident sequences--and consequently, the design and operational contributors to risk--and the effect of proposed changes to plant design or operation on the plant-risk profile.

One conceptual problem also exists relative to the use of DPD's to characterize uncertainties in the MP3 PSS. The problem is that modeling uncertainties associated with the CRAC II code characterization of public health consequences were translated by a rather opaque process into uncertain-ties on the source term used in the CRAC II analysis. Thus, the uncertainty on the source term was comprised of two DPD's, one which reflected uncertainty in the definition of the source term and one which reflected uncertainty asso-ciated with the CRAC II code modeling. The effect of using a source term DPD to characterize health consequence modeling uncertainty on either the median health consequence CCDF's or the predicted uncertainties is unknown.

Seismic Analysis Although the analysis of seismic risk in the MP3 PSS is conceptually similar to that used in the Zion and Indian Point risk asses;ments, a number of questions on the validity of both the methodology and the data persist.

Regarding the methodology, it appears that the fault tree-based approach to analyzing seismic risk is primarily a means of displaying and manipulating knowledge on the potential contributors to seismic risk gained in other anal-yses. The analysis seems to lack the investigative quality which is char-acteristic of carefully performed risk assessment. Specifically, the manner in which the laboriously prepared system models designed to characterize the effect of internally initiated events are considered in the seismic risk assessment is unclear. Also, the th'oroughness of the investigation of

, locationdependent CCF during seismic events is not apparent.

, Although these observations may not impact the quality of the seismic risk results, they certainly indicate potential difficulties in the review and future utilization of this portion of the PSS.

e 6

Regarding the data utilized in thE seismic risk evaluatiCn, evidence exists both of excessively conservative and of excessively optimistic features in the analysis. As pointed out in the PSS report, the analysis leading to definition of equipment and structural fragilities was performed in what j appears to be an extremely conservative manner relative to other current risk assessments. Conservatisms, both in the analysis and in the failure criteria, appear to significantly bias the analysis in the direction of overpredicting -

risk. An early assessment of the degree of conservatism in the fragility analysis, performed by Structural Mechanics Associates (SMA), has indicated that elimination of the conservative bias will reduce the CCDF for acute fatalities following severe seismic events by about an order of magnitude. On the optimistic sice, review by the same consultant has indicated that the failure to include consideration of the Decollement Zone developed by the USGS in the MP3 site seismic hazard (seismic recurrence interval) curve definition has significantly biased both the uncertainties and the median hazard curve in the optimistic direction (i.e., current uncertainties are too small and median hazard is too improbable).

The net effect of correcting both areas of conservatism and areas of optimism is expected to significantly reduce the currently predicted seismic risk for MP3. As an aside, it should be noted that these changes to the existing analysis will also significantly broaden the uncertainty bounds associated with seismic risk, thereby correcting an anomaly in the present results relating to the excessively small degree of certainty associated with predicted seismic risk.

2.3 POTENTIALLY IMPORTANT OMISSIONS As well as the Review Board was able to judge, no major omissions exist in the scope of the MP3 PSS relative to the state-of-the-art. No ex-plicit effort was made by NUSCo to' systematically review the current unresolved safety issues list with the objective of addressing these issues in the current PSS (e.g., the general issue of Systems Interaction.was treated no differently O

e 7

than in other risk assessments). However, some currently visible safety issues were addressed in the PSS, induding fire risk, ATWS, and pressurized thermal shock.

Implementation of the study was consistent with the stated scope and selected methodology.

• 2.4 ACHIEVEMENTS AND LIMITATIONS On balance, the MP3 PSS represents the product of a carefully planned program which was implemented in a competent and timely manner. Perhaps the greatest achievement was the ability of NUSCo to assemble an in-house team of technologists' capable of managing a diverse and somewhat inexperienced team of contractors in the completion of a complex project. This planning and imple-mentation was carried out with continuous attention to the dual objectives implicit in satisfying both the NRC and in-house needs.

With the one exception noted in the paragraph below, the results of the study represent a reasonable characterization of the overall risk from internally initiated events which is suitable for comparison with CCDF's l developed in the RSS. {

This does not imply that the methodolbgy used in the MP3 i PSS is at the same level of development as that utilized in the RSS. Nor does it imply that the risk defined in the RSS would remain unchanged if that '

earlier study were updated using new data and approaches employed in the MP3 PSS.

It does imply that the overall MP3 risk representation is suitable for comparison with the CCDF's from the RSS which are considered to be one measure of historical acceptability. One exception to this generalization exists as I described below.

Some questions exist relative to the median acute fatality CCDF for MP3.

' In particular, there is a ' factor of approximately four orders of mag-nitude between the median and the mean acute fatality CCDF's due to internal failures. Ir addition, the value of the peak early fatality seems small.

This-issue is discussed further in Section 3.2. .

~

8

The other major limitatier.s of the study are related to twc factors:

1

1) Risk assessments are inherently iterative processes in

! which knowledge gained during their conduct is folded back into refinement of the characterization of important issues. Because of the time schedule for this study, only a modest amount of feedback has been p,ossible. -

2) The specific form of a PSS for use as a utility in-house decision tool is defined by the nature of the decisions the

• study is intended to support. The specific programs and decisions into which the PSS is .to fit are only now being

- clarified and, therefore, the final form of the PSS for NUSCo purposes may need to be somewhat different from its t

current form.

Two examples of the impact of the first limitation on the current study are in evidence in the seismic analysis and the risk profile implicit in the dominant accident sequences. As discussed in Section 2.2, the current seismic analysis contains significant conservatisms and less significant areas of optimism. These deficiencies exist because the iterative process is currently incomplete in the seismic area, and NUSCO plans to correct this deficiency imnediately. The second example relates to the uniformity of conservatism in the dominant accident sequences arising from-internally ini-tiated events.

It appears that some dominant sequences (e.g., those involving loss of off-site power subsequent to transient or LOCA initiators) have a great deal more conservatism than other sequences. This implies that plant-betterment project decisions based on the MP3 risk profile as evidenced by the dominant sequences may be inappropriate due to imbalanced conservatism.

The second major limitation is, again, related to the current in-complete definition of the decision process which the PSS is to support.

Evidence of this limitation is the fact that approximately one third 'of the core melt frequency is associated:with unassigned components of CCF as treated by the BFR model.

This is not clearly a problem since it does not impact the adequacy of the study relative to its ' regulatory purpose, and.since the need for a more precise definition of the. specific design and operational contrib-utors to CCF will arise from a . utilization plan', which is presently being developed by NUSCo, to define the program and ' management-decision process in .

which the PSS will be employed.

9

3.0 SPECIFlc ISSUES DESERVING OF COMMENT In addition to the sumary of findings presented in Section 2, a number of specific issues are deserving of comment. These are sumarized in this section.

3.1 CONSISTENCY BETWEEN SCOPE AND OBJECTIVES -

With the exception of limitations and qualifications noted in Section 2.4, excellent consistency between PSS scope and the multiple study objectives was obtained. As noted earlier, finalization of the study for use in support of the NUSCo decision process must await definition of the nature of the decision process as well as the specific issues requiring evaluation in the early stage of utilization.

3.2 OBSERVATIONS ON LOW NUMBERS IN THE MP3 PSS One of the obviously striking features of the results of the MP3 PSS is the exceedingly low values both of median frequencies and of peak acute fatalities resulting from internally initiated accident sequences. A median frequency of a reactor accident causing one or more acute

• fatalities in the range of 10-13 to 10-12 cannot be easily cor.prehended. Because of the barriers to comprehension of these low numbers, some comment is appropriate.

In the performance of risk assessment studies, current practice supports an evaluation in which several typically low-frequency components of overall risk are multiplied together to arrive at final CCDF's for public health effects. These components include:

1) Frequency of core melt;

2) Frequency of containment failure and large radionuclide releases given core melt;

3) Frequency distribution of health consequences given a large radionuclide release and a spectrum of possible weather conditions.

'10

In the MP1 PSS, tbe predicted mean core melt frequency (4.5 x 10 per rea ctor-yea r) is a relatively low value, but entirely consistent with I predicted frequencies frcm other studies on compa rable plants. The mean I frequency of large release (defined as resulting from release categories M1A through M7) for MP3 is predicted to be approximately 7.8 x 10-6 per reactor-year. This value, which is approximately a factor of five less than the '

predicted mean core melt frequency, is both reasonable and consistent with the ,

RSS and many other studies on comparable plants. Finally, the mean frequency of occurrence of one or more acute fatalities was predicted to be approximately 1.0 x 10-7 per reactor-year. This value is, again, reasonably consistent with those predicted in other studies, and reflects the fact that timely evacuation of pecple near the site, together with the potential for favorable weather ~

conditions, provides a significant degree of public protection against reactor-accident-caused acute fatalities.

Some observations relative to the median and 90th percentile acute fatality CCDF's for MP3 are appropriate:

1) In recard to the CCDF due to internal plant failures, there

~

is a factor of approximately four orders of magnitude between the curves. This is an unusually large difference and appears to be due principally to the choice of log-unifom as opposed to log-normal distributions on tne procability of failure of the two RHR values in the "V" accident sequence.

2) In regard to the CCDF due to external events, there is only a factor of two between the median and 90th percentile curves. Other risk assessments have shown significantly larger differences due to the large uncertainties involved in seismic analyses. When the overall seismic analysis is redone, this question should be reexamined.

3) The peak value of the median early fatality CCDF due to internal failures seems to be significantly smaller than that predicted in other . risk assessments having similar source terms and source-term DPD's. This matter should be explored further.

In sumary, while the approach utilized in the MP3 PSS in characterizing risk is conceptually the same as that followed in the RSS, the improvements in details of the various models (for instance, lower probabilities of and longer times to containment failure) used .in the assessment have r'esulted in the 11

prediction of very low probabilities of the order of 10~ per reactor-year for the peak consequences. S.uch probabilities are so smali that they raise ques-tions about whether other matters not covered in this overall approach might yield higher probabilities. However, models to cover such matters do not presently exist and no other risk assessment studies have attempted to address such issues. .

3.3 HUMAN FACTORS TREATMENT

• An early cursory review of the treatment of th~e operator's role in accident management in the PSS revealed some errors. These errors, which were primarily related to treatment of the cognitive process, were corrected, and.

the final report seems te reflect a state-of-the-art treatment of operator errors.

3.4 CONTAINMENT STRENGTH EVALUATION The containment strength evaluation represented a significant anal-ytical effort with a resulting mean failure pressure in an historically consis-tent range. One difference between the MP3 study and the Zion study was the estimation of the uncertainty associated with the failu,re pressure. The uncertainty for MP3 has been estimated to be significantly larger than for Zion. This higher uncertainty seems intuitively to be more reasonable.

e a e

a O

e 12

[ Attachment 1 MP3 PSS LEVEL 3 REVIEW BOARD CHARTEE Northeast Utilities Service Company has impaneled a special Review Board to perform a critical review of the methodology and findings of the Millstone Point Unit 3 Probabilistic Safety Study (PSS). The two responsibilities of this Review Board are: '

1) To assess the process utilized to perform the PSS to assure that the methocology being employed is consistent with the

_ study objectives and with the state-of-the-art.

2) To assess the quality of the product of the PSS both by evaluating the consistency between the study as implemented and the defined methodology, and by reviewing the study results in light of the experience of the reviewers.

The desired output of this Review Board is a written statement from the entire Board which sumarizes the findings in the following areas:

1) The competence with which the project was carried out;

2) The comparability of the methodology employed to the present state-of-the-art in risk assessment;

3) Potentially important omissions from the scope of the study;

4) The achievements and limitations of the final PSS. l In addition to the sumary of findings, the Review Board will document the scope, depth, and format of the review.

The review effort preceding preparation of this written statement will include screening of detailed information provided to the Board; attendance at the Review Board meetings periodically scheduled to monitor the adequacy of the work being performed; attendance by Board Member's supporting staff at selected project meetings; and commissioning of specialized reviews by technical experts in certain critical areas as required. .

Although the specific areas in which detailed' technical reviews are required will be decided by the Review Board, Northeast Utilities Service, Company has a special interest in the following area's:

i

\

1) Consistency Between Scope and Objectives l In undertaking the MP3 PSS, Northeast Utilities Service Company (NUSCo) has defined several objectives including:

o To provide a description bcth of the overall risk profile and of the contributors to that profile which is consistent with the current state-of-the- -

art in risk assessment; o To provide models and tools which are sufficiently

)

complete and lucid to be utilized and updated by NUSCo in future evaluations of safety issues.

It is desired that the specific methodology of the PSS be evaluated to assure its ability to support achievement of the various NUSCo objectives.

2) Quality of Implementation of the Technical Approach NUSCo has established a multi-level review program to assure that technical work performed as part of the PSS is completed in a manner consistent with the stated methodology. Although the Review Board is not ex duplicate the activities in this review process, pected to it should verify that all important aspects of a program to assure technical quality of the PSS have been carefully and thoroughly implemented.

3) Format and Scrutability Several past risk assessment studies have been criticized for the lack of traceability in certain critical areas. In order to assess the adequacy of the Millstone Unit 3 Probabilistic Safety Study from this standpoint, the Review Board will evaluate the report to determine whether logical conclusiens are drawn based on traceable and consistent arguments. This evaluation will include tracing sequences from initiating events through public health consequences.

Special attention should be paid to clarity of information sources, and to the presentation of assumptions and engineering judgment.

l

~

4) Specific Technical Areas Experience with the conduct of risk assessments has shown that several specific technical. ingredients should be reviewed carefully to assure adequacy of their treatment.

These areas include:

a) Initiator completeness, '

b) Common cause failure analysis, 2

c) human factors, d) Fiant and system models including system success criteria, e) Treatment of uncertainties, f) Post-core-melt accident process analysis, g) Cor.tain nent strength evaluation, h) E. sternal events,

1) Health consequences. ,

In addition to these technical ingredients, NUSCO -

is current'y spensoring work on an evaluation of core cooling under severe accident conditions. This evaluation,

~ which is intended to investigate the potent-ial for recovery from degraded core conditions prior to widespread core melting, will be reported as part of the PSS if resulting insights are considered to be significant.

The general approach which the Review Board should take in assessing the areas listed above should include investigat-ing the following questions:

a) Is the methodology utilized justifiable based on the current state-of-the-art?

b) Has available operating, experience been considered both as a source of data and as a guide in defining applicable methodology?

c) How does the methodology compare with that used in the Reactor Safety Study and other Probabil,istic Risk Assessment studies?

d) Are key assumptions clearly stated and justified? -

e) Are uncertainties carefully delineated, quantified, and assessed as to their effects on the end results?

f) Are currently important safety and licensing issues treated explicitly in the PSS?

g) Is the treatment of sequences similar to those which -

have occurred at nuclear power plants lucid?

h) Are the results of th'e study reasonable in light of other PRA's, and are the effects of important features at MP3 clearly ~ displ'ayed?

i 9

3

(:here deemed necessary by the Review Board, these investigations can be carried out in more depth by experienced staff members from their respective organizations. Should specific outside review be required, than the Review Board will suggest these topics to N115Co along with candidates who could perform the more detailed assessments, and an estimat,e of the time required to .

complete the reviews. .

O e

w 4

I e

t e O e

B e

O 4 4

9 e

-4

. . .