ML20056C464
| ML20056C464 | |
| Person / Time | |
|---|---|
| Issue date: | 11/16/1992 |
| From: | Ward D Advisory Committee on Reactor Safeguards |
| To: | Carroll J NRC |
| References | |
| ACRS-CT-2053, NUDOCS 9306240185 | |
| Download: ML20056C464 (5) | |
Text
.
L T-ao 63 t
PW Ha@3 cc: Paul Boehnert 11/16/92 To:
JAY CARROLL, Chairman of Human Factors Subcommittee From: DAVE W o
ant COMMENTS ON 11/12/92 SENIOR MANAGEMENT MEETING
" PRODUCTS AND APPLICATIONS RESULTING FROM ORGANIZATIONAL FACTORS RESEARCH" Summary of the meetina:
I The meeting had originally been planned as a workshop to solicit from NRC Senior Managers information on how they, as individuals who have to actually do this, make judgments about the performance of licensee organizations. The idea was that these in-use methods would then be compared with methods for doing the same thing which had been developed in the research program. That seemed a most interesting but overly ambitious agenda for the workshop. It would put the Senior Managers on the spot to describe (and implicitly, to defend] their methods. As it happened, the purpose of the meeting evolved to something more routine and the workshop concept was dropped. Namely, the researchers were to describe to the Senior Managers what they had done and the Senior Managers were then to help the RES staff decide whether and how to " wind up the program". The first part of that was accomplished, but, while there was some usoful feedback and comment from the Senior Managers, the second part was not.
This research project began in 1987 and a large number of small subtasks were parceled out to people at universities, national labs, and others. Contacts were made with programs doing related work in the UK and in Sweden. Over the last year or two a process of integrating, and bringing coherence to, the many activities has been accomplished by four principals, namely researchers from Brookhaven (Sonya Haber], Penn State [ Rick Jacobs], UCLA (George Apostalakis], and a consulting outfit called APG [Doug Orvis].
The purpose of this Organizational Factors Research has now been focused into the
/
creation of two products:
1] A set of characteristics or attributes which can be assessed in a particular plant Q
and used as part of an evaluation of the effectiveness of the plant organization. This would be used in inspections, SALP, diagnostic evaluations, and event analysis.
h, 2] A means to quantify the contribution to risk of these attributes in a PRA for a gr.en plant. This would satisfy an early NRR request for a tool to quantify the risk h
contribution involved in the sort of things SALP evaluations reveal.
E
".3 The research has identified 20 organizational " dimensions" which are believed to "X
N characterize the ability of an NPP organization to safely operate the plant. These g
have been grouped into.5 ofganjzational " factors". There is general agreement 93o6240185 921116 f
\\
l 2
i cmong th] researchers on th3 20 dimensions. Thera is'not y t con:en;us on grouping into the 5 factors. However, there is agreement that 20 is probably too many for practical use, and especially for quantification.
The 20 dimensions have been verified to be apt descriptors of an organization's performance to the extent that they have shown to fit more than one plant and to have evolved from the research of more than one group. They also are reasonably satisfying intuitively -- for what that is worth. The dimensions have not, however, been validated as predictors of the actual safety performance of a plant in the sense that a high score on one, or some, or all of the 20 dimensions can be shown to significantly correlate with high safety performance measured by some other means.
The dimensions and factors are listed below (factors are the subheadings, in capital letters):
CULTURE Organizational Culture Ownership Safety Culture Time Urgency COMMUNICATIONS Communication - External Communication -Interdepartmental Communication - Intradepartmental DECISION-MAKING Centralization Goal Setting Organizational Learning Problem lesntification Resource Allocation ADMINISTRATIVE KNOWLEDGE Coordination of Work Formalization Organizational Knowledge Roles-Responsibilities HUMAN RESOURCE ALLOCATION Performance Evaluation Personnel Selection Technical Knowledge Training identifying, defining, and verifying these attributes as predictors of relative safety performance is one thing, measuring them in a particular plant is another. The research has developed three different techniques or instruments [e.g., surveys and observations of plant workers) for measuring the 20 dimensions. Ten of the dimensions are measured by all three techniques, five by two of the techniques. and five by only one. Several of the techniques are not yet fully developed.
3 The second part of the rese rch has cttempted t3 move beyond good /better/best qualitative evaluations to methods that can use the organizational dimensions or factors in actually quantifying risk increases or reductions predicted to resutt from higher or lower organizational capabilities. The first step in this effort has been to define a " work process' or standardized sequence of tasks which can be used to describe the jobs an organization carries out to achieve any particular goal. The next step is to make use of the data on organizational dimensions to quantify successes and failures along the logic paths of the work processes. This all seems to be a daunting task. It is. So also was the original development of PRA.
Much work remains:
V&V of the " dimensions" -- a major task; cant be done in a few weeks or months Reducing the 20 dimensions to a more practical number; then V&V of those Completion of measuring instruments; including field testing Completion of the quantification and PRA methods Imoortant comments at the meetino:
Tom Murley made an insightful observation vis a vis how NRR is now evaluating organizations. He said their implicit assumption is that the only important organizational problem is complacency in upper level management at licensees.
Then it is assumed that dinging the plants across the board, from HP to maintenance, will somehow keep the upper management generally on their toes. He suspects this to be an inadequate model. I agree.
Joe Murphy expressed concern about getting too far out in front of the industry with this program. EPRI and INPO have not expressed much interest. Reportedly, NUMARC is polling utilities to determine if anyone is working in this area.
In my opinion, it appears to be necessary for NRC to take the lead, as it has in many other programs in the past [ e.g., TH, heavy section steel failure, PRA, SA) when industry was not doing anything and there was believed to be a reactor safety need.
So, getting too far out in front is a red herring. The question is whether there is a safety need. If there is, and industry is dragging its feet, NRC has a responsibility to take the lead.
Tom Murley also raised a good question about George Apostalakis' proposed method. In his method one would look at sequences shown to be important in traditional PRAs and establish whether they were made even more important by possible organizational failures. This is OK, but would overlook possible new sequences important just because of organizational failures. Apostalakis admits mis is a flaw in his method, but says being more thorough is impossible with resources available for the project. [See comment below.)
i George Apostalakis compared his program to the seismic PRA program; both are attempts to incorporate common cause influences into the analysis of redundant systems in PRA. He said far more effort has gone into the seismic approach, all et SSMRP for example, than into what he is trying to do now, in my opinion, NPPs are much more at risk from organizational failures than from earthquakes. How can NRC justify such disproportionate expenditure of its resources?
4
- Tcm Leamon questioned whether the 20 dimensions have been shown to be independent and exclusive of each other. If not, some considerable analytical and V&V difficuttles could be presented down the road. Rick Jacobs acknowledged this to be a problem; the dimensions are probably not exclusive. He said time and resources have not been sufficient to permit a fuller investigation of this.
We have heard about a program at MIT which was to be funded privately and was i
to look at a number of things, including management and organizational i
effectiveness. Joe Murphy said he understands this effort not to be moving very fast or far; it has not gotten anticipated funding.
Jim Sniezek objected to a researcher's proposal that their product could be useful in SALP. He says this is running counter to the agency's intent to look at performance rather than process in making SALP evaluations. I see Jim's point but dont believe it is entirely valid. He would seem to reduce SALP to nothing more than looking at performance indicators.
In my opinion, purely performance-based regulation is an ideal whose time is not likely to come. The primary reason for this is that any Pl on which data can be gathered is just a surrogate which we hope, but can never prove, relates to the only actual measure of performance we really care about, i.e., no severe accidents.
Some oorsonal comments:
" Regulatory need", as a reason for research, should not be construed too narrowly.
The immediate products of the OFR program can probably be developed into something practical and useful, but they are not now at that state. However, just as 1
important as completing the particular development of these products, is the wedge that this program offers into understanding and ultimately doing something about the more generalissue of optimizing the organization and management of the human resources at NPPs to ensure adequate safety. We cannot honestly claim this area is now under control.
What is the basis for a statement that there is no regulatory need at this time?
OFR has been going on in a modest way for 5 years. Some fairly clear results are emerging, but the work is not finished. The Commission and NRC's Senior Management have been patient. There are strong indications that their patience has been rewarded and that still more patience is warranted. The program is not finished and an attempt to " wrap it up" now will result in loss of all benefits and an unjustified black eye for the effort, if the program is continued at its current level for about two more years there may then be a point reached where a useful phase has been completed. However, it would be best to plan on continuing an effort in this general area for an indefinite period. An arbitrary goal might be to end it five years after TH research is ended.
The chief benefit of the PRA part of this program is to force diccipline onto the whole effort. I dont object to generating numbers by expert opinion, but we should not gwe too much credence to them. It is much like Level 2 of a traditional PRA in that respect.
m m
5
- Kramer's wrap-up plan is not adequate. The RES lower level managers don't have the clout / courage to overcome apathy and antipathy within higher level staff. A strong ACRS statement is needed to get around the Staff to the Commission which I believe will have more vision. Enough has been accomplished to demonstrate that this is a doable program. But, it is far from finished.
Suaaestions for an ACRS oosilloni l believe the ACRS should recommend continuation of the program for two more years at about the present level of funding. Goals to be sought during this time would be:
1] Reduce the number of " dimensions' to a smaller, mutually exclusive and independent set.
2] Validate, to the extent possible, the final set of dimensions or factors against independent measures of safety performance.
3] Field test and optimize the measuring instruments.
4] Continue the effort to quantify the factors and to apply them to risk estimates.
This task does not have a simple end point, nor does the optimization of other PRA methods. A goal might be to provide for a method of applying OF to risk I
which is commensurate with the methods for seismic risk.
i
- 5) Toward the end of the two-year period assess the research and results for how best to continue a program in this area. The expectation should be ; rut a continuing effort of research in this general area will be appropriate.
4 l
i