ML20115B105

Evaluation of Detailed Control Room Design Review Suppl II to Summary Rept for TMI-1, Technical Evaluation Rept

ML20115B105
Person / Time
Site:	Three Mile Island
Issue date:	03/04/1985
From:	SCIENCE APPLICATIONS INTERNATIONAL CORP. (FORMERLY
To:	NRC
Shared Package
ML20115B100	List: ML20115B102 ML20115B105
References
CON-NRC-03-82-096, CON-NRC-3-82-96, RTR-NUREG-0737, RTR-NUREG-737 NUDOCS 8504170376
Download: ML20115B105 (13)

Text

1 EflCLOSURE 4 EVALUATION'0F THE DCRDR SUPPLEMENT II TO THE

SUMMARY

REPORT FOR THREE MILE ISLAND, UNIT 1 March 4, 1985 Prepared by Science Applications International Corporation 1710 Goodridge Drive

.McLean, Virginia 22102 Prepared for 4 U.S. Nuclear Regulatory Commission Washington, D.C. 20555 L Contract NRC-03-82-096 l

B504170376 ADOCKB50408 05000289 ,

i- PDR F .PDR . .;

l

FOREWORD This Supplemental to the Technical Evaluation Report (TER) was prepared by Science Applications International Corporation (SAIC) under Contract NRC-03-82-096, Technical Assistance'in Support of NRC Licensing Actions:

Program III. The evaluation was performed in support of the Division of Human Factors. Safety. Human Factors Engineering Branch (HFEB). SAIC previously evaluated GPUN's program plan and summary report submitted concurrently for. the DCRDR conducted for Three Mile Island - Unit 1.

Results of that evaluation are described in a Safety Evaluation Report (SER) prepared by HFEB and transmitted to the licensee. Subsequent to issuance of the-SER, a meeting and documentation audit were held with GPUN. GPUN submitted a supplement to its summary report in June 1984 which was considered in a subsequent evaluation performed by SAIC A pre-implemen-tation audit was conducted at the Three Mile Island site on October 4,1984 and.as a result,'GPUN submitted a second supplement to its summary report on -

December 14, 1984 This report. includes the SAIC evaluation of the licensee's DCRDR .

summary report, supplement II (Reference 1). Findings from the pre-implementation audit (Reference 2) were also considered in preparation of this evaluation.

l 1

EVALUATION OF THE DCRDR SUPPLEMENT II FOR THREE MILE ISLAND, UNIT 1 This report documents the Science Applications International Corpo-ration (SAIC) evaluation of Supplement II of the summary report of the DCRDR (Reference 1) submitted to the.NRC on December 14, 1984 by GPUN for Three LMileLIsland - Unit 1. ~

This evaluation also considers information obtained

from a pre-implementation audit held at the Three Mile' Island site on October 4, 1984 (Reference 2). Further information regarding DCRDR activities was acquired from a review of the program plan and summary report submitted concurrently by the licensee (Reference 3), the supplement to the summary report submitted on June 29,1984 (Reference 4), and a GPUN report prepared in 1980 (Reference 5). DCRDR activities were discussed with the -

licensee at a meeting held between the NRC and. GPUN on March 27, 1984 (Reference 6) and a documentation audit meeting held on April 11, 1984-(Reference 7). This report may be considered as an addendum to our earlier report dated August 15,1984 (Reference 8).

Results of the SAIC evaluation of Supplement II of the summary report of the DCRDR follow a brief overview of the background leading up to preparation and submission of the report by the licensee.

BACKGROUND GPUN essentially began its human factors review of the control room at TMI-1 in February 1980 prior to the issuance of the DCRDR requirements stated in NUREG-0737, Supplement 1 (Reference 9) and the methodology I suggested in NUREG-0700 (Reference 10) or other appropriate guidance. The program plan for TMI-1 was submitted to the NRC concurrent with the summary L

report in January 1984. This ' document included findings from the completed L portions of the review, highlighting changes that had been accomplished to date and activities still incomplete. NRC comments on this document were provided to the licensee via reference 8.

t l Further information regarding DCRDR activities was acquired at a

[ meeting held between GPUN and the NRC in March 1984 and a documentation i 1 b

L

. _ , _ . .u. _ _ - . . _ - - - _ . _ _ _ _ _ _ _ _ - . . . _ . _ _ _ . _ _ . _ . _ . _ _ . _ . _ _ . _ _

audit meeting held in April 1984. A supplement to the summary report was submitted by the licensee to the NRC in June 1984. This supplement documented a review of the functions and tasks implicit in new, abnormal transient operating procedures established in.1983-4 to complete GPUN's DCRDR for.TMI-1.

The pre-implementation audit was conducted at the TMI-Unit 1 on October 4, 1984.

The audit included observation of the TMI-1 control room and remote shutdown panel, briefings by and discussions with various members of the GPUN DCRDR team and their human factors consultant (MPR Associates), and walkthroughs of- an emergency operating procedure with operators. A review of the remote shutdown panel was also conducted in addition to a walkthrough of an emergency procedure used during the DCRDR.

More specifically, the October 1984 audit was-focused on soliciting ~

more information regarding the conclusions reached in our TER of August 15, 1984. Based on consideration of the pre-implementation audit findings and -

all documentation submitted by the licensee to date, it was decided that the licensee should prepare a revision to its supplement report to help facilitate a determination as to whether outstanding issues relative to NUREG-0737, Supplement I requirements had been fulfilled satisfactorily at TMI-1. The revision was requested in order to:

1. Provide additional resumes and define the functions, responsi-bilities, and levels of effort for staff involved in the 1983-4 review;

2. Describe the sequence and the process used to determine information and control requirements / characteristics. The licensee was to generically describe the process used to conduct in-depth interviews with control room operators during the 1983-4 review. GPUN was to provide some examples of HEDs generated as a result of the interview process and their resolutions.

Essentially, GPUN was to trace the~ analysis process that was used to generate an HED. The licensee was also to identify which E0Ps were included in the 1983-4 review.

2

e

3. Confirm that the same verification methodologies used in the 1980 review were also used during the 1983-4 review.

4. ' Provide implementation schedules for all HEDs identified with safety consequences from the 1983-4 review.

5. Provide documentation describing the ongoing human factors program at TMI-1 which relies to some extent on operator input.

6. . Confirm that the guidelines and human factors criteria used by GPUN to conduct its control room survey were comparable to those suggested in NUREG-0700.

Satisfactory resolution of these issues would assure that the DCRDR process fulfilled the requirements of NUREG-0737, Supplement 1. -

DISCUSSION The following provides a statement of the problem (Reference 3) related to each issue cited above, a summary of the licensee's response to the problem as found in Supplement II, and SAIC's evaluation of the response in terms of the licensee's fulfillment of the Supplement 1 to NUREG-0737 requirements.

ISSUE I: 1983-4 Review Team Based on information provided by the licensee in previous submittals and at meetings , the 1980-83 DCRDR activities were conducted by a team which included human factors consultants, GPUN staff, and personnel from MPR Associates. The organizations were qualified for tasks for which they were responsible and the structure and management of the DCRDR were flexible

.enough to permit a multidisciplinary effort.

Little reference was made to staffing or levels of effort involved in the licensee's 1983-4 review using upgraded E0Ps. This information was considered necessary to fully evaluate staffing for the complete control room design review. GPUN agreed to provide this information, including staff resumes and information on functions, responsibilities , and levels of 3

effort of staff involved in this review, in Supplement II to its summary report.

As documented in Supplement II, a mul tidi sci plina ry team was established to conduct the 1983-4 review. The team included MPR personnel and GPUN staff with expertise in engineering, safety analysis, and human factors engineering. Although levels of effort were not provided, it appears that the participating individuals were qualified for review tasks for which they were assigned. These individuals were assisted at various stages of the-review by control room operators, shift supervisors, and shift technical advisors.

ISSUE 2: Determination of Information and Control Characteristics GPUN began its system function and task analysis of selected procedures '

in 1980 prior to the issuance of NUREG-0737, Supplement 1 and the guidance suggested in NUREG-0700. Between 1983-4, a review was conducted which -

addressed the system function and task analysis (which had been used as a basis for development of emergency operating procedures) required by NUREG-0737, Supplement 1. As part of this review, GPUN, using the Abnormal Tran-sient Operating Guidelines (ATOGs) developed by B & W as a starting point, developed 10 plant-specific procedures for TMI-1 (ATPs).

Once these procedures were developed, a number of activities were conducted as part of the TMI-1 system function and tasks analysis. These included: 1) walkthroughs of the procedures; 2) training exercise observation; and 3) a desk top analysis of tasks implicit in the procedures.

Documentation of these processes suggeste_d that GPUN performed a verification of procedures and task performance capabilities for all upgraded E0Ps. Whether GPUN systematically determined information and control requirements and the characteristics of those requirements independent of the control room was not clear.

Information acquired at the October 1984 pre-implementation audit did help clarify and support methods used by the licensee during the DCRDR system function and task analysis activities. A walkthrough of a control room abnormal transient procedure (ATP) conducted on site demonstrated that all instruments and controls required to implement the steps of the 4

procedure were present in the control room. Characteristics of information and controls were determined to be adequate. These findings helped support and substantiate the methodology used by the licensee to determine the information and control requirements and characteristics.

However, in light of the fact that written documentation had not been provided in previous submittals, the licensee was asked during the audit to describe in writing the sequence and process used to determine information j and control requirements (i.e., characteristics) in a second supplement to its summary report. The walkthrough process also needed to be documented.

Examples of HEDs generated as a result of these processes and their solutions were to be provided as well.

Based on information provided in Supplement 11 to the summary report, GPUN conducted its system function and task analysis using a number of processes. GPUN Human Factors staff performed a task analysis of the developed ATPs to assess adequacy of existing control room displays and .

controls. Training exercises were observed in the Lynchburgh Simulator in order to promote- the reviewer's familiarity with the new procedures and to identify tasks where operator time was of the essence. A desk top review was conducted during which procedures and the control room were analyzed from the view point of maintaining control of the variables important to plant safety.

Walkthroughs also were conducted with a CRO, SRO,.and shift technical advisor. The findings and comments collected from the walkthroughs dealt with the degree to which control room displays and controls facilitated the performance of procedure tasks, the effectiveness of the procedures themselves, and the effectiveness of operator training.

From a review of Supplement II to the Summary Report, it is clear that GPUN did not perform a task analysis in a totally systematic fashion independent of the control room and procedures. As stated by GPUN, "Because the review was performed after the procedures had been put in place, a

' clean slate' prescriptive task analysis, as has been performed on several new plants to develop symptom-oriented emergency procedures, was inappropriate" (Reference 1, p g. 4). Rather GPUN used processes to determine if the control room supported the information and control 5

requirements of -the procedures (Reference ~1, p g. 1). GPUN has not, documented, however, what steps were taken during the review to provide assurance that the specifics of the procedures had not been inappropriately  ;

biased to suit the controls and displays of the TMI-1 control room or to suit the' operator or training practices in place at TMI-1 prior to implementation of the procedures.

Of the many processes used by GPUN, the walkthroughs and observation of training exercises resulted in an evaluation of th'e procedures and verification that the existing control instrumentation could support the

-tasks implicit in the ATPs. Furthermore, these activities provided some measure of training effectiv.eness. In light of GPUN's stated objectives for these activities, it is not surprising that HEDs generated and documented by GPUN during the review were primarily procedural in nature. GPUN contends.

that its task analysis and desk top analysis were conducted independent of the control room. The absence of data collection forms and written documentation regarding methodology, however, makes it impossible for us to '

fully evaluate the degree 'to which control and display requirements and characteristics were identified which then could have been validated in the control room.

Overall, we conclude that GPUN, in its 1984 review at TMI-1, did not conduct a function and task analysis that resulted in the identificat-ion of information. and control requirements and their associated characteristics totally independent of the control room. Therefore, GPUN has not fully met the requirement of Supplement I to NUREG-0737. However, since the onset of its DCRDR activities in 1980, GPUN has identified and corrected numerous HEDs at TMI-1. A review of the identified HEDs indicated that they were similar in type to those that might have been detected if GPUN had imple-mented a system function and task analysis which would have satisfied the I

requirement of NUREG-0737, Supplement 1 (Reference 8). . Furthermore, in 1983-4, GPUN upgraded, verified and modified as necessary the emergency operating procedures at TMI-1 and training of those procedures. Therefore, we believe GPUN has engaged in activities which have led to a significant improvement in the human factors engineering of the TMI-1 control room, thereby promoting the overall , objective of NUREG-0737, Supplement 1.

4 6

t

_.r_._ . ._ ,. ,_ _ _.._. . _

ISSUE 3: Verification that Design Improvements Would Provide Necessary Correction and Not Introduce New HEDs As part of GPUN's 1980 review, verification was accomplished by a process whereby full scale drawings of panels showing -design improvements were placed in the mockup. Walkthroughs were conducted and operator feedback was solicited to verify that corrections were compatible with task performance and did not introduce new deficiencies. The mockup of changes also was reviewed against human factors criteria.

GPUN'.s verification methodology used during the 1980 review was evaluate,d and found to meet the requirements of Supplement 1 to NUREG-0737.

The licensee had fail.ed, however, to describe the verification process used during the 1984 review although the presence of an ongoing human factors program at TMI-1 suggested that such verification was accomplished. The '

licensee wac, therefore, asked to describe processes employed and to describe the t,ngoing human factors program in this supplement to the summary -

report.

As described in Supplement 11 to the summary report, GPUN employs two human factors engineers full time to maintain a human factors program at THI-1. Primary functions of the staff include:

o review of all new plant modifications; o upgrading of existing man-machine interface; o human engineering of E0Ps; and o maintenance of control room labeling alarm tiles, mimics and demarcation by assigning location and issuing purchase specifi-cations.

Furthermore, all new systems or modifications to existing systems require a System Design Description. This document is reviewed by the human factors staff both to assure an adequate new design and to ensure that the changes will not adversely impact existing control room configurations. All such descriptions must comply with GPUN's in-house Human Engineering Guide 7

and include a section documenting the human factors review. Scope of the human factors review is based on the extent of the man-machine interface and will' include walkthroughs, task / function analyses, user surveys, or other similar procedures..

In conclusion, it appears that the operation of the human factors program at TMI-1, as described, has and will continue to be responsible for verifying that design improvements will provide necessary correction and not introduce new HEDs.

ISSUE 4: Scheduling of Unresolved HEDs NUREG-0737, Supplement 1 requires each licensee to submit a proposed schedule for implementing design changes. Based on documentation previously provided -by GPUN, the licensee has corrected HEDs with safety significance ~

identified from the 1980 review. Non-safety related HEDs and those identified during the 1983-4 review are being handled by the ongoing human -

factors modification program at TMI-1 described above. Implementation schedules'for HEDs identified with safety consequences from the 1983-4 review needed to be provided in the supplement to the summary report.

The licensee has provided solutions and/or schedules for safety-significant HEDs identified during the 1983-4 review. We concur with the proposed solutions for HEDs. _Also we concur with the justification for not correcting or partially correcting HEDs and schedules for correction.

ISSUE 5: Control Room Survey Guidelines and Human Factors Criteria NUREG-0737 Supplement I requires the conduct of a survey of the control room components to identify characteristics of instruments, equipment, layout and ambient conditions that do not conform to good engineering practices. To conduct its survey, at TMI-1, GPUN developed checklists from guidelines contained in MIL-STD-1472B (Reference 11) and human engineering references such as VanCott and Kinkade (1972) (Reference 12) and Woodson and Conover (1964) (Reference 13). The development of such guidelines was necessary as GPUN conducted its survey in 1980, prior to the issuance of the NRC DCRDR guidelines (NUREG-0700).

8

It~ appears from both discussions with GPUN and a review of documentation that the scope of the control room survey essentially was

-comprehensive and included consideration of al.1 primary control panels.

However, GPUN was asked to provide in a supplement to its summary report, documentation around a comparison if conducted, of its survey guidelines and NUREG-0700 guidelines. Such information was deemed necessary to determine if the specific guidelines and criteria used by the licensee were in accordance with accepted human factors guidelines and criteria such as those suggested by NUREG-0700.

GPUN failed to submit in its supplement information regarding the comparability of its survey guidelines to those suggested by NUREG-0700.

The completeness of the survey cannot be assessed fully until the licensee provides documentation to show a favorable comparison between its guidelines and those in NUREG-0700.

CONCLUSIONS -

Documentation included in GPUN's Supplement II to its summary report has helped demonstrate the licensee's satisfaction of the requirements l

specified in NUREG-0737, Supplement 1. The following is a summary of our comments on GPUN's compliance with each of the NUREG-0737, Supplement I re-quirements which had not been fully evaluated prior to receipt of this submission:

o It appears that a qualified team was established to conduct the DCRDR activities conducted in 1983-4 using upgraded E0Ps.

o The licensee's system function and task analysj s include.d walkthroughs, observation of training exercises, and some desk top analyses of upgraded plant emergency operating procedures.

Procedures and task performance capabilities were verified.

Training and procedural formats were evaluated. The process (es) to determine the characteristics of information and control needs independent of the control room was not completely described or documented. As a result, GPUN has not completely satisfied the NUREG-0737 Supplement I requirement for a system function and task analysis. However, GPUN's system function and task analysis 9

_ _ - _ - _ _ _ _ _ _ _ _ _ _ _ _ _ b

.C.

I!

I activities since 1980 have led to numerous changes being made in

' the control room. These changes have resulted in a significant improvement in the human factors engineering of the control room.

o An ongoing human factors program at TMI-1 was and is responsible for ensuring that selected design improvements will provide the

. necessary corrections ' without ' introducing new deficiencies. This satisfies the verification requirement of NUREG-0737, Supple-ment 1.

o GPUN has corrected almost all identified HEDs. A few, however, have not been corrected or are only partially corrected at this time. The licensee has provided schedules to show that HED corrective actions will be implemented in an acceptable time ,

frame. Adequate justifications have been provided for HEDs not corrected at this time.

o- GPUN has conducted a control room survey using guidelines developed in 1980. The licensee stated that it compared its guidelines to NUREG-0700 and found that the guidelines actually used were comparable if not more rigorous. Documentation of this comparison has not been-provided for NRC review. lii order to confirm the completeness of the survey effort,at TMI-1, the NRC has requested GPUN to submit information sitowing a favorable comparison between its survey guidelines and criteria and those suggested in NUREG-0700.

10

o REFERENCES

.1. "TMI-1 Detailed Control Room Design Review (DCRDR) Summary Report Supplement II",- GPUN Corporation, December 14,1984.

2. Pre-Implementation Audit at Three Mile Island-Unit 1 site, October 4, 1984. -

3. " Program Description and Summary Report of the Detailed Design Review of the Three Mile Island Unit 1 Control Room," GPUN Corporation, NRC

. Accession Number 8401230325 November 1983.

4. '" Supplementary Report of the Detailed Control Room Design Review for Three Mile Island Unit 1," GPUN Corporation June 1984.

5. "A Review of the Three Mile Island Unit 1 Control Room from a Human -

Factors Viewpoint," GPUN Corporation, December 1980.

.6. Meeting between the NRC and GPUN, Washington, D.C., March 27, 1984. .

7. Meeting between the NRC and GPUN, Washington, D.C., April 11, 1984.

8. " Evaluation of the Detailed Control Room Design Review Program Plan and Summary Report for Three Mile Island - ' Unit 1 (Rev. 3), SAIC, August 15, 1984.

9. NUREG-0737, " Clarification of TMI Action Plan Requirements," No'vember 1980 Supplement 1 December 1982.

10. NUREG-0700, " Guidelines for Control Room Design Review," Septemaer-1981.

11. Military Standard MIL-STD-14728, Human Engineering Design Criteria for Military Systems Equipment and Facilities, Department of Defense.

Washington, D.C., December 1974.

12. VanCott, H.P. and Kinkade, R.G.~, Editors, " Human Engineering Guide to (

Equipment Design," Revised Edition, Government Printing Office,1972.

13.- Woodson, W.G. and Conover, D.W., " Human Engineering Guide for Equipment Design," University of California Press,1964.

11

_ _ _ - - _